Search code examples
pythonmultithreadingkernelcpythongil

Under what circumstances would the OS be aware of threads created by Python?


My understanding is that in general the OS Kernel (e.g. Linux) does not really have visibility into any threads created in user space. Moreover, I believe in Linux specifically the OS kernel only sees "tasks" or processes, and not "threads" per se.

Additionally, I'm not sure if CPython would actually ever use multiple threads for anything unless one uses explicitly e.g. the threading package in Python or one of the C standard Python libraries chooses to create threads itself. I suppose the exception here is multiprocessing or any Python libraries that may also spawn multiple processes, in which case the OS Kernel sees them as separate.

Under what set of of circumstances would the OS Kernel be aware of any threads created by a Python script, including any of the modules it may use?


Solution

  • In Linux:

    task is a thread, process is an address space with one or more threads.

    User-space threads are directly represented in the kernel. There is POSIX API to create threads pthread_create and there is Linux implementation of threading APIs called NPTL which maps one user-level thread to kernel-level task (task_struct).

    CPython multithreading module implements threads using pthread_create.