Search code examples
pythonmultiprocessingcpythonpython-internals

Does cPython use multiple cores for built-in functions such as sort, any, all?


I understand that cPython has a GIL so that your script can't run on multiple cores without using the multiprocessing module. But is there anything to stop the built in functions such as sorting using multiple cores? I don't understand cPython structure but I think the question I'm asking is 'are builtin functions like sort, any and list comprehensions actually below the GIL?


Solution

  • The cPython GIL has to do with only allowing a single thread to run bytecode within a process -- it's not related to the non-abstracted CPU.

    That said, as of now, unless you're calling something to fork/use multiple processes or your OS/hardware is catching calls and doing this for you (highly unlikely), you will see all your operations happen on a single CPU core.

    Built-in functions that are implemented in C happen "below the GIL" as they're more direct calls to underlying APIs, but putting arguments and data to those functions happens within the GIL, as you're using bytecode to read and write.

    As an aside, if you want to better understand the cPython relationship to its host, I'd suggest the following high-level official overview and/or the PDF slides and the playground that I wrote for a conference.