Search code examples
pythonconcurrencyparallel-processingpython-multiprocessinggnu-parallel

Using multiprocessing and GNU Parallel at the same time?

I have a Python script that is currently using multiprocessing to perform tasks in parallel. My advisor recommended me to use GNU Parallel to speed it up since Python programs always execute on a single core. Should I keep the multiprocessing script as it is and use GNU Parallel on top of it? Or should I remove the multiprocessing part and then use GNU Parallel? Does it make a difference?

Solution

  • Does it make a difference?

    There is a really simple answer: Try it and measure.

    The performance of parallelization these days depends on so many factors and it really depends on your application (and sometimes on your hardware).