Search code examples
pythonlinuxparallel-processinggnu-parallel

spawn n workers with GNU parallel without input


I would like to trigger n jobs doing the following work:

# work.py
import time
import random
while True:
    print(random.randint(0,9))
    time.sleep(1)

where n is the number of available cores.

How can GNU parallel be used in such situation?


Solution

  • You should be able to use:

    seq $(parallel --number-of-cores) |
        parallel -N0 --line-buffered python3 -u ./work.py
    

    But I wonder why you don't use Python's built-in multiprocessing, rather than introduce an additional dependency.