Search code examples
pythonresourcesniceulimit

Python: ulimit and nice for subprocess.call / subprocess.Popen?

I need to limit the amount of time and cpu taken by external command line apps I spawn from a python process using subprocess.call , mainly because sometimes the spawned process gets stuck and pins the cpu at 99%.

nice and ulimit seem like reasonable ways to do this, but I'm not sure how they'd interact with subprocess.

  • The limits look something like:
    • Kill the process if it's taking more than 60 seconds
    • Limit it to 20% of cpu
  • I want to apply the resource limiting to the subprocess, not to the python process that's spawning the subprocesses.

Is there a way to apply nice and ulimit to the subprocess.call spawned process? Are there better python-native alternatives?

This is on a linux (ubuntu) system.

Solution

  • You can set limits for subprocesses with the ulimit and nice shell commands like this:

    import subprocess
subprocess.Popen('ulimit -t 60; nice -n 15 cpuhog', shell=True)

    This runs cpuhog with a limit of 60 seconds of CPU time and a niceness adjustment of 15. Note that there is no simple way to set a 20% CPU throttle as such. The process will use 100% CPU unless another (less nice) process also needs the CPU.