Search code examples
pythonperformancescipy

Performance of Scipy lsim/lsim2


In a python script of mine, I currently use the lsim function to simulate a system model. An issue I encountered recently is that lsim spawns a lot of subprocesses on multiple cores, and together they cause heavy CPU load. That fact also shows in the profiling log, where I attached the relevant snippet below. I run this script on a processing machine, which I share with multiple people. If I instead use lsim2, it seems that no subprocesses are spawned, however running the script takes unbearably long. Anyone has an idea how I can run lsim fast, while using fewer resources/just one core?

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
     3740   25.422    0.007   51.062    0.014 /grid/common/pkgs/python/v3.7.2/lib/python3.7/site-packages/scipy/signal/ltisys.py:1870(lsim)
 26753891   21.519    0.000   21.519    0.000 {built-in method numpy.dot}
       12    0.001    0.000   21.450    1.788 /grid/common/pkgs/python/v3.7.2/lib/python3.7/subprocess.py:431(run)
       12    0.000    0.000   21.265    1.772 /grid/common/pkgs/python/v3.7.2/lib/python3.7/subprocess.py:895(communicate)
       24    0.000    0.000   21.265    0.886 /grid/common/pkgs/python/v3.7.2/lib/python3.7/subprocess.py:985(wait)
       24    0.000    0.000   21.265    0.886 /grid/common/pkgs/python/v3.7.2/lib/python3.7/subprocess.py:1592(_wait)
       12    0.000    0.000   21.264    1.772 /grid/common/pkgs/python/v3.7.2/lib/python3.7/subprocess.py:1579(_try_wait)

Solution

  • Setting the environment variable OPENBLAS_NUM_THREADS=1 did the trick for me