Search code examples
pythonpandascudatheanokeras

How to know what is being written in Python during import module?

It is very strange on one of my Ubuntu when I import modules, for one module the pandas, it will take random time (from 0.9 to 160s ) to completed. Import pandas in 161s

I am not sure what cause the problem, but I found that during module loading python is writing something to the Disk with IOW/s= 2M .

So my question is:

Is there a way to track which file Python is writing to during execution?

Python 2.7 , Cuda 8.0 RC , cuDnn 5.1, Theano 9.0-dev, keras , pandas 0.18.1

After tracking I found python is writing to the /home/username/.nv/ComputeCache , so maybe one of the GPU related libs cause this problem. But I have no idea why only pandas is affected .

Thanks all, I find a workaround: import pandas first.

Solution

  • Use strace to trace system calls.

    $ strace -ttt -feopen,write -o log python -c 'open("foo", "w").write("blah")'
$ cat log
...
122157 1468546777.800508 open("foo", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 3
122157 1468546777.800733 write(3, "blah", 4) = 4
122157 1468546777.804145 +++ exited with 0 +++

    This should show what files are being opened and written to, and the time of each event.