Search code examples
pythonnumpycythondistutilsmingw-w64

Pyximport build errors


Right now I'm trying to compile and run Deep Q Learning. I've got everything set up and working except for one issue related to pyximport.

In ale_data_set.py, it imports pyximport to compile shift.pyx. The error I receive:

ImportError: Building module shift failed: ['ImportError: DLL load failed: A dynamic link library (DLL) initialization routine failed.\n']

When I run the lines at the top of ale_data_set.py, I get the same error:

import numpy as np

import pyximport; pyximport.install()

import shift

The thing is, sometimes I can get it run import shift just fine. Sometimes I'll be able to reload(pyximport), then call pyximport.install() and import shift just fine. Other times it doesn't work.

The interesting thing to me is that the times that it doesn't work, when I exit (via control-c), I get this:

Exception KeyboardInterrupt in <module 'threading' from 'C:\SciSoft
\WinPython-64bit-2.7.9.3\python-2.7.9.amd64\Lib\threading.pyc'> ignored

This seems to tell me that it's still compiling in the background, or at least the thread has not returned. It makes sense to me that the sometimes that it works, I just happen to be calling import shift after it's finished compiling.

Does anyone know what may be causing this?

If it's just a threading issue, is there a way for me to get the thread from compiling shift so I can wait it?

Edit: I've also tried setting the numpy include_dirs and that has not helped.

I've also checked to make sure there are no spaces in the path, that GCC is in the PATH, and that the MS compiler is also installed.


Solution

  • So I'm still not sure why it is not importing properly after calling pyximport, but I did notice that shift.pyd was being created seemingly properly. As such, I took the file (in the build output directory) and copied it over to *PythonDir*/Lib/site-packages. I can now just call import shift without using pyximport and it seems to be working just fine.