Search code examples
pythonvisual-c++cython

Issue when compiling Cython code with more than 1000 global constants


Let's say test.py has many constant definitions (use case: coming from an API):

CONSTANT0 = 0
CONSTANT1 = 1
CONSTANT2 = 2
...
CONSTANT3998 = 3998
CONSTANT3999 = 3999

Now building this with Cython on Windows:

import setuptools
from distutils.core import setup     # will use my default compiler MSVC++
from Cython.Build import cythonize
setup(ext_modules=cythonize("test.py", language_level="3"), script_args=['build'])

takes many minutes, and never finishes.

I see in task manager that link.exe takes 25% of my CPU during minutes, and the memory usage increases: 800 MB at the time I'm writing this!

How to solve this little Cython bug?


You can generate a dummy module test.py with 4000 constants with this code:

with open('test.py', 'w') as f:
    for i in range(1000):
        f.write(f'CONSTANT{i} = {i}\n')

Solution

  • This problem can be solved by removing the compiler optimizations, as seen in this issue.

    The compiler parameter /Od seems to be necessary:

    setup(ext_modules=cythonize(Extension('test', sources=['test.py'], extra_compile_args=['/Od'])),
        script_args=['build'])