Search code examples
pythoncwindowsbuildingpython-c-extension

Building c-extended Python package on Windows


I am trying to build the 'nwalign' Python package on a 64-bit Windows with 64-bit Python version machine. (Package can be found here: https://pypi.python.org/pypi/nwalign/?)

I tried building and installing using 'python setup.py install' the package both in Python 2.7 and 3.6 but I get the following linkage error (I've replaced some info by '...'):

(C:\...Anaconda3\envs\Python2) C:\...Desktop\nwalign-0.3.1>python setup.py 
...
building 'nwalign/cnwalign' extension
creating build\temp.win-amd64-2.7
creating build\temp.win-amd64-2.7\Release
creating build\temp.win-amd64-2.7\Release\nwalign
C:\...AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0\VC\Bin\amd64\cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBUG -IC:\...Anaconda3\envs\Python2\lib\site-packages\numpy\core\include -Inwalign -IC:\...Anaconda3\envs\Python2\include -IC:\...Anaconda3\envs\Python2\PC /Tcnwalign/cnwalign.c /Fobuild\temp.win-amd64-2.7\Release\nwalign/cnwalign.obj
cnwalign.c
c:\...anaconda3\envs\python2\lib\site-packages\numpy\core\include\numpy\npy_1_7_deprecated_api.h(12) : Warning Msg: Using deprecated NumPy API, disable it by #defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION
...
nwalign/cnwalign.c(7433) : warning C4267: 'initializing' : conversion from 'size_t' to 'int', possible loss of data
C:\...AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0\VC\Bin\amd64\link.exe /DLL /nologo /INCREMENTAL:NO /LIBPATH:C:\...Anaconda3\envs\Python2\libs /LIBPATH:C:\...Anaconda3\envs\Python2\PCbuild\amd64 /LIBPATH:C:\...Anaconda3\envs\Python2\PC\VS9.0\amd64 /EXPORT:initnwalign/cnwalign build\temp.win-amd64-2.7\Release\nwalign/cnwalign.obj /OUT:build\lib.win-amd64-2.7\nwalign\cnwalign.pyd /IMPLIB:build\temp.win-amd64-2.7\Release\nwalign\cnwalign.lib /MANIFESTFILE:build\temp.win-amd64-2.7\Release\nwalign\cnwalign.pyd.manifest
LINK : error LNK2001: unresolved external symbol initnwalign/cnwalign
build\temp.win-amd64-2.7\Release\nwalign\cnwalign.lib : fatal error LNK1120: 1 unresolved externals
error: command 'C:\\...AppData\\Local\\Programs\\Common\\Microsoft\\Visual C++ for Python\\9.0\\VC\\Bin\\amd64\\link.exe' failed with exit status 1120

The setup.py file:

from setuptools import setup, find_packages
from distutils.extension import Extension
#from Cython.Distutils import build_ext

version = '0.3.1'
import numpy
np_include = numpy.get_include()
try:
    import nwalign
    doc = nwalign.__doc__
except:
    doc = ""

setup(name='nwalign',
      version=version,
      description="Needleman-Wunsch global sequence alignment",
      long_description=doc,
      ext_modules=[ Extension("nwalign/cnwalign",
                      sources=["nwalign/cnwalign.c"],
                      include_dirs=[np_include, "nwalign"])],
      keywords='sequence bioinformatics alignment needleman-wunsch',
      url='http://bitbucket.org/brentp/biostuff/',
      #download_url='http://bitbucket.org/brentp/biostuff/get/tip.tar.gz',
      author='brentp',
      author_email='bpederse@gmail.com',
      license='BSD',
      test_suite='nose.collector',
      include_package_data=True,
      zip_safe=False,
      packages=['nwalign'],
      #package_dir={'': 'nwalign'},
      package_data = {'nwalign': ['*.pyx', "*.c"]},
      install_requires=[
          'numpy', 'cython'
      ],
      entry_points= {
          # -*- Entry points: -*-
          'console_scripts': ['nwalign = nwalign:main']
          },
    classifiers   = [
        'Development Status :: 3 - Alpha',
        'Intended Audience :: Developers',
        'Intended Audience :: Science/Research',
        'License :: OSI Approved :: BSD License',
        'Operating System :: OS Independent',
        'Programming Language :: Python',
        'Topic :: Scientific/Engineering :: Bio-Informatics',
        'Topic :: Scientific/Engineering',
        'Topic :: Text Processing'
        ],
)

I've tried directing my MSCV compiler to 64-bit using the appropriate bat-file but could not alleviate the issue. (I ran 'vcvarsall amd64' in the terminal for 'C:...\AppData\Local\Programs\Common\Microsoft\Visual C++ from Python\9.0')

The same issue and associated solution (Changing .def file and removing exports) brought up by another nwalign user here https://bitbucket.org/brentp/biostuff/issues/1/instalation-issue did not work for me as my build does not create a .def file, only a cnwalign.obj.

Any ideas?


Solution

  • The problem is because [MS.Docs]: /EXPORT (Exports a Function) linker directive is incorrectly used (/EXPORT:initnwalign/cnwalign: initnwalign/cnwalign is not a valid function identifier).
    That (as can be easily seen), is triggered by the extension name: nwalign/cnwalign.

    Quote from Python2.7.13's extension.py file (${PYTHON2.7.13_INSTALL_DIR}\Lib\distutils\extension.py:31 (part of class Extension)):

    Instance attributes:
       name : string
         the full name of the extension, including any packages -- ie.
         *not* a filename or pathname, but Python dotted name

    So, in setup.py, modifying the way Extension is instantiated, from:

    ext_modules=[ Extension("nwalign/cnwalign",
    

    to:

    ext_modules=[ Extension("nwalign.cnwalign",
    

    should avoid the error, and build the .pyd.