Disclaimer: I'm very new to python packaging with distutils. So far I've just stashed everything into modules, and packages manually and developed on top of that. I never wrote a setup.py file before.
I have a Fortran module that I want to use in my python code with numpy. I figured the best way to do that would be f2py, since it is included in numpy. To automate the build process I want to use distutils and the corresponding numpy enhancement, which includes convenience functions for f2py wrappers.
I do not understand how I should organize my files, and how to include my test suite.
What I want is the possibility to use ./setup.py for building, installing, and testing, and developing.
My directory structure looks as follows:
volterra
├── setup.py
└── volterra
├── __init__.py
├── integral.f90
├── test
│ ├── __init__.py
│ └── test_volterra.py
└── volterra.f90
And the setup.py file contains this:
def configuration(parent_package='', top_path=None):
from numpy.distutils.misc_util import Configuration
config = Configuration('volterra', parent_package, top_path)
config.add_extension('_volterra',
sources=['volterra/integral.f90', 'volterra/volterra.f90'])
return config
if __name__ == '__main__':
from numpy.distutils.core import setup
setup(**configuration(top_path='').todict())
After running ./setup.py build I get.
build/lib.linux-x86_64-2.7/
└── volterra
└── _volterra.so
Which includes neither the __init__.py file, nor the tests.
volterra/integral.f90) Can't I give a parameter which says, look for stuff in volterra/? The top_path, and package_dir parameters didn't do the trick.__init__.py file is not included in the build. Why is that?Here is a setup.py that works for me:
# pkg - A fancy software package
# Copyright (C) 2013 author (email)
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see http://www.gnu.org/licenses/gpl.html.
"""pkg: a software suite for
Hey look at me I'm a long description
But how long am I?
"""
from __future__ import division, print_function
#ideas for setup/f2py came from:
# -numpy setup.py: https://github.com/numpy/numpy/blob/master/setup.py 2013-11-07
# -winpython setup.py: http://code.google.com/p/winpython/source/browse/setup.py 2013-11-07
# -needing to use
# import setuptools; from numpy.distutils.core import setup, Extension:
# http://comments.gmane.org/gmane.comp.python.f2py.user/707 2013-11-07
# -wrapping FORTRAN code with f2py: http://www2-pcmdi.llnl.gov/cdat/tutorials/f2py-wrapping-fortran-code 2013-11-07
# -numpy disutils: http://docs.scipy.org/doc/numpy/reference/distutils.html 2013-11-07
# -manifest files in disutils:
# 'distutils doesn't properly update MANIFEST. when the contents of directories change.'
# https://github.com/numpy/numpy/blob/master/setup.py
# -if things are not woring try deleting build, sdist, egg directories and try again:
# https://stackoverflow.com/a/9982133/2530083 2013-11-07
# -getting fortran extensions to be installed in their appropriate sub package
# i.e. "my_ext = Extension(name = 'my_pack._fortran', sources = ['my_pack/code.f90'])"
# Note that sources is a list even if one file:
# http://numpy-discussion.10968.n7.nabble.com/f2py-and-setup-py-how-can-I-specify-where-the-so-file-goes-tp34490p34497.html 2013-11-07
# -install fortran source files into their appropriate sub-package
# i.e. "package_data={'': ['*.f95','*.f90']}# Note it's a dict and list":
# https://stackoverflow.com/a/19373744/2530083 2013-11-07
# -Chapter 9 Fortran Programming with NumPy Arrays:
# Langtangen, Hans Petter. 2013. Python Scripting for Computational Science. 3rd edition. Springer.
# -Hitchhikers guide to packaging :
# http://guide.python-distribute.org/
# -Python Packaging: Hate, hate, hate everywhere :
# http://lucumr.pocoo.org/2012/6/22/hate-hate-hate-everywhere/
# -How To Package Your Python Code:
# http://www.scotttorborg.com/python-packaging/
# -install testing requirements:
# https://stackoverflow.com/a/7747140/2530083 2013-11-07
import setuptools
from numpy.distutils.core import setup, Extension
import os
import os.path as osp
def readme(filename='README.rst'):
with open('README.rst') as f:
text=f.read()
f.close()
return text
def get_package_data(name, extlist):
"""Return data files for package *name* with extensions in *extlist*"""
#modified slightly from taken from http://code.google.com/p/winpython/source/browse/setup.py 2013-11-7
flist = []
# Workaround to replace os.path.relpath (not available until Python 2.6):
offset = len(name)+len(os.pathsep)
for dirpath, _dirnames, filenames in os.walk(name):
for fname in filenames:
if not fname.startswith('.') and osp.splitext(fname)[1] in extlist:
# flist.append(osp.join(dirpath, fname[offset:]))
flist.append(osp.join(dirpath, fname))
return flist
DOCLINES = __doc__.split("\n")
CLASSIFIERS = """\
Development Status :: 1 - Planning
License :: OSI Approved :: GNU Lesser General Public License v3 or later (LGPLv3+)
Programming Language :: Python :: 2.7
Topic :: Scientific/Engineering
"""
NAME = 'pkg'
MAINTAINER = "me"
MAINTAINER_EMAIL = "me@me.com"
DESCRIPTION = DOCLINES[0]
LONG_DESCRIPTION = "\n".join(DOCLINES[2:])#readme('readme.rst')
URL = "http://meeeee.mmemem"
DOWNLOAD_URL = "https://github.com/rtrwalker/geotecha.git"
LICENSE = 'GNU General Public License v3 or later (GPLv3+)'
CLASSIFIERS = [_f for _f in CLASSIFIERS.split('\n') if _f]
KEYWORDS=''
AUTHOR = "me"
AUTHOR_EMAIL = "me.com"
PLATFORMS = ["Windows"]#, "Linux", "Solaris", "Mac OS-X", "Unix"]
MAJOR = 0
MINOR = 1
MICRO = 0
ISRELEASED = False
VERSION = '%d.%d.%d' % (MAJOR, MINOR, MICRO)
INSTALL_REQUIRES=[]
ZIP_SAFE=False
TEST_SUITE='nose.collector'
TESTS_REQUIRE=['nose']
DATA_FILES = [(NAME, ['LICENSE.txt','README.rst'])]
PACKAGES=setuptools.find_packages()
PACKAGES.remove('tools')
PACKAGE_DATA={'': ['*.f95','*f90']}
ext_files = get_package_data(NAME,['.f90', '.f95','.F90', '.F95'])
ext_module_names = ['.'.join(osp.splitext(v)[0].split(osp.sep)) for v in ext_files]
EXT_MODULES = [Extension(name=x,sources=[y]) for x, y in zip(ext_module_names, ext_files)]
setup(
name=NAME,
version=VERSION,
maintainer=MAINTAINER,
maintainer_email=MAINTAINER_EMAIL,
description=DESCRIPTION,
long_description=LONG_DESCRIPTION,
url=URL,
download_url=DOWNLOAD_URL,
license=LICENSE,
classifiers=CLASSIFIERS,
author=AUTHOR,
author_email=AUTHOR_EMAIL,
platforms=PLATFORMS,
packages=PACKAGES,
data_files=DATA_FILES,
install_requires=INSTALL_REQUIRES,
zip_safe=ZIP_SAFE,
test_suite=TEST_SUITE,
tests_require=TESTS_REQUIRE,
package_data=PACKAGE_DATA,
ext_modules=EXT_MODULES,
)
To install, at the command line I use:
python setup.py install
python setup.py clean --all
The only issue I seem to have is a minor one. when I look in site-packages for my package it is installed inside the egg folder C:\Python27\Lib\site-packages\pkg-0.1.0-py2.7-win32.egg\pkg. Most other packages I see there have a C:\Python27\Lib\site-packages\pkg folder separate to the egg folder. Does anyone know how to get that separation?
As for testing, after installing, I type the following at the command line:
nosetests package_name -v
Try investigating python setup.py develop (Python setup.py develop vs install) for not having to install the package after every change.
As I commented in the code I found the following useful: