Add a pypi python package to buildroot

I'm trying to integrate python3-functionfs module into buildroot. I'm able to select it with make menuconfig but when I'm running make the package isn't even downloaded.

The package is available here : functionfs-0.3 pypi page
And the download url here : functionfs-0.3 download link
There is also the github repo here : functionfs git repository I'm using Buildroot 2017.02 version.

Here is my Config.in file :

config BR2_PACKAGE_PYTHON3_FUNCTIONFS
    bool "python3-functionfs"
    depends on BR2_PACKAGE_PYTHON3
    help
        Pythonic API for linux’s functionfs.
        functionfs is part of the usb gadget subsystem. Together with usb_gadget’s configfs integration, allows userland to declare and implement an USB device.

    https://pypi.python.org/pypi/functionfs

And here is my .mk file :

################################################################################
#
# python3-functionfs
#
################################################################################

PYTHON_FUNCTIONFS_VERSION = 0.3
PYTHON_FUNCTIONFS_SOURCE = functionfs-$(PYTHON_FUNCTIONFS_VERSION).tar.gz
PYTHON_FUNCTIONFS_SITE = https://pypi.python.org/packages/e3/2d/56e0d9ffe0da7c116a6724ee538375689dd59e34dbe1676edf6b66b52be4
PYTHON_FUNCTIONFS_LICENSE = GPLv3+
PYTHON_FUNCTIONFS_LICENSE_FILE = COPYING
PYTHON_FUNCTIONFS_SETUP_TYPE = setuptools

$(eval $(python-package))

The documentation also mentions at 17.8.3. Generating a python-package from a PyPI repository

If the Python package for which you would like to create a Buildroot package is available on PyPI, you may want to use the scanpypi tool located in utils/ to automate the process.

You can find the list of existing PyPI packages here.

scanpypi requires Python’s setuptools package to be installed on your host.

When at the root of your buildroot directory just do :

utils/scanpypi foo bar -o package

This will generate packages python-foo and python-bar in the package folder if they exist on https://pypi.python.org.

Find the external python modules menu and insert your package inside. Keep in mind that the items inside a menu should be in alphabetical order.

Please keep in mind that you’ll most likely have to manually check the package for any mistakes as there are things that cannot be guessed by the generator (e.g. dependencies on any of the python core modules such as BR2_PACKAGE_PYTHON_ZLIB). Also, please take note that the license and license files are guessed and must be checked. You also need to manually add the package to the package/Config.in file.

If your Buildroot package is not in the official Buildroot tree but in a br2-external tree, use the -o flag as follows:

utils/scanpypi foo bar -o other_package_dir

This will generate packages python-foo and python-bar in the other_package_directory instead of package.

Option -h will list the available options:

utils/scanpypi -h

However I don't have an util/ folder in buildroot main directory. The script is located at support/scripts/scanpypi but when I'm running it I have the following error :

$ support/scripts/scanpypi functionfs -o package
Traceback (most recent call last):
  File "support/scripts/scanpypi", line 47, in <module>
    import setuptools
  File "/usr/local/lib/python2.7/dist-packages/setuptools/__init__.py", line 11, in <module>
    from setuptools.extern.six.moves import filterfalse, map
  File "/usr/local/lib/python2.7/dist-packages/setuptools/extern/__init__.py", line 1, in <module>
    from pkg_resources.extern import VendorImporter
  File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 40, in <module>
    from pkgutil import get_importer
ImportError: cannot import name get_importer

This can be solved by renaming both support/scripts/pkgutil.py and support/scripts/pkgutil.pyc.

However I would like to understand what's going on when I'm trying to create the package by myself and it doesn't download.

Does someone know why functionfs-0.3.tar.gz is not downloaded when running make ?

Solution

Your package is not downloaded because there is a mismatch between your package name and the name of the variables in the .mk files. Basically, you have three things that must match:

The BR2_PACKAGE_<FOO> option in Config.in
The name of the file and directory must be package/<foo>/<foo>.mk
The variables in the .mk files must be named <FOO>_SOMETHING

You didn't say what the name of the .mk file was, but at the very least your option is named BR2_PACKAGE_PYTHON3_FUNCTIONFS while the make variables are named PYTHON_FUNCTIONFS_SOMETHING.

That explains why it isn't downloaded.

Then, regarding the scanpypi script, it is definitely in the utils/ directory in recent versions of Buildroot. It used to be in support/scripts a few releases ago. So basically you're looking at the Buildroot documentation that is online (and matches the latest release), but you're using an older Buildroot version. You can build the Buildroot documentation matching your Buildroot release by running make manual-html.