I am trying to create a simple project with dependencies such as requests and to create a distributed zip file with the following command : python3.6 setup.py bdist --format=zip. In the dist zip file there should be also package.zip created with the following bash script:
#!/bin/bash
pip install -r requirements.txt --target ./packages
if [[ -d packages ]]; then
cd packages
find . -name "*.pyc" -delete
find . -name "*.egg-info" | xargs rm -rf
zip -9mrv packages.zip .
mv packages.zip ..
cd ..
rm -rf packages
fi
The final zip file contain the following files:
Archive: dist/test_python_zip-1.0.linux-x86_64.zip
Length Date Time Name
--------- ---------- ----- ----
0 2019-05-27 12:37 src/
0 2019-05-27 12:37 __pycache__/
0 2019-05-27 12:37 commands/
187 2019-05-27 12:37 test_python_zip-1.0-py3.6.egg-info
0 2019-05-22 19:18 __init__.py
222 2019-05-27 12:34 __main__.py
0 2019-05-27 12:37 src/commands/
0 2019-05-27 12:37 src/commands/__pycache__/
28 2019-05-22 17:47 src/commands/buy.py
18 2019-05-22 19:02 src/commands/__init__.py
246 2019-05-22 18:39 src/commands/hello.py
30 2019-05-22 17:47 src/commands/start.py
125 2019-05-27 12:37 src/commands/__pycache__/__init__.cpython-36.pyc
337 2019-05-27 12:37 src/commands/__pycache__/hello.cpython-36.pyc
214 2019-05-27 12:37 src/commands/__pycache__/start.cpython-36.pyc
210 2019-05-27 12:37 src/commands/__pycache__/buy.cpython-36.pyc
95 2019-05-27 12:37 __pycache__/__init__.cpython-36.pyc
403 2019-05-27 12:37 __pycache__/__main__.cpython-36.pyc
0 2019-05-27 12:37 commands/__pycache__/
28 2019-05-22 17:47 commands/buy.py
18 2019-05-22 19:02 commands/__init__.py
246 2019-05-22 18:39 commands/hello.py
30 2019-05-22 17:47 commands/start.py
121 2019-05-27 12:37 commands/__pycache__/__init__.cpython-36.pyc
333 2019-05-27 12:37 commands/__pycache__/hello.cpython-36.pyc
210 2019-05-27 12:37 commands/__pycache__/start.cpython-36.pyc
206 2019-05-27 12:37 commands/__pycache__/buy.cpython-36.pyc
--------- -------
3307 27 files
The setup.py file content is:
from distutils.command.bdist_dumb import bdist_dumb
from distutils.core import setup
class custom_bdist_dumb(bdist_dumb):
def reinitialize_command(self, name, **kw):
cmd = bdist_dumb.reinitialize_command(self, name, **kw)
if name == 'install':
cmd.install_lib = '/'
return cmd
if __name__ == '__main__':
setup(
# our custom class override
cmdclass={'bdist_dumb': custom_bdist_dumb},
name='test_python_zip',
version="1.0",
py_modules=['__main__'],
packages=['lib', 'src/commands'])
The project tree is the following:
.
├── build
│ ├── bdist.linux-x86_64
│ ├── lib
│ │ ├── commands
│ │ │ ├── buy.py
│ │ │ ├── hello.py
│ │ │ ├── __init__.py
│ │ │ └── start.py
│ │ ├── __init__.py
│ │ ├── __main__.py
│ │ └── src
│ │ └── commands
│ │ ├── buy.py
│ │ ├── hello.py
│ │ ├── __init__.py
│ │ └── start.py
│ └── lib.linux-x86_64-2.7
│ └── commands
│ ├── buy.py
│ ├── hello.py
│ ├── __init__.py
│ └── start.py
├── dist
│ └── test_python_zip-1.0.linux-x86_64.zip
├── __main__.py
├── packages.sh
├── packages.zip
├── __pycache__
├── requirements.txt
├── setup.py
├── src
│ ├── commands
│ │ ├── buy.py
│ │ ├── hello.py
│ │ ├── hello.pyc
│ │ ├── __init__.py
│ │ ├── __init__.pyc
│ │ ├── __pycache__
│ │ └── start.py
│ ├── __init__.py
│ ├── __init__.pyc
│ ├── packages.zip
│ └── test_python_zip.egg-info
│ ├── dependency_links.txt
│ ├── PKG-INFO
│ ├── SOURCES.txt
│ └── top_level.txt
└── test_python_zip.egg-info
├── dependency_links.txt
├── PKG-INFO
├── SOURCES.txt
└── top_level.txt
Finally the main.py file content is:
import sys
if not ('packages.zip' in sys.path):
sys.path.insert(0, 'packages.zip')
print(sys.version)
print("\n \n")
print(sys.path)
from src.commands import hello
def main():
hello.call()
if __name__ == '__main__':
main()
The problem here is that the requirements dependency is not loaded from the package.zip file and if it is not installed I got the following error:
3.7.3 (default, Apr 3 2019, 05:39:12)
[GCC 8.3.0]
['lib', 'dist/test_python_zip-1.0.linux-x86_64.zip', '/usr/lib/python37.zip', '/usr/lib/python3.7', '/usr/lib/python3.7/lib-dynload', '/home/yborisov/.local/lib/python3.7/site-packages', '/usr/local/lib/python3.7/dist-packages', '/usr/lib/python3/dist-packages']
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "dist/test_python_zip-1.0.linux-x86_64.zip/__main__.py", line 8, in <module>
File "dist/test_python_zip-1.0.linux-x86_64.zip/src/commands/hello.py", line 1, in <module>
ModuleNotFoundError: No module named 'requests'
What should be the problem and how I can resolved it?
Instead of using package.zip in the final zip I used the package folder which I moved under src package. I added a MANIFEST.in with content
include packages __main__.py
recursive-include src/commands *
The setup.py was changed to
from distutils.command.bdist_dumb import bdist_dumb
from distutils.core import setup
from setuptools import find_packages
class CustomBdistDumb(bdist_dumb):
def reinitialize_command(self, name, **kw):
cmd = bdist_dumb.reinitialize_command(self, name, **kw)
if name == 'install':
cmd.install_lib = '/'
return cmd
setup(
# our custom class override
cmdclass={'bdist_dumb': CustomBdistDumb},
name='test_python_zip',
version="1.0",
package_data={
'': ['packages']
},
py_modules=['__main__'],
packages=find_packages(),
)
and the request module is loaded directly from the packages folder:
from src.packages import requests
URL = "https://api.ipify.org/?format=json"
def call():
# sending get request and saving the response as response object
r = requests.get(url=URL)
# extracting data in json format
data = r.json()
print(data)