I am looking for techniques to speed up building a deb pkg from my python application:
Source: my-shiny-app
Section: python
Priority: extra
Maintainer: me <me_write_me@me.com>
Build-Depends: debhelper (>= 9), python, dh-virtualenv (>= 0.6-1)
Standards-Version: 3.9.5
Package: my-shiny-app
Architecture: amd64
Pre-Depends: dpkg (>= 1.16.1), python2.7, ${misc:Pre-Depends}
Depends: ${python:Depends}, ${misc:Depends}, libzmq1
Description: my-app
The most of the time takes building python libraries that my application depends on.
UPDATE:
I have found a ticket in dh-virtualenv github. They suggest to use pip7. Pip7 supports local caching of wheel packages (copy&paste from the ticket):
#!/usr/bin/make -f
%:
dh $@ --with python-virtualenv
WHEEL_ROOT:= file:///var/wheelhouse
override_dh_virtualenv:
pip wheel -w=$(WHEEL_ROOT) -f $(WHEEL_ROOT) -r requirements.txt
dh_virtualenv \
--extra-pip-arg "--use-wheel" \
--extra-pip-arg "--find-links=$(WHEEL_ROOT)"
Problem is: the created deb package does not include python and libraries.
If building dependencies is the bottleneck, try upgrading to the latest version of pip. The later versions store pre-built packages in a local wheel cache making subsequent builds much faster.
If downloading packages takes long, you should copy your dependencies to a local folder on your build-server. Make sure you copy the packages to local folder on the file system. While pip supports access via --find-links=http://your-local-package-host/, collecting dependencies is still rather slow unless you set up https & proper caching headers for the package host.
If installing the app itself is taking long (hangs after "Processing /some/folder") then you're experiencing the slow copy pip installation issue. You could try to work around that by replacing pip install . with python setup.py sdist and pip install dist/my-shiny-app-1.0.tar.gz in dh-virtualenv.