What's the best strategy for managing third-party Python libraries with Google App Engine?
Say I want to use Flask, a webapp framework. A blog entry says to do this, which doesn't seem right:
$ cd /tmp/
$ wget http://pypi.python.org/packages/source/F/Flask/Flask-0.6.1.tar.gz
$ tar zxf Flask-0.6.1.tar.gz
$ cp -r Flask-0.6.1/flask ~/path/to/project/
(... repeat for other packages ...)
There must be a better way to manage third-party code, especially if I want to track versions, test upgrades or if two libraries share a subdirectory. I know that Python can import modules from zipfiles and that pip can work with a wonderful REQUIREMENTS file, and I've seen that pip has a
zip command for use with GAE.
(Jun 2021) This post is over a decade old, and so an updated answer is warranted now.
requirements.txt along with any desired version#s; they'll be automatically installed by Google upon deployment. (This is the same technique used if you decide to migrate your app to Google Cloud Functions or Cloud Run.)
requirements.txt as above
pip install -t lib -r requirements.txt
appengine_config.py as shown in step 5 on this page
libraries: section of your
app.yaml like this
requirements.txt nor use
pip install to install them locally unless you want to self-bundle because, say if you need a newer version of the built-in library.)
appengine_config.py like the above.
If you have a Python 2 app with both built-in and non-built-in 3P libraries, use the techniques in both #2 and #3 above (built-in libraries in
app.yaml and non-built-in libraries in
requirements.txt and run the
pip install cmd above). One of the improvements in the second generation runtimes like Python 3 is that all these games with 3P libraries go away magically (see #1 above).
Flask is a 3rd-party micro web framework, and it's an interesting case for this specific question. For Python 3, they all go into
requirements.txt, so you'd just add
flask to that file, and you're done. (Just deploy from there.)
For Python 2, it's even more interesting because it's a built-in library. Unfortunately, the version on App Engine servers is
0.12. Who wants to use that when we're at/beyond
2.0.3 now?!? So instead of putting it in
app.yaml like other built-in libraries, you'd pretend the built-in version doesn't exist and put it in
requirements.txt then run
pip2 install -t lib -r requirements.txt to bundle/vendor it with your application code. (However, the final version for Python 2 is
1.1.4, so that's what gets installed.)