Search code examples
pythonherokutorch

When deploying to heroku it keeps downloading packages that are not in my requirements.txt. Mainly pytorch


Torch is too large to push to heroku so I just went ahead and decided to take it out of my project. I uninstalled torch and torch vision and then ran pip freeze -> requirements.txt

My requirements.txt is:

appnope==0.1.3
asttokens==2.0.8
backcall==0.2.0
cachetools==5.2.0
certifi==2022.9.24
charset-normalizer==2.1.1
click==8.0.4
contourpy==1.0.5
cycler==0.11.0
decorator==5.1.1
executing==1.1.1
fire==0.4.0
Flask==2.2.2
Flask-Cors==3.0.10
fonttools==4.37.4
google-auth==2.12.0
google-auth-oauthlib==0.4.6
grpcio==1.49.1
gunicorn==20.1.0
heroku3==5.1.4
idna==3.4
ipython==8.5.0
itsdangerous==2.1.2
jedi==0.18.1
Jinja2==3.1.2
jmespath==1.0.1
kiwisolver==1.4.4
Markdown==3.4.1
MarkupSafe==2.1.1
matplotlib==3.6.1
matplotlib-inline==0.1.6
nautical-calculations==1.0.1
numpy==1.23.3
oauthlib==3.2.1
opencv-python==4.6.0.66
packaging==21.3
pandas==1.5.0
parso==0.8.3
pexpect==4.8.0
pickleshare==0.7.5
Pillow==9.2.0
prompt-toolkit==3.0.31
protobuf==3.19.6
psutil==5.9.2
ptyprocess==0.7.0
pure-eval==0.2.2
pyasn1==0.4.8
pyasn1-modules==0.2.8
pybboxes==0.1.5
Pygments==2.13.0
pyparsing==3.0.9
python-dateutil==2.8.2
pytz==2022.4
PyYAML==6.0
requests==2.28.1
requests-oauthlib==1.3.1
rsa==4.9
s3transfer==0.6.0
sahi==0.10.7
scipy==1.9.2
seaborn==0.12.0
Shapely==1.8.4
six==1.16.0
stack-data==0.5.1
tensorboard==2.10.1
tensorboard-data-server==0.6.1
tensorboard-plugin-wit==1.8.1
termcolor==2.0.1
terminaltables==3.1.10
thop==0.1.1.post2209072238
tqdm==4.64.1
traitlets==5.4.0
typing_extensions==4.4.0
urllib3==1.26.12
wcwidth==0.2.5
Werkzeug==2.2.2

Then to push to heroku I do:

git init
heroku create
git add .
git push heroku master

But heroku keeps downloading torch (Downloading torch-1.12.1-cp310-cp310-manylinux1_x86_64.whl (776.3 MB)) which causes my compiled slug size to be too large. the slug size is 964.9.

I just don't want torch to be installed but I don't know how to make it not push to heroku.

Complete debloy log

(venv) konradlach@Konrads-Air Hackathon % git push heroku master                  
Enumerating objects: 61, done.
Counting objects: 100% (61/61), done.
Delta compression using up to 8 threads
Compressing objects: 100% (55/55), done.
Writing objects: 100% (61/61), 14.38 MiB | 25.92 MiB/s, done.
Total 61 (delta 2), reused 0 (delta 0), pack-reused 0
remote: Compressing source files... done.
remote: Building source:
remote: 
remote: -----> Building on the Heroku-22 stack
remote: -----> Determining which buildpack to use for this app
remote: -----> Python app detected
remote: -----> No Python version was specified. Using the buildpack default: python-3.10.7
remote:        To use a different version, see: https://devcenter.heroku.com/articles/python-runtimes
remote: -----> Installing python-3.10.7
remote: -----> Installing pip 22.2.2, setuptools 63.4.3 and wheel 0.37.1
remote: -----> Installing SQLite3
remote: -----> Installing requirements with pip
remote:        Collecting absl-py==1.2.0
remote:          Downloading absl_py-1.2.0-py3-none-any.whl (123 kB)
remote:        Collecting appnope==0.1.3
remote:          Downloading appnope-0.1.3-py2.py3-none-any.whl (4.4 kB)
remote:        Collecting asttokens==2.0.8
remote:          Downloading asttokens-2.0.8-py2.py3-none-any.whl (23 kB)
remote:        Collecting backcall==0.2.0
remote:          Downloading backcall-0.2.0-py2.py3-none-any.whl (11 kB)
remote:        Collecting cachetools==5.2.0
remote:          Downloading cachetools-5.2.0-py3-none-any.whl (9.3 kB)
remote:        Collecting certifi==2022.9.24
remote:          Downloading certifi-2022.9.24-py3-none-any.whl (161 kB)
remote:        Collecting charset-normalizer==2.1.1
remote:          Downloading charset_normalizer-2.1.1-py3-none-any.whl (39 kB)
remote:        Collecting click==8.0.4
remote:          Downloading click-8.0.4-py3-none-any.whl (97 kB)
remote:        Collecting contourpy==1.0.5
remote:          Downloading contourpy-1.0.5-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (295 kB)
remote:        Collecting cycler==0.11.0
remote:          Downloading cycler-0.11.0-py3-none-any.whl (6.4 kB)
remote:        Collecting decorator==5.1.1
remote:          Downloading decorator-5.1.1-py3-none-any.whl (9.1 kB)
remote:        Collecting executing==1.1.1
remote:          Downloading executing-1.1.1-py2.py3-none-any.whl (22 kB)
remote:        Collecting fire==0.4.0
remote:          Downloading fire-0.4.0.tar.gz (87 kB)
remote:          Preparing metadata (setup.py): started
remote:          Preparing metadata (setup.py): finished with status 'done'
remote:        Collecting Flask==2.2.2
remote:          Downloading Flask-2.2.2-py3-none-any.whl (101 kB)
remote:        Collecting Flask-Cors==3.0.10
remote:          Downloading Flask_Cors-3.0.10-py2.py3-none-any.whl (14 kB)
remote:        Collecting fonttools==4.37.4
remote:          Downloading fonttools-4.37.4-py3-none-any.whl (960 kB)
remote:        Collecting google-auth==2.12.0
remote:          Downloading google_auth-2.12.0-py2.py3-none-any.whl (169 kB)
remote:        Collecting google-auth-oauthlib==0.4.6
remote:          Downloading google_auth_oauthlib-0.4.6-py2.py3-none-any.whl (18 kB)
remote:        Collecting grpcio==1.49.1
remote:          Downloading grpcio-1.49.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.7 MB)
remote:        Collecting gunicorn==20.1.0
remote:          Downloading gunicorn-20.1.0-py3-none-any.whl (79 kB)
remote:        Collecting heroku3==5.1.4
remote:          Downloading heroku3-5.1.4.tar.gz (160 kB)
remote:          Preparing metadata (setup.py): started
remote:          Preparing metadata (setup.py): finished with status 'done'
remote:        Collecting idna==3.4
remote:          Downloading idna-3.4-py3-none-any.whl (61 kB)
remote:        Collecting ipython==8.5.0
remote:          Downloading ipython-8.5.0-py3-none-any.whl (752 kB)
remote:        Collecting itsdangerous==2.1.2
remote:          Downloading itsdangerous-2.1.2-py3-none-any.whl (15 kB)
remote:        Collecting jedi==0.18.1
remote:          Downloading jedi-0.18.1-py2.py3-none-any.whl (1.6 MB)
remote:        Collecting Jinja2==3.1.2
remote:          Downloading Jinja2-3.1.2-py3-none-any.whl (133 kB)
remote:        Collecting jmespath==1.0.1
remote:          Downloading jmespath-1.0.1-py3-none-any.whl (20 kB)
remote:        Collecting kiwisolver==1.4.4
remote:          Downloading kiwisolver-1.4.4-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.6 MB)
remote:        Collecting Markdown==3.4.1
remote:          Downloading Markdown-3.4.1-py3-none-any.whl (93 kB)
remote:        Collecting MarkupSafe==2.1.1
remote:          Downloading MarkupSafe-2.1.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB)
remote:        Collecting matplotlib==3.6.1
remote:          Downloading matplotlib-3.6.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.8 MB)
remote:        Collecting matplotlib-inline==0.1.6
remote:          Downloading matplotlib_inline-0.1.6-py3-none-any.whl (9.4 kB)
remote:        Collecting nautical-calculations==1.0.1
remote:          Downloading nautical_calculations-1.0.1-py3-none-any.whl (6.0 kB)
remote:        Collecting numpy==1.23.3
remote:          Downloading numpy-1.23.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.1 MB)
remote:        Collecting oauthlib==3.2.1
remote:          Downloading oauthlib-3.2.1-py3-none-any.whl (151 kB)
remote:        Collecting opencv-python==4.6.0.66
remote:          Downloading opencv_python-4.6.0.66-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (60.9 MB)
remote:        Collecting packaging==21.3
remote:          Downloading packaging-21.3-py3-none-any.whl (40 kB)
remote:        Collecting pandas==1.5.0
remote:          Downloading pandas-1.5.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (12.1 MB)
remote:        Collecting parso==0.8.3
remote:          Downloading parso-0.8.3-py2.py3-none-any.whl (100 kB)
remote:        Collecting pexpect==4.8.0
remote:          Downloading pexpect-4.8.0-py2.py3-none-any.whl (59 kB)
remote:        Collecting pickleshare==0.7.5
remote:          Downloading pickleshare-0.7.5-py2.py3-none-any.whl (6.9 kB)
remote:        Collecting Pillow==9.2.0
remote:          Downloading Pillow-9.2.0-cp310-cp310-manylinux_2_28_x86_64.whl (3.2 MB)
remote:        Collecting prompt-toolkit==3.0.31
remote:          Downloading prompt_toolkit-3.0.31-py3-none-any.whl (382 kB)
remote:        Collecting protobuf==3.19.6
remote:          Downloading protobuf-3.19.6-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.1 MB)
remote:        Collecting psutil==5.9.2
remote:          Downloading psutil-5.9.2-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (282 kB)
remote:        Collecting ptyprocess==0.7.0
remote:          Downloading ptyprocess-0.7.0-py2.py3-none-any.whl (13 kB)
remote:        Collecting pure-eval==0.2.2
remote:          Downloading pure_eval-0.2.2-py3-none-any.whl (11 kB)
remote:        Collecting pyasn1==0.4.8
remote:          Downloading pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
remote:        Collecting pyasn1-modules==0.2.8
remote:          Downloading pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
remote:        Collecting pybboxes==0.1.5
remote:          Downloading pybboxes-0.1.5-py3-none-any.whl (24 kB)
remote:        Collecting Pygments==2.13.0
remote:          Downloading Pygments-2.13.0-py3-none-any.whl (1.1 MB)
remote:        Collecting pyparsing==3.0.9
remote:          Downloading pyparsing-3.0.9-py3-none-any.whl (98 kB)
remote:        Collecting python-dateutil==2.8.2
remote:          Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
remote:        Collecting pytz==2022.4
remote:          Downloading pytz-2022.4-py2.py3-none-any.whl (500 kB)
remote:        Collecting PyYAML==6.0
remote:          Downloading PyYAML-6.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (682 kB)
remote:        Collecting requests==2.28.1
remote:          Downloading requests-2.28.1-py3-none-any.whl (62 kB)
remote:        Collecting requests-oauthlib==1.3.1
remote:          Downloading requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
remote:        Collecting rsa==4.9
remote:          Downloading rsa-4.9-py3-none-any.whl (34 kB)
remote:        Collecting s3transfer==0.6.0
remote:          Downloading s3transfer-0.6.0-py3-none-any.whl (79 kB)
remote:        Collecting sahi==0.10.7
remote:          Downloading sahi-0.10.7-py3-none-any.whl (97 kB)
remote:        Collecting scipy==1.9.2
remote:          Downloading scipy-1.9.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (33.7 MB)
remote:        Collecting seaborn==0.12.0
remote:          Downloading seaborn-0.12.0-py3-none-any.whl (285 kB)
remote:        Collecting Shapely==1.8.4
remote:          Downloading Shapely-1.8.4-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (2.0 MB)
remote:        Collecting six==1.16.0
remote:          Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
remote:        Collecting stack-data==0.5.1
remote:          Downloading stack_data-0.5.1-py3-none-any.whl (24 kB)
remote:        Collecting tensorboard==2.10.1
remote:          Downloading tensorboard-2.10.1-py3-none-any.whl (5.9 MB)
remote:        Collecting tensorboard-data-server==0.6.1
remote:          Downloading tensorboard_data_server-0.6.1-py3-none-manylinux2010_x86_64.whl (4.9 MB)
remote:        Collecting tensorboard-plugin-wit==1.8.1
remote:          Downloading tensorboard_plugin_wit-1.8.1-py3-none-any.whl (781 kB)
remote:        Collecting termcolor==2.0.1
remote:          Downloading termcolor-2.0.1-py3-none-any.whl (5.4 kB)
remote:        Collecting terminaltables==3.1.10
remote:          Downloading terminaltables-3.1.10-py2.py3-none-any.whl (15 kB)
remote:        Collecting thop==0.1.1.post2209072238
remote:          Downloading thop-0.1.1.post2209072238-py3-none-any.whl (15 kB)
remote:        Collecting tqdm==4.64.1
remote:          Downloading tqdm-4.64.1-py2.py3-none-any.whl (78 kB)
remote:        Collecting traitlets==5.4.0
remote:          Downloading traitlets-5.4.0-py3-none-any.whl (107 kB)
remote:        Collecting typing_extensions==4.4.0
remote:          Downloading typing_extensions-4.4.0-py3-none-any.whl (26 kB)
remote:        Collecting urllib3==1.26.12
remote:          Downloading urllib3-1.26.12-py2.py3-none-any.whl (140 kB)
remote:        Collecting wcwidth==0.2.5
remote:          Downloading wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
remote:        Collecting Werkzeug==2.2.2
remote:          Downloading Werkzeug-2.2.2-py3-none-any.whl (232 kB)
remote:        Collecting botocore<2.0a.0,>=1.12.36
remote:          Downloading botocore-1.27.89-py3-none-any.whl (9.2 MB)
remote:        Collecting torch
remote:          Downloading torch-1.12.1-cp310-cp310-manylinux1_x86_64.whl (776.3 MB)
remote:        Building wheels for collected packages: fire, heroku3
remote:          Building wheel for fire (setup.py): started
remote:          Building wheel for fire (setup.py): finished with status 'done'
remote:          Created wheel for fire: filename=fire-0.4.0-py2.py3-none-any.whl size=115926 sha256=269b5d2211a3589ee4ea7db46be081e29f44aac7addd12c20cc00386eb9e0bec
remote:          Stored in directory: /tmp/pip-ephem-wheel-cache-efcv6587/wheels/26/9a/dd/2818b1b023daf077ec3e625c47ae446aca587a5abe48e05212
remote:          Building wheel for heroku3 (setup.py): started
remote:          Building wheel for heroku3 (setup.py): finished with status 'done'
remote:          Created wheel for heroku3: filename=heroku3-5.1.4-py2.py3-none-any.whl size=161472 sha256=50373decc7d4f19365f7825f3113923da726a31ba76243fcc1b26bfa8763aa22
remote:          Stored in directory: /tmp/pip-ephem-wheel-cache-efcv6587/wheels/3e/4b/f7/58d717b05f6be3ddda7dc0c35ec61bc6cf828ca32c0472f478
remote:        Successfully built fire heroku3
remote:        Installing collected packages: wcwidth, tensorboard-plugin-wit, pytz, pyasn1, pure-eval, ptyprocess, pickleshare, executing, backcall, appnope, urllib3, typing_extensions, traitlets, tqdm, terminaltables, termcolor, tensorboard-data-server, six, Shapely, rsa, PyYAML, pyparsing, Pygments, pyasn1-modules, psutil, protobuf, prompt-toolkit, Pillow, pexpect, parso, oauthlib, numpy, nautical-calculations, MarkupSafe, Markdown, kiwisolver, jmespath, itsdangerous, idna, gunicorn, fonttools, decorator, cycler, click, charset-normalizer, certifi, cachetools, absl-py, Werkzeug, torch, scipy, requests, python-dateutil, pybboxes, packaging, opencv-python, matplotlib-inline, Jinja2, jedi, grpcio, google-auth, fire, contourpy, asttokens, thop, stack-data, sahi, requests-oauthlib, pandas, matplotlib, heroku3, Flask, botocore, seaborn, s3transfer, ipython, google-auth-oauthlib, Flask-Cors, tensorboard
remote:        Successfully installed Flask-2.2.2 Flask-Cors-3.0.10 Jinja2-3.1.2 Markdown-3.4.1 MarkupSafe-2.1.1 Pillow-9.2.0 PyYAML-6.0 Pygments-2.13.0 Shapely-1.8.4 Werkzeug-2.2.2 absl-py-1.2.0 appnope-0.1.3 asttokens-2.0.8 backcall-0.2.0 botocore-1.27.89 cachetools-5.2.0 certifi-2022.9.24 charset-normalizer-2.1.1 click-8.0.4 contourpy-1.0.5 cycler-0.11.0 decorator-5.1.1 executing-1.1.1 fire-0.4.0 fonttools-4.37.4 google-auth-2.12.0 google-auth-oauthlib-0.4.6 grpcio-1.49.1 gunicorn-20.1.0 heroku3-5.1.4 idna-3.4 ipython-8.5.0 itsdangerous-2.1.2 jedi-0.18.1 jmespath-1.0.1 kiwisolver-1.4.4 matplotlib-3.6.1 matplotlib-inline-0.1.6 nautical-calculations-1.0.1 numpy-1.23.3 oauthlib-3.2.1 opencv-python-4.6.0.66 packaging-21.3 pandas-1.5.0 parso-0.8.3 pexpect-4.8.0 pickleshare-0.7.5 prompt-toolkit-3.0.31 protobuf-3.19.6 psutil-5.9.2 ptyprocess-0.7.0 pure-eval-0.2.2 pyasn1-0.4.8 pyasn1-modules-0.2.8 pybboxes-0.1.5 pyparsing-3.0.9 python-dateutil-2.8.2 pytz-2022.4 requests-2.28.1 requests-oauthlib-1.3.1 rsa-4.9 s3transfer-0.6.0 sahi-0.10.7 scipy-1.9.2 seaborn-0.12.0 six-1.16.0 stack-data-0.5.1 tensorboard-2.10.1 tensorboard-data-server-0.6.1 tensorboard-plugin-wit-1.8.1 termcolor-2.0.1 terminaltables-3.1.10 thop-0.1.1.post2209072238 torch-1.12.1 tqdm-4.64.1 traitlets-5.4.0 typing_extensions-4.4.0 urllib3-1.26.12 wcwidth-0.2.5
remote: -----> Discovering process types
remote:        Procfile declares types -> web
remote: 
remote: -----> Compressing...
remote:  !     Compiled slug size: 964.9M is too large (max is 500M).
remote:  !     See: http://devcenter.heroku.com/articles/slug-size
remote: 
remote:  !     Push failed
remote: Verifying deploy...
remote: 
remote: !       Push rejected to mighty-meadow-30317.
remote: 
To https://git.heroku.com/mighty-meadow-30317.git
 ! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to 'https://git.heroku.com/mighty-meadow-30317.git'

Solution

  • I ended up uninstalling everything from pip using What is the easiest way to remove all packages installed by pip? and reinstalling all the packages I actually needed and it fixed the issue!