Django keeps using wrong storage backend when trying to upload static files to S3

Question

Update: Error was not caused by django-storages, but by django-heroku

It appears that django-heroku overrides some variables which caused this behavior. See my answer below.

---

I have been using S3 as a storage service for other Django apps, however for a new project, Django refuses to pick up the correct backend and keeps putting the files on my local filesystem instead of upload.

To be exact, after I install boto3 and adjust settings.py, then run python manage.py collectstatic, that command keeps moving the static files to <my_project_path>/staticfiles instead of starting the upload to S3.

This is the output of python manage.py collectstatic:

You have requested to collect static files at the destination
location as specified in your settings:

    <my_project_path>/staticfiles

This will overwrite existing files!
Are you sure you want to do this?

Type 'yes' to continue, or 'no' to cancel:

My settings:

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'django.contrib.sites',

    # Vendor
    'storages',
    'allauth',
    'allauth.account',
    'allauth.socialaccount',
    'sass_processor',
    'crispy_forms',

    [..]
]

# static & media files
USE_S3 = config('USE_S3', cast=bool, default=True)

AWS_ACCESS_KEY_ID = config('AWS_ACCESS_KEY_ID')
AWS_SECRET_ACCESS_KEY = config('AWS_SECRET_ACCESS_KEY')
AWS_DEFAULT_ACL = None
AWS_STORAGE_BUCKET_NAME = config('AWS_STORAGE_BUCKET')
AWS_S3_CUSTOM_DOMAIN = '%s.s3.amazonaws.com' % AWS_STORAGE_BUCKET_NAME
AWS_S3_OBJECT_PARAMETERS = {
    'CacheControl': 'max-age=86400',
}
AWS_LOCATION = 'static'

STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles')
MEDIA_ROOT = os.path.join(BASE_DIR, 'media')

if USE_S3:
    MEDIA_URL = 'https://%s/%s/' % (AWS_S3_CUSTOM_DOMAIN, 'media')
    DEFAULT_FILE_STORAGE = '<my_app>.storage_backends.MediaStorage'
    STATIC_URL = 'https://%s/%s/' % (AWS_S3_CUSTOM_DOMAIN, AWS_LOCATION)
    STATICFILES_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'

else:
    MEDIA_URL = '/media/'
    STATIC_URL = '/static/'
    STATICFILES_STORAGE = 'whitenoise.storage.CompressedManifestStaticFilesStorage'

STATICFILES_DIRS = (
    os.path.join(BASE_DIR, 'static'),
)
STATICFILES_FINDERS = [
    'django.contrib.staticfiles.finders.FileSystemFinder',
    'django.contrib.staticfiles.finders.AppDirectoriesFinder',
    'sass_processor.finders.CssFinder',
]

My requirements.txt:

...
boto3==1.11.12
botocore==1.14.15
django-storages==1.9.1
jmespath==0.9.4
s3transfer==0.3.3
...

I tried copying the settings from the other project where upload works, re-checked the django-storages docs, even installed the exact versions of boto3 etc. to exclude an error because of a newer version, checked my settings.py if somewhere I forgot a conflicting setting, but I just can't figure out why Django doesn't even try the upload.

Any pointers very much appreciated!

Answer 1

After more digging I found the culprit. Turns out the django-heroku package caused the unwanted behavior.

The package automatically sets STATIC_ROOT='/static/', overriding the AWS location in the settings.py.

You can disable this behavior by setting staticfiles=False when setting up django-heroku like this:

import django_heroku
django_heroku.settings(locals(), staticfiles=False)

I also realized in the other apps I had S3 successfully running, I didn't even use django-heroku, which is what caused my confusion and was the one difference in setup I had missed in my previous checks.