x-amz-acl being set incorrectly when using collectstatic with latest version of Boto, Pipeline, and Storages

Question

I'm in the process of bringing my app's dependencies up-to-date. I made the following changes to requirements.txt:

• boto: upgraded to 2.34.0
• django-pipeline: upgraded to 1.4.2
• django-require: upgraded to 1.0.6
• django-storages: was on latest (1.1.8)

Now when I try to run collectstatic into my S3 bucket, I get the following error:

boto.exception.S3ResponseError: S3ResponseError: 400 Bad Request

<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>InvalidArgument</Code><Message></Message><ArgumentName>x-amz-acl</ArgumentName>
<ArgumentValue>/tmp/tmpDyVin1</ArgumentValue><RequestId>xxx</RequestId>
<HostId>yyy</HostId></Error>

Not surprised, since clearly /tmp/tmpDyVin1 is not a valid value for x-amz-acl.

The issues stems from specifying my own class for handling static files that combines Boto, Require, and Pipeline:

settings.py

STATICFILES_STORAGE = 'myapp.storage.OptimizedS3BotoStorage'

storage.py

from pipeline.storage import PipelineMixin
from require.storage import OptimizedFilesMixin
from storages.backends.s3boto import S3BotoStorage

class ReleaseVersionCachedFilesMixin(CachedFilesMixin):

    def hashed_name(self, name, content=None):
        ...

class OptimizedS3BotoStorage(PipelineMixin, OptimizedFilesMixin, ReleaseVersionCachedFilesMixin, S3BotoStorage):
    pass

This worked perfectly and consistently with the old versions of all these modules. Digging through the new code, I see that the problem lies in the interaction with these three:

• When S3BotoStorage.__init__() is called, the first parameter, acl, is passed in as the value of this tmp directory. This overrides the previous value of public-read and leads to the problem above.
• This __init__() routine is called by CachedFilesMixin.__init__(), which receives args = ('/tmp/tmpnNUVD9',).
• That __init__() is called by PipelineMixin.__init__(), which does this:

def __init__(self, location=None, *args, **kwargs): if not settings.PIPELINE_ENABLED and location is None: location = tempfile.mkdtemp() super(PipelineMixin, self).__init__(location, *args, **kwargs)

So, the problem is that Pipeline passes in location as the first argument, which gets propagated down and becomes the acl.

Answer 1

It might seem like the solution to this nasty-looking problem is just to move PipelineMixin to the end of the list of arguments in my custom storage class, but that breaks the r.js optimizer. The solution is to set this PIPELINE_ENABLED flag to True.

(The documentation says this flag defaults to not settings.DEBUG, but that may not be a meaningful dependency for your environment. The flag wasn't used this way in PipelineMixin until 1.4, but it's not documented in the directives for upgrading to 1.4.)