Search code examples
pythonamazon-s3boto

Error on s3 file upload with python


Trying to upload an image to a bucket on s3 amazon.

The image is uploaded via form and the image is stored in 'file':

file = request.files['file']

I also try to rename it as 'filename' which holds the id of the user uploading the image.

My app runs on Heroku

Here is the code:

        filename = str(userName.id)

        # S3_BUCKET, AWS_ACCESS_KEY & AWS_SECRET_KEY = HEROKU envar from config.py

        conn = boto.connect_s3(app.config['AWS_ACCESS_KEY'], app.config['AWS_SECRET_KEY'])
        bucket = conn.get_bucket(app.config['S3_BUCKET'])

        key = '%s.jpg' % filename
        k = Key(bucket)
        k.key = key
        k.set_contents_from_filename(file.filename)

When I try to upload an image named '888888.jpg' I get the following error on heroku logs:

    2015-02-12T22:56:33.214246+00:00 heroku[router]: at=info method=POST path="/registering" host=monkey-me.herokuapp.com request_id=261d25bd-10bc-4703-aa38-badbcb4e634e fwd="213.243.186.53" dyno=web.1 connect=1ms service=809ms status=500 bytes=456
2015-02-12T22:56:33.181111+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request
2015-02-12T22:56:33.181113+00:00 app[web.1]:     rv = self.dispatch_request()
2015-02-12T22:56:33.181101+00:00 app[web.1]: Traceback (most recent call last):
2015-02-12T22:56:33.181105+00:00 app[web.1]:     response = self.full_dispatch_request()
2015-02-12T22:56:33.181082+00:00 app[web.1]: Exception on /registering [POST]
2015-02-12T22:56:33.181103+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/flask/app.py", line 1817, in wsgi_app
2015-02-12T22:56:33.181106+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/flask/app.py", line 1477, in full_dispatch_request
2015-02-12T22:56:33.181110+00:00 app[web.1]:     reraise(exc_type, exc_value, tb)
2015-02-12T22:56:33.181107+00:00 app[web.1]:     rv = self.handle_user_exception(e)
2015-02-12T22:56:33.181114+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/flask/app.py", line 1461, in dispatch_request
2015-02-12T22:56:33.181117+00:00 app[web.1]:   File "/app/app.py", line 361, in registering
2015-02-12T22:56:33.181121+00:00 app[web.1]:     with open(filename, 'rb') as fp:
2015-02-12T22:56:33.181119+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/boto/s3/key.py", line 1358, in set_contents_from_filename
2015-02-12T22:56:33.181116+00:00 app[web.1]:     return self.view_functions[rule.endpoint](**req.view_args)
2015-02-12T22:56:33.181109+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/flask/app.py", line 1381, in handle_user_exception
2015-02-12T22:56:33.181122+00:00 app[web.1]: IOError: [Errno 2] No such file or directory: u'888888.jpg'
2015-02-12T22:56:33.181118+00:00 app[web.1]:     k.set_contents_from_filename(file.filename)

What am I doing wrong(it's the first time I work with S3)


Solution

  • You need to store the file on disk before using upload_part_from_filename. You could also use cStringIO to load the file into memory and upload that to S3 instead of writing to disk first, but keep in mind large files will need to be chunk uploaded or you will fill up your memory using the below method. Here is an example of how you could use cStringIO

    # Python 2.7
    import cStringIO
    
    buff = cStringIO.StringIO()
    f = request.FILES['file']
    buff.write(f.read())
    buff.seek(0)
    k.set_contents_from_file(buff)