Search code examples
pythongoogle-app-enginewebapp2

Google App Engine: store uploaded file in Google Cloud Storage


I have managed to create a file in GCS in the post method of a webapp2 handler of a Google App Engine application. I can't see how to copy the content of a posted file in the file created in GCS. Here is my code

 inFile =  self.request.POST.multi['file'].file
 gcs_file = gcs.open(filename,
                    'w',
                    content_type='text/plain',
                    options={'x-goog-meta-foo': 'foo',
                             'x-goog-meta-bar': 'bar'},
                    retry_params=write_retry_params)
  while 1:
        line = inFile.readline()
        if not line: break
        gcs_file.write(line)
  gcs_file.close()

At the end of the process the file in GCS is 0 byte

UPDATE There is a reason why I am not using the blobstore. When using the Blobstore you have to create an url and sand it back to the client. it is the client that performs the actual upload. INSTEAD I need to encrypt the file on the server before to put it in the GCS. Thus I need to receive the file from the client, encrypt it on the server and store it in the GCS.


Solution

  • I've successfully POST'ed a file to GCS with the following method:

    def post(self):
        write_retry_params = gcs.RetryParams(backoff_factor=1.1)
        filename = '/{MY-BUCKET-NAME}/static.txt'
    
        gcs_file = gcs.open(
            filename,
            'w',
            content_type='text/plain',
            options={'x-goog-meta-foo': 'foo',
                     'x-goog-meta-bar': 'bar'},
            retry_params=write_retry_params)
    
        inFile = self.request.POST.multi['file'].file
        while 1:
            line = inFile.readline()
            if not line:
                break
            gcs_file.write(line)
            logging.info('Wrote line: {}'.format(line))
    
        gcs_file.close()
    

    Here's the little log message from Console:

    I 09:20:32.979 2015-05-07  200      84 B   1.13s /static
        76.176.106.172 - - [07/May/2015:09:20:32 -0700] "POST /static HTTP/1.1" 200 84 - "curl/7.37.1" "{MY-APP}" ms=1131 cpu_ms=1355 cpm_usd=0.000009 loading_request=1 instance=00c61b117cee89e66d34a42c5bbe3cf2b0bb06b5 app_engine_release=1.9.20
    I 09:20:32.832 Wrote line: stack
    I 09:20:32.833 Wrote line: overflow
    

    Here's the test.txt file I uploaded:

    stack
    overflow
    

    And the cURL command I used:

    curl -F"file=@/Users/{name}/test.txt" http://{MY-APP}/videostatic
    

    If you're still getting 0 bytes from readline() or read() I'm going to have to assume that your client is not sending the correct multipart message.