I implemented browser based resumable uploads into Google's Cloud Storage using an XMLHttpRequest send to a server-side created resumable upload url. This works completely fine when disabling web security, which I did during development.
But now in the real world, CORS keeps making trouble. I tried this with other browsers (without success), too, but sticked to chrome for further testing.
Note: A fake.host entry in /etc/HOSTS is used to trick chrome into avoiding localhost-restrictions. Nevertheless, same happens with the "real" domain of our online test server.
The request is started using a normal XMLHttpRequest call:
var xhr = this.newXMLHttpRequest();
xhr.open('PUT', url, true);
xhr.setRequestHeader('Content-Type', this.currentInputFile.type);
xhr.setRequestHeader('Content-Range', 'bytes ' + startByte + '-' + (this.currentInputFile.size - 1) + '/' + this.currentInputFile.size);
xhr.onload = function(e) {
...
};
...
if (startByte > 0) {
xhr.send(this.currentInputFile.slice(startByte));
} else {
xhr.send(this.currentInputFile);
}
The browser then successfully initiates a preflight request:
Remote Address:173.194.71.95:443
Request URL:https://www.googleapis.com/upload/storage/v1/b/my-bucket-name/o?uploadType=resumable&name=aa%20spacetestSMALL_512kb.mp4&upload_id=XXXXXXXXX
Request Method:OPTIONS
Status Code:200 OK
Request Headers:
:host:www.googleapis.com
:method:OPTIONS
:path:/upload/storage/v1/b/my-bucket-name/o?uploadType=resumable&name=aa%20spacetestSMALL_512kb.mp4&upload_id=XXXXXXXXX
:scheme:https
:version:HTTP/1.1
accept:*/*
accept-encoding:gzip,deflate
accept-language:en-US,en;q=0.8,de;q=0.6
access-control-request-headers:content-range, content-type
access-control-request-method:PUT
origin:https://fake.host
referer:https://fake.host/upload.xhtml
user-agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.94 Safari/537.36
x-client-data:YYYYYY
Query String Parameters
uploadType:resumable
name:aa spacetestSMALL_512kb.mp4
upload_id:XXXXXXXXX
Response Headers
access-control-allow-credentials:true
access-control-allow-headers:content-range, content-type
access-control-allow-methods:PUT
access-control-allow-origin:https://fake.host
alternate-protocol:443:quic
content-length:0
content-type:text/html; charset=UTF-8
date:Fri, 05 Sep 2014 14:11:21 GMT
server:UploadServer ("Built on Aug 18 2014 11:58:36 (1408388316)")
status:200 OK
version:HTTP/1.1
... and starts the PUT-request until all data is transferred. But afterwards chrome silently logs an error without completing/ending the request:
XMLHttpRequest cannot load https://www.googleapis.com/upload/storage/v1/b/my-bucket-name…XXXXXXXX. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'https://fake.host' is therefore not allowed access.
This is what chrome logs about the PUT request:
Request URL:https://www.googleapis.com/upload/storage/v1/b/my-bucket-name/o?uploadType=resumable&name=aa%20spacetestSMALL_512kb.mp4&upload_id=XXXXXXXXX
Request Headers
Provisional headers are shown
Content-Range:bytes 0-3355302/3355303
Content-Type:video/mp4
Origin:https://fake.host
Referer:https://fake.host/upload.xhtml
User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.94 Safari/537.36
X-DevTools-Emulate-Network-Conditions-Client-Id:YYYYYYY
Query String
uploadType:resumable
name:aa spacetestSMALL_512kb.mp4
upload_id:XXXXXXXXX
Notably, when adding the same url in http://client.cors-api.appspot.com/client and issuing any request, all but the OPTIONS request types fail, too. It seems like the cloud storage api only issues the correct response headers for OPTION requests, but not PUT/POST/GET/... requests.
So am I doing something impossible? Is there something broken? Is this a bug in the cloud storage api? I've spend hours googling and reading SO answers, without any luck so far.
For now, I could periodically check if the download transferred 100% of the data and just ignore the http request outcome, as the file is in fact completely uploaded to the storage bucket. But this is a rather ugly workaround which I really don't want to use if the real issue can be solved.
When requesting a resumable upload url, you MUST include the origin the browser will send when trying to use that upload url, or else the subsequent uploading will fail, just as is happening in the question (the OPTIONS call will look good, but the PUT will not).
It must exactly match the browser's origin (which you can get as location.origin in javascript).
This is the step "Initiating a resumable upload session" in this documentation: https://cloud.google.com/storage/docs/json_api/v1/how-tos/resumable-upload
If you're requesting the resumable upload url on the server side, you'll probably need the client side (the browser) to pass you its origin (eg: location.origin).
fwiw I was using Google's Cloud Storage library for python for this step, and needed to add the origin like this:
myblob.create_resumable_upload_session(mycontenttype, origin=browserorigin)
Note that you definitely do not need to set up CORS for your bucket.