I'm attempting to get an Android app to interact with Google Docs/Drive using the Documents List API: https://developers.google.com/google-apps/documents-list
I've run in to a bit of trouble with uploading files using the GData Resumable Upload Protocol, i.e.: https://developers.google.com/google-apps/documents-list#uploading_a_new_document_or_file_with_both_metadata_and_content
I'm using a 512KiB chunk size. Files smaller than the chunk size upload successfully, but files larger than the chunk size will fail before the first chunk is completed. This is also true even when I increase the chunk size to 1MiB or 2MiB. A 768KiB file will fail with a chunk size of 512KiB, but succeed with a chunk size of 1MiB.
Larger-than-chunk-size files do make it past the first step, i.e., POSTing XML to the "resumable create media" link. I then see an SSLException thrown in the writeTo() method of the HttpContent implementation used to send the chunk. This exception occurs midway through the FIRST chunk, for example:
javax.net.ssl.SSLException: Write error: ssl=0x2a6318: I/O error during system call, Broken pipe
at org.apache.harmony.xnet.provider.jsse.NativeCrypto.SSL_write(Native Method)
at org.apache.harmony.xnet.provider.jsse.OpenSSLSocketImpl$SSLOutputStream.write(OpenSSLSocketImpl.java:713)
at libcore.net.http.FixedLengthOutputStream.write(FixedLengthOutputStream.java:41)
at com.google.api.client.http.AbstractInputStreamContent.writeTo(AbstractInputStreamContent.java:89)
Note that to see the above, I had to actually capture IOExceptions in my writeTo() method (rethrowing them after logging them). Otherwise I'd simply see a generic "400 Bad Request" response due to the content being abruptly cut off.
I've tried a few different test devices, e.g., Galaxy Nexus (VZW/CDMA, 4.0.2, stock), Droid4 (Moto 2.3.6, stock) DroidX (stock).
Headers on the request uploading the first chunk:
Accept-Encoding: gzip
Authorization: GoogleLogin auth=XXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Content-Length: 524288
Content-Range: 0-524287/1635085
Content-Type: image/png
GData-Version: 3.0
User-Agent: XXXXXXXXXXXXXX
Thanks for any suggestions.
UPDATE: adding code snippet, this is the bit that PUTs the next chunk.
private boolean putNext()
throws CancelException, DirectoryException {
try {
ByteArrayContent content = new ByteArrayContent(mediaType, buffer, 0, bufferLength);
HttpRequest request = conn.getHttpRequestFactory().buildPutRequest(new GenericUrl(nextLocation), content);
HttpHeaders requestHeaders = request.getHeaders();
requestHeaders.setContentLength(String.valueOf(bufferLength));
requestHeaders.setContentType(mediaType);
if (sent != 0 || bufferLength != size) {
// Only set content range when file will span multiple chunks.
requestHeaders.setContentRange(sent + "-" + (sent + bufferLength - 1) + "/" + size);
}
HttpResponse response = request.execute();
sent += bufferLength;
bufferLength = 0;
HttpHeaders responseHeaders = response.getHeaders();
int statusCode = response.getStatusCode();
if (statusCode == GoogleDriveConnection.RESPONSE_308_RESUME_INCOMPLETE) {
nextLocation = responseHeaders.getLocation();
return false;
} else if (statusCode >= 200 && statusCode < 400) {
Document responseDom = DomUtil.load(response.getContent());
return true;
} else {
Log.w(LOG_TAG, "Google Drive upload error: Invalid response code to upload request: " + statusCode);
throw DirectoryException.networkErrorHost(null, catalog.getHostName());
}
} catch (IOException ex) {
Log.w(LOG_TAG, "Google Drive write error.", ex);
throw DirectoryException.networkErrorHost(ex, catalog.getHostName());
} catch (SAXException ex) {
throw DirectoryException.networkErrorHost(ex, catalog.getHostName());
}
}
Another possibly important note: the SSLException failure is triggered by the presence of the Content-Range header. If I omit it, the first chunk of bigger-than-chunk-size files is uploaded successfully without the exception. The server returns a 201 Created and the file is created with a truncated 512KiB size.
Thanks again!
UPDATE 2
I now have a pure-Java app available to demonstrate the issue. This is packaged as an Eclipse project with included libraries but should be trivial to use elsewhere. http://android.nextapp.com/test/DriveUpload.zip
An auth token is required to test it. It requires three command line parameters to run:
[filename] [content/type] [AuthToken]
(I'm currently cutting/pasting authtokens from debug output in the real app)
Here's the trimmed output of this application:
TOKEN=********
FILE=/tmp/1199Panigale.jpg
Headers: POST Request
-- User-Agent: DriveUpload
-- GData-Version: 3.0
-- Authorization: GoogleLogin auth=********
-- X-Upload-Content-Type: image/png
-- X-Upload-Content-Length: 1635085
[POST Request] --------------------------------------------------------
<entry xmlns="http://www.w3.org/2005/Atom">
<title>1199Panigale.jpg</title>
</entry>
Executing post request: https://docs.google.com/feeds/upload/create-session/default/private/full/folder%3Aroot/contents?convert=false
Post complete, response=200
Headers: POST Response
-- Server: HTTP Upload Server Built on Apr 23 2012 11:11:29 (1335204689)
-- Location: https://docs.google.com/feeds/upload/create-session/default/private/full/folder%3Aroot/contents?convert=false&upload_id=********2
-- Date: Sat, 28 Apr 2012 04:51:47 GMT
-- Pragma: no-cache
-- Expires: Fri, 01 Jan 1990 00:00:00 GMT
-- Cache-Control: no-cache, no-store, must-revalidate
-- Content-Length: 0
-- Content-Type: text/html
Preparing PUT request #0
Headers: Put Request
-- User-Agent: DriveUpload
-- GData-Version: 3.0
-- Authorization: GoogleLogin auth=********
-- Content-Type: image/png
-- Content-Range: 0-524287/1635085
[writeTo]
[getCotent]
Read: 4096, total: 4096
Read: 4096, total: 8192
Read: 4096, total: 12288
Read: 4096, total: 16384
[---skip a few---]
Read: 4096, total: 270336
Read: 4096, total: 274432
Read: 4096, total: 278528
Read: 4096, total: 282624
Read: 4096, total: 286720
Read: 4096, total: 290816
Apr 27, 2012 9:49:02 PM org.apache.http.impl.client.DefaultRequestDirector tryExecute
INFO: I/O exception (java.net.SocketException) caught when processing request: Broken pipe
Apr 27, 2012 9:49:02 PM org.apache.http.impl.client.DefaultRequestDirector tryExecute
INFO: Retrying request
[writeTo]
[getCotent]
Read: 4096, total: 4096
Read: 4096, total: 8192
Read: 4096, total: 12288
Read: 4096, total: 16384
Read: 4096, total: 20480
[---skip a few---]
Read: 4096, total: 274432
Read: 4096, total: 278528
Read: 4096, total: 282624
Read: 4096, total: 286720
Read: 4096, total: 290816
Apr 27, 2012 9:49:02 PM org.apache.http.impl.client.DefaultRequestDirector tryExecute
INFO: I/O exception (java.net.SocketException) caught when processing request: Broken pipe
Apr 27, 2012 9:49:02 PM org.apache.http.impl.client.DefaultRequestDirector tryExecute
INFO: Retrying request
[writeTo]
[getCotent]
Read: 4096, total: 4096
Read: 4096, total: 8192
Read: 4096, total: 12288
Read: 4096, total: 16384
Read: 4096, total: 20480
[---skip a few---]
Read: 4096, total: 278528
Read: 4096, total: 282624
Read: 4096, total: 286720
Read: 4096, total: 290816
Read: 4096, total: 294912
Apr 27, 2012 9:49:02 PM org.apache.http.impl.client.DefaultRequestDirector tryExecute
INFO: I/O exception (java.net.SocketException) caught when processing request: Broken pipe
Apr 27, 2012 9:49:02 PM org.apache.http.impl.client.DefaultRequestDirector tryExecute
INFO: Retrying request
[writeTo]
[getCotent]
Read: 4096, total: 4096
Read: 4096, total: 8192
Read: 4096, total: 12288
Read: 4096, total: 16384
Read: 4096, total: 20480
[---skip a few---]
Read: 4096, total: 278528
Read: 4096, total: 282624
Read: 4096, total: 286720
java.net.SocketException: Broken pipe
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109)
at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
at sun.security.ssl.OutputRecord.writeBuffer(OutputRecord.java:314)
at sun.security.ssl.OutputRecord.write(OutputRecord.java:303)
at sun.security.ssl.SSLSocketImpl.writeRecordInternal(SSLSocketImpl.java:768)
at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:756)
at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:108)
at org.apache.http.impl.io.AbstractSessionOutputBuffer.write(AbstractSessionOutputBuffer.java:153)
at org.apache.http.impl.io.ContentLengthOutputStream.write(ContentLengthOutputStream.java:114)
at driveupload.UploadStream$1.writeTo(UploadStream.java:149)
at org.apache.http.entity.HttpEntityWrapper.writeTo(HttpEntityWrapper.java:96)
at org.apache.http.impl.client.EntityEnclosingRequestWrapper$EntityWrapper.writeTo(EntityEnclosingRequestWrapper.java:108)
at org.apache.http.impl.entity.EntitySerializer.serialize(EntitySerializer.java:120)
at org.apache.http.impl.AbstractHttpClientConnection.sendRequestEntity(AbstractHttpClientConnection.java:264)
at org.apache.http.impl.conn.AbstractClientConnAdapter.sendRequestEntity(AbstractClientConnAdapter.java:224)
at org.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:255)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:647)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:464)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:732)
at driveupload.DriveUpload.executeRequest(DriveUpload.java:71)
at driveupload.UploadStream.putNext(UploadStream.java:191)
at driveupload.UploadStream.write(UploadStream.java:225)
at java.io.OutputStream.write(OutputStream.java:116)
at driveupload.DriveUpload.test(DriveUpload.java:127)
at driveupload.DriveUpload.main(DriveUpload.java:44)
The problem turns out to be that I was failing to prefix the "Content-Range" header with "bytes". I haven't a clue as to why I didn't find this on one of the previous half-dozen times I checked the headers for consistency with the spec. :)
Thanks for your help, and sorry to bother you with this non-issue.
I did notice though that the spec indicates the response to the PUT requests will provide a "Location" header for the next upload. I'm not seeing one, but it works fine if I stick with the previously issued location. My code now simply updates the location URL if one were to be provided, and continues with the previously issued one if not.