Jclouds multipart upload to Google Cloud Storage failing with 400 Bad Request

Question

I am building an application that uses Jclouds (version 2.1.0) to access Google Cloud Storage and upload/download files. I am encountering an issue with multipart uploads. Not sure if I am doing something wrong, or if this is a bug.

What I am seeing is whenever I do a multipart upload, I get a "400 Bad Request" response. Here is the code I am using to upload:

String fileContents = Files.toString(new File("google_creds.json"), Charset.defaultCharset());

Supplier<Credentials> credentialSupplier = new GoogleCredentialsFromJson(fileContents);

Iterable<Module> modules = ImmutableSet.<Module>of(
        new SLF4JLoggingModule());

BlobStoreContext context = ContextBuilder.newBuilder("google-cloud-storage")
        .credentialsSupplier(credentialSupplier)
        .modules(modules)
        .buildView(BlobStoreContext.class);

BlobStore client = context.getBlobStore();

byte[] testBytes = new byte[102400];

Blob blob = client.blobBuilder("testupload").payload(testBytes)
        .contentLength(testBytes.length).build();


// Multipart upload fails
 client.putBlob("filecatalyst-unit-tests", blob, multipart());

And here is the logging output that I see:

>> "{"sourceObjects":[{"name":"3fda0043-e340-4ccb-a582-9e3b78cf85e1_00000001","generation":1520026856033315,"objectPreconditions":{"ifGenerationMatch":1520026856033315}}],"destination":{"name":"testupload","size":102400,"contentType":"application/unknown","storageClass":"STANDARD","metadata":{}}}"
>> POST https://www.googleapis.com/storage/v1/b/filecatalyst-unit-tests/o/testupload/compose HTTP/1.1
>> Accept: application/json
>> Authorization: Bearer ***********************
>> Content-Type: application/json
>> Content-Length: 293
<< HTTP/1.1 400 Bad Request
<< Alt-Svc: hq=":443"; ma=2592000; quic=51303431; quic=51303339; quic=51303338; quic=51303337; quic=51303335,quic=":443"; ma=2592000; v="41,39,38,37,35"
<< Server: UploadServer
<< X-GUploader-UploadID: AEnB2UpqKslm87V4wto81YsNphtPFtybHjWgffGAC3XPE-xQAXSjDldst5s5nuWPyhXjEGOwWE5kDETjNA1EedbuEw9Vbe8CtA
<< Vary: X-Origin
<< Vary: Origin
<< Date: Fri, 02 Mar 2018 21:40:56 GMT
<< Content-Type: application/json; charset=UTF-8
<< Content-Length: 178
<< "{[\n]"
<< " "error": {[\n]"
<< "  "errors": [[\n]"
<< "   {[\n]"
<< "    "domain": "global",[\n]"
<< "    "reason": "invalid",[\n]"
<< "    "message": "Invalid argument"[\n]"
<< "   }[\n]"
<< "  ],[\n]"
<< "  "code": 400,[\n]"
<< "  "message": "Invalid argument"[\n]"
<< " }[\n]"
<< "}[\n]"

Here is a formatted version of the JSON body for the compose request:

{  
   "sourceObjects":[  
      {  
         "name":"83116375-0ba1-4d1a-aec3-8dd29dab0da9_00000001",
         "generation":1520263556434885,
         "objectPreconditions":{  
            "ifGenerationMatch":1520263556434885
         }
      }
   ],
   "destination":{  
      "name":"testupload",
      "size":33554432,
      "contentType":"application/unknown",
      "storageClass":"STANDARD",
      "metadata":{  

      }
   }
}

Have tried with varying size files, and I get the same result. If I do not use multipart upload, it works fine.

Answer 1

JCLOUDS-1389 tracks this issue and jclouds 2.1.1 and 2.2.0 will include the fix.