Search code examples
google-cloud-storageaws-sdk-net

Uploading to Google Cloud Storage using AWS SDK for .NET `Expected hash not equal to calculated hash`


I am trying to upload to Google Cloud Storage using the AWS SDK For .NET. I have enabled interoperability mode on GCS and put AWSSecretKey and AWSAccessKey in my App.config. I can list objects, but uploading is not working for me.

The code I am using is

            AmazonS3Config c = new AmazonS3Config();
            c.ServiceURL = "https://storage.googleapis.com";
            client = new AmazonS3Client(c);
            PutObjectRequest request = new PutObjectRequest()
            {
                ContentBody = "asdf",
                BucketName = bucketName,
                Key = keyName
            };
            client.PutObject(request);

But the result I get is:

Unhandled Exception:
Amazon.Runtime.AmazonClientException: Expected hash not equal to calculated hash
  at Amazon.S3.Internal.AmazonS3ResponseHandler.CompareHashes (System.String etag, System.Byte[] hash) [0x0003f] in <c4952b4c8b494e049894eba742a57891>:0
  at Amazon.S3.Internal.AmazonS3ResponseHandler.ProcessResponseHandlers (Amazon.Runtime.IExecutionContext executionContext) [0x00129] in <c4952b4c8b494e049894eba742a57891>:0
  at Amazon.S3.Internal.AmazonS3ResponseHandler.PostInvoke (Amazon.Runtime.IExecutionContext executionContext) [0x00000] in <c4952b4c8b494e049894eba742a57891>:0
  at Amazon.S3.Internal.AmazonS3ResponseHandler.InvokeSync (Amazon.Runtime.IExecutionContext executionContext) [0x00007] in <c4952b4c8b494e049894eba742a57891>:0
  at Amazon.Runtime.Internal.PipelineHandler.InvokeSync (Amazon.Runtime.IExecutionContext executionContext) [0x0000e] in <b8d5dfd6d5c84f7e87493f5d86f0adaa>:0
  at Amazon.Runtime.Internal.ErrorHandler.InvokeSync (Amazon.Runtime.IExecutionContext executionContext) [0x0001f] in <b8d5dfd6d5c84f7e87493f5d86f0adaa>:0
  at Amazon.Runtime.Internal.PipelineHandler.InvokeSync (Amazon.Runtime.IExecutionContext executionContext) [0x0000e] in <b8d5dfd6d5c84f7e87493f5d86f0adaa>:0
  at Amazon.Runtime.Internal.CallbackHandler.InvokeSync (Amazon.Runtime.IExecutionContext executionContext) [0x00007] in <b8d5dfd6d5c84f7e87493f5d86f0adaa>:0
  at Amazon.Runtime.Internal.PipelineHandler.InvokeSync (Amazon.Runtime.IExecutionContext executionContext) [0x0000e] in <b8d5dfd6d5c84f7e87493f5d86f0adaa>:0
  at Amazon.Runtime.Internal.Signer.InvokeSync (Amazon.Runtime.IExecutionContext executionContext) [0x00006] in <b8d5dfd6d5c84f7e87493f5d86f0adaa>:0
  at Amazon.Runtime.Internal.PipelineHandler.InvokeSync (Amazon.Runtime.IExecutionContext executionContext) [0x0000e] in <b8d5dfd6d5c84f7e87493f5d86f0adaa>:0
  at Amazon.Runtime.Internal.EndpointDiscoveryHandler.InvokeSync (Amazon.Runtime.IExecutionContext executionContext) [0x00030] in <b8d5dfd6d5c84f7e87493f5d86f0adaa>:0
  at Amazon.Runtime.Internal.PipelineHandler.InvokeSync (Amazon.Runtime.IExecutionContext executionContext) [0x0000e] in <b8d5dfd6d5c84f7e87493f5d86f0adaa>:0
  at Amazon.Runtime.Internal.CredentialsRetriever.InvokeSync (Amazon.Runtime.IExecutionContext executionContext) [0x00007] in <b8d5dfd6d5c84f7e87493f5d86f0adaa>:0
  at Amazon.Runtime.Internal.PipelineHandler.InvokeSync (Amazon.Runtime.IExecutionContext executionContext) [0x0000e] in <b8d5dfd6d5c84f7e87493f5d86f0adaa>:0
  at Amazon.Runtime.Internal.RetryHandler.InvokeSync (Amazon.Runtime.IExecutionContext executionContext) [0x00057] in <b8d5dfd6d5c84f7e87493f5d86f0adaa>:0
  at Amazon.Runtime.Internal.PipelineHandler.InvokeSync (Amazon.Runtime.IExecutionContext executionContext) [0x0000e] in <b8d5dfd6d5c84f7e87493f5d86f0adaa>:0
  at Amazon.Runtime.Internal.CallbackHandler.InvokeSync (Amazon.Runtime.IExecutionContext executionContext) [0x00007] in <b8d5dfd6d5c84f7e87493f5d86f0adaa>:0
  at Amazon.Runtime.Internal.PipelineHandler.InvokeSync (Amazon.Runtime.IExecutionContext executionContext) [0x0000e] in <b8d5dfd6d5c84f7e87493f5d86f0adaa>:0
  at Amazon.S3.Internal.AmazonS3KmsHandler.InvokeSync (Amazon.Runtime.IExecutionContext executionContext) [0x00007] in <c4952b4c8b494e049894eba742a57891>:0
  at Amazon.Runtime.Internal.PipelineHandler.InvokeSync (Amazon.Runtime.IExecutionContext executionContext) [0x0000e] in <b8d5dfd6d5c84f7e87493f5d86f0adaa>:0
  at Amazon.Runtime.Internal.EndpointResolver.InvokeSync (Amazon.Runtime.IExecutionContext executionContext) [0x00007] in <b8d5dfd6d5c84f7e87493f5d86f0adaa>:0
  at Amazon.Runtime.Internal.PipelineHandler.InvokeSync (Amazon.Runtime.IExecutionContext executionContext) [0x0000e] in <b8d5dfd6d5c84f7e87493f5d86f0adaa>:0
  at Amazon.S3.Internal.AmazonS3PostMarshallHandler.InvokeSync (Amazon.Runtime.IExecutionContext executionContext) [0x00007] in <c4952b4c8b494e049894eba742a57891>:0
  at Amazon.Runtime.Internal.PipelineHandler.InvokeSync (Amazon.Runtime.IExecutionContext executionContext) [0x0000e] in <b8d5dfd6d5c84f7e87493f5d86f0adaa>:0
  at Amazon.Runtime.Internal.Marshaller.InvokeSync (Amazon.Runtime.IExecutionContext executionContext) [0x00006] in <b8d5dfd6d5c84f7e87493f5d86f0adaa>:0
  at Amazon.Runtime.Internal.PipelineHandler.InvokeSync (Amazon.Runtime.IExecutionContext executionContext) [0x0000e] in <b8d5dfd6d5c84f7e87493f5d86f0adaa>:0
  at Amazon.S3.Internal.AmazonS3PreMarshallHandler.InvokeSync (Amazon.Runtime.IExecutionContext executionContext) [0x00007] in <c4952b4c8b494e049894eba742a57891>:0
  at Amazon.Runtime.Internal.PipelineHandler.InvokeSync (Amazon.Runtime.IExecutionContext executionContext) [0x0000e] in <b8d5dfd6d5c84f7e87493f5d86f0adaa>:0
  at Amazon.Runtime.Internal.CallbackHandler.InvokeSync (Amazon.Runtime.IExecutionContext executionContext) [0x00007] in <b8d5dfd6d5c84f7e87493f5d86f0adaa>:0
  at Amazon.Runtime.Internal.PipelineHandler.InvokeSync (Amazon.Runtime.IExecutionContext executionContext) [0x0000e] in <b8d5dfd6d5c84f7e87493f5d86f0adaa>:0
  at Amazon.S3.Internal.AmazonS3ExceptionHandler.InvokeSync (Amazon.Runtime.IExecutionContext executionContext) [0x00012] in <c4952b4c8b494e049894eba742a57891>:0
  at Amazon.Runtime.Internal.PipelineHandler.InvokeSync (Amazon.Runtime.IExecutionContext executionContext) [0x0000e] in <b8d5dfd6d5c84f7e87493f5d86f0adaa>:0
  at Amazon.Runtime.Internal.ErrorCallbackHandler.InvokeSync (Amazon.Runtime.IExecutionContext executionContext) [0x00012] in <b8d5dfd6d5c84f7e87493f5d86f0adaa>:0
  at Amazon.Runtime.Internal.PipelineHandler.InvokeSync (Amazon.Runtime.IExecutionContext executionContext) [0x0000e] in <b8d5dfd6d5c84f7e87493f5d86f0adaa>:0
  at Amazon.Runtime.Internal.MetricsHandler.InvokeSync (Amazon.Runtime.IExecutionContext executionContext) [0x0002b] in <b8d5dfd6d5c84f7e87493f5d86f0adaa>:0
  at Amazon.Runtime.Internal.RuntimePipeline.InvokeSync (Amazon.Runtime.IExecutionContext executionContext) [0x00006] in <b8d5dfd6d5c84f7e87493f5d86f0adaa>:0
  at Amazon.Runtime.AmazonServiceClient.Invoke[TResponse] (Amazon.Runtime.AmazonWebServiceRequest request, Amazon.Runtime.Internal.InvokeOptionsBase options) [0x0007d] in <b8d5dfd6d5c84f7e87493f5d86f0adaa>:0
  at Amazon.S3.AmazonS3Client.PutObject (Amazon.S3.Model.PutObjectRequest request) [0x0001c] in <c4952b4c8b494e049894eba742a57891>:0
  at GettingStartedGuide.S3Sample.Main (System.String[] args) [0x00059] in <52a410cf4a3b4fedb3f3e1dd35779339>:0

The same code works OK with S3 (minus setting the ServiceURL)


Solution

  • The GCS docs say:

    In the Cloud Storage XML API, chunked transfer encoding and V4 signatures cannot currently be used simultaneously. Some Amazon S3 tools use chunked transfer encoding along with signatures by default; you should disable chunked transfer encoding in such cases.

    I found that to resolve this error, I needed to add UseChunkEncoding = false to the PutObjectRequest:

                PutObjectRequest request = new PutObjectRequest()
                {
                    ContentBody = "asdf",
                    BucketName = bucketName,
                    Key = keyName,
                    UseChunkEncoding = false,  // <<<<<<
                };
                client.PutObject(request);