Search code examples
c#amazon-s3md5restsharppre-signed-url

How to add Content-Md5 Header to RestSharp file upload request?


I am trying to upload a list of strings as a csv to a presigned URL in AWS S3.

I have successfully upload this using the default .NET HttpClient, but I am unable to use RestSharp RestClient to do the same thing.

The Error returned by AWS is: BadDigest - The Content-MD5 you specified did not match what we received.

These are code sections I have been using. Both have the same inputs:

  • ids: a list of strings
  • md5: byte[], calculated with md5.ComputeHash() in System.Security.Cryptography library
  • url: presigned URL

Upload using HttpClient: (successful)

using (var ms = new MemoryStream())
{
    using (var sw = new StreamWriter(ms, new UTF8Encoding(false), 8192, true))
    {
        sw.Write(string.Join(Environment.NewLine, ids));
    }
    ms.Position = 0;

    var file = ms.ToArray();
    var content = new ByteArrayContent(file);
    content.Headers.ContentType = new MediaTypeHeaderValue("text/csv");
    content.Headers.ContentLength = file.Length;
    content.Headers.ContentMD5 = md5;

    var client = new HttpClient();
    var result = client.PutAsync(url, content).Result;
    var response = result.Content.ReadAsString();
}

Upload using RestSharp: (failed - Bad Digest)

using (var ms = new MemoryStream())
{
    using (var sw = new StreamWriter(ms, new UTF8Encoding(false), 8192, true))
    {
        sw.Write(string.Join(Environment.NewLine, ids));
    }
    ms.Position = 0;

    var file = ms.ToArray();
    var md5Based64 = Convert.ToBase64String(md5).Replace("-", "");
    var options = new FileParameterOptions
    {
        DisableFilenameEncoding = true,
    };
    var uploadRequest = new RestRequest()
        .AddFile("", file, "", "text/csv", options)
        .AddOrUpdateHeader("Content-Md5", md5Based64)
        .AddOrUpdateHeader("Content-Length", file.Length)
        .AddOrUpdateHeader("Content-Type", "text/csv");

    var restClient = new RestClient(
        new Uri(url), 
        configureSerialization: cfg =>
        {
            cfg.UseNewtonsoftJson();
        }, 
        useClientFactory: true
    );
    var response = restClient.ExecuteAsync(uploadRequest, Method.Put).Result;
}

I feel like the conversion between md5 and md5Based64 might be the key here or maybe manually adding header AddOrUpdateHeader("Content-Md5", md5Based64) is causing it to failed but I am not sure what else to do.

Any suggestions on how to fix this are welcomed. Let me know if you notice anything missing or want to be clarified.


Solution

  • RestSharp.AddFile encodes your request as multipart/form-data, while your pre-signed URL and the hashing algorithm both expect raw file data in the request body.

    Instead of AddFile, use AddStringBody:

    If you have a pre-serialized payload like a JSON string, you can use AddStringBody to add it as a body parameter. You need to specify the content type, so the remote endpoint knows what to do with the request body.