HttpClient returns 0 bytes

Question

GetByteArrayAsync and GetStreamAsync both returns 0 bytes or empty stream. I want to write the bytes received to a file.

I have tried using byte[] stream = await _httpClient.GetByteArrayAsync("http://gscs-b2c.lge.com/downloadFile?fileId=BZhqND5RTnHcyMRDCeJJQ") and _httpClient.GetStreamAsync("http://gscs-b2c.lge.com/downloadFile?fileId=BZhqND5RTnHcyMRDCeJJQ"), but both of them return either 0 bytes or empty stream.

I have tried from PostMan, as a Get request, it returned a valid PDF file. I have tried it from the browser, it returns a valid PDF file. But here in the code it return empty file (0 bytes). I checked other answers but still I could not fix this. Interestingly it works for other pdf files from other sites.

Following is the sample code in .net Core 2.2 that I am using

_client = new HttpClient();
_client.DefaultRequestHeaders.UserAgent.ParseAdd("Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36");
_client.Timeout = TimeSpan.FromMinutes(2);

System.Net.ServicePointManager.SecurityProtocol = System.Net.SecurityProtocolType.Tls | System.Net.SecurityProtocolType.Tls11 | System.Net.SecurityProtocolType.Tls12;

var downloadPdfLink = "http://gscs-b2c.lge.com/downloadFile?fileId=BZhqND5RTnHcyMRDCeJJQ";
using (Stream stream = await _client.GetStreamAsync(downloadPdfLink))
{
    using (var fs = new FileStream("./file/myfile.pdf",FileMode.CreateNew,FileAccess.Write))
    {
        //stream.Seek(0, SeekOrigin.Begin);
        stream.CopyTo(fs);
    }
}

The expected result should be that the returned stream is written to "myfile.pdf". Can you please help me understand what is the problem here, what am I doing wrong?

Answer 1

When downloading the .pdf file via a web browser, I have noticed in the Developer Tools that additional Headers get added to the Request: Accept-Encoding: gzip, deflate, amongst others, which will in return carry the file in the Content of the Response, rather than leaving it with a Length of 0 like in your original code.

There are several ways in fixing your code:

HttpClient.DefaultRequestHeaders

_client.DefaultRequestHeaders.Add("Accept-Encoding", "gzip");
_client.DefaultRequestHeaders.Add("Accept-Encoding", "deflate");

HttpClientHandler.AutomaticDecompression

var handler = new HttpClientHandler();
handler.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
var _client = new HttpClient(handler);

HttpRequestHeaders.AcceptEncoding

var request = new HttpRequestMessage(HttpMethod.Get, downloadPdfLink);
request.Headers.AcceptEncoding.Add(new StringWithQualityHeaderValue("gzip"));
request.Headers.AcceptEncoding.Add(new StringWithQualityHeaderValue("deflate"));
using (HttpResponseMessage response = await _client.SendAsync(request))
{
    using (Stream stream = await response.Content.ReadAsStreamAsync())
    {
        Directory.CreateDirectory("./file");
        using (var fs = new FileStream("./file/myfile.pdf", FileMode.Create, FileAccess.Write))
        {
            //stream.Seek(0, SeekOrigin.Begin);
            stream.CopyTo(fs);
        }
    }
}

And, as a friendly reminder, don't forget to Dispose() your HttpClient, once you are done with it.