How to retrieve a truly large file using System.Net.Http.HttpClient?
Related: Large HTTPResponseMessage causes .NET Core server process to run out of memory
Calling HttpClient.GetStreamAsync() doesn't work because I need to look at the response headers.
The answer doesn't apply though. The file is too big to read into RAM at once. That shouldn't be a problem because I can use ReadAsStreamAsync() to get asynchronous stream, right? Nope! That call is really nice and tries to materialize the whole thing into a MemoryStream, which is impossible because it exceeds the maximum size of a MemoryStream.
How do I actually get the stream so I can read from it?
.NET version is .NET 7
Must request for the HttpClient to not buffer; thus must call SendAsync (or Send). The critical point is HttpCompletionOption.ResponseHeadersRead:
var request = new HttpRequestMessage(HttpMethod.Get, url);
var response = await client.SendAsync(request, HttpCompletionOption.ResponseHeadersRead);
using var response = await client.SendAsync(request, HttpCompletionOption.ResponseHeadersRead);
var stream = response.Content.ReadAsStreamAsync();