c# JSON REST response via 3 different approaches (WebRequest, RESTSharp, HttpClient) is empty, but Postman and browser works

Question

When calling a specific endpoint in C# which works without issues in Postman (or via Firefox), I'm getting an empty response.

The url I'm calling is returning a collection of data. In the url parameters I can specify how much of said data I want. I've inspected the response size in Postman, and when I limit the amount of data requested in my C# call such that the response is around 700kb, then I get a JSON response back.

However, if I exceed this size in the C# call, then the response is empty '{ }' and the ContentLength returned = -1 (the statusCode returned is 200, so this seems fine at least). This same request which fails in C# works fine within Postman and Firefox however...

I somehow suspect this occurs because either the deserializer's buffer is not big enough OR because the response is still in transit and the code somehow continues executing before it has read the whole response body...

See below for the 3 implementations which I've tested:

1:

var httpClient = new HttpClient();
var responseMessage = await httpClient.GetAsync(requestUrl, HttpCompletionOption.ResponseHeadersRead);
if (responseMessage.StatusCode == System.Net.HttpStatusCode.OK)
{
      using (var httpStream = await responseMessage.Content.ReadAsStreamAsync())
      {
           using (var sr = new StreamReader(httpStream))
           {
               Info(await sr.ReadToEndAsync()); //Info logs the string to a file
           }
      }
}

2 (RESTSharp):

var client = new RestClient();
var request = new RestRequest(requestUrl, Method.GET, DataFormat.Json);
Info(request.Content); //Info logs the string to a file

3:

var httpWebRequest = (HttpWebRequest)WebRequest.Create(requestUrl);
httpWebRequest.ContentType = "application/json; charset-utf8";
var httpWebResponse = httpWebRequest.GetResponse() as HttpWebResponse;
var binReader = new BinaryReader(responseStream);
const int bufferSize = 4096;
byte[] responseBytes;
using (MemoryStream ms = new MemoryStream())
{
   byte[] buffer = new byte[bufferSize];
   int count;
   while ((count = binReader.Read(buffer, 0, buffer.Length)) != 0)
   {
         ms.Write(buffer, 0, count);
   }
   responseBytes = ms.ToArray();
}
Info(Encoding.UTF8.GetString(responseBytes, 0, responseBytes.Length)); //Info logs the string to a file

I'm not modifying the HttpClient.MaxResponseContentBufferSize property, but for good measure I've also tried changing this value, to no avail.

How can I resolve this?

Answer 1

I've found the issue, it was being caused by the backend service to which I was connecting. Thank you again @Panagiotis Kanavos