Xamarin iOS WebException: App crashes after HttpWebRequest is complete

Question

The problem

So we are creating a Xamarin iOS app that will connect to a REST API running on ASP .NET. To perform the HttpRequests it uses a library we share between the iOS app and an android app.

On the first look everything works fine. The iOS app calls an asynchronous Method from our library, receives data from the server and correctly displays it in a table view. However for some reason it doesn't terminate the connection once it's done resulting in a WebException some time later (seemingly random from immediately after the request is performed up until minutes later) saying Error getting response stream (ReadDoneAsync2): ReceiveFailure. Interestingly enough when we call the exact same method in our library from a console application everything works just fine.

Library Code

This is the relevant code from our library (We don't know where the error occurs, as the main UI Thread crashes. Debugging also doesn't reveal anything suspicious. I'll therefore include all code below that will we executed on the client during the API call):

ClubProvider:

public class ClubProvider : BaseProvider
{
    /// <summary>
    /// Creates a new <see cref="ClubProvider"/> using the specified <paramref name="uri"/>.
    /// </summary>
    internal ClubProvider(string uri)
    {
        if (!uri.EndsWith("/"))
        {
            uri += "/";
        }
        Uri = uri + "clubs";
    }

    public async Task<ClubListResponse> GetClubListAsync()
    {
        return await ReceiveServiceResponseAsync<ClubListResponse>(Uri);
    }
}

BaseProvider (where the actual HttpWebRequest is performed)

public abstract class BaseProvider
{
    public virtual string Uri { get; private protected set; }

    /// <summary>
    /// Reads the response stream of the <paramref name="webResponse"/> as a UTF-8 string.
    /// </summary>
    private async Task<string> ReadResponseStreamAsync(HttpWebResponse webResponse)
    {
        const int bufferSize = 4096;
        using Stream responseStream = webResponse.GetResponseStream();
        StringBuilder builder = new StringBuilder();
        byte[] buffer = new byte[bufferSize];
        int bytesRead;
        while ((bytesRead = await responseStream.ReadAsync(buffer, 0, bufferSize)) != 0)
        {
            builder.Append(Encoding.UTF8.GetString(buffer, 0, bytesRead));
        }
        return builder.ToString();
    }

    /// <summary>
    /// Performs the <paramref name="webRequest"/> with the provided <paramref name="payload"/> and returns the resulting <see cref="HttpWebResponse"/> object.
    /// </summary>
    private async Task<HttpWebResponse> GetResponseAsync<T>(HttpWebRequest webRequest, T payload)
    {
        webRequest.Host = "ClubmappClient";
        if (payload != null)
        {
            webRequest.ContentType = "application/json";
            string jsonPayload = JsonConvert.SerializeObject(payload);
            ReadOnlyMemory<byte> payloadBuffer = Encoding.UTF8.GetBytes(jsonPayload);
            webRequest.ContentLength = payloadBuffer.Length;
            using Stream requestStream = webRequest.GetRequestStream();
            await requestStream.WriteAsync(payloadBuffer);
        }
        else
        {
            webRequest.ContentLength = 0;
        }
        HttpWebResponse webResponse;
        try
        {
            webResponse = (HttpWebResponse)await webRequest.GetResponseAsync();
        }
        catch (WebException e)
        {
            /* Error handling
            ....
            */
        }
        return webResponse;
    }

    /// <summary>
    /// Performs the <paramref name="webRequest"/> with the provided <paramref name="payload"/> and returns the body of the resulting <see cref="HttpWebResponse"/>.
    /// </summary>
    private async Task<string> GetJsonResponseAsync<T>(HttpWebRequest webRequest, T payload)
    {
        using HttpWebResponse webResponse = await GetResponseAsync(webRequest, payload);
        return await ReadResponseStreamAsync(webResponse);
    }

    /// <summary>
    /// Performs an <see cref="HttpWebRequest"/> with the provided <paramref name="payload"/> to the specified endpoint at the <paramref name="uri"/>. The resulting <see cref="IResponse"/> will be deserialized and returned as <typeparamref name="TResult"/>.
    /// </summary>
    private protected async Task<TResult> ReceiveServiceResponseAsync<TResult, T>(string uri, T payload) where TResult : IResponse
    {
        HttpWebRequest webRequest = WebRequest.CreateHttp(uri);
        webRequest.Method = "POST";
        string json = await GetJsonResponseAsync(webRequest, payload);
        TResult result = JsonConvert.DeserializeObject<TResult>(json);
        return result;
    }

    /// <summary>
    /// Performs an <see cref="HttpWebRequest"/> to the specified endpoint at the <paramref name="uri"/>. The resulting <see cref="IResponse"/> will be deserialized and returned as <typeparamref name="TResult"/>.
    /// </summary>
    private protected async Task<TResult> ReceiveServiceResponseAsync<TResult>(string uri) where TResult : IResponse
    {
        return await ReceiveServiceResponseAsync<TResult, object>(uri, null);
    }
}

Call from iOS client

public async override void ViewDidLoad()
{
    // ...
    ServiceProviderFactory serviceProviderFactory = new ServiceProviderFactory("https://10.0.0.40:5004/api");
    ClubProvider clubProvider = serviceProviderFactory.GetClubProvider();
    ClubListResponse clubListResponse = await clubProvider.GetClubListAsync();
    var clublist = new List<ClubProfileListData>();
    foreach (var entry in clubListResponse.ClubListEntries)
    {
        clublist.Add(new ClubProfileListData(entry.Name, entry.City + "," + entry.Country, entry.MusicGenres.Aggregate(new StringBuilder(), (builder, current) => builder.Append(current).Append(",")).ToString()));
    }
    // ...
}

At first this seems to work just fine but it crashes later on with this error message:

Taking a look at the performed request with Wireshark reveals that the connection is never terminated:

The connection stays open until the app crashes and we close the simulator. Interestingly enough the app doesn't always crash immediately. We load the received data into a TableView and every once in a while the app doesn't crash after loading the data. It only crashes when we start scrolling through the results. This doesn't make sense to us though as all network streams should be closed by now right? (After all we are using using statements for all ResponseStreams. Therefore all streams should automatically be disposed when returning from the awaited Task :C ) As if it would be trying to stream the data as needed.

Testing the library code using a Console Application

Now the obvious reason for this could be that we forgot to close some stream in our library however the following code succeeds with no error whatsoever:

class Program
{
    static void Main(string[] args)
    {
        new Thread(Test).Start();
        while (true)
        {
            Thread.Sleep(1000);
        }
    }

    public static async void Test()
    {
        ServiceProviderFactory serviceProviderFactory = new ServiceProviderFactory("https://10.0.0.40:5004/api");
        ClubProvider clubProvider = serviceProviderFactory.GetClubProvider();
        ClubListResponse clubListResponse = await clubProvider.GetClubListAsync();
        foreach (ClubListResponseEntry entry in clubListResponse.ClubListEntries)
        {
            Console.WriteLine(entry.Name);
        }
        Console.WriteLine("WebRequest complete!");
        Console.ReadLine();
    }
}

Taking a look at the captured packets we see that the connection is closed as expected once the request is completed:

The question

So why is this? Why does our library code work as intended in our .NET Core Console Application but fails to disconnect when called by the iOS app? We have the suspicion that this could be due to the async/await calls (as described here). However we do get an exception so we aren't sure if this is really the same bug described in the question linked above. Now before we rewrite all our library code we'd like to eliminate all other possible causes for this weird behavior. Also we successfully use async calls to load some images without crashing the application so we're really just guessing at this point :C

Any help would be greatly appreciated.

Answer 1

Alright further testing revealed that the app crashing actually didn't have anything to do with our API calls whatsoever. The problem actually was us loading the images to be shown next to our datasets asynchronously on demand. Turns out that during testing of the image loading we only had like 20 entries in our ListView. This was fine because iOS could load all images at once. However when loading 1500+ datasets from our API the ListView started buffering, only loading images as needed and that's when the application crashed. Probably because the original image stream wasn't available anymore or something like that.

Also as an interesting side note: iOS does actually close the network connection to the server but only after a 100 second timeout of no packets sent while the Windows .NET Core Console Application closes it immediately. And we never waited this long. Oh well :D