C# - TcpClient first read after ConnectionAborted socket error is very slow

Question

I'm currently writing code that interacts with a device with TCP/IP. After 15 seconds of inactivity the device is written to send a FIN packet to disconnect. I'm using Polly for retries where I make sure to dispose of the TcpClient object, instantiate a new instance, and then reconnect to the IP/port before retrying the IO operation again.

In this case I'm trying to read a response from the connected socket. The retry policy works and I get the response but I have noticed that when it does have to retry (due to a ConnectionAborted socket exception) it takes approximately 15 seconds before reading bytes into the buffer completes. I don't do a lot of network programming so I'm relatively ignorant in this area and I have no idea why it takes so long.

// Retry policy and the TcpClient are private member variables in this case
_readRetryPolicy = Policy.Handle<IOException>()
    .RetryAsync(3, onRetryAsync: async (_, _) =>
    {
        Disconnect();
        await ConnectAsync(_ip!, _port).ConfigureAwait(false);
    });

public async Task<byte[]> ReadResponseAsync()
{
    if (_tcpClient is not TcpClient { Connected: true })
    {
        Disconnect();
        await ConnectAsync(_ip!, _port).ConfigureAwait(false);

        // Possible StackOverflowException but I'm just hacking at this problem right now
        return await ReadResponseAsync().ConfigureAwait(false);
    }

    var buffer = new byte[32];
    var responseBytesStream = Enumerable.Empty<byte>();
    int numBytesRead = 0;
    do
    {
        numBytesRead = await _readRetryPolicy.ExecuteAsync(() => _tcpClient.GetStream().ReadAsync(buffer).AsTask()).ConfigureAwait(false);
        responseBytesStream = responseBytesStream.Concat(buffer);
    }
    while (_tcpClient.GetStream().DataAvailable);

    return responseBytesStream.ToArray();
}

public void Disconnect()
{
    if (_tcpClient is TcpClient c) c.Dispose();
    _tcpClient = null;
}

Answer 1

Each socket connection is an independent stream of communication (technically, two streams, one going each way). So if you have an application that sent a command and then the connection was dropped, the application can reconnect but it probably needs to re-send that command. Retrying just the read is insufficient.

For reliability reasons, your application should always retry connections (re-sending commands as necessary until they are ACKed). But it's also not a bad idea to add in a heartbeat if your protocol supports it. This is an actual message (data) that is sent, not the TCP-level keepalive packets. This kind of message may be called keepalive, heartbeat, or noop. If you send that message every 8-10 seconds, you should avoid the disconnection most of the time.

In general, I discourage writing TCP/IP applications because there's a lot of pitfalls. It's a lot harder than it first appears. But if you don't have a choice in your protocol, then I do recommend watching my video tutorial on developing a reliable C# application communicating over TCP/IP.