Search code examples
algorithmnetworkingtcpnagle

Nagle's Algorithm, ACK Delay and Rlogin echo

I was suggested to ask here too, since specific questions about protocols are on-topic, but in case anyone is interested the question also has a small bounty on ServerFault.

I'm reading about TCP data flow, Delayed ACK and Nagle's Algorithm.

So far I understand that:

  1. The Delayed ACK implementation on TCP creates a delay on the acknowledgement of segments received to give the opportunity for the application to write some data along with the acknowledgement, thus avoiding sending an empty ACK packet and contributing to network congestion.
  2. The Nagle's Algorithm implementation states that you can't send a small TCP segment while another small segment is still not acknowledged. This avoids the traffic being loaded with several tinygrams.

On some interactive applications, like Rlogin for instance, Nagle's Algorithm and Delayed ACKs can "conflict":

Rlogin sends the keyboard input to the server as we type them and some keys (like F1) generates more than one byte (F1 = Escape + left bracket + M). Those bytes can be sent in different segments if they are delivered to TCP one by one.

The server doesn't reply with an echo until it has the whole sequence, so all the ACKs would be delayed (expecting some data from the application). The client on the other hand, would wait for the first byte acknowledgement before sending the next one (respecting the Nagle's Algorithm). This combination ends up resulting in a "laggy" Rlogin.

The tcpdump of the F1 and F2 key being sent on a Rlogin is represented below:

    type Fl key
1   0.0                 slip.1023 > vangogh. login: P 1:2(1) ack 2
2   0.250520 (0.2505)   vangogh.login > slip.1023: P 2:4(2) ack 2
3   0.251709 (0.0012)   slip.1023 > vangogh.login: P 2:4(2) ack 4
4   0.490344 (0.2386)   vangogh.login > slip.1023: P 4:6(2) ack 4
5   0.588694 (0.0984)   slip.1023 > vangogh.login: . ack 6
    type F2 key
6   2.836830 (2.2481)   slip.1023 > vangogh.login: P 4:5(1) ack 6
7   3.132388 (0.2956)   vangogh.login > slip.1023: P 6:8(2) ack 5
8   3.133573 (0.0012)   slip.1023 > vangogh.login: P 5:7(2) ack 8
9   3.370346 (0.2368)   vangogh.login > slip.1023: P 8:10(2) ack 7
10  3.388692 (0.0183)   slip.1023 > vangogh.login: . ack 10

Now the doubt: Even though the page I read states that the server doesn't reply with an echo before it has the whole key sequence, the packets captured through tcpdump shows that the keys are being echoed on their respective ACKs (the first reply is 2 bytes long because the echo from ESC is two characters - caret + left bracket).

If data is being sent from the application to TCP (the echo response) why are the ACKs being delayed? According to what was stated, about the server waiting the full sequence before echoing it, wasn't the ACKs supposed to contain no echo up to the last ACK, that would contain the whole sequence echo?

EDIT: Here is the output of tcpdump for the modified Rlogin, without the Nagle's Algorithm (TCP_NODELAY flag):

        type Fl key
1   0.0     slip.1023 > vangogh.login: P 1:2(1) ack 2
2   0.002163 (0.0022)   slip.1023 > vangogh.login: P 2:3(1) ack 2
3   0.004218 (0.0021)   slip.1023 > vangogh.login: P 3:4(1) ack 2
4   0.280621 (0.2764)   vangogh.login > slip.1023: P 5:6(1) ack 4
5   0.281738 (0.0011)   slip.1023 > vangogh.login: . ack 2
6   2.477561 (2.1958)   vangogh.login > slip.1023: P 2:6(4) ack 4
7   2.478735 (0.0012)   slip.1023 > vangogh.login: . ack 6
        type F2 key
8   3.217023 (0.7383)   slip.1023 > vangogh.login: P 4:5(1) ack 6
9   3.219165 (0.0021)   slip.1023 > vangogh.login: P 5:6(1) ack 6
10  3.221688 (0.0025)   slip.1023 > vangogh.login: P 6:7(1) ack 6
11  3.460626 (0.2389)   vangogh.login > slip.1023: P 6:8(2) ack 5
12  3.489414 (0.0288)   vangogh.login > slip.1023: P 8:10(2) ack 1
13  3.640356 (0.1509)   slip.1023 > vangogh.login: . ack 10

It can be noticed on segment 4, the ~0.2ms delay, even though Nagle's Algorithm is turned off (so all the special key bytes arrive together and can be processed together). On segment 6 we cannot identify the ~0.2ms delay because the server takes a while to reprocess and repacket the retransmission of some lost segments, but this delay is above 2s.

On the F2 key example, on segment 11 we can also notice the ~0.2ms delay. We can't identify it on segment 12 because it was probably sent right after segment 11.

This does indicates that @MattTimmersans answer is right, and it could be that the book makes a misinterpretation of the segments delay. They could be indeed a characteristic of the network medium instead of the ACKs being delayed because of data not being sent down to the TCP stack.

Reference: http://people.na.infn.it/~garufi/didattica/CorsoAcq/Trasp/Lezione9/tcpip_ill/tcp_int.htm

Solution

  • Since the ACKs in segments 2,4,5,7 come with data, they are not delayed ACKs (weren't initiated by the ACK timer elapsing).

    I believe the 0.25s delays associated with those are just the round trip time between hosts. I notice that this data was logged in 1993. If I remember correctly, I think a 250ms ping was not uncommon in those days.

    If the tcpdump were run on vangogh instead of slip, it looks like it would see the echo coming quickly after the ESC and [M packets.

    The example shows that Nagle's algorithm can suck even without ACK delay, because it adds an extra round-trip time to the exchange.