I'm using mosquitto 1.4.10.
I have a client that publishes multiple messages to the same topic in batches (eg. 50 messages per batch). The client will send each batch every few seconds (eg. every 5 seconds). After 1 or a few batches are sent, on the next batch, suddenly mosquitto will kill the connection to the client.
I've provided a packet capture from the machine the broker is running on that shows the behaviour which I'll describe below.
Client: 172.16.0.114 Mosquitto Broker: 10.16.208.139 (port 8883)
The numbers below are the packet numbers in the capture file.
1: Client connects to broker
17?: Client starts publishing first batch of messages
386: First batch of messages complete
387: Client starts publishing second batch of messages
388: Missing packet?
389: Mosquitto ACKs packet 387
390: Mosquitto starts spamming TCP DUP ACK packets for 387
549: Mosquitto stops spamming TCP DUP ACK packets for 387
550-578: Lots of retransmissions
784: Second batch of messages complete
785: Third batch of messages starts
788: Mosquitto sends FIN ACK, closing the connection (WHY?)
790-880: Mosquitto responds to remaining messages with RST
The main question is why would mosquitto suddenly close the connection after starting to receiving the 3rd batch of messages. The logs in mosquitto at this point where the 3rd batch starts is here:
18:52:24
1491418344: OpenSSL Error: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
18:52:24
1491418344: Socket error on client admin, disconnecting.
This was being caused by a bug in mosquitto that would kill existing SSL connections if another non-SSL connection was attempted. Specifically, the TCP health check performed by the AWS load balancer was sending a small TCP packet which triggered the bug to kill my client's connection.
See issue on the mosquitto github.