Search code examples
zeromqamqp

Buffering messages for dead subscriber with zeromq


I using a pub-sub pattern with tcp. When one of my subscriber dies (kill -9 for example) et been restarted with the same IDENTITY it does not get the previous messages.

What are the solutions so when it restart it gets the messages sent? (I understand 0mq does not handle that)

  1. run publisher
  2. run sub0 (subscribe to socket)
  3. run sub1 (subscribe to socket)
  4. pkill -9 sub0 (simulate daemon dying)
  5. publisher send message
  6. run sub0 again (same ZMQ_IDENTITY)

sub0 does not receive the lost message.


Solution

  • This is entirely the responsibility of your application. Take a look at The Guide... particularly Chapter 5 on advanced pub/sub patterns, and even more specifically Getting an out of band snapshot.

    The upshot is that your publishing server actually has two sockets, one for publishing, and one for other system-level communication. Anytime it publishes a new messages, it also adds that message to a local cache... it never forgets the messages it sends. Anytime your subscribing client re-connects to the server, it's 2nd socket sends a request to the server to get all messages it missed (or, as in the case of the linked example, the entire current state of the data), which are sent back over that 2nd socket pair. In this way the subscriber is up to date with all messages when it starts to get new ones over the normal subscriber channel.