I'm trying to send image numpy arrays using the ZMQ pub/sub model. I followed the documentation here:
However, I realized that there would be a buildup of frames in my subscriber as the subscriber wouldn't be able to process the frames at the rate at which the publisher is sending due to the operations done on the subscriber side for the frames.
Since the documentation uses a multipart message to send the image array, I am unable to use the CONFLATE option on set_sockopt as that makes it only keep a part of the multipart.
My question: Is there a way for the subscriber to keep only the latest multipart message?
In the end I sort of solved this by repeatedly subscribing and unsubscribing from the ZMQ socket.
# This is run every time the subscriber receive function is called
socket.setsockopt(zmq.SUBSCRIBE, '')
md = socket.recv_json()
msg = socket.recv()
socket.setsockopt(zmq.UNSUBSCRIBE, '')
Essentially, I made it so that my subscriber socket doesn't care about the other messages that come in other than the one message that it has received until the next time it tries to grab a message.
I don't believe that this is the best solution for this problem, as there are costs involved when repeatedly subscribing and unsubscribing. Hoping that there might be a better way to do this but so far I haven't been able to find it.