Confused with using SendHighWatermark in NetMQ

Question

I'm making tests for my prototype with NetMQ, in particular I'm learning the influence of HighWatermark options.

I have understood and tested the following case:

• server with ROUTER socket isn't started yet
• meanwhile client with DEALER socket is sending 20 messages (SendHighWatermark is 10)

In this case I see that client can send only 10 messages and then it's blocking on send.

Then I made another test with slow processing - ROUTER sleeps after receiving each message. In this case I expected DEALER to send 10 messages quickly and another 10 messages with delay. But DEALER sends all messages with no delays.

ROUTER's code:

const string processingEndpoint = "tcp://127.0.0.1:6668";
using (var context = NetMQContext.Create())
using (var router = context.CreateRouterSocket())
{
    router.Bind(processingEndpoint);
    var msg = router.ReceiveMultipartMessage();
    Thread.Sleep(5000);  // emulate slow processing
}

DEALER's code:

const string processingEndpoint = "tcp://127.0.0.1:6668";
string clientIdentity = "fast dealer";
using (var context = NetMQContext.Create())
using (var dealer = context.CreateDealerSocket())
{
    client.Options.Identity = System.Text.Encoding.Unicode.GetBytes(clientIdentity);
    client.Options.SendHighWatermark = 10;
    client.Connect(processingEndpoint);

    for (var i = 0; i < 20; i++)
    {
        var msg = new NetMQMessage();
        msg.Append(string.Format("{0}_Payload{1}", clientIdentity, i));
        client.SendMultipartMessage(msg);
        Console.WriteLine("Sent msg {0}", i);
    }
}

Why in my case the dealer can send without delays when processing is slow?

Answer 1

Few reasons, in the second test the socket is already connected, so you the send high watermark of the dealer and receive high watermark of the router. If you set it to 10 as well you total of 20. However you also have socket buffering which by default are 8mb