This is a question about message passing. This relates specifically to an in-house application written in C#. But it has a home grown "message passing" system resembling erlang.
Okay, we hope that it will be possible to learn from erlang folks or documentation to find an elegant solution to a couple of message passing challenges. But alas, after reading erlang documentation online and forums these topics don't seem to be addressed--that we can find.
So the question is: In erlang, when does the queue to send messages to a process get full? And does erlang handle the queue full situation? Or are the queues for message passing in erlang boundless--that is only limited by system memory?
Well in our system it involves processing a stream of financial data with potentially billions of tuples of information being read from disk, each tuple of financial information is called a "tick" in the financial world.
So it was necessary to set a limit to the queue size of each "process" in our system. We arbitrarily selected 1000 items max in each queue. Now those queues quickly get filled entirely by the tick messages.
The problem is that the processes also need to send other types of messages to each other besides just ticks but the ticks fill up the queues preventing any other types of message from getting passed.
As a "band aid" solution (which is messy) allow multiple queues per process for each message type. So a process will have a tick queue, and a command queue, and fill queue, and so on.
But erlang seems so much cleaner by having a single queue to each "process" that carries different message types. But again, how does it deal with the the queue getting hogged by a flood of only one of the message types?
So perhaps this is a question about the internals of erlang. Does erlang internally have separate limits on message types in a queue? Or does it internally have a separate queue per type of message?
In any case, how are sending processes aware when a queue is too full to receive certain types of message? Does the send fail? Does that mean error handling becomes necessary in erlang for inability to send?
In our system, it tracks when queues get full and then prevents any processes from running which will attempt to add to a full queue until that queue has more space. This avoids messy error handling logic since processes, once invoked, are guaranteed to have room to send one message.
But again, if we put multiple types of messages on that same queue. Other message types will be blocked that must get through.
It has become my perhaps mistaken impression that erlang wasn't designed to handle this situation so perhaps it doesn't address the problem of a queues getting filled with a flood of a single message type.
But we hope someone know how to answer this point to good reference information or book that covers this specific scenario.
Erlang sends all messages to a single queue with the system memory being the upper limit on the queue size. If you want to prioritize messages you have to scan the entire queue for the high priority messages before fetching a low priority one.
There are ways to get around this by spawning handler processes which throttle and prioritize traffic, but the erlang VM as such has no support for it.