I'm looking into a small network system prototype, that at it's lowest level, has one software "parent" process that communicates back and forth with 5 software "children" processes.
I am using ZeroMQ to communicate between processes.
My question is a question of multi-threaded handling vs singled-threaded handling.
In this kind of system would a single thread in the parent that handles sending, receiving, and processing messages to and from the children be more efficient then 5 threads (1 thread per process)?
For singled-threaded, I'm concerned that while the parent works to process one message, the messages will start piling up.
For multi-threaded, I'm concerned of context switching and performance hits if this system architecture is expanded. Think 50 parents at 5 threads a piece, so 250 threads minimum.
The threads are written to ZeroMQ standards without locks, critical sections, shared memory, etc.
I use Linux and C++.
You can run a message queue on the parent, which should also allow it to not get overwhelmed by the children while processing the events in the order they arrive. Furthermore, you can expand this to a simple send-acknowledge model, where children will wait for their message to be acknowledged, before sending further messages, so you can allow the parent to control the rate at which it receives messages.
Regarding the number of threads to run, I agree with you, as you scale up the complexity of the parent program will increase.
The main factor, I believe, will be whether the threads need to share any data, or communicate, with each other. If that is not the case, the problem is simplified as you can use 1 thread per 1 or more children and just grab messages from the queue.