I have a class object which is acting as a server. It receives request from anywhere and pushes the request in its request queue (Producer). Now there is a consumer thread running which is popping the request from the request queue and based on the request calling appropriate class method to furnish the request. Now the consumption of the request from the queue and launching of appropriate function is being executed in a synchronous manner. What I want is the consumer thread pops up a request from the queue and launches the appropriate function in asynchronous manner so that the consumer can pop next request from the queue immediately.
One solution I have tried with this is the consumer pops up a request from the queue and create a boost::thread and start appropriate function in a new thread. I have saved thread pointers in std::vector as well as also tried boost::thread_group. So far so good. But there is a problem in this solution.
Once I have furnished more than 150 requests, there are more 150 threads and after that pthread does not create new thread giving error "pthread_create: Resource temporarily unavailable", which I believe means the stack of the current process has ran out so new threads cannot be created.
Question #1 My request handlers does not contain while (1), and those are just doing some work and exiting and are not waiting for anything at all, thats why I am expecting my initial threads has completed their processing and exited from the thread handler function. Considering this if the thread has completed its processing and exited shouldn't it got cleaned up its stuff from the stack?
One solution to this problem is I can set the stack size of the thread, but that will still raise this error after say 1000 threads.
So my requirement is I must clean up the completed threads after some time (i.e. say when the thread pointers vector has exceeded 100 or after every 1 minute or something like that).
Question #2 Besides launching the new thread as I have mentioned above what is the other asynchronous function call mechanism I should try. Is boost::function + boost::bind asynchronous? Is this a good solution to the situation I have mentioned? Say my system is supposed to be online 24/7/365 and receiving say >1000 request each day.
Update #1 So I found one problem in my design. I have mentioned in my question #1 that my request handler contains just plain calls which I found is not true. It is downloading a file from a server synchronously, which is essentially a blocking operation. I should download the file asynchronously.
There is no use of making threads which your underlying system can not handle concurrently if the request handler is not doing any blocking operation.
So as Alex has mentioned having more than one consumer threads (I think 5 are enough) to pop a request from the queue and have a asynchronous file download will solve my issue.
One solution is to have multiple consumer threads, which each pop a work item off the queue and deal with it synchronously. It enables you to manage concurrency (avoid over subscription), whilst still processing multiple items at a time. You also remove the overhead of launching a new thread on every item, which I'd predict is one of your bottlenecks.
You should ensure your queue is designed for multiple consumers.
Never used this implementation, but a threadpool might help.