Search code examples
c++multithreadingboostboost-asioblocking

Design for Boost ASIO , Worker threads SQl queries for 'practical' web server

I am looking for solution to develop an efficient web server framework where:

  1. One or few IO threads handle client HTTP connections and TCP IO.
  2. Multiple threads do business processing (SQL queries, file IO, etc.)

All blogs solution I have seen are solving 10000 connections with worker threads doing almost zero business logic (i.e just writing data using async_write). Is the Boost.Asio's HTTP Server 3 a solution for my problem? If so, how many threads should I use per core?

I am also interested in knowing how HTTP Server 3 compares to the 1 acceptor thread + thread pool model used by mongoose.

Solution

  • Sorry, I don't have enough reputation to use the comments, so I have to post a new answer to give any input.

    O.P. is asking a specific question about http://www.boost.org/doc/libs/1_53_0/doc/html/boost_asio/example/http/server3/. This is an example http server that starts a thread pool to handle the http requests.

    O.P.'s original question might be restated: "given a server machine with a set of resources A and a workload that consumes B resources per request, how many threads should I allocate in the thread pool?"

    There's a thread here: Reasonable number of threads for thread pool running web service requests with a similar discussion (with respect to Java thread pools) but that discussion doesn't seem to come to any conclusive answer.

    Here's an example of a short tutorial on capacity planning in the "old fashioned 1970's mainframe style" that I learned in school: http://www.cs.umb.edu/~eb/goalmode/primer.pdf.

    In this case you might create a simple model like:

    You have an average rate of requests arriving, X. For each request you are consuming a certain average amount of cpu (in units of time) S_c, and an average amount of time spent waiting for disk requests to complete, S_d. So each thread is taking an average time S_c + S_d before it is returned to the thread pool. (You would need to measure this.) Thus on average you would expect that you would need at least N = X * (S_c+S_d) threads to avoid incoming threads queueing at an empty thread pool. You might actually want to allocate some small multiple of N (e.g. 3N) threads to be able to deal with bursts of one kind or another.

    But the number of threads in the pool isn't really the interesting limit. The interesting limit is either the total amount of CPU or the total amount of disk bandwidth you have available. Suppose each request requires 3 seconds of CPU processing, and that you have a system with 12 cores. Thus, in any 3 second period you should expect to handle 12 simultaneous requests. Thus an average arrival rate greater than 12/3 = 4 requests per second would saturate your CPU. (Similar calculation for your disk bandwidth.)

    So really what you are going to end up figuring out is: given my expected arrival rate of requests, X, and the amount of CPU and disk consumed by each request, how much cpu and disk should I purchase?