I am writing a simple program that sends messages back and forth between a single computer running many clients and multiple servers using Twisted. Replies to these messages require fairly long calculations, which is why I implemented deferToThread() and Callback() on both the client and the server as to not block messages from coming in.
However, I would like to maximize the number of simultaneous active connections that can be handled concurrently. I think, as it stands now, my program can handle 10 simultaneous active connections (as in 10 threads performing a calculation for a reply on either the client or the server). Does it make sense that this is the default number of active connections and can I increase it using the suggestThreadPoolSize() option for the reactor as described here?
If your program is CPU-bound, then the Python global interpreter lock will prevent you from having any multi-CPU concurrency, so fiddling with the thread pool size will not have much practical effect on the speed with which your program can deal with new connections.
Instead, you should use reactor.spawnProcess to start a subprocess, and shuttle your CPU-intensive work off to a pool of subprocesses. Something such as Ampoule might help you do this more conveniently.
However, even multiple processes will only buy you as much CPU concurrency as you have ... CPUs. (Well, cores, or hyperthreads, really; but, "logical" CPUs, at least.) Accepting more work when you do not have the available resources to actually do that work is going to build up huge buffers in memory and eventually crash your program. Eventually you need to apply backpressure.
Twisted does provide all the tools you need to do this right now, although we are working on making it even easier.
Oh - and this should go without saying, but if your program isn't CPU-bound, then you should just stop using threads ;-).