Search code examples
c++cmultithreadingwinapicontext-switch

Reduce Context Switches Between Threads With Same Priority

I am writing an application that use a third-party library to perform heavy computations.

This library implements parallelism internally and spawn given number threads. I want to run several (dynamic count) instances of this library and therefore end up with quite heavily oversubscribing the cpu.

Is there any way I can increase the "time quantum" of all the threads in a process so that e.g. all the threads with normal priority rarely context switch (yield) unless they are explicitly yielded through e.g. semaphores?

That way I could possibly avoid most of the performance overhead of oversubscribing the cpu. Note that in this case I don't care if a thread is starved for a few seconds.

EDIT:

One complicated way of doing this is to perform thread scheduling manually.

  1. Enumerate all the threads with a specific priority (e.g. normal).
  2. Suspend all of them.
  3. Create a loop which resumes/suspends the threads every e.g. 40 ms and makes sure no mor threads than the current cpu count is run.

Any major drawbacks with this approach? Not sure what the overhead of resume/suspending a thread is?

Solution

  • There is nothing special you need to do. Any decent scheduler will not allow unforced context switches to consume a significant fraction of CPU resources. Any operating system that doesn't have a decent scheduler should not be used.

    The performance overhead of oversubscribing the CPU is not the cost of unforced context switches. Why? Because the scheduler can simply avoid those. The scheduler only performs an unforced context switch when that has a benefit. The performance costs are:

    1. It can take longer to finish a job because more work will be done on other jobs between when the job is started and when the job finishes.

    2. Additional threads consume memory for their stacks and related other tracking information.

    3. More threads generally means more contention (for example, when memory is allocated) which can mean more forced context switches where a thread has to be switched out because it can't make forward progress.

    You only want to try to change the scheduler's behavior when you know something significant that the scheduler doesn't know. There is nothing like that going on here. So the default behavior is what you want.