I'm debugging a complex multi-threaded program that may still have race condition bugs. I'm currently debugging it by running the same unit tests on the non-deterministic code repeatedly until it fails.
I know there is no simple solution, but is there a heuristic tool which makes the system's scheduler (Linux) change the preemption points at each run, increasing the likelihood that bugs are found?
I knew of a tool called Jinx Debugger that did this, but it is no longer available.
The usual solution is to carefully analyze the code, and "prove" locking correctness. This is a bit of an art.
For experimental approaches, you could instrument the code with function calls that do random sleeps, and run for long time. Your function could only sleep some small percentage of the time, for a random couple of milliseconds.