The GDB manual states that when using all-stop mode for debugging a multithreaded application, it is not possible to advance every thread in lock-step by exactly one statement. This makes sense since a step in GDB essentially allows all threads to be scheduled by the OS (however the OS decides to do this) until the next statement is reached by the thread for which the step was called.
My question is this: Is it reasonable to assume that the average scheduling behavior of the OS in between GDB steps is comparable to the average scheduling behavior of the OS when not stepping (while still using GDB to keep as many variables constant as possible), or does the stepping muck with the scheduling enough that the advancement of threads is not (on average) the same as without stepping?
If the stepping does affect the behavior, how can I get an accurate representation of multithreaded program flow and program state at discrete points in my program? Will recording and playing back be viable?
Is it reasonable to assume that the average scheduling behavior of the OS in between GDB steps is comparable to the average scheduling behavior of the OS when not stepping
Not really. The "not stepping" average behavior will have threads either running out of their time quanta, or blocking on system calls. In the "stepping" case, the threads are unlikely to every run out of their time quanta (because the time distance between steps is likely to be very short). So the average behavior is likely to be very different.
how can I get an accurate representation of multithreaded program flow and program state at discrete points in my program?
In general, you shouldn't care about multithreaded program flow. It is impossible to debug multithreaded programs that way.
When doing multithreaded programming, you must care about preserving invariants (every resource that can be accessed by multiple threads is protected against data races, etc.). If you do, your program will just work (TM). If you don't, you are unlikely to find all ways that the program misbehaves anyway.