Is there a low-level system API for Linux and for Darwin, to get a "snapshot" of the scheduler, to see which threads were at a given time running (active), preempted, waiting on a resource, or similar?
For example the Ubuntu "System Monitor" has an "Waiting channel" column for processes, is there a way to get a similar output for all threads inside a process?
I have a multi-threaded queuing system where jobs are dispatched to the threads, with a visual timeline showing when the jobs are started/finished. In order to optimize it I want to add some visual indication of when the jobs are actually running (on one processor core for example), vs when they are preempted.
top -H shows threads and their states. Following the -H switch in its source will be one way to find out. From a quick look, the function int cmd_threads(globalstate *gstate) in commands.c seems to toggle the setting.
UPDATE: Procfs provides that info to user space. /proc/<pid>/task/<thread-id>/status gives exactly what you need. That's what top uses too. Here is sample output from my system:
$ cat /proc/418/task/448/status | grep State
State: S (sleeping)
There's plenty more useful detail under that directory.