On a GNU/Linux machine, if one wants to do "real-time"(sub millisecond time critical) tasks, you almost invariably have to go through the lengthy, complex, and problem prone process of patching the kernel to expose adequate support[1] [2].
The biggest problem is, many systems where real-time tasking is most useful do not have the fundamental hardware requirements to even allow these patches to work, namely a high resolution timer peripheral. Or if they do, it is specific to the hardware, so as such needs to be specifically implemented in the patch on a case by case basis. This is true even if the CPU/instruction clock rate is more than fast enough to give the required time granularity and then some.
So, my question is, what are some of the best second place ways/tricks to get as close as possible to the above real-time goal? Things that one can simply do in the applications source code, without intimate knowledge of underlying hardware or too much "kernel hacking".
Elevating process priority, starting an extra thread for "critical" tasks, and (in C) using variants of nanosleep() are the best looking answers/tricks I have come up with so far. I hope to find more.
The sched_setscheduler(2) and friends allow you to use two different soft real-time schedulers, SCHED_FIFO and SCHED_RR. Processes running under these schedulers are prioritised higher than regular processes. So as long as you only have a few of these processes, and control the priorities between them, you can actually get pretty decent real-time responses.
The the difference between SCHED_FIFO and SCHED_RR is:
With the "real-time" schedulers, there are up to 100 different priorities (POSIX only requires 32 distinct levels, so one should use sched_get_priority_min(2) and sched_get_priority_max(2) to get the actual number. The schedulers both work by preempting processes and threads with lower priority; the difference is in how they handle tasks with the same priority.
SCHED_FIFO is a first in first out scheduler (hence the name). This means that the task that hits the run queue first is allowed to run until it is done, voluntarily gives up its space on the run queue, or is preempted by a higher priority task.
SCHED_RR is a round robin scheduler. This means that tasks with the same priority are only allowed to run for a certain time quantum. If the task is still running when this time quantum runs out the task is preempted, and the next task in the run queue (with same priority) is allowed to run for up to its time quantum. As with SCHED_FIFO, higher priority tasks preempt lower priority ones, how ever, when a task which was preempted by a higher priority task is allowed to run again, then it's only allowed to run for the time left in its quantum. See the Notes section in sched_rr_get_interval(2) for how to set the time quantum for a task.