Read performance counters periodically in linux

Is there a way to read performance counters periodically in linux?

Something like perf stat with the ability to sample every X cycles is what I'm looking for.

Basically I would like to be able to read the instruction counter (number of instructions executed) every X amount of cpu cycles for some program.

Solution

It seems that the perf tool in Linux works by recording an event when the counters reach a specific value, rather than sampling at regular intervals.

Command perf record -e cycles,instructions -c 10000 stores an event every 10000 cycles and every 10000 instructions. It can be run against a new command or an existing pid. It records to perf.data in current directory.

Analyzing the data is another matter. Using perf script gets you quite close:

ls 16040 2152149.005813: cycles:          c113a068  ([kernel.kallsyms])
ls 16040 2152149.005820: cycles:          c1576af0  ([kernel.kallsyms])
ls 16040 2152149.005827: cycles:          c10ed6aa  ([kernel.kallsyms])
ls 16040 2152149.005831: instructions:          c1104b30  ([kernel.kallsyms])
ls 16040 2152149.005835: cycles:          c11777c1  ([kernel.kallsyms])
ls 16040 2152149.005842: cycles:          c10702a8  ([kernel.kallsyms])
...

You need to write a script that takes a bunch of lines from that output and counts the number of 'cycles' and 'instructions' events in that set. You can adjust the resolution by changing the parameter -c 10000 in the recording command.

I verified the analysis by running perf stat and perf record against ls /. Stat reported 2 634 205 cycles, 1 725 255 instructions, while script output had 410 cycles events and 189 instructions events. The smaller the -c value, the more overhead there seems to be in the cycles reading.

There is also a -F option to perf record, which samples at regular intervals. However, I could not find a way to retrieve the counter values when using this option.

Edit: perf stat apparently works on pids also, and captures data until ctrl-c is pressed. It should be quite easy to modify the source so that it always captures for N seconds and then run it in a loop.