How to read PMC(Performance Monitoring Counter) of x86 intel processor

Question

My desktop is Intel x86_64 processor with Ubuntu operating system.

I know there is perf tool to get a list of statistics of a program. But what I am trying to do is read performance counter directly without using the perf tool.

1. First Question

First Questions is I downloaded this code from Github: Github Code Reference.

It compiled successfully with linux-headers-5.3.0-40-generic kernel without any errors. Once I use "insmod" the .ko file, the system hangs. The .ko file is not inserted when I checked the dmesg, so I have to cease it after I do "insmod" the .ko file. Does it happen because I attempted unauthorized access? If there are suggestions that I can try, I am glad to hear that.

The corresponding code is below.

static void set_pce(void *arg)
{
    int to_val = (arg != 0);
    u_int64_t cr4_val;

    cr4_val = __read_cr4();
    if (to_val) {
        cr4_val |= X86_CR4_PCE;
    } else {
        cr4_val &= ~X86_CR4_PCE;
    }

    __write_cr4(cr4_val);
}

static int __init user_rdpmc_init(void){
    int cpu;

    num_cpus = num_online_cpus();

    printk(KERN_INFO "Enabling RDPMC from ring 3 for %d CPUs\n", num_cpus);
    for (cpu = 0; cpu < num_cpus; cpu++) {
    smp_call_function_single(cpu, set_pce, (void *) 1, 1);
    }

   return 0;
}

2. Second Question

Second question is I am using linux-headers-5.3.0-40-generic kernel version in my Ubuntu desktop. I downloaded kernel code version 5.5.3 from kernel.org. I followed the perf code given in the 5.5.3 kernel code thoroughly and discovered that core.c file under linux-5.5.3/arch/x86/events/intel directory actually does setting and reading the performance counters. I used the core.c file contents to make it as a module to read the performance counter. When I compile it, it creates a bunch of errors because I use linux-headers-5.3.0-40-generic to build the module but my ubuntu kernel doesn't have all header files linked to the core.c file from kernel code from kernel.org.

How can I make my Ubuntu kernel use all the files linked to core.c from kernel.org and build the .ko file?

Or Is there any module source code that has x86 performance counter reading that I can use as a reference?

Thank you for your help in advance.

Answer 1

I know there is perf tool to get a list of statistics of a program. But what I am trying to do is read performance counter directly without using the perf tool.

If you do not want to use perf tool, you can try to use oprofile tool or intel vtune or https://github.com/RRZE-HPC/likwid or https://github.com/opcm/pcm. Or you can use perf_event_open syscall which is how perf tool works (you can study or modify perf tool sources from https://mirrors.edge.kernel.org/pub/linux/kernel/tools/perf/ - and perf tool version may not be equal to kernel version).

If you want to access msr registers as root, use modprobe msr (this is standard kernel module, already compiled for your kernel in ubuntu) and wrmsr and rdmsr tools (msr-tools deb/ubuntu package, by intel), like in slide 27 of Performance Monitoring Chris Dahnken Intel SSG EMEA HPCTC presentation.

I don't understand why do you want to work with performance counters without perf tool. If you want to get counter readings from inside of your program, for example before and after some loops, you can use perf_event_open syscall (with specific ioctls) directly. (Or try to use perf stat + same ioctls PERF_EVENT_IOC_* or try to learn perf + JIT integration)

Or you can use existing kernel module which will export msr register access to root user - the msr.ko. And msr tools - https://01.org/msr-tools. Or with this msr+pmc example https://technicalandstuff.wordpress.com/2015/05/15/using-intels-pcm-in-linux-and-inside-c/ + https://software.intel.com/en-us/articles/intel-performance-counter-monitor (https://github.com/opcm/pcm)

There are also some examples of perf counters usage in https://github.com/RRZE-HPC/likwid.

You can also use PAPI library to access counters from your code, it will handle most of perf_event_open stuff for you. http://icl.cs.utk.edu/projects/papi/wiki/PAPITopics:Getting_Started

First Questions is I downloaded this code https://github.com/softdevteam/user_rdpmc ... "insmod" the .ko file, the system hangs.

There are too low "Stars" rating and the code is too old (2016) to really doing any investigations on the hang. Direct access of PMC may interfere with NMI watchdog (do echo 0 > /proc/sys/kernel/nmi_watchdog as root) or other perf session. It is safer to use perf_event_open syscall.

Second question ... discovered that core.c file under linux-5.5.3/arch/x86/events/intel directory actually does setting and reading the performance counters

This file is part of perf_event_open syscall implementation (perf_events subsystem of the kernel, https://github.com/torvalds/linux/tree/master/kernel/events + https://github.com/torvalds/linux/tree/master/arch/x86/events).

To use this code you can use the perf tool or perf_event_open syscall.

You should not compile the perf_events subsystem of the kernel as separate module because it is already compiled into your kernel (intel/amd specific part can be partially ko) and the Subsystem itself does not support compilation as module:

https://github.com/torvalds/linux/tree/master/kernel/events

Makefile: obj-y := core.o ring_buffer.o callchain.o

How can I make my Ubuntu kernel use all the files linked to core.c from kernel.org and build the .ko file?

Your ubuntu kernel already have all perf_events subsystem files compiled, some are linked into the kernel image and other are .ko files already installed like intel-rapl-perf.ko

$ grep _PERF_ /boot/config-`uname -r`
$ ls -l /lib/modules/`uname -r`/kernel/arch/x86/events/intel