I have a test program
void task()
{
long sum=0;
while(true) {
for(int i=0; i<100000;++i)
for(int j=0; j < 10000; ++j)
sum += j
}
}
int main()
{
boost::thread t(task);
t.join();
return true;
}
and I use perf record -t -e intel_pt// to profile the thread. I see a lot of calls to nmi, like 2(?) calls every some milliseconds intervals. Is there anyways to get rid of this type of interrupts? Centos 7, kernel 3.10.0, kernel.watchdog = 0.
EDIT based on comment from @Zulan : gcc version: 4.8.5.
compile line: g++ -I//include/ -L//lib -lboost_system -lboost_thread test.cpp
Processor Intel Platinum 8xxx/Gold 6xxx series.
perf script output snippnet:
a.out 25192 [009] 3976.829805: 1 branches: ffffffff816ac7f5 nmi ([kernel.kallsyms]) => ffffffff816ad500 do_nmi ([kernel.kallsyms])
a.out 25192 [009] 3976.829805: 1 branches: ffffffff816ad54f do_nmi ([kernel.kallsyms]) => ffffffff816ad5f3 do_nmi ([kernel.kallsyms])
a.out 25192 [009] 3976.829805: 1 branches: ffffffff816ad5fa do_nmi ([kernel.kallsyms]) => ffffffff8103dc10 is_debug_stack ([kernel.kallsyms])
a.out 25192 [009] 3976.829805: 1 branches: ffffffff8103dc62 is_debug_stack ([kernel.kallsyms]) => ffffffff816ad5ff do_nmi ([kernel.kallsyms])
a.out 25192 [009] 3976.829805: 1 branches: ffffffff816ad60e do_nmi ([kernel.kallsyms]) => ffffffff816ad558 do_nmi ([kernel.kallsyms])
a.out 25192 [009] 3976.829805: 1 branches: ffffffff816ad573 do_nmi ([kernel.kallsyms]) => ffffffff8113a3f0 rcu_nmi_enter ([kernel.kallsyms])
a.out 25192 [009] 3976.829805: 1 branches: ffffffff8113a41e rcu_nmi_enter ([kernel.kallsyms]) => ffffffff810ba170 __smp_mb__before_atomic ([kernel.kallsyms])
a.out 25192 [009] 3976.829805: 1 branches: ffffffff810ba17a __smp_mb__before_atomic ([kernel.kallsyms]) => ffffffff8113a423 rcu_nmi_enter ([kernel.kallsyms])
a.out 25192 [009] 3976.829805: 1 branches: ffffffff8113a427 rcu_nmi_enter ([kernel.kallsyms]) => ffffffff810ba180 __smp_mb__after_atomic ([kernel.kallsyms])
...
a.out 25192 [009] 3976.829807: 1 branches: ffffffff8113a4aa rcu_nmi_exit ([kernel.kallsyms]) => ffffffff8113a48a rcu_nmi_exit ([kernel.kallsyms])
a.out 25192 [009] 3976.829807: 1 branches: ffffffff8113a48c rcu_nmi_exit ([kernel.kallsyms]) => ffffffff816ad592 do_nmi ([kernel.kallsyms])
a.out 25192 [009] 3976.829807: 1 branches: ffffffff816ad5ed do_nmi ([kernel.kallsyms]) => ffffffff816ad760 do_nmi ([kernel.kallsyms])
a.out 25192 [009] 3976.829807: 1 branches: ffffffff816ad76e do_nmi ([kernel.kallsyms]) => ffffffff816ac7fa nmi ([kernel.kallsyms])
a.out 25192 [009] 3976.829807: 1 branches: ffffffff816ac804 nmi ([kernel.kallsyms]) => ffffffff816ac23c restore_args ([kernel.kallsyms])
a.out 25192 [009] 3976.829808: 1 branches: ffffffff816ac26c irq_return ([kernel.kallsyms]) => 40576f task (a.out)
another one:
a.out 25192 [009] 3976.829808: 1 branches: ffffffff816b6718 irq_work_interrupt ([kernel.kallsyms]) => ffffffff8102f620 smp_irq_work_interrupt ([kernel.kallsyms])
a.out 25192 [009] 3976.829808: 1 branches: ffffffff8102f629 smp_irq_work_interrupt ([kernel.kallsyms]) => ffffffff81090d20 irq_enter ([kernel.kallsyms])
a.out 25192 [009] 3976.829808: 1 branches: ffffffff81090d29 irq_enter ([kernel.kallsyms]) => ffffffff8113a360 rcu_irq_enter ([kernel.kallsyms])
a.out 25192 [009] 3976.829808: 1 branches: ffffffff8113a3a0 rcu_irq_enter ([kernel.kallsyms]) => ffffffff8113a3b8 rcu_irq_enter ([kernel.kallsyms])
a.out 25192 [009] 3976.829808: 1 branches: ffffffff8113a3c0 rcu_irq_enter ([kernel.kallsyms]) => ffffffff81137e10 rcu_eqs_exit_common.isra.31 ([kernel.kallsyms])
...
a.out 25192 [009] 3976.829809: 1 branches: ffffffff8113a27d rcu_irq_exit ([kernel.kallsyms]) => ffffffff8113a260 rcu_irq_exit ([kernel.kallsyms])
a.out 25192 [009] 3976.829809: 1 branches: ffffffff8113a26e rcu_irq_exit ([kernel.kallsyms]) => ffffffff81090e4e irq_exit ([kernel.kallsyms])
a.out 25192 [009] 3976.829809: 1 branches: ffffffff81090e50 irq_exit ([kernel.kallsyms]) => ffffffff8102f653 smp_irq_work_interrupt ([kernel.kallsyms])
a.out 25192 [009] 3976.829809: 1 branches: ffffffff8102f654 smp_irq_work_interrupt ([kernel.kallsyms]) => ffffffff816b671d irq_work_interrupt ([kernel.kallsyms])
a.out 25192 [009] 3976.829809: 1 branches: ffffffff816b671d irq_work_interrupt ([kernel.kallsyms]) => ffffffff816ac1ed ret_from_intr ([kernel.kallsyms])
a.out 25192 [009] 3976.829809: 1 branches: ffffffff816ac234 retint_swapgs ([kernel.kallsyms]) => ffffffff816ac23c restore_args ([kernel.kallsyms])
a.out 25192 [009] 3976.829809: 1 branches: ffffffff816ac26c irq_return ([kernel.kallsyms]) => 40576f task (a.out)
grep'g for irq_return() from perf script output shows 3 irq per 20ms.
a.out 25192 [009] 3976.809174: 1 branches: ffffffff816ac26c irq_return ([kernel.kallsyms]) => 405764 task (a.out)
a.out 25192 [009] 3976.809178: 1 branches: ffffffff816ac26c irq_return ([kernel.kallsyms]) => 405764 task (a.out)
a.out 25192 [009] 3976.829808: 1 branches: ffffffff816ac26c irq_return ([kernel.kallsyms]) => 40576f task (a.out)
a.out 25192 [009] 3976.829809: 1 branches: ffffffff816ac26c irq_return ([kernel.kallsyms]) => 40576f task (a.out)
a.out 25192 [009] 3976.850317: 1 branches: ffffffff816ac26c irq_return ([kernel.kallsyms]) => 405764 task (a.out)
a.out 25192 [009] 3976.850393: 1 branches: ffffffff816ac26c irq_return ([kernel.kallsyms]) => 405764 task (a.out)
a.out 25192 [009] 3976.850395: 1 branches: ffffffff816ac26c irq_return ([kernel.kallsyms]) => 405764 task (a.out)
a.out 25192 [009] 3976.870945: 1 branches: ffffffff816ac26c irq_return ([kernel.kallsyms]) => 405764 task (a.out)
a.out 25192 [009] 3976.870984: 1 branches: ffffffff816ac26c irq_return ([kernel.kallsyms]) => 40576f task (a.out)
a.out 25192 [009] 3976.870985: 1 branches: ffffffff816ac26c irq_return ([kernel.kallsyms]) => 40576f task (a.out)
a.out 25192 [009] 3976.891571: 1 branches: ffffffff816ac26c irq_return ([kernel.kallsyms]) => 40576f task (a.out)
a.out 25192 [009] 3976.891609: 1 branches: ffffffff816ac26c irq_return ([kernel.kallsyms]) => 40576f task (a.out)
a.out 25192 [009] 3976.891611: 1 branches: ffffffff816ac26c irq_return ([kernel.kallsyms]) => 40576f task (a.out)
in grub, isolcpus, nohz_full, rcu_nocbs are set on particular cpus, and nmi_watchdog=0. test program ran with taskset -c a.out.
First of all, if you want to trace kernel you will get a lot of kernel function calls in the trace, including but not limited to NMI related ones.
If you want to trace your user space program like you posted above, use
perf record -t -e intel_pt//u ...
Better to consult with documentation https://elixir.bootlin.com/linux/latest/source/tools/perf/Documentation/intel-pt.txt