I am trying to grok what myApp is very busy with (90% cpu single thread). It is a server that i shouldn't restart. I've collected samples by
perf record -p 5068 -F 99 --call-graph dwarf sleep 10
And perf report gives me this:
+ 100.00% 0.00% myApp [unknown] [.] 0xffffffffffffffff ◆
+ 80.67% 0.67% myApp myApp [.] pipeline_run ▒
+ 67.71% 0.00% myApp myApp [.] QueryProcessor::process
I spent some time googling and reading docs, and I suspect that 0xffffffffffffffff could not be resolved because perf does not know where stack bottom is as it didn't start that process. But could someone confirm it or point me to the right direction?
In my case this was caused by perf using a too small stack dump size. That will cause the bottom of the truncated stack to become 0xffffffffffffffff, which will make perf report and friends think that the function 0xffffffffffffffff is used a lot.
When I used the maximum stack dump capture size, I got rid of most of the 0xffffffffffffffff. If you pass --call-graph dwarf the default stack dump size of 8192 will be used. To maximize it, change to --call-graph dwarf,65528.
From perf record --help:
When "dwarf" recording is used, perf also records (user) stack dump
when sampled. Default size of the stack dump is 8192 (bytes).
User can change the size by passing the size after comma like
"--call-graph dwarf,4096".
If you try with a value larger than 65528 you get
callchain: Incorrect stack dump size (max 65528): 128000