I've been profiling some code which likely spends a lot of it's execution time in system calls. Timing some functions manually and with callgrind, callgrind reports system call times of around 20, 30 or 40 times longer than simply timing the function (which would ofcourse include CPU time as well).
--collect-systime=ON was used to collect this sysCall time for each function.
As far as I know, callgrind works by counting the CPU instructions and for timing system calls simply lets the OS do the work and doesn't interfere. I don't understand why the time spent in sysCalls increases when profiling with callgrind, can anyone elaborate?
Is callgrind still a useful tool to profile time spent in sysCalls?
Can you try --collect-systime=usec and --collect-systime=nsec to see if they are significantly different? usec might be a bit faster.
When --collect-systime is specified, Valgrind will be calling one of the various time syscalls for each of the client application syscalls. I would expect that to add a substantial overhead, particularly if your client application is calling very many "fast" syscalls.