Search code examples
linuxdebugginggdbcoredump

Linux service crashes

I have a linux service (c++, with lots of loadable modules, basically .so files picked up at runtime) which from time to time crashes ... I would like to get behind this crash and investigate it, however at the moment I have no clue how to proceed. So, I'd like to ask you the following:

  1. If a linux service crashes where is the "core" file created? I have set ulimit -c 102400, this should be enough, however I cannot find the core files anywhere :(.
  2. Are there any linux logs that track services? The services' own log obviously is not telling me that I'm going to crash right now...
  3. Might be that one of the modules is crashing ... however I cannot tell which one. I cannot even tell which modules are loaded. Do you know how to show in linux which modules a service is using?
  4. Any other hints you might have in debugging a linux service?
Solution

  • Under Linux, processes which switch user ID, get their core files disabled for security reasons. This is because they often do things like reading privileged files (think /etc/shadow) and a core file could contain sensitive information.

    To enable core dumping on processes which have switched user ID, you can use prctl with PR_SET_DUMPABLE.

    Core files are normally dumped in the current working directory - if that is not writable by the current user, then it will fail. Ensure that the process's current working directory is writable.