Search code examples
unixlinux-kernelsyslogsyslog-ng

What does this message from syslog@localhost signifies?


Message from syslogd@saskappcu at Mar 18 13:24:54 ...
 kernel:BUG: soft lockup - CPU#30 stuck for 61s! [events/30:161]

Message from syslogd@saskappcu at Mar 18 13:24:54 ...
 kernel:Process events/30 (pid: 161, ti=f4ea4000 task=f4e5faa0 task.ti=f4ea4000)

Message from syslogd@saskappcu at Mar 18 13:24:54 ...
 kernel:Stack:

Message from syslogd@saskappcu at Mar 18 13:24:54 ...
 kernel:Call Trace:

Message from syslogd@saskappcu at Mar 18 13:24:54 ...
 kernel:Code: 00 89 51 04 89 0a 89 43 20 89 43 24 8b 43 08 39 d8 74 23 83 40 7c 01 31 c9 8b 7b 0c 8b 15 58 09 ac c0 8b 02 39 c2 75 09 eb 31 90 <8b> 00 39 d0 74 2a 3b 78 0c 75 f5 89 d8 ba 00 00 04 00 e8 b9 a0

Message from syslogd@saskappcu at Mar 18 13:24:58 ...
 kernel:BUG: soft lockup - CPU#8 stuck for 61s! [buildop:2223]

Message from syslogd@saskappcu at Mar 18 13:24:58 ...
 kernel:Process buildop (pid: 2223, ti=e9724000 task=f3ba0aa0 task.ti=e9724000)

Message from syslogd@saskappcu at Mar 18 13:24:58 ...
 kernel:Stack:

Message from syslogd@saskappcu at Mar 18 13:24:58 ...
 kernel:Call Trace:

Message from syslogd@saskappcu at Mar 18 13:24:58 ...
 kernel:Code: 26 00 89 c8 f0 81 28 00 00 00 01 74 05 e8 2c fe ff ff c3 8d b4 26 00 00 00 00 66 ba 00 01 f0 66 0f c1 10 38 f2 74 0e f3 90 8a 10 <eb> f6 66 83 38 00 75 f4 eb e5 c3 8d 74 26 00 f0 81 28 00 00 00

Solution

  • A softlockup is defined as a bug that causes the kernel to loop in kernel mode for more than 20 seconds without giving other tasks a chance to run.

    The log

    kernel:BUG: soft lockup - CPU#30 stuck for 61s! [events/30:161]
    

    is generated by the following line in kernel/kernel/watchdog.c

    pr_emerg("BUG: soft lockup - CPU#%d stuck for %us! [%s:%d]\n",
             smp_processor_id(),
             duration,
             current->comm,
             task_pid_nr(current));
    

    It means that

    • The CPU core 30 in the system,
    • has been busy executing kernel code for the past 61 seconds.
    • The current thread on the system being events/30,
    • whose process-id is 161.

    For more details, checkout kernel/Documentation/lockup-watchdogs.txt.