Search code examples
wordpressamazon-web-servicesapacheubuntubitnami

How to find root cause for server shut down frequently


I'm on ubuntu apache. Lately it shuts down frequently and back on after a restart. I kept analyzing apache2 error log to find the cause. Previously it was reporting PHP code error. But after fixing it, it throws different error now.

What can I conclude based on these errors? Which probably would have caused the downtime and how to fix it?

AH: scoreboard is full, not at MaxRequestWorkers.Increase ServerLimit.
AH00484: server reached MaxRequestWorkers setting, consider raising the MaxRequestWorkers setting

AH00687: Negotiation: discovered file(s) matching request: /opt/bitnami/apps/file_that_doesn't-exist
(70007)The timeout specified has expired: [client 148.251.79.134:60170] AH01075: Error dispatching request to : (polling)

AH00045: child process 5062 still did not exit, sending a SIGTERM
AH00046: child process 5299 still did not exit, sending a SIGKILL
AH01909: localhost:443:0 server certificate does NOT include an ID which matches the server name

I've done enough google search to understand each of this error. Most importantly I woudl like to kow which of these error would have cause the server to go down? And what is the way fixing it?


Solution

  • Bitnami Engineer here,

    AH: scoreboard is full, not at MaxRequestWorkers.Increase ServerLimit.
    AH00484: server reached MaxRequestWorkers setting, consider raising the MaxRequestWorkers setting
    

    It seems the server is reaching the limits and that can be the reason of the issues you are running into. It can reach the limits either because the instance is really small and you need to increase its memory/CPU or because you are being attacked.

    • You can check if you are being attacked by running these commands
    cd /opt/bitnami/apache2/logs/
    tail -n 10000 access_log | awk '{print $1}'| sort| uniq -c| sort -nr| head -n 10
    

    Are those IPs familiar? Is there any IP that is requesting your site too many times?

    You can find more information about how to block it here

    • You can also increase the MaxRequestWorkers parameter in Apache by editing the /opt/bitnami/apache2/conf/bitnami/httpd.conf file or you can also increase the instance type using the AWS console so the server has more resources from now on.