2017/06/29 18:37:56 [crit] 2470#2470: ngx_slab_alloc() failed: no memory in upstream zone "backends"
2017/06/29 18:37:56 [error] 2470#2470: cannot add new server to upstream "<redacted>", memory exhausted
I am receiving a stream of critical errors in my log that indicate various upstream zones cannot be added to various upstreams because memory is exhausted.
That said, I have plenty of free memory. I'm guessing I need to increase some setting somewhere, but for the life of me, The Google doesn't seem to be able to tell me what I need to increase.
We use nginx as a reverse proxy for service discovery with our AWS ECS Docker container cluster.
Turns out the problem was that I had the zone "backends" set too small.. I increased that from 64k to 256k and all is fine now. It also turns out that I should probably be using a different zone for each upstream rather than a shared one for all my upstreams.
This answer was compliments of the nginx professional support team. We're using a licensed version of nginx. Awesome support!