I'm trying to run third-party server software on GCE (Compute Engine). It talks HTTP and/or HTTPS, runs in one Managed Instance Group and gets traffic from Appengine.
This particular program behaves very badly if it gets too many requests per instance. Therefore I want the the load balancer to refuse requests if the load is too great. But I can't get any of the ones I have tested to behave this way. Whatever "Balancing Mode" I set, it seems like it always distributes the requests to the healthy instances in my group. This leads to too many requests per instance under peak (before auto-scaling kicks in) and means that all requests fail.
Is there a way to get any of the load balancers to refuse excessive traffic? For example return an HTTP error right away?
You are looking for a functionnality that the load balancer can't do, but you may be able to find what you are looking in cloud armor: https://cloud.google.com/armor/
Some of the options are still in Beta/Alpha and is still going to evoluate.
If that still not good enough you won't have a choice and should create a tcp LB to direct traffic to something else like a HAproxy to do what you want.