Search code examples
.netazureload-balancingazure-load-balancer

Load balancing an external service in Azure

We've got a .NET service hosted in Azure (currently on a VM, later as a Web Job) that performs HTTP/HTTPS requests to an external, public service through a third-party API. For scalability and reliability (because the service is unreliable and likely to go down for longer periods when too many requests are made), we'd like to load-balance this service.

I think this is much better solved via infrastructure than via code, but I don't know enough about the infrastructure offered by Azure to know whether this can even be solved via infrastructure. I've read the overview docs about the Azure load balancer, but they only talk about balancing requests into Azure (public and internal load balancing), not requests going out of it to the Internet.

So, does Azure offer some infrastructure allowing me to load-balance requests going from my application to the Internet?

Concrete requirements:

  • Map a single HTTP endpoint to a number of configured endpoints, ideally using configured weights (i.e., choose endpoint A in 66% of the cases, endpoint B in 22%, endpoint C in 12%).
  • When one of the configured endpoints fails, transparently fail over to another endpoint and ignore the failing one for some time.

Update (2018-05-06): @kim's answer below has brought Azure Traffic Manager to my attention, which uses DNS-based load balancing and polling for automatic failover. This is probably a standard way of doing load balancing, but the ideal solution for my problem would be a proxy that receives requests, replays them to one of the load-balanced endpoints, detects failure right for the single request, and fails over immediately, retrying the request with another endpoint. That way, the calling application wouldn't ever notice something is wrong unless no working endpoint was available. I understand this is probably too specific to my problem for it to have a standard infrastructure solution, but I'm adding this anyway, just in case...

Solution

  • I haven't ever had to do what you are describing but here are some of my thoughts. The following services could be of use for you:

    What you could try is to place a traffic manager in-front of your APIs to load balance them and monitor that they are up and running.

    Then, place an API management service on-top of the traffic manager to guard the API and provide access control for it. If ever in the future, you need re-locate your API, you can easily reconfigure the API Management service to point somewhere else.