I've set up a ServiceBus Relay with Microsoft Azure. I'm connecting to this with a HTTP backend handling incoming GET and POST requests hitting the Relay.
Now, this works just fine. My issue though is that the max concurrency I can get is 2 simultaneous requests, i.e my backend will only ever receive 2 requests at the same time, if there are more requests they will be stalled until the previous requests have been handled & completed. I've tried increasing concurrency with a service behavior but this seems to be ignored, but lowering the concurrency to 1 seems to be working with the settings. So I'm guessing this is some kind of limit imposed by the actual relay!? But could that really be? Only 2? In my scenario this makes the relay unusable as the requests have to do time consuming processing (2 seconds) which can not be delayed and requires an immediate response. I therefore can not simply store the request and process them like a queue.
How can I scale this out? Add more relays and connect to them all?
You'll need to set ServicePointManager.DefaultConnectionLimit to an appropriate value to allow more concurrent connections. Try doing this as early as possible in your application code:
ServicePointManager.DefaultConnectionLimit = int.MaxValue;
This will allow your application to open an unlimited (in practice) number of concurrent HTTP connections.
The default limit is due to how the HTTP protocol was originally designed.
The following quote is from section 8.1.4 of the HTTP/1.1 RFC:
A single-user client SHOULD NOT maintain more than 2 connections with any server or proxy [...] These guidelines are intended to improve HTTP response times and avoid congestion.
Therefore, a compliant client would typically have to ensure that it never has more than two open connections to a single server.
However, this have changed and you're allowed to override this behavior. The following quote is from section 6.4 of RFC 7230 which is an update of the HTTP/1.1 protocol:
Previous revisions of HTTP gave a specific number of connections as a ceiling, but this was found to be impractical for many applications. As a result, this specification does not mandate a particular maximum number of connections but, instead, encourages clients to be conservative when opening multiple connections.