Force reqwest to periodically re-establish connections for load balancing

Question

I'm writing a high-throughput (>1000 requests per second) REST service in Rust, using reqwest to asynchronously connect to a load-balanced upstream service. To reduce latency of requests to that upstream service, I'm using a long-lived reqwest::Client with a connection pool.

The challenge is that reqwest's long-lived connections "break" the load-balancing of the upstream service, since - as long as the number of connections in the pool is sufficient - no connections will be established to machines added to that upstream service to increase its capacity. This leads to my service over-utilizing some machines of that upstream service and under-utilizing others (my applications is the main user of that upstream service, so I can't rely on other users of that service to balance my lopsided usage).

Is there any mechanism in reqwest to periodically close and re-establish connections, in order to ensure that they are balanced across all machines of the upstream service as evenly as possible (or any way to manually implement such behavior on top of reqwest)?

In the reqwest documentation, I've seen ConnectionBuilder::timeout() to ensure connections are closed and later re-established when upstream machines are no longer accessible. There's also ConnectionBuilder::pool_idle_timeout() to eventually close connections from that pool if they are idle long enough. But I've found nothing on closing connections - either automatically or explicitly - that are behaving fine.

Answer 1

In the end, I've implemented the desired behavior on top of reqwest::Client:

I've implemented a pool of reqwest::Clients where each Client is stored along with its creation time stamp. When requesting an entry from the pool, the pool will check the age of available entries, will dispose of those that are too old and will return a remaining one (or a new one if none remains).

This isn't perfect, since it effectively reduces each reqwest::Client connection pool to a single connection. But it adds the desired additional behavior, is simple, well encapsulated and seems to work well in practice.