Streaming multiple videos through FastAPI to Web Browser causes HTTP requests stalling

Question

I have a large FastAPI application. There are many different endpoints, including one that is used to proxy video streams. The usage is something like this: the endpoint receives the video stream URL, opens it and returns it through streaming response.

If I proxy 5 video streams, then everything is fine. If I proxy 6 streams, then my FastAPI app stops accepting any further requests, until I close one of the opened streams in the browser.

I will need to proxy a much larger number of video streams, but the problem occurs already when proxying 6. How fix this?

I am attaching a minimally reproducible example, but I cannot attach the links to the video streams themselves. I will note that there are no problems with the original streams, they work directly in any quantity.

import uvicorn
import httpx
from fastapi import FastAPI
from fastapi.responses import StreamingResponse

app = FastAPI()


async def proxy_mjpeg_stream(url: str):

    async with httpx.AsyncClient() as client:
        try:
            async with client.stream("GET", url) as stream:
                async for chunk in stream.aiter_bytes():
                    if chunk:
                        yield chunk
        except httpx.ReadTimeout as ER:
            print(f'ERROR ReadTimeout - {url=} - {ER}')


@app.get("/get_stream")
async def get_stream(url: str):
    if url:
        headers = {
            "Content-Type": 'multipart/x-mixed-replace; boundary=frame',
        }
        return StreamingResponse(proxy_mjpeg_stream(url), headers=headers)


if __name__ == "__main__":
    uvicorn.run(app)

Answer 1

As you noted that you are testing the application through a web browser, you should be aware that every browser has a specific limit for parallel connections to a given hostname (as well as in general). That limit is hard coded—in Chrome and FireFox, for instance, that is 6—have a look here.

So, if one opened 6 parallel connections to the same domain name through Chrome/FireFox, all subsequent requests would get stalled and queued, until a connection is available (that is, when the request is completed/a response is received).

You could use a Python client using httpx, as demonstrated in this answer, in order to test your application instead. Also, as noted in the linked answer, when opening a new connection to the same endpoint, you should do that from a tab that is isolated from the browser's main session; otherwise, succeeding requests might be blocked by the browser (i.e., on client side), as the browser might be waiting for a response to the previous request from the server, before sending the next request (e.g., for caching purposes).

Finally, please have a look at this answer on how to create a reusable HTTP client at application startup, instead of creating a new instance every time the endpoint is called.