Search code examples
websocketstreaminghttp2bidirectional

Use HTTP/2 instead of WebSocket

Lot's of articles and the most popular answer said that's not possible to transfer binary data with HTTP/2 and I'm a bit confused since:

  1. There is Stream API in browser
  2. We can use "Body is a readable byte stream" and "Send ReadableStream in request body"
  3. Also I found an example of bi-directional fetch example. And here is an example of ndjson streaming

I only see the problem that's not possible to reserve dedicated TCP connection like it was with WebSocket and we're limited with 6 connections by default and 100 streams for each, so 600 streams in total.

Would really appreciate for clarification why we still need WebSocket API. Thanks!

Solution

  • Lot's of articles and the most popular answer said that's not possible to transfer binary data with HTTP/2

    Just to clarify: HTTP/2, the wire protocol, can transfer binary data without issues. The (old-ish) answer you mention was referring to the JavaScript APIs available in browsers at the time, but times have changed, and now it is possible to handle binary data with the browser's JavaScript APIs.

    I only see the problem that's not possible to reserve dedicated TCP connection like it was with WebSocket and we're limited with 6 connections by default and 100 streams for each, so 600 streams in total.

    The default behavior of browsers for HTTP/2 is to open only 1 connection, because they can multiplex requests into it. Browsers try very hard to open just one connection, so that even in case of WebSocket over HTTP/2 they use the same connection that is used for regular HTTP requests.

    Would really appreciate for clarification why we still need WebSocket API.

    I'm assuming you're trying to compare a legitimate WebSocket usage, that is messages sent by the client to the server, but most commonly messages sent from the server to the client, as it would happen in a chat application, or an online game, etc.

    If you control the server, and you enable HTTP/2, I guess there are many similarities with WebSocket, BUT...

    If you have to use HTTP/1.1, I guess many of the features mentioned in the links of your question are not available (for example, the article that shows how to stream the request body via fetch() also mentions this is not available in HTTP/1.1).

    Intermediaries may decide to buffer HTTP/2 DATA frames, so if you stream HTTP/2 request or response bodies, expecting each chunk to arrive to the recipient as soon as possible, you may have surprises: an intermediary can buffer the whole response and send it at once. For the intermediary, your stream of chunks that simulates WebSocket messages would be indistinguishable from an HTML page.

    This buffering issue should be resolved by using TLS to encrypt the traffic, as an intermediary would unlikely buffer encrypted bytes (although it could, but then it would be a problem for WebSocket too).

    WebSocket can send to the client unsolicited messages. This means that the client only needs to establish the connection, and does not need to communicate to the server at all to receive messages.

    Slightly differently, with HTTP/2 you would need to establish the connection, and then make at least 1 request to the server to establish an "infinite response body" for that request that delivers chunks of data from the server.

    For both WebSocket and HTTP/2, you need to invent a protocol format for your "messages", delivered either as WebSocket messages or as HTTP/2 chunks. For example, imagine a board game where the opponent makes a move; the message should contain some metadata (e.g. the game id, opponent name, opponent game time, etc.) and the data (the move).

    With WebSocket, you are guaranteed that the whole message arrives to the JavaScript APIs as a single entity, even if it is larger than few KiB. This is because WebSocket defines a message framing that delineates where a message starts and ends, and browsers do the work of coalescing all the WebSocket frames that compose a message, and then deliver to the JavaScript APIs only the message content.

    With HTTP/2 streaming, you would have to parse the chunks yourself and coalesce them together until a full message is recomposed. You have to do the framing work that in WebSocket is done by the browser for you.

    To be clear, in the examples reported in your links, this is the idiomatic way to receive the response body as a stream:

    const response = await fetch(url);
const reader = response.body.getReader();

while (true) {
  const {value, done} = await reader.read();
  if (done) {
    break;
  }
  console.log('Received', value);
}

    However, if you want to simulate WebSocket, done will never be true because you want to keep the response "infinite" to continue streaming messages from the server to the client.

    If the server sent a chunk of JSON bytes { game: { id: 1, players: ["foo", "bar"] } }, then the client may receive either:

    • value={ game: { id: 1, pl, so only about half of the message sent by the server; or
    • value={ game: { id: 1, players: ["foo", "bar"] } } { something: "else", that is one and a half of the two messages sent by the server.

    WebSocket can automatically compress each message for you. Textual data like JSON compresses very well.

    With HTTP/2, you can use stream transformers to decompress the message, but you have to be careful with the framing of each message, which you would have to define.

    All in all, when trying to use HTTP/2 as WebSocket replacement, you would need to reinvent a framing format, to coalesce chunks into messages, and likely implement of message compression decoding.

    This is basically reinventing WebSocket.

    My opinion is that for simple cases HTTP/2 streaming might work as a WebSocket replacement, but as soon as you hit a moderately more complex case, WebSocket still has its reason.