Having made a program which streams PNG images to the browser by means of a multipart/x-mixed-replace Content-Type header, I noticed that only the frame before-last is displayed in the <img> tag, as opposed to the most recently sent one.
This behaviour is very annoying, as I'm only sending updates when the image changes to save on bandwidth, which means that the wrong frame will be on screen while I'm waiting for it to update.
Specifically, I am using Brave Browser (based on chromium), but as I have tried with both "shields" up and down, I assume this problem occurs also in other chromium-based browsers at least.
Searching for the problem yields only one relevant result (and many non-relevant ones) which is this HowToForge thread, with no replies. Likewise, I also thought the issue is to do with buffering, but I made sure to flush the buffer to no avail, much alike to the user in the thread. The user does report that it works on one of their servers though and not the other, which then lead me to believe that it may be to do with a specific HTTP header or something along those lines. My first guess was Content-Length because the browser can tell when the image is complete from that, but it didn't seem to have any effect.
So essentially, my question is: Is there a way to tell the browser to show the most recent multipart/x-mixed-replace and not the one before? And, if this isn't standard behaviour, what could the cause be?
And of course, here's the relevant source code, though I imagine this is more of a general HTTP question than one to do with the code:
package routes
import (
"crypto/md5"
"fmt"
"image/color"
"net/http"
"time"
brain "path/to/image/generator/module"
)
func init() {
RouteHandler{
function: func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "multipart/x-mixed-replace; boundary=frame")
w.Header().Set("Cache-Control", "no-cache") // <- Just in case
w.WriteHeader(200)
// If the request contains a token and the token maps to a valid "brain", start consuming frames from
// the brain and returning them to the client
params := r.URL.Query()
if val, ok := params["token"]; ok && len(val) > 0 {
if b, ok := SharedMemory["brains"].(map[string]*brain.Brain)[val[0]]; ok && !b.CheckHasExit() {
// Keep a checksum of the previous frame to avoid sending frames which haven't changed. Frames cannot
// be compared directly (at least efficiently) as they are slices not arrays
previousFrameChecksum := [16]byte{}
for {
if !b.CheckHasExit() {
frame, err := b.GetNextFrame(SharedMemory["conf"].(map[string]interface{})["DISPLAY_COL"].(color.Color))
if err == nil && md5.Sum(frame) != previousFrameChecksum {
// Only write the frame if we succesfully read it and it's different to the previous
_, err = w.Write([]byte(fmt.Sprintf("--frame\r\nContent-Type: image/png\r\nContent-Size: %d\r\n\r\n%s\r\n", len(frame), frame)))
if err != nil {
// The client most likely disconnected, so we should end the stream. As the brain still exists, the
// user can re-connect at any time
return
}
// Update the checksum to this frame
previousFrameChecksum = md5.Sum(frame)
// If possible, flush the buffer to make sure the frame is sent ASAP
if flusher, ok := w.(http.Flusher); ok {
flusher.Flush()
}
}
// Limit the framerate to reduce CPU usage
<-time.After(time.Duration(SharedMemory["conf"].(map[string]interface{})["FPS_LIMITER_INTERVAL"].(int)) * time.Millisecond)
} else {
// The brain has exit so there is no more we can do - we are braindead :P
return
}
}
}
}
},
}.Register("/stream", "/stream.png")
}
start() runs in body onload)function start() {
// Fetch the token from local storage. If it's empty, the server will automatically create a new one
var token = localStorage.getItem("token");
// Create a session with the server
http = new XMLHttpRequest();
http.open("GET", "/startsession?token="+(token)+"&w="+(parent.innerWidth)+"&h="+(parent.innerHeight));
http.send();
http.onreadystatechange = (e) => {
if (http.readyState === 4 && http.status === 200) {
// Save the returned token
token = http.responseText;
localStorage.setItem("token", token);
// Create screen
var img = document.createElement("img");
img.alt = "main display";
// Hide the loader when it loads
img.onload = function() {
var loader = document.getElementById("loader");
loader.remove();
}
// Start loading
img.src = "/stream.png?token="+token;
// Start capturing keystrokes
document.onkeydown = function(e) {
// Send the keypress to the server as a command (ignore the response)
cmdsend = new XMLHttpRequest();
cmdsend.open("POST", "/cmd?token="+(token));
cmdsend.send("keypress:"+e.code);
// Catch special cases
if (e.code === "Escape") {
// Clear local storage to remove leftover token
localStorage.clear();
// Remove keypress handler
document.onkeydown = function(e) {}
// Notify the user
alert("Session ended succesfully and the screen is inactive. You may now close this tab.");
}
// Cancel whatever it is the keypress normally does
return false;
}
// Add screen to body
document.getElementById("body").appendChild(img);
} else if (http.readyState === 4) {
alert("Error while starting the session: "+http.responseText);
}
}
}
A part inside a multipart MIME message starts with the MIME header and ends with the boundary. There is a single boundary before the first real part. This initial boundary closes the MIME preamble.
Your code instead assumes that a part starts with the boundary. Based on this assumption you first send the boundary, then the MIME header and then the MIME body. Then you stop sending until the next part is ready. Because of this the end of one part will only be detected once you send the next part, since only then you send the end boundary of the previous part.
To fix this your code should initially send one boundary to end the MIME preamble. For each new part it should then send the MIME header, the MIME body and then the boundary to end this part.