TensorflowJS - Perform inference in an inactive tab

Question

I am running inference on a webcam feed using TensorflowJS. The code can be found here.

<script type="text/javascript">
    let model, webcam, labelContainer;

    // Load the image model and setup the webcam
    async function init() {
        const modelURL = URL + "model.json";
        const metadataURL = URL + "metadata.json";

        // load the model and metadata
        // Refer to tmImage.loadFromFiles() in the API to support files from a file picker
        // or files from your local hard drive
        // Note: the pose library adds "tmImage" object to your window (window.tmImage)
        model = await tmImage.load(modelURL, metadataURL);
        maxPredictions = model.getTotalClasses();

        // Convenience function to setup a webcam
        const flip = true; // whether to flip the webcam
        webcam = new tmImage.Webcam(500, 500, flip); // width, height, flip
        await webcam.setup(); // request access to the webcam
        await webcam.play();
        window.requestAnimationFrame(loop);

    }

    async function loop() {
        webcam.update(); // update the webcam frame
        window.requestAnimationFrame(loop);
    }


</script>

Is there a way for predictions to be made even when you do not have the browser tab active? By this, I mean that the tab is not selected and the browser window may be minimised.

Answer 1

Problem

The requestAnimationFrame is called before the next browser repaint. As the tab is in the background, no repaint is happening. To quote Chrome Developer updates:

Per the documentation, Chrome does not call requestAnimationFrame() when a page is in the background.

Solution

Instead of using requestAnimationFrame you can use a setTimeout function instead:

setTimeout(loop, 20); // fixed 20ms delay

However, Chrome will also start throttling background tabs after 10 seconds. It is possible to solve that problem by starting Chrome with the flag --disable-background-timer-throttling. For more information about this background throttling, check out the information regarding Budget-based background timer throttling by the Chrome developers.