Search code examples
javascriptwebrtcchromiumwebcamgoogle-chrome-os

Wrong image colour when streaming from 16-bit depth webcam on Chrome OS

I want to display the video stream from a USB thermal camera - for the record, it's a FLIR Lepton PureThermal 2 - on my web app, running on a Chrome OS (version 85.0.4183.84 - latest available).

At first I assumed the device could be streamed as a standard webcam, so I came up with the following code:

<!DOCTYPE html>
<html lang="en">

<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>Webcam test</title>
</head>

<body>
  <video id="videoElement" width="800" height="600" muted autoplay></video>
</body>

<script type="text/javascript">
  let videoElement = document.getElementById('videoElement');

  if (navigator.mediaDevices && navigator.mediaDevices.getUserMedia) {

    let videoDevices = [];
    navigator.mediaDevices.enumerateDevices()
    .then(devices => {
      videoDevices = devices.filter(device => {
        if (device.kind !== "videoinput") return;
        console.debug(`Identified video input: ${JSON.stringify(device)}`);
        return true;
      });

      let selectedCamera = videoDevices[videoDevices.length - 1]; // usually the thermal camera will be the last one
      console.debug(`Streaming device ${JSON.stringify(selectedCamera)}...`);

      return navigator.mediaDevices.getUserMedia({
        video: {
          deviceId: { exact: selectedCamera.deviceId },
          width: {exact: 160 }, height: {exact: 120 }, // forcing QQVGA resolution
        },
      });
    })
    .then(stream => {
      videoElement.srcObject = stream;
      videoElement.play();
    })
    .catch(error => console.error(error));
  }

</script>

</html>

The code above works as expected on any browser on Windows 10, but I'm not getting the image I was expecting on Chrome OS. See the comparison below: enter image description here

If you look closely, Chrome is in fact streaming something, it's just not clear enough. Changing the image's level and contrast gives me this: enter image description here

Just for the record, the code works in all browsers below (tested on Windows 10 only):

  • Chrome 85.0.4183.83
  • Firefox 80.0
  • Opera 70.0.3728.133
  • Edge 85.0.564.41
  • Edge Legacy (probably version 40)

What am I missing in my code to make it work on Chrome OS work like it does for Chrome on Windows?

Solution

  • I managed to make it work on Chrome OS by doing the following:

    • As soon as the video starts playing, save the bytes from the current frame into a texture buffer (with the help of WebGL)
    • For every pixel in the frame, multiply the red channel by 65535, returning the temperature in centikelvin
    • Convert it to celsius and manipulate the pixels to show a grey scale image within a temperature range
    • Draw the pixel on a canvas
    • Request a new frame (restarting the loop)

    See sample code here: https://codesandbox.io/s/upbeat-wood-umo9w?file=/index.html

    Result: enter image description here

    Special thanks to Aleksandar Stojiljkovic and Kurt Kiefer.

    The same result could be achieved by applying a custom shader to the texture instead.

    I'm still not sure why Chrome OS handles the video differently, so I've raised an issue on Chromium issue tracker anyway.