Javascript - imageCapture.takePhoto() function to take pictures

Question

I am building an web application for my experiment purpose. The aim here is to capture ~15-20 frames per second from the webcam and send it to the server. Once the frame is captured, it is converted to base64 and added to an array. After certain time, it is sent back to the server. Currently I am using imageCapture.takePhoto() to achieve this functionality. I get blob as a result which is then changed to base64. The application runs for ~5 seconds and during this time, frames are captured and sent to the server.

What are the more efficient ways to capture the frames through webcam to achieve this?

Answer 1

You can capture still images directly from the <video> element used to preview the stream from .getUserMedia(). You set up that preview, of course, by doing this sort of thing (pseudocode).

const stream = await navigator.getUserMedia(options)
const videoElement = document.querySelector('video#whateverId')
videoElement.srcObject = stream
videoElement.play()

Next, make yourself a canvas object and a context for it. It doesn't have to be visible.

const scratchCanvas = document.createElement('canvas')
scratchCanvas.width = video.videoWidth
scratchCanvas.height = video.videoHeight
const scratchContext = scratchCanvas.getContext('2d')

Now you can make yourself a function like this.

function stillCapture(video, canvas, context) {
    context.drawImage( video, 0, 0, video.videoWidth, video.videoHeight)  
    canvas.toBlob(
           function (jpegBlob) {
               /* do something useful with the Blob containing jpeg */
           }, 'image/jpeg')
}

A Blob containing a jpeg version of a still capture shows up in the callback. Do with it whatever you need to do.

Then, invoke that function every so often. For example, to get approximately 15fps, do this.

const howOften = 1000.0 / 15.0
setInterval (stillCapture, howOften, videoElement, scratchCanvas, scratchContext)

All this saves you the extra work of using .takePhoto().