Search code examples
imagevideobackgroundtwiliovideo-processing

Video with virtual background goes black when image src value comes from an different origin

I'm trying to load an image from different origin to use it as virtual background. First I fetch the image and I'm able to show it in another element in my page (looks like CORS is not the problem ??) but somehow when attaching the image in the virtualBackground the video goes black, if I use an image from same origin it loads without issues.

There is no errors in te console either.

I'm using twilio/video-processors.js v 2.0.0 and twilio-video v2.28.1. Please check the piece of code below:

const loadImage = (imgSource: string): Promise<HTMLImageElement> =>
      new Promise((resolve, reject) => {
        const img = new Image();
        img.src = imgSource;
        // img.src = '/specificResources/my-bg.jpg'; // When I use this one it works
        img.onload = () => resolve(img);
        console.log('IMAGE', img); // Prints img element
        img.onerror = (e) => {
          console.log('onerror', e);
          reject(e);
        };
      });
    try {
      const imgUri =
        'https://my.domain.com/images/fdef355eb7a7';
      const data = await fetchResource(imgUri);
      const backgroundImage = await loadImage(data.location); // data.location is another url
      console.log('[backgroundImage]', backgroundImage); // Prints img element
      const virtualBackground = new VirtualBackgroundProcessor({
        assetsPath: '/twilioVideoProcessorAssets',
        backgroundImage,
        fitType: ImageFit.Contain,
        pipeline: Pipeline.Canvas2D,
      });
      await virtualBackground.loadModel();
      console.log('Adding processor');
      videoTrack.addProcessor(virtualBackground); 
    } catch (error) {
      console.error('Error loading model', error);
    }
Solution

  • Turns out that it was a CORS issue after all, the server needed to include some attributes in the response header of the resource.

    Access-Control-Allow-Origin=<origin> 
Vary: Origin

    Somehow hard to figure out because it wasn't throwing any CORS error or any other kind of error in the console and video suddenly goes black so can misinterpret that something else is going on, unless you add crossOrigin="anonymous" in the image object you will notice.

    So maybe there is room to improve to throw some error or exception message in the console.

    Here is some useful docs: