Input data is not a valid image. - Azure Cognitive Services - Javascript - Canvas

Question

I am trying to use the Image Description API of Azure and tried to send the data by converting through the canvas(base64), I am not able to figure out the reason behind the error, "Input data is not a valid image." with a status code 400 in response. Following is the code:

imageDescribe = async () => {
try {
    canvas.getContext('2d').drawImage(video, 0, 0, 200,200);  
    // convert canvas image as byte64 string
    const imageBuffer = canvas.toDataURL().replace(/^data:image\/(png|jpeg);base64,/, '');
    
    const subscription = 'xxx';
    const endpoint = 'https://eastus.api.cognitive.microsoft.com/vision/v2.0/analyze?visualFeatures=Description&language=en';
    
    const response = fetch(endpoint, {
        method: 'POST',
        headers: {
            'Content-Type': 'application/octet-stream',
            'Ocp-Apim-Subscription-Key': subscription
        },
        body: imageBuffer
    });
    const data = await response.json();
    console.log(data);
    console.log(response.status);



} catch (error) {
    console.log(error);

}

}

Answer 1

I tested your code against my Azure resource and found a couple of things to fix.

1. The request body must be binary data

Use canvas.toBlob((blob) => {...}).

2. Async calls

Both toBlob and fetch should be used as async calls.

canvas.toBlob((blob) =>{
     fetch(endpoint, {
            method: 'POST',
            headers: {
                'Content-Type': 'application/octet-stream',
                'Ocp-Apim-Subscription-Key': subscription
            },
            body: blob
            })
            .then((response) => response.json())
            .then((data) => console.log(data));
        }); 

So putting it all together, here is the working code,

function getImageDescription() {
       //you might want to make sure video is a good image.
      canvas.getContext('2d').drawImage(video, 0, 0, 200,200);  
  
      const subscription = 'xxx';
      const endpoint = 'https://eastus.api.cognitive.microsoft.com/vision/v2.0/analyze?visualFeatures=Description&language=en';

      canvas.toBlob((blob) =>{

        fetch(endpoint, {
            method: 'POST',
            headers: {
                'Content-Type': 'application/octet-stream',
                'Ocp-Apim-Subscription-Key': subscription
            },
            body: blob
            })
            .then((response) => response.json())
            .then((data) => console.log(data))
            .catch((error) => console.error('Error:', error));
        });
}

Here is the actual response.