How to get a tensor from an image

I have an image and I would like to get the tensor from it.

Some of the images are already on the frontend server be whereas others will be served by the server

Solution

To do that, one needs to use fromPixels

In case the image is already displayed in an html page You can consider doing a querySelector to get the image first and then you can use fromPixels

html

<img id="my-image" src="mydata.jpg">

const image = document.querySelector("#my-image")
const t = tf.fromPixels(image)

If the image is not present in the html page, you can use the constructor Image to create it and then pass it as parameter to fromPixels

const im = new Image()
im.onload = () => {
  const a = tf.fromPixels(im, 4)
  a.print()
  console.log(a.shape)
}
im.src = "url-of-the-image"
document.body.appendChild(im)

onload makes sure that the image has finished downloading before converting it to a tensor.

If the image url and the url on which the frontend page is served are different there will be a cross-origin issue when creating the tensor. If the server serves the image with an access control that allows the frontend to retrieve that image, then setting the crossOrigin attribute of the image will solve the issue, otherwise there will nothing that can be done to get the tensor from that image.

const im = new Image()
im.crossOrigin = "anonymous";
im.src = "https://i.imgur.com/lVlPvCB.gif"
document.body.appendChild(im)
im.onload = () => {
  const a = tf.fromPixels(im, 4)
  a.print()
  console.log(a.shape)
}

<html>
  <head>
    <!-- Load TensorFlow.js -->
    <script src="https://cdnjs.cloudflare.com/ajax/libs/tensorflow/0.12.4/tf.js"> </script>
  </head>

  <body>
  </body>
</html>