Tensorflow js server side classification with mobilenet and blazeface

Question

I'm trying to use tensorflowjs with an already built model, at the beginning I had a problem with blazeface because it didn't find face on photo (that display just faces) and so I'm trying with mobilenet and same probleme the result are non sens. So pretty sur it came from the format of image I'm sending.

So this is my code:

module.exports = {
    blazeface: require('@tensorflow-models/blazeface'),
    mobilenet: require('@tensorflow-models/mobilenet'),
    tf: require("@tensorflow/tfjs-node"),
    fs: require("fs"),
    sizeOf: require("image-size"),

    async structureData(imageLink) {
        let data = this.fs.readFileSync(imageLink);//return a buffer
        const dimension = await this.sizeOf(imageLink);
        const tensorflowImage = {
            data: data,
            width: dimension.width,
            height: dimension.height
        };
        console.log(tensorflowImage)
        return tensorflowImage;
    },

    async detectFace(imageLink) {
        let image = await this.structureData(imageLink);
        const model = await this./*blazeface*/mobilenet.load();
        const returnTensors = false;
        const predictions = await model.classify(image);
        console.log(predictions);
    }
}

So this code do not result of an error but the result are this =>

[
  {
    className: 'theater curtain, theatre curtain',
    probability: 0.03815646469593048
  },
  {
    className: 'web site, website, internet site, site',
    probability: 0.0255243219435215
  },
  { className: 'matchstick', probability: 0.02520526386797428 }
]

and with any photo (here it's a banana in white background). So I'm pretty sur i need to rebuilt the structureData function but I don't know how ... I also tried this with uint32array

    async structureData(imageLink) {
        let data = this.fs.readFileSync(imageLink);
        data = new Uint32Array(data);
        const dimension = await this.sizeOf(imageLink);
        const tensorflowImage = {
            data: data,
            width: dimension.width,
            height: dimension.height
        };
        console.log(tensorflowImage)
        return tensorflowImage;
    },

But I'm getting this error.

Error: pixels passed to tf.browser.fromPixels() must be either an HTMLVideoElement, HTMLImageElement, HTMLCanvasElement, ImageData in browser, or OffscreenCanvas, ImageData in webworker or {data: Uint32Array, width: number, height: number}, but was Object

remember that i'm using node and so I can't (or don't think I can) us HTMLimageelement

Thanks a lot :)

Answer 1

By using a tensor, a video or image element as parameter to the model, it will be able to do the classification.

let data = this.fs.readFileSync(imageLink);
tensor = tf.node.decodeImage(data, 3)

await model.classify(tensor)