Search code examples
machine-learninggoogle-cloud-platformgoogle-cloud-automl

GCP AutoML Tables predict online takes several seconds

The following takes several seconds. I know this is not conceptual but I'm cross posting from a GCP issue just in case someone happened to step on the same scenario.

const { PredictionServiceClient } = require('@google-cloud/automl');
const predictionServiceClient = new PredictionServiceClient({ apiEndpoint });
const prediction = await predictionServiceClient.predict({
    name: modelFullId,
    params,
    payload,
});

This API in real time takes close to 10s when cold and 5s when hot. Is there a way to fasten this up other than to export the model and run it yourselves?

Solution

  • Yes, you can export the model and use with tensorFlowJS.

    https://cloud.google.com/vision/automl/docs/edge-quickstart

    https://github.com/tensorflow/tfjs/tree/master/tfjs-automl

    1. Export the model and download model.json, dict.txt, *.bin files to your local.

    2. Load the model into tensorFlowJS and use it.