Export a Model's Labels and Confidence

Question

I have trained a model via Google Cloud Vision. On the Evaluate tab, it shows me a list of labels it trained on, along with the confidence value (see below). Is there any way I can download this information as a CSV? EXPORT DATA unfortunately doesn't do that, it only exports the image location along with some meta-data, no confidence info.

Answer 1

For you to extract the evaluation data, you can send a GET request to projects.locations.models.modelEvaluations.list. You will need locationId and modelId for this.

To get the locationId send GET request to projects.locations.list. GET request:

curl -X GET -H "Content-Type: application/json" -H "Authorization: Bearer "$(gcloud auth application-default print-access-token) https://automl.googleapis.com/v1/projects/your-project-id/locations

projects.locations.list returns:

{
  "locations": [
    {
      "name": "projects/your-project-id/locations/eu",
      "locationId": "eu"
    },
    {
      "name": "projects/your-project-id/locations/us-central1",
      "locationId": "us-central1"
    }
  ]
}

To get the modelId send GET request to projects.locations.models.list. GET request:

curl -X GET -H "Content-Type: application/json" -H "Authorization: Bearer "$(gcloud auth application-default print-access-token) https://automl.googleapis.com/v1/projects/your-project-id/locations/us-central1/models

name: contains the modelId in this example it is ICN1234567890. projects.locations.models.list returns:

{
  "model": [
    {
      "name": "projects/your-project-id/locations/us-central1/models/ICN1234567890",
      "displayName": "flowers_20201217111947",
      "datasetId": "ICN8001048258791079936",
      "createTime": "2020-12-17T03:20:27.198250Z",
      "deploymentState": "UNDEPLOYED",
      "updateTime": "2021-09-07T08:59:02.243603Z",
      "imageClassificationModelMetadata": {
        "stopReason": "MODEL_EARLY_STOPPED",
        "modelType": "mobile-versatile-1",
        "nodeQps": 3.2,
        "trainBudgetMilliNodeHours": "4000",
        "trainCostMilliNodeHours": "1882"
      }
    },...

Now build the GET request for the evaluation data. Use projects.locations.models.modelEvaluations.list:

curl -X GET -H "Content-Type: application/json" -H "Authorization: Bearer "$(gcloud auth application-default print-access-token) https://automl.googleapis.com/v1/projects/your-project-id/locations/us-central1/models/ICN1234567890/modelEvaluations

modelEvaluations.list returns. See response snippet below:

   {
      "name": "projects/your-project-id/locations/us-central1/models/ICN1234567890/modelEvaluations/8145357881766117082",
      "annotationSpecId": "5162564433687347200",
      "createTime": "2020-12-17T05:49:10.557128Z",
      "classificationEvaluationMetrics": {
        "auPrc": 0.99306494,
        "confidenceMetricsEntry": [
          {
            "recall": 1,
            "precision": 0.17166212
          },
          {
            "confidenceThreshold": 0.05,....

You can refer to this to see the complete fields returned for modelEvaluation. See comparison of UI and values via the modelEvaluations.list: