Google Cloud Vision returns empty response

Question

I am trying to build a C# library that will act as a wrapper for a set of Google APIs. When working with Google Vision API, I have found the API returns an empty response set for certain queries. For example, when I try to run FACE_ANNOTATION on car.png, the response I get back is:

{
  "responses": [
    {}
  ]
}

I have eliminated all the basic issues like storing the image in a Google Cloud bucket, public access for the image, valid API key, enabling the API from the Google API Dashboard.

Below is a segment of the code where I make the request:

httpClient.DefaultRequestHeaders.Accept.Clear();
httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));

// The API address to which we will make the HTTP POST query
String request_query = "v1/images:annotate?" + $"key={APIKey}";
HttpResponseMessage response = await httpClient.PostAsJsonAsync(request_query, imageRequests);

Stream stream = await response.Content.ReadAsStreamAsync();
StreamReader streamReader = new StreamReader(stream);
String response_str = streamReader.ReadToEnd();        
Console.WriteLine(response_str);

if (response.IsSuccessStatusCode) {
    try {
        imageResponseList = JsonConvert.DeserializeObject<AnnotateImageResponseList>(response_str);

     } catch (JsonSerializationException e) {
         Debug.WriteLine(e.StackTrace);
     }
}

Here is the request body (imageRequests as it's called in my code above) that is sent to the API:

{
  "requests": [
    {
      "image":
       {
         "content":"",
         "source":
         {
           "imageUri":"gs://<google_cloud_bucket>/car.png"
         }
       },
       "features":[
         {
           "type":0,
           "maxResults":100,
           "model":"builtin/stable"
         } 
       ],
       "imageContext":null
    }
  ]
}

Now, I am aware that there is already a C# client that can be used directly, but the project I am working on needs me to access the REST API through HTTP requests.

Any help would be appreciated.

Answer 1

The API request is malformed for what you are trying to do.

You are specifying the content and source fields. The Vision API images.annotate documentation specifies that:

If both content and source are provided for an image, content takes precedence and is used to perform the image annotation request.

The content you have specified is empty, the API has nothing to work with so the response is empty too. Remove the content to avoid the issue.

The type field is also giving problems. When trying the API in the the images.annotate documentation, it suggested the following for the type field:

Expected one of ["TYPE_UNSPECIFIED", "FACE_DETECTION", "LANDMARK_DETECTION", "LOGO_DETECTION", "LABEL_DETECTION", "TEXT_DETECTION", "DOCUMENT_TEXT_DETECTION", "SAFE_SEARCH_DETECTION", "IMAGE_PROPERTIES", "CROP_HINTS", "WEB_DETECTION"].

You can also set the field to something like "type": "0" or "type": 0 to point to that specific position on the expected values array, but I think the other option are more descriptive. The 0 value is equivalent to "TYPE_UNESPECIFIED" and it will yield an empty answer too.

This request is working for me:

{
  "requests": [
    {
      "image": {
        "source": {
          "imageUri": "gs://<google_cloud_bucket>/car.png"
        }
      },
      "features": [
        {
          "type": "FACE_DETECTION",
          "maxResults": 100,
          "model": "builtin/stable"
        }
      ],
      "imageContext": null
    }
  ]
}