Google Lens APIs/SDK

Question

I was going through the Google Cloud Vision API to search for related products in the image. However, I found that I need to first create a product set (and upload the images) and then search on the set that I've created.

Is there any API via which I can provide an image (without creating any product set) and search google (or any specific website like amazon) for the related products (and their eCommerce links) in the image? Basically replicating what google lens does for its app. If not, is there any google lens mobile SDK for the same use-case, ie finding the related products (and their eCommerce URI) provided an image??

Update 1

I researched a bit more on this, and found Web Detection API. This does serve the purpose somewhat. However, I cant filter the results from a specific website (for eg:- Amazon). Is there a way to filter results from a specific domain?

Answer 1

With Vision API, you can detect from the web properties, faces, and text in the images.

An option is to use filtering to return more specific results when listing resources, evaluations, or operations. Using ?filter="[filter_name=]filter-value returns all JSON objects that match the filter-value. You can see more information in this link.

Another option is to filter before showing the results. You can see this example, which returns links from the Web and cloud storage that contain certain colors in the images.

You can see this example:

def detect_properties_uri(uri):
    """Detects image properties in the file located in Google Cloud Storage or
    on the Web."""
    from google.cloud import vision
    client = vision.ImageAnnotatorClient()
    image = vision.Image()
    image.source.image_uri = uri
 
    response = client.image_properties(image=image)
    props = response.image_properties_annotation
    print('Properties:')
 
    for color in props.dominant_colors.colors:
        print('frac: {}'.format(color.pixel_fraction))
        print('\tr: {}'.format(color.color.red))
        print('\tg: {}'.format(color.color.green))
        print('\tb: {}'.format(color.color.blue))
        print('\ta: {}'.format(color.color.alpha))
 
    if response.error.message:
        raise Exception(
            '{}\nFor more info on error messages, check: '
            'https://cloud.google.com/apis/design/errors'.format(
                response.error.message))
# [END vision_image_property_detection_gcs]

The next step is to iterate through the results like this example and filter the links that contain the domain. Looking at the domain inside the web links.

for entity in web_detection.web_entities:
    if (str(entity.description).find(domain) >-1)
        print(entity.description)

In this case, you will iterate each result and filter trying to find the domain inside the description and display it on the screen.

Another option would be to manipulate the result in JSON format. In that chase, you need to:

• Parse JSON
• Filter the JSON

You can see this example:

import json
 
input_json = """
[
    {
        "type": "1",
        "name": "name 1"
    },
    {
        "type": "2",
        "name": "name 2"
    },
    {
        "type": "1",
        "name": "name 3"
    }
]"""
 
# Transform json input to python objects
input_dict = json.loads(input_json)
 
# Filter python objects with list comprehensions
output_dict = [x for x in input_dict if x['type'] == '1']
 
# Transform python object back into json
output_json = json.dumps(output_dict)
 
# Show json
print output_json