Search code examples
javascriptapirestimage-processingsearch-engine

Get labels for images using Google Custom Search api

If I drag an image to google and search, I get results of other images that are similar.

I'd like javascript or C++ or C# code to send an arbitrary image and get back the top 10 similar images along with the labels (if any) that are associated with each.

I'm even ok with pre-uploading "my" image so it accessible via url. So I'd like to do:

customsearch.google.com?myurl=123.com/snap.jpg&limit=10

And i'd like to get back:

results:[
    {"a.com/2003nissan.jpg":["nissan","red","2003","car"]},
    {"cd.com/maxima_2001.jpg":["Maxima","2001","dealer","used"]},
    {"b.com/fordf150.jpg":["f-150","truck","2007","driver"]}
]

That is of course the simplified example. I'm fine with wading through all the api and getting the results in any format, but what I need to know is

  • Is this possible?
  • If so, what basic steps do I take (eg, get an api key, load X.js, ?)
  • Any example or tutorial would be greatly appreciated.

Finally, if Bing, or anything else, can do the same, I'm not prejudiced.

Solution

  • While not a webservice (yet) OverFeat is a fully trained deep neural network. It's available for free as a command line app. You just pass it the image, and it tells you the top 5 labels it thinks are related. It was trained on a database of I think 1 million images, and it has 1000 different labels that it chooses from.

    It even has an example of running it with a webcam. I'll be trying it out soon, but I do believe this will give pretty good results.