Search code examples
rgoogle-search-api

Why is the Same Google Search API Producing Different Results in R vs Browser

Sumbitting the exact same Google Search API query in the browser (Chrome) and in R returns a different number of results. What could be the reason for this? The only obvious difference is that I am submitting the query in the browser from my computer (UK based) while R results come from a GCE VM based in NL. Can this be the cause even though I have specified the country for the search in the query string?

# Pasted in the browser address bar
https://www.googleapis.com/customsearch/v1?q=%22KALLIGIANNIS%22%20Rethymno&num=10&lr=lang_en&cx=SSS&gl=gr&cr=countryGR&dateRestrict=date:r:20150831:20170831&key=XXX&alt=json

# Get request in R
httr::GET('https://www.googleapis.com/customsearch/v1?q=%22KALLIGIANNIS%22%20Rethymno&num=10&lr=lang_en&cx=SSS&gl=gr&cr=countryGR&dateRestrict=date:r:20150831:20170831&key=XXX&alt=json')

The results in the browser show:

"searchInformation": {
  "searchTime": 0.133114,
  "formattedSearchTime": "0.13",
  "totalResults": "109",
  "formattedTotalResults": "109"

The results in R

oneresult <- GET('https://www.googleapis.com/customsearch/v1?q=%22KALLIGIANNIS%22%20Rethymno&num=10&lr=lang_en&cx=SSS&gl=gr&cr=countryGR&dateRestrict=date:r:20150831:20170831&key=XXX&alt=json')

content(oneresult)[[5]]

$searchTime
[1] 0.584238

$formattedSearchTime
[1] "0.58"

$totalResults
[1] "59"

$formattedTotalResults
[1] "59"
Solution

  • The google search algorithm is a black box. It yields different results depending on geo-location, and additional parameters, not all of them are known.

    For example, using the browser (not via googleapis) in regular mode versus incognito might also yield different results.

    My guess is that you are right in your hypothesis (that the difference is caused by the location the search is originating from).