Search code examples
curlgoogle-search

Is it possible to get Google search result specifying period via curl or wget?

I know how to get google search result via curl like:

curl -sA "Chrome" -L 'http://www.google.com/search?hl=en&q=candy&num=100&start=0' -o abc.html

But, bit curious, I could not get valid search result when I specify period of results.

curl -sA "Chrome" -L "https://www.google.com/search?q=candy&num=100&source=lnt&tbs=cdr%3A1%2Ccd_min%3A1%2F1%2F2010%2Ccd_max%3A12%2F31%2F2010&tbm=" -o /mytmp/out.html

I mean the above command is the similar query when using "search tool" -> "specify 2010/1/1 to 2010/12/31" on a browser like following image. But, via curl, specifying-period feature does not work. I confirmed that the URL in the command works well on a normal browser like Firefox.

How can I get google search result specifying period via curl or wget?

enter image description here

Solution

  • If you view the source code then you'll find that the date is there (and I believe it is part of javascript).

    Jan 1, 2010 \x26#8211; Dec 31, 2010

    When you do it using curl, It is very likely that you are missing the javascript library file (which your browser downloads automatically).

    I have used -H "User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:44.0) Gecko/20100101 Firefox/44.0" with your curl request.