Search code examples
javascriptpythonhyperlinkurllib2

javascript code bing search - get more than 30 results


I am doing a search over the bing search webpage. Assume I search for the term "Julia Roberts", then the url with the results is roughly the following one:

http://www.bing.com/images/search?q=julia+roberts

As I am downloading the code using python, I use the following source:

import urllib

def geturl(link,my_file):
    try:
        urllib.request.urlretrieve(link, my_file)
    except:
        pass

link = "http://www.bing.com/images/search?q=julia+roberts"
my_file = "my_file.txt"

get_url(link,my_file)

The problem is, such source includes only the first 30 results (or 50).

How do I get the source that includes, say, the 200 first results?


Solution

  • I am not sure you expecting below result , please try in your console window for

    "http://www.bing.com/images/search?q=julia+roberts" for this page.

    JS:

    var a=document.querySelectorAll('div a img');
    
    for(i in a){
    console.log(a[i].src); // get all the image source src path
    //console.count("called"); // it will retrieve 246 image src path 
    }