Search code examples
javahtmljsoup

How to get links from HTML, correct usage of `doc.select`


I would like to get links from the HTML code. Code from this page: https://www.valorebooks.com/books/fiction/fantasy

 <div class="sub_bar sub_bar_no_pointer"> 
         <span class="showing">Showing 1 - 50 of 28705 - Browse More Fantasy Books for Sale</span> 
         <div class="paginator" id="pg"> 
          <a href="/books/fiction/fantasy" class="active">1</a> 
          <a href="/books/fiction/fantasy?page=2">2</a> 
          <a href="/books/fiction/fantasy?page=3">3</a> 
          <a href="/books/fiction/fantasy?page=4">4</a> 
          <a href="/books/fiction/fantasy?page=5">5</a> 
          <span class="paginatorText">...</span> 
          <a href="/books/fiction/fantasy?page=575">575</a> 
          <span class="paginatorText">|</span> 
          <a href="/books/fiction/fantasy?page=2" class="spriteButton arrow     next icon-right-open"></a>
         </div> 
        </div> 

I've found an example how to get the links, but I'm not sure what should I write in doc.select("div.paginator");. Is it correct or not, maybe I should write this in another way.

    Elements myLink = doc.select("div.paginator");      

    int number = 0;
    for (Element links : myLink) {
        Elements a = myLink.select("a[href]");                      // get links
        number = Integer.parseInt(a.get(a.size() - 2).text());
    }
    for (int i = 0; i < 20; i++) {
        getData(url + i);
    }
}

Solution

  • It's not clear what you're trying to achieve.

    To print a string of all the links, you might do this:

    System.out.println(doc.select("div.paginator").select("a[href]").toString());
    

    For an Elements array of the links:

    Elements theLinks = doc.select("div.paginator").select("a[href]");