Search code examples
javaweb-scrapingjsoup

How to get the absolute url of img elements from an unordered list in JSoup


I am trying to extract the absolute urls from the image attributes in the unordered list. But am unable to do so. I only want the image links in the unordered list not the whole site.

<ul class="bjqs">
    <li><img src="images/slider_img/8.jpg" title="Automatically generated caption" /></li>
    <li><img src="images/slider_img/aeration-1.jpg" title="Automatically generated caption" /></li>
    <li><img src="images/slider_img/DesalinationIII-1.jpg" title="Automatically generated caption" /></li>
    <li><img src="images/slider_img/Energy-the-global-view-1.jpg" title="Automatically generated caption" /></li>
    <li><img src="images/slider_img/Exergy-1.jpg" title="Automatically generated caption" /></li>
    <li><img src="images/slider_img/Exergy-2.jpg" title="Automatically generated caption" /></li>
    <li><img src="images/slider_img/Medical-Engg-1.jpg" title="Automatically generated caption" /></li>
    <li><img src="images/slider_img/Phovoltaic-Systems-1.jpg" title="Automatically generated caption" /></li>
    <li><img src="images/slider_img/Phovoltaic-Systems-2.jpg" title="Automatically generated caption" /></li>
    <li><img src="images/slider_img/solar-1.jpg" title="Automatically generated caption" /></li>
    <li><img src="images/slider_img/solar-2.jpg" title="Automatically generated caption" /></li>
    <li><img src="images/slider_img/solar-3.jpg" title="Automatically generated caption" /></li>
    <li><img src="images/slider_img/solar-4.jpg" title="Automatically generated caption" /></li>
    <li><img src="images/slider_img/solar-5.jpg" title="Automatically generated caption" /></li>
    <li><img src="images/slider_img/solar-6.jpg" title="Automatically generated caption" /></li>
    <li><img src="images/slider_img/solar-7.jpg" title="Automatically generated caption" /></li>
</ul>

Below is what I have tried so far. But this only gives me the first link. Not the others.

public static void main(String[] args) throws IOException {
    Document doc = Jsoup.connect("example.org").get();

    Elements ec = doc.select("ul.bjqs");

    for (Element e : ec) {
        System.out.println(e.select("li").select("img").attr("src"));
    }
}

Solution

  • You first selector is wrong. Use this to get a list of all images:

    Elements images = doc.select("ul.bjqs > li > img");
    for (Element e : images) {
        System.out.println(e.attr("src"));
    }
    

    This will print all relative URLs in your HTML. To get the absolute URLs you can use URI.resolve() with your base URL:

    String url = "http://example.org/";
    Document doc = Jsoup.connect(url).get();
    
    URI uri = new URI(url);
    Elements images = doc.select("ul.bjqs > li > img");
    for (Element e : images) {
        String relative = e.attr("src");
        System.out.println(uri.resolve(relative));
    }
    

    This will print the absolute URLs of the images.