Search code examples
javahtmljsoup

How to grab this specific element with JSoup?

I am trying to webscrape basic info about users on this website. All I want to do is grab the user's name and their respective reputation. Below is a copy of the page's source:

    /**Update information about latest job in String array LatestJobInfo.*/
    public void getLatestInfo() {
        Document latest_job = getJob(getLatestID());
        try {

            //Find client
            Elements clientname_fetch = latest_job.select("div.item-listing__username.mt-0.mb-0");
            System.out.println(clientname_fetch);


        } catch (Exception e) {
            e.printStackTrace();
        }

    }

https://pastebin.com/dtxvHBEh

I am attempting to scrape the information from lines 288 and 292. I have tried the following and literally got nothing. I think I need to access these elements through their parents first, but I am having trouble doing that. If someone can give me a quick guide on the syntax of grabbing deep elements like these, that would be perfect.

Here is what I have tried: Elements clientname_fetch = latest_job.select("div.item-listing__username.mt-0.mb-0"); Elements clientname_fetch = latest_job.select("h5");

Solution

  • The first selector should be .item-listing__username and the second one is .item-listing__avatar__rep.
    In order to get them, I've opened my browser's dev tools (F12), selected the inspector tool (circled with #1), moved the cursor to the target element and clicked it (#2, at this point I could not choose the exact element but it was not a problem). The browser highlighted the line that contains that element (#3) and from there I've navigated down until I've found the username and score -> right click -> copy css selector and that's it.
    enter image description here