Search code examples
seleniumselenium-webdriverweb-scrapingxpathwebdriverwait

How to get the header info of the first news on the page using Selenium


I am trying to navigate to each of the links and get their first news on the page.but can't get a unique xpath for the header info across all pages

Refer the attached pic, it shows 2 elements. how can i make it as unique to fetch only the header / title news. enter image description here


Solution

  • There are many headers on that page and they are changing. I mean some news are coming and being removed from this page.
    If you want to get all the news title texts you can get all the headers elements and then iterate over them and extract their texts.
    Something like this:

    List<WebElement> headers = driver.findElements(By.xpath("//h3[@class='gs-c-promo-heading__title gel-pica-bold nw-o-link-split__text']"));
    for (WebElement header : headers){    
        ((JavascriptExecutor) driver).executeScript("arguments[0].scrollIntoView(true);", header);
        System.out.println(header.getText());
    }