How can I get values from Selenium WebElement more than faster?

Question

This is my test code to get value from Selenium WebElement.

import java.util.List;    
import org.apache.commons.lang3.ObjectUtils.Null;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebDriverException;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.firefox.FirefoxDriver;

public class Scan extends WebDriverException {
    private long start = 0;
    private WebDriver driver = null;

    public static void main(String[] args) {

        Scan scan = new Scan();
        scan.driver = new FirefoxDriver();
        scan.driver.get("https://en.wikipedia.org/");
        scan.scanAllElements();
        scan.driver.quit();
    }

    public void scanAllElements() {
        // get all elements
        List<WebElement> elms = driver.findElements(By.xpath("//*"));
        System.out.println("elms size:" + elms.size());

        // start timer
        this.start = System.currentTimeMillis();

        // scan all elements and get some value.
        for (WebElement elm : elms) {
            elm.getTagName();
            elm.getAttribute("class");
            elm.getAttribute("id");
            elm.getAttribute("href");
            elm.getText();
            elm.getSize();
            elm.getLocation();
        }

        // check the time
        stopTimer(elms.size());
    }

    public void stopTimer(int elmsSize) {
        long end = System.currentTimeMillis();
        long ms = end - this.start;
        long sec = ms / 1000;
        long min = sec / 60;

        System.out.println("--- Speed Test ---");
        System.out.println(ms + " ms");
        System.out.println(sec + " s");
        System.out.println(min + " min " + (sec % 60) + " s ");
        System.out.println("1 loop average time:" + (ms / elmsSize) + " ms");

    }

}

The result is this.It takes long time. I'd like to make it fast.

elms size:1031
--- Speed Test ---
123468 ms
123 s
2 min 3 s 
each loop average time:119 ms

What I did.

1.Skip some elements

If the value is not that what I want. It's skip to get another getting values with (continue).

2.filter by xpath

This sample get all elemnts (//*). So I filtered elements when I get it which was good way. But I have still some hundred elements and I need minimise the process time.

3.Multithread

I tested Runnable Callable Stream.

Runnable and Callable solved speed problem. The Process time became to around 40%. But many elements became to null !!

Stream minimise only 10% time and it's also some elements are null.

If you have any idea to get it more than faster, please tell me!!

Answer 1

You can using Javascript, code below will return an ArrayList of Map with tag name, id, href, class keys almost instantly:

ArrayList<Maps> list = (ArrayList) ((JavascriptExecutor) driver).executeScript("return [...document.querySelectorAll(\"*\")].map(e=>{return {tagName:(e.tagName==undefined?null:e.tagName),class:(e.className==undefined?null:e.className),id:(e.id==undefined?null:e.id),href:(e.href==undefined?null:e.href)}})");

All you need is to add js code to get location and size. For text you can use textContent.
Before execute the script be sure page is loaded.