I'm trying to get values from a table in a webpage that has only divelements. No tr td. Below is an example (not the real one) how the HTML look like.
<div class="table" style="style="transform: translate3d(0px, 0px, 0px); opacity: 1;">
<div class="row heading">
<div class="row" style="opacity: 1;">
<div class="cell">
<div class="row1">value1</div>
<span class="Tooltip" style="style="position: relative;">...</span>
</div>
<div class="cell1">
<div class="classname"></div>
<div class="row1">value2</div>
</div>
<div class="cell1">
<div class="classname"></div>
<div class="row1">value3</div>
</div>
<div class="cell1">
<div class="classname"></div>
<div class="row1">value4</div>
</div>
<div class="cell1">
<div class="classname"></div>
<div class="row1">value5</div>
</div>The table has 50 rows in it 5 columns each. I tried 2 approaches to get all the values from the table.
Using CSSSelector
for(int i=2;i<=38;i++) {
WebDriverWait wait = new WebDriverWait(driver, 60);
wait.until(ExpectedConditions.visibilityOfElementLocated(By.cssSelector("#root > div > div.Home > div.home-left > div.table > div:nth-child("+i+") > div:nth-child(1) > div")));
String valueone = driver.findElement(By.cssSelector("#root > div > div.Home > div.home-left > div.table > div:nth-child("+i+") > div:nth-child(1) > div")).getText();
clist.add(valueone);
for(int j=2;j<=5;j++) {
val = driver.findElement(By.cssSelector("#root > div > div.Home > div.home-left > div.table > div:nth-child("+i+") > div:nth-child("+j+") > div.total")).getText();
clist.add(val);
}
}
Using Xpath:
for(int i=2;i<=38;i++) {
String c;
WebDriverWait wait = new WebDriverWait(driver, 30);
wait.until(ExpectedConditions.visibilityOfElementLocated(By.xpath("//*[@id=\"root\"]/div/div[3]/div[1]/div[5]/div["+i+"]/div[1]/div")));
valueone = driver.findElement(By.xpath("//*[@id=\"root\"]/div/div[3]/div[1]/div[5]/div["+i+"]/div[1]/div")).getText();
clist.add(valueone);
}
for(int j=2;j<=38;j++) {
c = driver.findElement(By.xpath("//*[@id=\"root\"]/div/div[3]/div[1]/div[5]/div["+j+"]/div[2]/div[2]")).getText();
clist.add(c);
}
for(int k=2;k<=38;k++) {
c = driver.findElement(By.xpath("//*[@id=\"root\"]/div/div[3]/div[1]/div[5]/div["+k+"]/div[3]/div")).getText();
clist.add(c);
}
for(int l=2;l<=38;l++) {
c = driver.findElement(By.xpath("//*[@id=\"root\"]/div/div[3]/div[1]/div[5]/div["+l+"]/div[4]/div[2]")).getText();
clist.add(c);
}
Issue: When I try to iterate using css selector, I get this exception (couldn't find 12th row) -
Exception in thread "main" org.openqa.selenium.TimeoutException: Expected condition failed: waiting for visibility of element located by By.cssSelector: #root > div > div.Home > div.home-left > div.table > div:nth-child(12) > div:nth-child(1) > div (tried for 30 second(s) with 500 milliseconds interval)
at org.openqa.selenium.support.ui.WebDriverWait.timeoutException(WebDriverWait.java:95)
at org.openqa.selenium.support.ui.FluentWait.until(FluentWait.java:272)
at App.GetData.main(GetData.java:46)
Caused by: org.openqa.selenium.NoSuchElementException: no such element: Unable to locate element: {"method":"css selector","selector":"#root > div > div.Home > div.home-left > div.table > div:nth-child(12) > div:nth-child(1) > div"}
And when I try to iterate using X Path, I get this exception (couldn't find 13th row) -
Exception in thread "main" org.openqa.selenium.TimeoutException: Expected condition failed: waiting for visibility of element located by By.xpath: //*[@id="root"]/div/div[3]/div[1]/div[5]/div[13]/div[1]/div (tried for 30 second(s) with 500 milliseconds interval)
at org.openqa.selenium.support.ui.WebDriverWait.timeoutException(WebDriverWait.java:95)
at org.openqa.selenium.support.ui.FluentWait.until(FluentWait.java:272)
at App.GetData.main(GetData.java:72)
Caused by: org.openqa.selenium.NoSuchElementException: no such element: Unable to locate element: {"method":"xpath","selector":"//*[@id="root"]/div/div[3]/div[1]/div[5]/div[13]/div[1]/div"}
I tried ImplicitWait, ExplicitWait and even Thread.sleep (which I know is not recommended). Nothing seems to be working. Irony is if I try 20 times, Selenium identifies all the elements without issue in 1 try.
Why is that css selector is finding hard to find an element and xpath is finds the element but finds hard to find another element?
I have faced this issue often in Selenium, when it is able to find the same element without issue on one run and struggles to find it on other runs? Is there a permanent solution/approach for this issue?
Found out what is causing the issue, finally! I was playing around with the HTML and found out that, upon initial loading only the initial 12 rows are getting loaded (even though all the rows are displayed) and only upon further scrolling, rest of the rows are getting loaded. So did the following before traversing through the rows and now all the rows are now getting identified both by cssSelector and Xpath.
((JavascriptExecutor) driver).executeScript("window.scrollTo(0, document.body.scrollHeight)");