Element not found when executing findElement in Selenium

Question

I'm trying to fill in multiple forms that come after each other, all the forms get filled swiftly with no errors because I make sure to add

WebDriverWait wait = new WebDriverWait(driver, 20);
wait.until(ExpectedConditions.elementToBeClickable(By.xpath("")));

before doing anything on a new page, and I know I'm on the correct page.

On the last form, I encounter this error :

Exception in thread "main" org.openqa.selenium.NoSuchElementException: Unable to locate element: //*[@id="formtovalidate"]/fieldset[1]/div/label/input For documentation on this error, please visit: https://www.seleniumhq.org/exceptions/no_such_element.html

So I went to check on the browser by taking a screenshot and the browser is on the correct page with the correct form, I also checked the xpath values and even tried other attributes.. nothing seemed to work.

So I went ahead and printed out the PageSource which showed a totally different page (not the previous page), I also noticed the this page flashed for a second before the final form appeared.

I also tried driver.navigate().refresh() but that didn't work. I kept searching and looking but nothing appeared. I also changed browsers, that did nothing..

This is the method I'm trying to execute:

private void method() {

WebDriverWait wait = new WebDriverWait(driver, 20);
wait.until(ExpectedConditions.elementToBeClickable(By.xpath("//*[@id=\"formtovalidate\"]/fieldset[1]/div/label/input")));
driver.findElement(By.xpath("//*[@id=\"formtovalidate\"]/fieldset[1]/div/label/input")).sendKeys(email); }

---

Update

Here's the form screenshot:

Here's the execution results:

Code:

String body_text = driver.findElement(By.tagName("body")).getText();
System.out.println(body_text);

Result: The form but in text

Code:

String body_innerHTML = driver.findElement(By.tagName("body")).getAttribute("innerHTML");
System.out.println(body_innerHTML);

Result: A different page :(

<zendesk-ticketing-form base-url="https://www.runescape.com/a=870/c=K0aO9WO69EI" css-cachebust="129" sitekey="6Lcsv3oUAAAAAGFhlKrkRb029OHio098bbeyi_Hv" grecaptcha="" has-valid-session="true" weblogin-url="https://secure.runescape.com/m=weblogin/a=870/c=K0aO9WO69EI/loginform?mod=www&amp;ssl=1&amp;dest=zendesk/support-form?form=360000065898">
<div class="x-display-none ie-error-display" data-js-ie-error="">
    <section class="c-article">
        <div class="c-article__content">
            <h1>Error: Unsupported Browser</h1>
            <p>
                We do not support your web browser. Please use a supported web browser by choosing one below.
                <br>
                <a href="https://www.mozilla.org/firefox/" target="_blank" rel="noopener">FireFox</a>
                <br>
                <a href="https://www.google.com/chrome/" target="_blank" rel="noopener">Chrome</a>
            </p>
        </div>
    </section>
</div>

Code:

 String pagesource = driver.getPageSource();
        System.out.println(pagesource);

Result: Same as the previous one.. different page..

Firefox Page Source: https://pastebin.com/Kv15V2SK

Firefox Inspect Element of the page screenshot: http://prntscr.com/qvi6hc

This is weird, as the page source is different to the form!

Answer 1

I couldn't find time to solve your problem. If you want to do it on your own, please Search this on Google, "Shadow Root, Selenium", I had this kind of error before. What I know is, you cannot directly reach an element that stays inside of a shadow root, This is why you are not getting the source code inside of it.

What you need to do is go through the element step by step:

You have to expand the shadow root,

Here is shadow root expand function:

public static WebElement expand_shadow_element(WebElement element)
    {
        WebElement shadow_root = (WebElement)((JavascriptExecutor)driver).executeScript("return arguments[0].shadowRoot", element);
        return shadow_root;
    }

You can imagine this function like

.switchTo.frame()

for now..

After some researches you will understand the shadow root.

I hope I got the problem right..

Try this function, If you cannot, I will help you later on. Good Luck.