Search code examples
javascriptplaywrightflakyness

What is flaky in Playwright locator DOM state getters?

We are using Chai JS assertions with Playwright, e.g.

await page.locator('input.checkbox').check();
assert.isTrue(await page.locator('button.submit').isEnabled());

However, locator.isEnabled() documentation says, that:

If you need to assert that an element is disabled, prefer expect(locator).toBeEnabled() to avoid flakiness. ...

Examining the documentation, I would say the only difference between locator.isEnabled() and expect(locator).toBeEnabled() is the default timeout only (but it can be changed). So, it seems to me, that using e.g.

assert.isTrue(await page.locator('button.submit').isEnabled({timeout: 5000});
await expect(page.locator('button.submit')).toBeEnabled({timeout: 5000});

Should be the same. Or what's different here?

Solution

  • You shouldn't use chai or any other manual assertions. Instead, use ones from Playwright, which will retry until the assertion passes or the assertion timeout is reached.

    await expect(page.locator("button.submit")).toBeEnabled();

    More info - See https://playwright.dev/docs/best-practices#use-web-first-assertions

    // 👍
await expect(page.getByText('welcome')).toBeVisible();
// 👎
expect(await page.getByText('welcome').isVisible()).toBe(true);

    Docs explicitly say that Playwright await expect(locator).toBeEnabled() will work with the locator and retry assertion/checking locator state instead of checking the single value of isEnabled().

    If it's still unclear - page.locator('button.submit').isEnabled() checks that the button is enabled at a specific time when your test code reaches it. And because your web app might not be loaded/browser might not render it as fast as the test code reaches it - you have a flaky test.

    Imagine there's an element that doesn't exist. <h10>

    console.log(await page.locator("h10").isEnabled({ timeout: 7000 }));

    In this case, Playwright will try to find this element for 7s only after throwing an error. It has nothing related to the element state—enabled/disabled/etc.

    But if we talk about elements that exist - isEnabled will return the current state when the code is executed. So if it's action by that time - it won't change even if timeout is set.

    console.log(await page.locator("button").isEnabled({ timeout: 7000 }));

    This is why you need to use native Playwright asserts to recheck the state of locators.