I'm trying to use a model based test approach, integrating xstate with playwright. Our system uses a workflow where 2 users can log into the system and respond to each others actions to progress through the workflow.
When I try to log out and back in again using the same browser, the browser logs back in as the first user, as if it was unable to clear that authenticated session when I logged out and defaulted back to it even though I entered a different user's details
Adding an afterEach that fully closes the browser between tests seemed to be a solution to this for single login tests, as this would clear all cookies and I was able to get through to my second test, logged in as the second user, however this quickly failed again, as I was not closing the browser in between different logins between individual test steps.
The problem is I can't find a way to be able to control/reference the browser or open multiple browsers in the test model, as the browser instance is created later in the test hooks. I would like to find a way to be able to close and re open the browser window in between steps, ideally incorporating this into my log in/logout functions OR maybe i'm missing some step when I logout to clear the session properly and then log back in with a new user
Below are my hooks and then what xstate is using to run the tests and is at the bottom of my script
test.beforeEach(async ({ browser }) => {
page = await browser.newPage();
await page.goto(URL);
testNum++
});
test.afterEach(async ({browser}) => {
for(let context of browser.contexts())
await context.close()
});
test.describe('CE workflows', () => {
const testPlans = testModel.getSimplePathPlans();
testPlans.forEach((plan, i) => {
plan.paths.forEach((path, i) => {
test(path.description, async ({}) => {
await path.test(page);
});
});
});
})
These are the login/logout functions I bookend each test step with
async function login(email) {
await page.goto(URL);
await page.getByTestId('loginEmail').fill(email);
await page.getByTestId('loginPassword').fill('password');
await page.getByTestId('btnLogin').click();
if (await page.getByRole('button', { name: 'Allow cookies' }).isVisible())
await page.getByRole('button', { name: 'No thanks' }).click();
}
async function logout() {
await page.getByTestId('btnOptions').click();
await page.getByRole('button', { name: 'Logout' }).click();
}
This is the first edge in my model Each step is similair to this in that it logs in, carries out the users action, then logs out, ready to be passed to the second user
const testModel = createModel(CEWorkflowMachine, {
events: {
CREATE_CE_CLIENT: async () => {
await login()
await page.goto(URL);
await page.getByRole('link', { name: 'Create compensation event' }).click();
await page.getByLabel('Title').fill(`Test for model based CE ${testNum}`);
await page.getByLabel('Description').fill('Test Description');
await page.getByLabel('Project manager\'s assumptions').fill('Test Assumption');
await page.getByText('The Project Manager gives an').click();
await page.getByText('Notify').click();
await logout()
},
There are few flags here which one of or multiple are likely leading to this behaviour. Ultimately, if it's logging in as the previous user, you are having issues with test isolation or some async/order of execution issue.
Confused test isolation
You currently do this to manage browser pages:
test.beforeEach(async ({ browser }) => {
page = await browser.newPage();
await page.goto(URL);
testNum++
})
This implies page is a variable available in scope. If it is not defined by let page, the situation would be worse as that would mean it's actually completely global.
Even if not global, it's asking for trouble to keep around a mutable reference. You'll have problems if you have in-file parallelisation or want to add that later. Add in that you can accidentally reference this var; or close over it with a closure; and factor in that lots of things are async and you can conclude this is highly risky and hellish to manage. Doing this is highly unusual and you won't see it in any official docs.
Whilst it's possible to have this and perfectly manage it, you are opening yourself up to a whole category of bugs that are now possible. page is attached to a browser context, and this is the construct playwright offers to isolate tests. If you have any chance of reusing it between tests at all there is a problem.
I strongly suspect this is probable in your case. The state machines are "closing over" the page variable you have setup. They may well still be referencing that old page object reference even if you reassign page.
You should scrap this widely scoped page var completely. Instead, make sure you are using page as passed to the test handlers/hooks:
test.beforeEach(async ({ page }) => {
await page.goto(URL);
});
test.describe('CE workflows', () => {
const testPlans = testModel.getSimplePathPlans();
testPlans.forEach((plan, i) => {
plan.paths.forEach((path, i) => {
test(path.description, async ({page}) => {
await path.test(page);
});
});
});
})`
You shouldn't need the manual browser closing if your tests are correctly isolated. So I removed that. The page provided to the test methods (and hooks) had already been scoped to the relevant test automatically. Ensure this is passed through to any shared methods and you are not using a global or widely scoped page anywhere.
That means your state machines will need to be provided page. You need to actually use the page that was passed to the machine (which now is passed the right one from test()). Note this is how it's done in the official docs.
const testModel = createModel(CEWorkflowMachine, {
events: {
CREATE_CE_CLIENT: async (page) => {
await login(page)
await page.goto(URL);
await page.getByRole('link', { name: 'Create compensation event' }).click();
await page.getByLabel('Title').fill(`Test for model based CE ${testNum}`);
await page.getByLabel('Description').fill('Test Description');
await page.getByLabel('Project manager\'s assumptions').fill('Test Assumption');
await page.getByText('The Project Manager gives an').click();
await page.getByText('Notify').click();
await logout(page);
}
And of course those login/logout methods need to accept page as a parameter and use that.
No need to call browser.newPage since the one provided to the tests/hooks is already a fresh one.
Logout not waiting
When you await a click() on the logout and login buttons, that will resolve as soon as the button is clicked. It won't automatically wait until the next page.
Your logon case is OK, because afterwards it of course waits to interact with the logged on page.
But for logout, depending on if the flow is complex or long, this means it clicks the logout button and immediately ends the test and kills the browser context. It's possible the logout did not have time to complete.
You can fix this by instructing Playwright to wait.
await Promise.all([
page.waitForNavigation(),
page.getByRole('button', { name: 'Logout' }).click()
])
However, theoretically this should not matter, since the browser contexts should be different between tests. But if there's leakage, this could compound the issue.