Playwright anti-patterns: 5 flaky code shapes we reject

A flaky Playwright test is usually not Playwright’s fault. It is a test that waits on time instead of on the app, and it does that in one of a few recognisable code shapes. A person writes them under deadline; an AI model writes them just as readily.

TestWatch writes Playwright tests with an AI agent, so it checks every generated test for these 5 shapes before the test is accepted. A test that contains one goes back to the agent with a message that names the line and the fix. Below is each shape: the code the check rejects, its message word for word, and the rewrite it accepts. Every snippet on this page goes through that same check each time we test the site, so the messages are the ones it really returns.

Why these shapes break

Playwright’s web-first assertions, like await expect(locator).toHaveText('Saved'), retry until the page matches or the assertion times out. That retry is what makes a test independent of how fast the machine is. Each shape below gives it up: it sleeps for a fixed time, waits for the wrong signal, reads the page once, never waits at all, or makes a decision from a single read. That is also why such a test passes on your laptop and fails on a slower CI runner, the subject of why Playwright tests pass locally but fail in CI.

Fixed sleeps: waitForTimeout

Two seconds is a guess. On a fast machine the test wastes most of it; on a busy runner the save takes three and the next line fails. Delete the sleep and wait for the result the next step depends on.

1await page.getByRole('button', { name: 'Save' }).click();
2await page.waitForTimeout(2000);
3await expect(page.getByRole('status')).toHaveText('Saved');

1await page.getByRole('button', { name: 'Save' }).click();
2await expect(page.getByRole('status')).toHaveText('Saved');

Waiting for networkidle

networkidle means no network connections for at least 500 ms. A page that polls for updates, sends analytics or holds a websocket never reaches it, and a client-rendered page can reach it before the list is on screen. Playwright’s documentation marks this option as discouraged and says to rely on assertions instead.

1await page.goto('/orders');
2await page.waitForLoadState('networkidle');
3await page.getByRole('link', { name: 'Order 1042' }).click();

1await page.goto('/orders');
2await expect(page.getByRole('heading', { name: 'Orders' })).toBeVisible();
3await page.getByRole('link', { name: 'Order 1042' }).click();

Asserting a snapshot: expect(await locator.textContent())

This reads the text once, at whatever moment the line runs, and compares the copy. If the cart counter updates a moment later, the test fails. The retrying matcher reads the element again until it matches.

1await page.getByRole('button', { name: 'Add to cart' }).click();
2expect(await page.getByTestId('cart-count').textContent()).toBe('1');

1await page.getByRole('button', { name: 'Add to cart' }).click();
2await expect(page.getByTestId('cart-count')).toHaveText('1');

A web-first assertion without await

Web-first assertions are asynchronous, and Playwright’s docs say they must be awaited. Without await, the test moves on, or finishes, before the check has run. It can pass with the success message nowhere on the page.

1await page.getByRole('button', { name: 'Place order' }).click();
2expect(page.getByText('Thanks! Your order has been placed.')).toBeVisible();

1await page.getByRole('button', { name: 'Place order' }).click();
2await expect(page.getByText('Thanks! Your order has been placed.')).toBeVisible();

Branching on isVisible()

isVisible() answers right now, without waiting. If the welcome dialog appears 200 ms after sign-in, the if has already decided there is none and skips the tour. The dialog then sits over the dashboard: a later click it covers waits and times out, while a check that the heading is visible still passes. Whether the test fails depends on timing, not on the app.

A test should know what the app will show. If this account gets the welcome tour, assert it and skip it on every run; if it never does, leave those lines out. When it varies between runs, that is test data to fix, for example a fresh account per run, not a branch to add.

1await page.getByRole('button', { name: 'Sign in' }).click();
2if (await page.getByRole('dialog', { name: 'Welcome' }).isVisible()) {
3  await page.getByRole('button', { name: 'Skip tour' }).click();
4}
5await expect(page.getByRole('heading', { name: 'Dashboard' })).toBeVisible();

1await page.getByRole('button', { name: 'Sign in' }).click();
2await expect(page.getByRole('dialog', { name: 'Welcome' })).toBeVisible();
3await page.getByRole('button', { name: 'Skip tour' }).click();
4await expect(page.getByRole('heading', { name: 'Dashboard' })).toBeVisible();

Popups that aren’t part of your app are a different case. For a third-party cookie banner or chat widget, Playwright’s page.addLocatorHandler (version 1.42 and later) clicks it away whenever it gets in the way. In TestWatch the test fixture already blocks or dismisses those, so a generated test never contains dismissal code, and our check rejects every branch on a one-shot state read, even one right after waiting for either outcome.

What the check allows

A rule that flags too much gets ignored, so the check leaves these alone:

Retrying a computed value. expect.poll re-runs the function until the value matches, so reading a count inside it is fine:

1await expect.poll(async () => page.getByRole('row').count()).toBeGreaterThan(2);

A longer timeout on one step. A report that takes a minute to build may need more than the default budget. An explicit timeout on that assertion is fine; a fixed sleep before it isn’t:

1await expect(page.getByText('Report ready')).toBeVisible({ timeout: 60_000 });

A count as a loop bound. Reading how many rows there are and visiting each is iteration, not a decision:

1const n = await page.getByRole('row').count();
2for (let i = 0; i < n; i++) {
3  await expect(page.getByRole('row').nth(i)).toBeVisible();
4}

Does it make a difference?

In our own eval, a test counts as brittle when it was accepted and catches the planted bugs but fails on the unchanged app, breaks on a harmless change, or passes only sometimes. Counted with today’s rules, our earliest runs (September 2026) had 7% brittle tests over 58 trials; the run on 18 October 2026 had 0% over 87. This check is one of several changes made in between, so that shows a direction, not what caused it. The cases and every trial are on How we measure TestWatch.

What this doesn’t catch

The check reads code; it doesn’t run it. It won’t notice a locator that matches the wrong element, a test that depends on data another test left behind, or an app that is genuinely slow. And it can’t tell you whether the assertion checks the right thing, which is the bigger question for a generated test: our AI Playwright test generator page covers the other checks a test has to pass before we accept it.

Questions