Writing Playwright End-to-End Tests With LLMs in October 2026: Show the Model the Real Page, Ban Fixed Sleeps, and Prove Every Test Can Fail

Ask a chat model to "write a Playwright test for our login page" and you will get something that looks finished in a few seconds. It will have a test() block, a few page.click() calls, and an assertion at the end. Run it a few times in CI and the problems show up: selectors that match the wrong element, hard-coded sleeps that fail on slow runners, and assertions that pass even when the feature is broken.

End-to-end (E2E) tests drive a real browser through your real app, so they catch problems that unit tests can't. They are also the most expensive tests to keep healthy. An LLM can save you a lot of typing here, but only if you give it the right inputs and hold its output to a few firm rules. This guide walks through a workflow that works with any current model and with Playwright's TypeScript API. The same ideas carry over to Cypress or Selenium.

Why E2E tests are hard for a model to get right

A model writing a unit test can see the function it is testing. A model writing an E2E test usually can't see your page. It guesses at button labels, form field names, URLs, and how long things take to load. Every guess is a place the test can go wrong. Three failure patterns come up again and again:

Brittle selectors. The model invents CSS classes like .login-btn or long XPath chains that break the next time a designer touches the markup.
Fixed waits. Calls like page.waitForTimeout(3000) make tests slow when the app is fast and flaky when it's slow.
Weak assertions. A test that only checks the URL changed, or that a page "has some text," can pass while the feature is broken.

The fix for all three is the same: give the model real information about your page, and tell it exactly which patterns are allowed.

Step 1: Give the model the real page, not a description

The best input is the page's accessibility tree, because that's what Playwright's recommended locators are built on. You can get a text snapshot of it from a quick script:

import { chromium } from 'playwright';

const browser = await chromium.launch();
const page = await browser.newPage();
await page.goto('http://localhost:3000/login');
console.log(await page.locator('body').ariaSnapshot());
await browser.close();

The output is a YAML-like list of roles and accessible names, such as - textbox "Email" and - button "Sign in". Paste that into the prompt. If you can't run a script, paste the rendered HTML of the relevant section instead (from your browser's developer tools, not the source template, since the template may not match what users see).

Playwright also ships a code generator, npx playwright codegen http://localhost:3000, that records your clicks and writes matching code. A good pattern is to record the happy path with codegen, then hand the recording to the model and ask it to clean it up, add assertions, and cover edge cases. That way the selectors come from the real page instead of the model's imagination.

Step 2: State the rules in the prompt

Models follow explicit constraints far better than implied ones. A short rules block at the top of the prompt does most of the work:

Write Playwright tests in TypeScript using @playwright/test.
Rules:
- Locate elements with getByRole, getByLabel, getByText, or getByTestId only.
  No CSS classes, no XPath, no nth-child.
- Never use waitForTimeout or any fixed sleep.
- Use web-first assertions (await expect(locator).toBeVisible(), toHaveText,
  toHaveURL) so Playwright retries until the condition holds.
- Each test must be independent: no test may rely on state left by another.
- Every test must assert the user-visible outcome, not just that a click happened.
- Only use element names that appear in the accessibility snapshot below.
  If something you need is missing, say so instead of inventing it.

The last rule matters most. Telling the model it may say "I can't find a confirmation message in the snapshot" gives it a way out other than making one up.

Step 3: Describe the behavior, including what should fail

Write the scenario the way a tester would, with the expected result for each case. For a login page, that might be:

Valid email and password: user lands on /dashboard and sees a heading with their name.
Wrong password: user stays on /login and sees the error "Incorrect email or password."
Empty email: the form doesn't submit and the email field shows a validation message.

Giving the exact expected text lets the model write precise assertions. Without it, you'll get vague checks like expect(page.locator('body')).toContainText('error'), which passes for almost any page that mentions errors.

Step 4: Review the output against a checklist

Here is the kind of test you want back:

import { test, expect } from '@playwright/test';

test('wrong password shows an error and stays on login', async ({ page }) => {
  await page.goto('/login');
  await page.getByLabel('Email').fill('test-user@example.com');
  await page.getByLabel('Password').fill('not-the-password');
  await page.getByRole('button', { name: 'Sign in' }).click();

  await expect(page.getByRole('alert')).toHaveText('Incorrect email or password.');
  await expect(page).toHaveURL(/\/login$/);
});

Before you commit anything the model wrote, read it with these questions in mind:

Does every locator use a role, label, visible text, or test ID that actually exists on the page?
Is there any waitForTimeout, setTimeout, or retry loop the model wrote by hand? Remove it and use a web-first assertion.
Does each test end with an assertion about what the user sees, not just a navigation?
Are credentials, API keys, or real customer emails hard-coded? Move them to environment variables or test fixtures.
Do any tests depend on running in a certain order? Each should set up its own data.

A test only earns trust once you've seen it catch a real failure. Image: Crash test dummies.jpg by National Institute of Standards and Technology, Public domain, via Wikimedia Commons.

Step 5: Prove each test can fail

A test that never fails isn't protecting anything. For each new test, break the behavior it checks and confirm the test goes red. Change the expected error text, comment out the redirect, or point the test at the wrong account. If the test still passes, its assertion is too weak. This takes a few minutes per test and catches the most common LLM mistake: an assertion that looks right but checks the wrong thing.

Step 6: Run it several times before you trust it

Flaky tests often pass the first time. Run new tests repeatedly, in parallel, before merging:

npx playwright test login.spec.ts --repeat-each=10 --workers=4

If any run fails, open the trace (npx playwright show-trace on the saved trace file, with tracing turned on in your config) to see exactly what the page looked like at each step. Paste the failing step and the error into the model and ask for a cause, not a patch. A common bad "fix" from a model is to add a longer timeout. Push back and ask what the test is actually waiting for.

Step 7: Handle setup with fixtures, not UI clicks

Models like to log in through the UI at the start of every test. That works, but it's slow and makes every test depend on the login page. Ask the model to use Playwright's storage-state pattern instead: log in once in a setup project, save the session to a file, and load it in the tests that need an authenticated user. Similarly, create test data through your API or a seed script rather than by clicking through admin screens. Be specific in the prompt about which API endpoints exist; otherwise the model will invent them.

An end-to-end test is a dress rehearsal: the whole show, start to finish, before the audience arrives. Image: Dress rehearsal scene from Act II of "Miss Nobody from Starland" by Silver and Friedman, Public domain, via Wikimedia Commons.

Step 8: Keep tests readable for the next person

Once you have a handful of tests for the same area, ask the model to refactor repeated steps into small helper functions or page objects. Review that refactor the same way: run the full suite before and after, and confirm the same tests pass and fail. Keep test names in plain language that says what the user does and what should happen, because those names are what you'll read in a CI failure report.

A prompt template you can reuse

Context: [app name], running at http://localhost:3000, Playwright with TypeScript.
Accessibility snapshot of the page:
[paste ariaSnapshot output]

Scenarios and expected results:
1. ...
2. ...

Rules: [paste the rules block from Step 2]

Output: one spec file. After the code, list any element or behavior you
needed but could not find in the snapshot.

The short version

An LLM is a fast way to draft E2E tests, but it can't see your app unless you show it. Feed it the real accessibility tree or a codegen recording, require role-based locators and web-first assertions, ban fixed sleeps, and spell out the expected result of each scenario. Then do the two checks the model can't do for you: break the feature to prove each test fails, and run the suite many times to prove it doesn't fail by accident. Tests that survive both are worth keeping.

Search This Blog

Technology On the Net