Playwright is free. Your test suite isn’t.
Every Playwright test is code your team writes, reviews and repairs. Pie’s agents explore your web, iOS and Android apps and generate the tests, so those hours go back into your product.
Stay on Playwright if your team wants every test as code in your repo, or a handful of specs covers your app.
Pick Pie for everything else. Your team gets tests across web, iOS and Android without writing a script or repairing a locator.
Pie vs Playwright at a glance
- What it is
- An open-source framework for web testing and browser automation, under the Apache 2.0 license.
- Who writes the tests
- Your team, in TypeScript, Python, .NET or Java.
- What a test is
- Code that finds elements with locators and checks them with assertions.
- Where it runs
- Chromium, Firefox and WebKit, on Windows, macOS and Linux, locally or in CI.
- AI help
- Planner, generator and healer agents that write and repair Playwright tests, plus an MCP server.
- What it is
- An agentic testing platform for web, iOS and Android apps.
- Who writes the tests
- Pie’s agents explore your app and generate them. Your team adds the cases that matter in plain English.
- What a test is
- Plain-English steps and checks. No scripts to write, no locators to break.
- Where it runs
- Web apps in a browser, with other browsers on request. iOS apps on the iOS Simulator, Android apps on the Android Emulator.
- AI help
- The whole product. Agents explore, write and run the tests, and group what they find into issues.
One refund check, written twice.
On the right is a real Pie test from a demo billing app we test with. It opens a partly refunded invoice and follows the refund to its credit memo. On the left is the same check as a Playwright spec, our translation, written with the user-facing locators Playwright’s docs recommend.
import { test, expect } from '@playwright/test';
test('partly refunded invoice links to its credit memo', async ({ page }) => {
await page.goto('/login');
await page.getByLabel('Email').fill(process.env.QA_EMAIL!);
await page.getByLabel('Password').fill(process.env.QA_PASSWORD!);
await page.getByRole('button', { name: 'Sign in' }).click();
await page.getByRole('link', { name: 'Configure' }).first().click();
await page.getByRole('button', { name: 'Payments' }).click();
await page.getByRole('link', { name: 'Invoices' }).click();
await expect(page.getByRole('heading', { name: 'Invoices' }))
.toBeVisible();
await page.getByRole('button', { name: 'Filters' }).click();
await page.getByLabel('Refunded').selectOption('Partly refunded');
await page.getByRole('button', { name: 'Apply filters' }).click();
await expect(page.getByTestId('invoice-status').first())
.toHaveText('Partly refunded');
await page.getByRole('link', { name: 'Open' }).first().click();
await expect(page.getByText('Refund amount')).toBeVisible();
const memos = page.getByRole('region', {
name: 'Credit memos against this invoice',
});
await memos.getByRole('link', { name: 'Open' }).first().click();
await expect(page.getByText('Payout method')).toBeVisible();
await expect(page.getByText('Refund issued')).toBeVisible();
await page.goBack();
}); 17 locators, and every one is code your team maintains.
Pie test- Enter the account email and password, then tap ‘Sign in’.
- Tap ‘Configure’ on any institute from the instance configuration list to enter its management view.
- Tap the ‘Payments’ dropdown in the sidebar and select ‘Invoices’.
- Tap the ‘Filters’ button, select ‘Partly refunded’ from the Refunded dropdown, and tap ‘Apply filters’.
- Tap ‘Open’ on the first invoice record in the filtered list.
- Scroll down to the ‘Credit memos against this invoice’ section and tap ‘Open’ on the associated credit memo.
- Tap the back button or navigation link to return to the customer’s payments overview.
- The Invoices view is displayed upon selection from the Payments menu.
- The invoice list is filtered to show only invoices with a ‘Partly refunded’ status after applying the filter.
- The detailed view of the selected invoice displays the refund amount and a list of associated credit memos.
- The credit memo details show the payout method and the date the refund was issued.
No locators. Seven steps anyone on your team could review.
The spec’s locator stops matching, the run goes red, and someone edits the file or asks the healer agent to. Pie re-reads the screen on each run, so a renamed label isn’t a locator edit for your team.
The Pie test is copied word for word from Pie, “tap” and all. The spec is ours, written to match it step for step.
Four reasons to stop writing specs.
- 01
What it costs to run
The license is free. The bill arrives as your team’s hours. In one study, flaky tests alone took at least 2.5% of a team’s productive time. Pie’s agents write and run your tests, so those hours go back into your product.
Source - 02
Who writes the tests
At Amazon, our CTO Adithya Aggarwal’s teams spent about six months on each test suite, then nobody maintained it and it got scrapped. Nobody likes writing tests, so Pie’s agents explore your app and generate them.
- 03
“But Playwright has agents now”
It does. They save typing, and your team still reviews, owns and maintains every line they write. Pie’s tests are plain-English steps, so you have no test code to maintain.
Source - 04
One place for web, iOS and Android
Playwright stops at the browser. A native app means a second framework and a second suite, the cost nobody prices at the start. Pie tests web, iOS and Android from one place.
Source
Pie takes five rows. Playwright keeps two.
Playwright wins on the license fee and the code you keep. Pie wins everywhere your team spends hours.
| What you’re comparing | Playwright | Pie | Advantage |
|---|---|---|---|
| License cost | Free and open source. | Custom pricing. Contact Pie for a quote. | Playwright |
| Cost to run it | Your team’s hours to write, review and repair specs, plus the CI to run them and a second framework for native apps. | No scripts to write and no locators to break, so your team spends those hours on your product. | Pie |
| Writing a new test | Someone on your team writes a spec in code, or reviews one the generator agent wrote. | Agents generate tests by exploring your app. Anyone on your team can add one in plain English. | Pie |
| When the UI changes | A locator that stops matching fails the run until someone, or the healer agent, fixes it. | No locators to update. Pie re-reads the screen on every run. | Pie |
| Native iOS and Android | Not covered, apart from experimental Android support for Chrome and WebView. Pair it with Appium. | Built in, on the iOS Simulator and the Android Emulator. | Pie |
| Fixing the bug | The healer repairs tests. Fixing the bug in your app stays with your team. | Pie can open a Fix PR with a proposed fix in a repo you connect. Merging it is always your team’s call. | Pie |
| Pull requests | Specs run inside your CI job on every commit. | Pie can test your pull requests on a preview URL or a build, and comments on the PR with a GO / NO-GO verdict. | Both |
| Test code you own | Every test is code in your repo. | Tests live in Pie as plain-English steps your whole team can read and edit. | Playwright |
Two cases where Playwright is the right call
If one of these describes your team, stay on Playwright.
You want every test as code
Playwright keeps every test in your repository, in a language your team already writes, inside your own CI job, with no vendor and no contract. If you want to review each test like the rest of the codebase, Playwright gives you exactly that.
SourceA handful of specs covers your app
A small, stable app with a few critical paths won’t run up much of a bill. Playwright’s free license, parallel runs and Trace Viewer will serve you well until your app outgrows them.
SourceWhich fits your team, Playwright or Pie?
Pick Playwright if
- Your team wants every test as code it reviews like the rest of the codebase.
- Your app is small and stable, and a handful of specs covers it.
Pick Pie to
- Test more of your app without writing another spec.
- Cover web, iOS and Android from one place.
- Give your team back the hours it spends repairing locators.
- Check your pull requests the way your users will use them.
Switch from Playwright one flow at a time
No big rewrite. Your specs keep running until Pie covers the same flow.
- 1
Send Pie the app
Give Pie a URL for your web app, or an iOS simulator build or an Android APK for mobile. If the app has a login, add a test login through a supported path.
- 2
Let Pie explore
Pie explores your app’s reachable flows and generates tests. You get results in hours, not weeks.
- 3
Add your edge cases
Type the cases only your team knows into Pie Chat in plain English. Pie turns them into tests and runs them with the ones it found.
- 4
Retire specs as Pie covers them
Compare Pie’s results with your Playwright runs, and delete each spec once Pie covers the flow. Connect GitHub or GitLab and Pie can test your pull requests too, given a preview URL or a build for each change.
FAQ
Did we miss a question?
Email us directly →
Your team has better things to fix.
Book a demo and bring the flow your suite skips. We’ll show you how Pie would cover it, and which specs you could retire first.
Sources
- Apache 2.0 license; “a framework for Web Testing and Automation”github.com/microsoft/playwright
- “Chromium, WebKit and Firefox on Windows, Linux and macOS, locally or in CI, headless or headed”playwright.dev/docs/intro
- TypeScript, Python, .NET and Java; one API for Chromium, Firefox and WebKitplaywright.dev
- Parallel by default; auto-waiting before actions; retrying assertions; Trace Viewerplaywright.dev/docs/test-parallel · /docs/actionability · /docs/trace-viewer
- Planner, generator and healer test agentsplaywright.dev/docs/test-agents
- “we recommend prioritizing user-facing attributes and explicit contracts”playwright.dev/docs/best-practices
- Codegen records your actions and generates testsplaywright.dev/docs/codegen-intro
- Device emulation; “experimental support for Android automation”playwright.dev/docs/emulation · /docs/api/class-android
- Cost of Flaky Tests in CI, ICST 2024 industry track. Five years of history, about 30 developers, one commercial project; “at least 2.5% of the productive developer time”. Not specific to Playwright.conf.researchr.org/details/icst-2024/icst-2024-industry/1
- World Quality Report 2025-26 (Capgemini, Sogeti, OpenText), Fig 15. 50% cite maintenance burden and flaky scripts as a test automation challenge.capgemini.com/insights/research-library/world-quality-report-2025-26