Playwright is free. Your test suite isn’t.

Every Playwright test is code your team writes, reviews and repairs. Pie’s agents explore your web, iOS and Android apps and generate the tests, so those hours go back into your product.

Playwright

Stay on Playwright if your team wants every test as code in your repo, or a handful of specs covers your app.

Pie

Pick Pie for everything else. Your team gets tests across web, iOS and Android without writing a script or repairing a locator.

The basics

Pie vs Playwright at a glance

PlaywrightOpen-source framework
What it is
An open-source framework for web testing and browser automation, under the Apache 2.0 license.
Who writes the tests
Your team, in TypeScript, Python, .NET or Java.
What a test is
Code that finds elements with locators and checks them with assertions.
Where it runs
Chromium, Firefox and WebKit, on Windows, macOS and Linux, locally or in CI.
AI help
Planner, generator and healer agents that write and repair Playwright tests, plus an MCP server.
PieAgentic testing platform
What it is
An agentic testing platform for web, iOS and Android apps.
Who writes the tests
Pie’s agents explore your app and generate them. Your team adds the cases that matter in plain English.
What a test is
Plain-English steps and checks. No scripts to write, no locators to break.
Where it runs
Web apps in a browser, with other browsers on request. iOS apps on the iOS Simulator, Android apps on the Android Emulator.
AI help
The whole product. Agents explore, write and run the tests, and group what they find into issues.
The same test, both ways

One refund check, written twice.

On the right is a real Pie test from a demo billing app we test with. It opens a partly refunded invoice and follows the refund to its credit memo. On the left is the same check as a Playwright spec, our translation, written with the user-facing locators Playwright’s docs recommend.

Playwright specrefund-audit.spec.ts
import { test, expect } from '@playwright/test';

test('partly refunded invoice links to its credit memo', async ({ page }) => {
  await page.goto('/login');
  await page.getByLabel('Email').fill(process.env.QA_EMAIL!);
  await page.getByLabel('Password').fill(process.env.QA_PASSWORD!);
  await page.getByRole('button', { name: 'Sign in' }).click();

  await page.getByRole('link', { name: 'Configure' }).first().click();
  await page.getByRole('button', { name: 'Payments' }).click();
  await page.getByRole('link', { name: 'Invoices' }).click();
  await expect(page.getByRole('heading', { name: 'Invoices' }))
    .toBeVisible();

  await page.getByRole('button', { name: 'Filters' }).click();
  await page.getByLabel('Refunded').selectOption('Partly refunded');
  await page.getByRole('button', { name: 'Apply filters' }).click();
  await expect(page.getByTestId('invoice-status').first())
    .toHaveText('Partly refunded');

  await page.getByRole('link', { name: 'Open' }).first().click();
  await expect(page.getByText('Refund amount')).toBeVisible();

  const memos = page.getByRole('region', {
    name: 'Credit memos against this invoice',
  });
  await memos.getByRole('link', { name: 'Open' }).first().click();
  await expect(page.getByText('Payout method')).toBeVisible();
  await expect(page.getByText('Refund issued')).toBeVisible();
  await page.goBack();
});

17 locators, and every one is code your team maintains.

Pie test
Steps
  1. Enter the account email and password, then tap ‘Sign in’.
  2. Tap ‘Configure’ on any institute from the instance configuration list to enter its management view.
  3. Tap the ‘Payments’ dropdown in the sidebar and select ‘Invoices’.
  4. Tap the ‘Filters’ button, select ‘Partly refunded’ from the Refunded dropdown, and tap ‘Apply filters’.
  5. Tap ‘Open’ on the first invoice record in the filtered list.
  6. Scroll down to the ‘Credit memos against this invoice’ section and tap ‘Open’ on the associated credit memo.
  7. Tap the back button or navigation link to return to the customer’s payments overview.
Checks
  • The Invoices view is displayed upon selection from the Payments menu.
  • The invoice list is filtered to show only invoices with a ‘Partly refunded’ status after applying the filter.
  • The detailed view of the selected invoice displays the refund amount and a list of associated credit memos.
  • The credit memo details show the payout method and the date the refund was issued.

No locators. Seven steps anyone on your team could review.

Now rename ‘Filters’ to ‘Filter’.

The spec’s locator stops matching, the run goes red, and someone edits the file or asks the healer agent to. Pie re-reads the screen on each run, so a renamed label isn’t a locator edit for your team.

The Pie test is copied word for word from Pie, “tap” and all. The spec is ours, written to match it step for step.

Where they differ

Four reasons to stop writing specs.

  1. 01

    What it costs to run

    The license is free. The bill arrives as your team’s hours. In one study, flaky tests alone took at least 2.5% of a team’s productive time. Pie’s agents write and run your tests, so those hours go back into your product.

    Source
  2. 02

    Who writes the tests

    At Amazon, our CTO Adithya Aggarwal’s teams spent about six months on each test suite, then nobody maintained it and it got scrapped. Nobody likes writing tests, so Pie’s agents explore your app and generate them.

  3. 03

    “But Playwright has agents now”

    It does. They save typing, and your team still reviews, owns and maintains every line they write. Pie’s tests are plain-English steps, so you have no test code to maintain.

    Source
  4. 04

    One place for web, iOS and Android

    Playwright stops at the browser. A native app means a second framework and a second suite, the cost nobody prices at the start. Pie tests web, iOS and Android from one place.

    Source
Side by side

Pie takes five rows. Playwright keeps two.

Playwright wins on the license fee and the code you keep. Pie wins everywhere your team spends hours.

What you’re comparingPlaywrightPieAdvantage
License cost Free and open source. Custom pricing. Contact Pie for a quote. Playwright
Cost to run it Your team’s hours to write, review and repair specs, plus the CI to run them and a second framework for native apps. No scripts to write and no locators to break, so your team spends those hours on your product. Pie
Writing a new test Someone on your team writes a spec in code, or reviews one the generator agent wrote. Agents generate tests by exploring your app. Anyone on your team can add one in plain English. Pie
When the UI changes A locator that stops matching fails the run until someone, or the healer agent, fixes it. No locators to update. Pie re-reads the screen on every run. Pie
Native iOS and Android Not covered, apart from experimental Android support for Chrome and WebView. Pair it with Appium. Built in, on the iOS Simulator and the Android Emulator. Pie
Fixing the bug The healer repairs tests. Fixing the bug in your app stays with your team. Pie can open a Fix PR with a proposed fix in a repo you connect. Merging it is always your team’s call. Pie
Pull requests Specs run inside your CI job on every commit. Pie can test your pull requests on a preview URL or a build, and comments on the PR with a GO / NO-GO verdict. Both
Test code you own Every test is code in your repo. Tests live in Pie as plain-English steps your whole team can read and edit. Playwright
The exceptions

Two cases where Playwright is the right call

If one of these describes your team, stay on Playwright.

You want every test as code

Playwright keeps every test in your repository, in a language your team already writes, inside your own CI job, with no vendor and no contract. If you want to review each test like the rest of the codebase, Playwright gives you exactly that.

Source

A handful of specs covers your app

A small, stable app with a few critical paths won’t run up much of a bill. Playwright’s free license, parallel runs and Trace Viewer will serve you well until your app outgrows them.

Source
Who should pick which

Which fits your team, Playwright or Pie?

Pick Playwright if

  • Your team wants every test as code it reviews like the rest of the codebase.
  • Your app is small and stable, and a handful of specs covers it.

Pick Pie to

  • Test more of your app without writing another spec.
  • Cover web, iOS and Android from one place.
  • Give your team back the hours it spends repairing locators.
  • Check your pull requests the way your users will use them.
Getting started

Switch from Playwright one flow at a time

No big rewrite. Your specs keep running until Pie covers the same flow.

  1. 1

    Send Pie the app

    Give Pie a URL for your web app, or an iOS simulator build or an Android APK for mobile. If the app has a login, add a test login through a supported path.

  2. 2

    Let Pie explore

    Pie explores your app’s reachable flows and generates tests. You get results in hours, not weeks.

  3. 3

    Add your edge cases

    Type the cases only your team knows into Pie Chat in plain English. Pie turns them into tests and runs them with the ones it found.

  4. 4

    Retire specs as Pie covers them

    Compare Pie’s results with your Playwright runs, and delete each spec once Pie covers the flow. Connect GitHub or GitLab and Pie can test your pull requests too, given a preview URL or a build for each change.

FAQ

Did we miss a question?
Email us directly →

Is Pie a Playwright alternative?
Yes. Pie replaces the specs your team writes and maintains with tests its agents generate by exploring your app. Stay on Playwright if you want every test as code in your repo, or a handful of specs covers your app.
Is Playwright free?
The license is. Playwright is open source under Apache 2.0. The running costs show up in engineer hours to write, review and repair specs, in the CI to run them, and in a second framework if you ship native apps. In one study of about 30 developers over five years, flaky tests alone took “at least 2.5% of the productive developer time”, and half the organizations in the World Quality Report 2025-26 name maintenance burden and flaky scripts as a test automation challenge. Pie’s agents write and run the tests, with no locators for your team to repair.
Does Playwright have AI agents now?
Yes. Playwright ships three test agents. A planner explores your app and writes a test plan, a generator turns the plan into Playwright tests, and a healer repairs failing tests. They save typing, and your team still owns and maintains the code they write. Pie’s tests are plain-English steps that Pie runs against what’s on the screen.
Can Pie test pull requests the way Playwright does in CI?
Pie can test your pull requests. Connect GitHub or GitLab and give Pie a preview URL or a build for each change, and Pie comments on the PR with a GO / NO-GO verdict. Playwright runs inside your own CI job instead.
How is Pie different from Playwright’s test recorder?
The recorder writes a spec from what you click, and you own and maintain that spec from then on. Pie decides what to click. It explores your app, finds the flows worth testing and generates the tests, with no locators for your team to keep in sync.
Can Pie test mobile apps as well as web?
Yes. Pie tests native iOS apps on the iOS Simulator and Android apps on the Android Emulator, along with web apps. Playwright emulates mobile browsers, so a team that also ships native apps needs a second tool such as Appium. With Pie you test all three from one place.
How do I move from Playwright to Pie?
One flow at a time. Send Pie your app, let it generate tests, and type your edge cases into Pie Chat. Keep your specs running until Pie covers the same flow, then retire them.
Who on my team runs Pie?
Whoever owns quality today, such as a QA lead or an engineer who owns releases. Pie does the exploring, writing and re-running. That person decides what matters and what counts as a bug, and anyone on the team, including a product manager, can read a test or add one in plain English.
What does Pie cost?
Custom pricing. Contact Pie for a quote.

Your team has better things to fix.

Book a demo and bring the flow your suite skips. We’ll show you how Pie would cover it, and which specs you could retire first.

Sources

  1. Apache 2.0 license; “a framework for Web Testing and Automation”github.com/microsoft/playwright
  2. “Chromium, WebKit and Firefox on Windows, Linux and macOS, locally or in CI, headless or headed”playwright.dev/docs/intro
  3. TypeScript, Python, .NET and Java; one API for Chromium, Firefox and WebKitplaywright.dev
  4. Parallel by default; auto-waiting before actions; retrying assertions; Trace Viewerplaywright.dev/docs/test-parallel · /docs/actionability · /docs/trace-viewer
  5. Planner, generator and healer test agentsplaywright.dev/docs/test-agents
  6. “we recommend prioritizing user-facing attributes and explicit contracts”playwright.dev/docs/best-practices
  7. Codegen records your actions and generates testsplaywright.dev/docs/codegen-intro
  8. Device emulation; “experimental support for Android automation”playwright.dev/docs/emulation · /docs/api/class-android
  9. Cost of Flaky Tests in CI, ICST 2024 industry track. Five years of history, about 30 developers, one commercial project; “at least 2.5% of the productive developer time”. Not specific to Playwright.conf.researchr.org/details/icst-2024/icst-2024-industry/1
  10. World Quality Report 2025-26 (Capgemini, Sogeti, OpenText), Fig 15. 50% cite maintenance burden and flaky scripts as a test automation challenge.capgemini.com/insights/research-library/world-quality-report-2025-26