Puppeteer drives the browser. Pie tests the product.

Test your web app without building a framework first. Pie’s agents explore it, write the tests and report what they find.

Pie

Pie is an agentic testing platform with the tests, the runs and the reports included. Agents explore your web, iOS and Android apps, so there’s no framework to build first.

Puppeteer

Puppeteer is a free, open-source JavaScript library that controls Chrome or Firefox. For testing, your team adds a test runner, assertions and the scripts.

The basics

Pie vs Puppeteer at a glance

PieAgentic testing platform
What it is
An agentic testing platform for web, iOS and Android apps.
Who writes the tests
Pie’s agents explore your app and generate them. Your team adds the cases that matter in plain English.
What a test is
Plain-English steps and checks. No scripts to write, no locators to break.
Where it runs
Web apps in a browser. iOS apps on the iOS Simulator, Android apps on the Android Emulator.
AI help
The whole product. Agents explore, write and run the tests, and group what they find into issues.
PuppeteerBrowser automation library
What it is
A free, open-source JavaScript library that controls Chrome or Firefox.
Who writes the tests
Your team, in JavaScript or TypeScript, with a test runner and assertions it picks.
What a test is
A script of selectors and browser commands your team keeps in its repo.
Where it runs
In Chrome or Firefox, on your machine or your CI. Phones are emulated in the desktop browser.
AI help
Experimental WebMCP support. A separate project, Chrome DevTools MCP, lets coding agents drive Chrome.
The same test, both ways

One refund check, written twice.

On the right is a real Pie test from a demo billing app we test with. It opens a partly refunded invoice and follows the refund to its credit memo. On the left is the same check as a Puppeteer script, our translation, using Puppeteer’s locators and Node’s built-in test runner.

Puppeteer scriptrefund-audit.test.mjs
import { test } from 'node:test';
import assert from 'node:assert/strict';
import puppeteer from 'puppeteer';

test('partly refunded invoice links to its credit memo', async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  await page.goto(process.env.BASE_URL + '/login');
  await page.locator('#email').fill(process.env.QA_EMAIL);
  await page.locator('#password').fill(process.env.QA_PASSWORD);
  await page.locator('::-p-text(Sign in)').click();

  await page.locator('::-p-text(Configure)').click();
  await page.locator('::-p-text(Payments)').click();
  await page.locator('::-p-text(Invoices)').click();
  await page.waitForSelector('h1');

  await page.locator('::-p-text(Filters)').click();
  await page.select('#refunded', 'partly-refunded');
  await page.locator('::-p-text(Apply filters)').click();
  const status = await page.$eval(
    '[data-testid="invoice-status"]', (el) => el.textContent);
  assert.equal(status, 'Partly refunded');

  await page.locator('::-p-text(Open)').click();
  await page.waitForSelector('::-p-text(Refund amount)');

  await page.locator(
    '[aria-label="Credit memos against this invoice"] ::-p-text(Open)'
  ).click();
  await page.waitForSelector('::-p-text(Payout method)');
  await page.waitForSelector('::-p-text(Refund issued)');
  await page.goBack();
  await browser.close();
});

16 selectors, plus a test runner and assertions your team brings.

Pie test
Steps
  1. Enter the account email and password, then tap ‘Sign in’.
  2. Tap ‘Configure’ on any institute from the instance configuration list to enter its management view.
  3. Tap the ‘Payments’ dropdown in the sidebar and select ‘Invoices’.
  4. Tap the ‘Filters’ button, select ‘Partly refunded’ from the Refunded dropdown, and tap ‘Apply filters’.
  5. Tap ‘Open’ on the first invoice record in the filtered list.
  6. Scroll down to the ‘Credit memos against this invoice’ section and tap ‘Open’ on the associated credit memo.
  7. Tap the back button or navigation link to return to the customer’s payments overview.
Checks
  • The Invoices view is displayed upon selection from the Payments menu.
  • The invoice list is filtered to show only invoices with a ‘Partly refunded’ status after applying the filter.
  • The detailed view of the selected invoice displays the refund amount and a list of associated credit memos.
  • The credit memo details show the payout method and the date the refund was issued.

No selectors. Seven steps anyone on your team could review.

Now rename ‘Filters’ to ‘Filter’.

The selector stops matching, the run goes red, and someone edits the script. Pie re-reads the screen on each run, so a renamed label isn’t a selector edit for your team.

The Pie test is copied word for word from Pie, “tap” and all. The Puppeteer script is ours, written to match it step for step.

Where they differ

Four reasons to stop testing with Puppeteer scripts.

  1. 01

    A library is the first piece of a test suite

    Puppeteer’s docs call it a library that gives you an API to control Chrome or Firefox. A test suite also needs a runner, assertions, reports and CI, and your team assembles those. Pie runs, reports and checks pull requests out of the box.

    Source
  2. 02

    Every script is yours to repair

    With Puppeteer, your team writes every script and repairs it when a selector stops matching. In one study, flaky tests alone took at least 2.5% of a team’s productive time. Pie’s tests have no selectors to repair.

    Source
  3. 03

    Unscripted flows go untested

    A flow nobody scripted is a flow nobody tests. Pie’s agents explore your running app and write tests for the flows they find, and your team adds the cases only it knows.

  4. 04

    A phone-sized browser isn’t a phone app

    Puppeteer’s phone support is emulation. It sets a device’s screen size and user agent in a desktop browser. Pie tests native iOS and Android apps as well as your web app, with API and accessibility testing in beta.

    Source
Side by side

A browser library next to a testing platform.

Puppeteer gives you an API to drive Chrome. Pie gives you the tests, the runs and the reports.

What you’re comparingPiePuppeteerAdvantage
Cost to run it No scripts to write and no locators to break, so your team spends those hours on your product. Your team’s hours to build the test setup, write each script and repair it, plus your own CI. Pie
Writing a new test Agents generate tests by exploring your app. Anyone on your team can add one in plain English. Your team writes it in JavaScript or TypeScript, with a runner and assertions it chose. Pie
When the UI changes No locators to update. Pie re-reads the screen on every run. Selectors live in your scripts. When one stops matching, someone edits the file. Pie
Native iOS and Android Built in, on the iOS Simulator and the Android Emulator. Phone screens are emulated in a desktop browser, by screen size and user agent. Pie
Reading a failure Pie groups related findings into issues, and can open a Fix PR in a repo you connect. Merging it is always your team’s call. Debug with a visible browser, slow motion and console logs. Reports come from the runner you add. Pie
Pull requests Pie can test your pull requests on a preview URL or a build, and comments on the PR with a GO / NO-GO verdict. Scripts run in your CI job, through the runner you pick. Both
License cost Custom pricing, tailored to your team’s requirements and testing volume. Contact Pie for a quote. Free and open source, under the Apache 2.0 license. Puppeteer
Browser automation beyond testing Pie is a testing product. It tests your app the way a user meets it. Screenshots, PDFs, crawling, request interception, performance traces and Chrome extension testing. Puppeteer
The exceptions

When the job isn’t testing

The job is automation, more than testing

Puppeteer generates screenshots and PDFs, crawls single-page apps and automates forms. Pie is a testing product, so those jobs stay with Puppeteer.

Source

You need Chrome’s internals

Puppeteer talks to Chrome over the DevTools Protocol. It can intercept requests, record a performance trace and load a Chrome extension to test it. Pie tests what your users see and do.

Source
Who should pick which

Which fits your team, Pie or Puppeteer?

Pick Pie to

  • Test your app without building a framework around a browser library.
  • Test your native iOS and Android apps, not a phone-sized browser.
  • Give your team back the hours it spends on selectors and flaky runs.
  • Check your pull requests the way your users will use them.
  • Get UI, API and accessibility testing without assembling a stack.

Pick Puppeteer if

  • The job is PDFs, screenshots, crawling or other browser automation.
  • You need Chrome’s internals, such as request interception or extension testing.
Getting started

Switch from Puppeteer one flow at a time

No big rewrite. Your test scripts keep running until Pie covers the same flow.

  1. 1

    Point Pie at your app

    Give Pie your web app’s URL. If the app has a login, add a test login through a supported path.

  2. 2

    Let Pie explore

    Pie explores your app’s reachable flows and generates tests. You get results in hours, not weeks.

  3. 3

    Add your edge cases

    Type the cases only your team knows into Pie Chat in plain English. Pie explores each flow, drafts the test and runs it once you confirm.

  4. 4

    Retire test scripts as Pie covers them

    Compare Pie’s results with your Puppeteer runs, and delete each test script once Pie covers the flow. The scripts that make PDFs or crawl pages stay where they are.

FAQ

Did we miss a question?
Email us directly →

Is Pie a Puppeteer alternative?

For testing, yes. Pie replaces the test scripts your team writes and maintains with tests its agents generate by exploring your app.

Stay on Puppeteer if the job is browser automation, such as PDFs, crawling or Chrome internals, more than testing.

Is Puppeteer a testing framework?

Puppeteer’s docs describe it as “a JavaScript library which provides a high-level API to control Chrome or Firefox”. UI testing is one of its uses.

A test suite on Puppeteer also needs a test runner, assertions and reporting, which your team picks and wires together.

Is Puppeteer free?

Yes. Puppeteer is open source under the Apache 2.0 license. The cost is your team’s time to write the scripts and keep them passing, plus the CI they run on.

In one study of about 30 developers over five years, flaky tests alone took “at least 2.5% of the productive developer time”.

Can Puppeteer test mobile apps?
Puppeteer emulates a phone inside a desktop browser by setting the screen size and user agent, which suits mobile web layouts. Pie tests native iOS apps on the iOS Simulator and Android apps on the Android Emulator, from a build you upload. Real devices are available on demand.
Which browsers does Puppeteer support?
Chrome and Firefox. Puppeteer drives Chrome over the DevTools Protocol and Firefox over WebDriver BiDi. Pie tests web apps in a browser.
Does Pie test APIs and accessibility too?

Yes, and both are in beta. Pie runs whole API flows call by call, and checks each screen its tests reach for accessibility problems.

Your UI, API and accessibility testing sit in one place.

What do Puppeteer users complain about?

Getting Chrome to run where the tests run. Puppeteer’s own troubleshooting guide says, “Getting headless Chrome up and running in Docker can be tricky.”

Puppeteer is a browser automation library, so the tests, the runner and the reports are yours to build. In Pie they come with the product, and the agents write the tests.

How do I move from Puppeteer to Pie?
One flow at a time. Give Pie your app’s URL, let it generate tests, and type your edge cases into Pie Chat. Keep your Puppeteer test scripts running until Pie covers the same flow, then retire them.
Who on my team runs Pie?

Whoever owns quality today, such as a QA lead or an engineer who owns releases. Pie does the exploring, writing and re-running, and the owner decides what matters and what counts as a bug.

Anyone on the team, including a product manager, can read a test or add one in plain English.

What does Pie cost?

Pie’s pricing is custom, tailored to your team’s requirements and testing volume.

You buy one product: agents explore your web, iOS and Android apps, write and run the tests, group related findings into issues and can open a Fix PR.

Book a demo, tell us what you ship and how often you release, and we’ll send you a quote.

Retire the test scripts. Keep shipping.

Book a demo and bring your web app’s URL. We’ll show you the tests Pie writes for it, with no runner to set up.

Sources

  1. Puppeteer docs. “a JavaScript library which provides a high-level API to control Chrome or Firefox”; listed uses include form automation, UI testing, screenshots and PDFs, and crawling single-page apps.pptr.dev/guides/what-is-puppeteer
  2. Puppeteer on GitHub. Apache 2.0 license; “JavaScript API for Chrome and Firefox”.github.com/puppeteer/puppeteer
  3. Puppeteer API, page.emulate. Emulates a device’s screen metrics and user agent; a shortcut for setting the user agent and the viewport.pptr.dev/api/puppeteer.page.emulate
  4. Puppeteer docs, WebDriver BiDi. The default protocol for Firefox; features not yet supported over BiDi include accessibility, coverage and tracing.pptr.dev/webdriver-bidi
  5. Puppeteer guides. Request interception can abort, continue or answer each request; Chrome extensions can be loaded and tested.pptr.dev/guides/network-interception · pptr.dev/guides/chrome-extensions
  6. Puppeteer debugging guide. Headful mode, slow motion, DevTools, forwarded console events and protocol logs.pptr.dev/guides/debugging
  7. Puppeteer WebMCP guide, an experimental API that is “subject to change”. Chrome DevTools MCP is a separate project that “Uses puppeteer to automate actions in Chrome”.pptr.dev/guides/webmcp · github.com/ChromeDevTools/chrome-devtools-mcp
  8. Cost of Flaky Tests in CI, ICST 2024 industry track. Five years of history, about 30 developers, one commercial project; “at least 2.5% of the productive developer time”. Not specific to Puppeteer.conf.researchr.org/details/icst-2024/icst-2024-industry/1
  9. Puppeteer docs, Troubleshooting. “Getting headless Chrome up and running in Docker can be tricky.”pptr.dev/troubleshooting