The difference
All Green. Still Broken. Pie Uses the Change Before You Merge.
CI ran the tests written before this change. Review and AI review read the diff. Nobody used the change, so the broken checkout went out with a wall of green.
Every other check on the pull request
- CI214 Tests Passedthe ones written before this change
- Code ReviewApproved“Looks right.”
- AI ReviewLooks Good2 suggestions on naming
- PreviewDeployednobody has opened it
Nobody Used the Change.
Pie on the same pull request
- Opened the Preview
- Applied a Promo Code
- Went to Payment
Pie PR BotbotCaught Before Merge. With a Replay of Every Step.
- 01
It Tests What Changed.
Pie reads the pull request and writes a few checks for it, in plain English. No test scripts to write, no locators to break.
- 02
It Uses the App Like a Customer.
It signs in with your test login, taps, types and reads the screen on your preview: web, iOS or Android.
- 03
It Tells You Whose Problem It Is.
This pull request, existing code, or the preview setup. When Pie can’t be sure, the row says so, and why.
How it works
How to Test a Pull Request Before Merging
Four steps. By hand, someone on your team does them for every pull request. With Pie, you tag it.
-
1Run the Change
By handDeploy the branch to a preview or staging URL.
PieTakes the URL from your comment or your deploy tool, and waits while it deploys.
-
2Sign In Like a Customer
By handA test account, its data, the feature flag on.
PieUses the test login you keep in Pie, only while a test runs.
-
3Use the Change
By handApply the code. Pay. Refresh. Try the edge cases.
PieWrites targeted checks from the diff and runs them on the preview.
-
4Say What You Saw
By handWrite it up on the pull request, with proof.
PiePosts one comment: a verdict, a row per change, a replay of every test.
Your pace
Why Nobody Clicks Through Every Pull Request.
Checking each one by hand adds up fast, and there are more of them: on teams with high AI adoption, developers merge 98% more pull requests. Faros AI, 2025
25
hours a week of hands-on checking
63% of one person’s week
- $84,500
- a year, at your hourly cost
With Pie
Pie tests those 25 pull requests on their previews. Your team reads 25 comments a week and decides what merges.
Book a demo →Bring your number: 25 pull requests a week.
Hours = pull requests × minutes ÷ 60. A year = hours × hourly cost × 52. A week = 40 hours. We quote Pie separately.
What comes back
A Red X Isn’t an Answer. Every Change Gets One.
Pie tests each change your pull request makes and works out what a result means before it tells you.
- Test Pass Verified
The change works on your preview.
- Test Pass Every case reached?No Partly verified
Names the case the tests didn’t reach.
- Test Stopped Why?Preview setup Couldn’t verify
Fix the setup and rerun.
- Test Fail Whose is it?Existing code Possible issue
Worth a look, not this PR’s doing.
- Test Fail Whose is it?This PR Broken by this PR
The bug is linked once it’s approved as real: steps, screenshots, a replay, and with code access on, a fix pull request.
When the tests pass but miss a case
From one of our own pull requests. Both tests passed, and Pie still wouldn’t call the changes verified.
Problems come first in the comment, so you read what needs you before the ticks. If the preview can’t reach a change at all, the row says Not testable on this preview and what would make it testable.
Pie PR Botbotcommit 3a59b326Pie Web Functional Testing
- Partly verified Test sidebar shows the saved credential instead of a guessed oneDefault and None states were shown, but no test showed an explicitly saved credential.
- Partly verified Credential popovers show name and usernameA step-reference popover showed both; one opened from a saved selection was not shown.
- Not testable on this preview CI runs the new unit testsTo make it testable: run the unit test step in CI.
Both tests passed.
Setup
What You Need to Start: A Preview URL and a Test Login.
-
Install the Pie app on the repositories you pick.
-
From your comment, your deploy tool’s link, or a pattern we set up.
-
With the data it needs and the feature flag on.
-
Comment @pie-pr-bot test on it, or switch it on for a repository.
Web, iOS and Android
Web previews in a browser; iOS and Android from a simulator or emulator build.
GitLab Too
Comment /pie on the merge request.
Behind a Firewall?
Let Pie’s traffic through, on the preview only.
Customers
Faster Releases, Without Breaking What Works.
“As we scale our products and offerings it’s really important to make sure we don’t ship something that breaks the existing experience for our users.”
“After evaluating multiple vendors, Pie was the only solution that consistently executed our most complex end-to-end workflows with reliability across builds.”
Security
Yes, Security Will Ask. Here’s What to Tell Them.
- Only the Repositories You Pick.You install the Pie GitHub App on the repositories you choose. If your organization restricts third-party apps, an admin approves it.
- We Never Use Your Source Code to Train AI Models.Source code here means the files and code changes in your repositories.
- Logins Reach Pie’s Agents Only While a Test Runs.You keep them in Pie’s credential settings.
- Pie Has a SOC 2 Type 2 Report.Our most recent report is available on request. Data is encrypted in transit.