Checking builds
Five Hours a Week by Hand. Before Testing Even Starts.
Finding out whether a build even works eats a piece of every day. Give Pie the smoke check and your testers start on builds that work. Slide in your numbers to see what it costs today.
- hours a month checking builds today
- 21.7
- what that costs a year
- $16,900
Book a demo → In the demo, we’ll work out what Pie takes off it.
When it runs
The Build Lands. The Check Starts Itself.
Set it up with us once and the smoke check comes off your team’s list. Each build gets the same tests.
-
When a New Build Lands
Your CI sends a new iOS or Android build and the suite starts from it, once we’ve set this trigger up with you.
-
When a Deploy Finishes
Your CI or deploy script fires an API trigger, and the suite runs after the deploy.
-
On Your Schedule
Once a day or before a release, at the time you pick, with your regression suite on a schedule of its own.
Pie Runs Your Smoke Suite
Nobody has to start it or watch it.
A sample run
Your Team Hears
1 passed9 failed1 issue
Slack gets the alert, the issue is ready for Jira and the proof stays with the run.
What comes back
Not Just Red or Green. The Step Where It Broke.
Know whether a build is worth testing before anyone opens it. Every failure says where it stopped and why, with a screenshot of each step. Every suite’s runs come back like this one, a real run from Pie testing Pie.
- Proof at every step.Each step keeps its screenshot, so you see what Pie saw.
- Pass or fail, test by test.See what held and what didn’t without opening the build.
- It says why it stopped.This test never got past sign-in, and the run says so.
Your smoke suite
Your Smoke Suite Is Tests You Already Run.
Pick the tests that cover what must never break and Pie runs just those. Still checking by hand? We import your cases, or you describe the flow in plain English and Pie drafts the test. Your team confirms each one.
- A short list beside the full one.Here, a 5-test suite sits beside a 54-test full pass. Your smoke suite sits the same way.
- Name it, pick the tests.A test can sit in more than one suite, so nothing gets rewritten.
The basics
New to Smoke Testing? Start Here.
Get up to speed in eight short answers, from what it is to how to automate it. The guides go deeper.
Smoke testing is a short set of tests that checks whether a new build works at all, before anyone spends time on it. It opens, you can sign in, and the flows that must never break still work.
You’ll also hear it called a build verification test or a build acceptance test.
Our guide to smoke testing covers what to include and how to run it.
A new iOS build lands. The smoke suite launches the app and tries to sign in, and the Continue button doesn’t respond. Every test after sign-in stops at the same step.
Nobody spends a morning on that build. The team gets one issue, and the next build gets the same check.
Smoke testing checks that a new build works at all. Sanity testing checks that one fix worked. Regression testing checks that everything that used to work still does.
| Smoke | Sanity | Regression | |
|---|---|---|---|
| Asks | Is this build worth testing? | Did the fix work? | Did anything that worked break? |
| Runs | The flows that must never break | One area | The whole suite |
| When | On a new build | After a fix | Nightly or before a release |
Only what must never break. That’s usually launch, sign-in, the main path through your app and the payment step if you have one.
The suites we see run from about ten tests to about fifty, and many sit near twenty. One that keeps growing has become a regression suite, so move the extras to your regression pass.
When a new build is ready, before anyone starts testing it. That can mean each build your CI makes, each deploy to staging or once a day.
Some teams also run one on production after a release, to confirm the deploy went out clean.
The name comes from hardware. Engineers powered on a new circuit board and watched for smoke. If it smoked, nothing else got tested.
A software smoke test works the same way. It’s the crudest check you can run, and a failure saves you from everything that would have come after it.
Yes, and it’s a good place to start, because the checks are few and they repeat on each build. Choosing what goes in the suite and deciding what a failure means stay with your team.
Playwright and Cypress are free and exact, as long as someone keeps the scripts working when the UI moves. An agent like Pie runs tests written in plain English, with no locators to break.
A new build is a new file, so a run starts from the build instead of a URL. Each platform can have its own suite.
Pie tests iOS apps on the iOS Simulator (a simulator .app build) and Android apps on the Android Emulator (an .apk).
Our mobile app testing guide covers the rest.
How customers run it
On Each Build, Every Day or Twice a Week.
A few of the setups we see, with names left out. How often yours runs depends on how often you ship.
- Each iOS buildA retail team · about 40 tests
The team uploads a new iOS build and the smoke suite runs from it. A separate run checks production after a release.
- Every day at 5pmA software team · 50 tests
The team picked the time so the suite runs ahead of that day’s deploy. A full regression runs once a week.
- Twice a weekA consumer app team · 20 tests
It runs midweek and on Friday, on iOS and Android. The same 20 tests also sit inside the team’s 40-test regression.
Security
Yes, Security Will Ask. Here’s What to Tell Them.
- You Pick the Test Account.Pie signs in with the test login you give it. A setup script can call your test API to create the data first.
- We Never Use Your Source Code to Train AI Models.Source code here means the files and code changes in your repositories.
- Logins Reach Pie’s Agents Only While a Test Runs.You keep them in Pie’s credential settings.
- Pie Has a SOC 2 Type 2 Report.Our most recent report is available on request. Data is encrypted in transit.