Guide

What Is Smoke Testing? How It Works, When to Run It, and How to Automate It in 2026

Smoke testing is a small set of checks that confirm a build's most critical functions work before deeper testing begins. Here's how it works, how it differs from sanity and regression testing, and how to automate it in CI/CD.

Adithya Aggarwal

CTO & Co-founder

11 min read

Posted Jun 8, 2026

What Is Smoke Testing?

Smoke testing is a small, fast subset of test cases that verifies the most critical functions of a software build work, run as a gate before any deeper testing begins. Its only job is to decide whether a build is stable enough to be worth testing further. If the core functions pass, the build is accepted; if any of them fail, the build is rejected on the spot.

The ISTQB glossary defines a smoke test as “a subset of all defined test cases that cover the main functionality of a component or system, to ascertain whether the most crucial functions of a program work, but not bothering with finer details.” That last clause is the whole philosophy: smoke testing trades depth for speed. It is intentionally shallow.

You will also see smoke testing called build verification testing (BVT) or confidence testing. These are the same practice under different names. The label “build verification” is most common in continuous integration, where the check runs automatically against every build to confirm it is sound before the pipeline invests in slower stages.

Where Does the Name “Smoke Testing” Come From?

The term “smoke testing” comes from hardware, not software. When an engineer assembled a new circuit board or a plumber pressurized a new pipe system, the first test was the crudest one possible: turn it on and see if smoke pours out. If it smoked, it failed instantly and there was no point checking anything else. If it did not, the device had earned the right to more careful inspection.

Software borrowed the metaphor exactly. A smoke test is the “power it on and see if it catches fire” check for a build. It does not tell you the software is correct; it tells you the software is not catastrophically broken. That distinction matters, because the value of smoke testing is in fast rejection, not in thorough confirmation.

The practice entered mainstream software engineering through daily-build discipline. In his influential 1996 IEEE Software article “Daily Build and Smoke Test,” Steve McConnell described the routine that teams on large Microsoft projects used: build the product every day and run a smoke test against it immediately, so that “the product is smoke tested every day” and breakage never accumulates silently. The idea predates modern CI by years, but it is the direct ancestor of the automated build gates we run now.

How Does Smoke Testing Work?

Smoke testing works by running a curated set of high-priority checks against a fresh build and treating the result as a binary gate: pass means proceed, fail means stop. The smoke test does not try to find every bug. It confirms that the application starts, the critical paths respond, and nothing fundamental is on fire before the team commits time to deeper testing.

A typical smoke test cycle has four steps. First, a new build is produced, whether from a commit, a pull request, or a deployment. Second, the smoke suite runs against that build, exercising a handful of critical functions end to end. Third, the results are evaluated as a whole, and any failure in the suite fails the build. Fourth, the outcome routes the build, a pass sends it to the next testing stage, and a fail sends it straight back to engineering with the broken function flagged.

The economics are the reason smoke testing exists. The longer a defect survives, the more it costs to fix, a relationship documented as far back as the NIST 2002 report on the economics of software testing, which estimated inadequate testing infrastructure cost the US economy tens of billions of dollars a year. A two-minute smoke test that rejects a broken build before an hour of regression testing is one of the highest-leverage checks a team can run.

Smoke vs Sanity vs Regression Testing

Smoke testing, sanity testing, and regression testing are often confused because all three confirm that software still works, but they answer different questions at different moments. Smoke testing asks “is this build stable enough to test at all?” Sanity testing asks “does this specific change work?” Regression testing asks “did this change break anything that used to work?” Getting the distinction right keeps each stage fast and purposeful.

The clearest way to separate them is breadth versus depth and timing. Smoke testing is broad and shallow, run first on every new build. Sanity testing is narrow and focused, run after a targeted fix to verify just the affected area. Regression testing is broad and deep, run after the smoke test passes to re-verify the whole feature set. Here is how they line up:

Note that smoke and sanity testing are both subsets of regression testing in intent, since each one trades full coverage for speed in a specific situation. If you want the deeper sweep that sits downstream of the smoke gate, see our guide on how to build a regression test suite.

When Should You Run Smoke Tests?

Smoke tests should run every time a new build exists and always before any slower testing stage. In practice that means three trigger points: on every commit or pull request in your CI pipeline, after every deployment to a staging or production environment, and as the first gate ahead of regression, integration, or performance suites. The principle is constant, run the cheap check before the expensive one.

Running smoke tests first is what makes the rest of the pipeline economical. Modern delivery research from the DORA State of DevOps program consistently finds that elite performers deploy on demand, multiple times a day, while low performers ship between once a week and once a month, and the elite teams do it by gating every change with fast automated tests rather than slow manual review. A smoke test is the fastest of those gates: it catches the build that will waste everyone’s afternoon, and it catches it in the first two minutes.

The deployment trigger matters as much as the commit trigger. A build can pass every test in CI and still break in staging because of a misconfigured environment variable, a missing migration, or a broken dependency. A post-deployment smoke test, sometimes called a production smoke test, confirms the deployed system actually answers before real users find out it does not.

What Should a Smoke Test Suite Include?

A smoke test suite should include only the checks that confirm the application’s most critical functions work, kept small enough to finish in a few minutes. The right size is usually a handful to a few dozen checks, never hundreds. The selection rule is simple: if this function is broken, is the build worthless? If yes, it belongs in the smoke suite. If no, it belongs in regression.

For most applications the critical-path shortlist looks similar:

• Application launch. The app starts, loads, and renders its primary screen without crashing or hanging.
• Authentication. A user can log in and log out. Almost nothing else matters if this is broken.
• Core navigation. The main routes resolve and the primary screens load rather than 404ing or white-screening.
• The revenue flow. The one or two paths tied directly to the business, checkout, subscription, booking, or whatever your product sells.
• Critical integrations. A smoke-level check that the database, payment provider, or key API responds at all.

The discipline is in what you leave out. A smoke suite that tries to validate edge cases, error messages, or visual details has misunderstood its job. Those checks are valuable, but they belong in deeper stages. The smoke test exists to answer one question fast, and every check you add that does not serve that question makes the answer slower and the gate less useful.

Should Smoke Tests Be Manual or Automated?

Smoke testing should be automated anywhere builds are produced frequently, because the entire value of a smoke test comes from running it on every single build without delay, and manual execution cannot keep that pace. A smoke test you run “when someone remembers” is not a gate; it is an occasional ritual. The practice only pays off when it is reliable, repeatable, and triggered automatically.

Manual smoke testing still has legitimate uses. On a brand-new product where flows change daily, scripting a smoke suite may not be worth it yet. Exploratory smoke checks before a high-stakes release, where a human eye catches things no assertion was written for, are also valuable. But these are supplements, not the system. In any team practicing continuous integration, the smoke suite belongs in the pipeline.

The catch with automated smoke testing is maintenance. The moment you script a smoke test against UI selectors, it inherits the same fragility as any other selector-based test: a renamed class, a restructured DOM, or a layout change can break it. Industry estimates put test maintenance at 60 to 80 percent of total automation effort, and a smoke suite is not exempt. You can see what that looks like on your own suite with our test maintenance cost calculator. A gate that frequently fails for reasons unrelated to the build is worse than no gate, because teams learn to ignore it. Keeping a smoke test fast is easy; keeping it trustworthy over time is the hard part.

How Do You Automate Smoke Testing in CI/CD?

You automate smoke testing by wiring a small, fast test suite into your CI/CD pipeline so it runs as the first stage on every build and blocks promotion on failure. The mechanics are straightforward; the discipline is in keeping the suite small, fast, and stable enough that the team trusts a red result.

A reliable setup follows five practices:

1. Run it first and run it fast. Place the smoke stage ahead of regression, integration, and performance suites in your pipeline. A failure here should short-circuit everything downstream within minutes.
2. Make it a blocking gate. A failed smoke test must stop the build from advancing. A non-blocking smoke test that merely warns gets ignored within a week.
3. Keep it deterministic. Smoke tests must not be flaky. A smoke suite that fails randomly trains your team to re-run it on red, which destroys the entire point of the gate. If yours flakes, fix the root causes before adding any new checks.
4. Smoke test after deployment, too. Add a post-deploy smoke run against staging and production so environment problems surface immediately, not in a support ticket.
5. Budget for maintenance, or remove it. A selector-based smoke suite needs ongoing repair. If no one owns that upkeep, the suite rots into noise and gets disabled.

Those third and fifth points are where most smoke suites quietly die. They start fast and trustworthy, then accumulate flakiness and selector breakage until the team mutes them. The durable solution is to stop binding smoke tests to the implementation details that change most often.

Smoke Testing Without the Maintenance Tax

Pie runs smoke testing as an autonomous gate that does not decay over time, because its tests are bound to behavior rather than selectors. Where a hand-scripted smoke suite breaks every time a class is renamed or a layout shifts, Pie’s agents identify elements the way a user sees them, so the same UI changes that break selector-based tests leave a Pie smoke suite running.

Three capabilities make that concrete for smoke testing specifically:

• Autonomous discovery explores your application, maps the real user flows, and prioritizes the high-risk ones, exactly the critical paths a smoke suite should cover. You do not hand-pick and hand-script the smoke cases; the platform identifies what matters and generates the suite. The first full suite for an average app lands in roughly 30 minutes.
• Self-healing, vision-based execution keeps the smoke gate trustworthy. Because elements are identified by what the user sees rather than by a CSS path, a UI change re-identifies cleanly instead of failing red, which removes the maintenance that usually erodes a smoke suite into noise.
• Cross-platform reach runs the same behavior-based smoke checks across web and native iOS and Android, so a mobile release gets the same fast gate as a web build without a second framework to maintain.

Pie is not a replacement for every kind of test you run, and a smoke test is only one stage in a healthy pipeline. But the failure mode that kills most automated smoke suites, slow rot from selector maintenance and flakiness, is exactly the failure mode an autonomous, vision-based platform is built to remove. The gate stays fast, and it stays trustworthy.

The Cheapest Test That Saves the Most Time

Smoke testing endures because the math is unbeatable: a two-minute check that rejects a broken build before an hour of deeper testing is one of the highest-return practices in software delivery. The concept is simple, the value is large, and the only real risk is letting the suite grow slow or letting it rot into flaky noise that the team learns to ignore.

Keep your smoke suite small, run it first on every build, make it a blocking gate, and hold it to a strict standard of stability. Do that, and it will catch the embarrassing breakage before it costs anyone an afternoon. The teams that get the most out of smoke testing in 2026 are the ones who stopped maintaining brittle scripts to keep the gate alive, and let the suite adapt to their app instead. If that is the gate you want, Pie was built to run it.

Adithya Aggarwal

CTO & Co-founder

Building the future of autonomous QA. Previously led mobile infrastructure at scale. LinkedIn →