Insights

Can You Trust AI-Generated Tests?

AI-written tests often copy expected results from the code, so they pass while bugs ship. See what three models did with a known bug, and what to keep instead.

Dhaval Shreyas

CEO & Co-founder at Pie

10 min read Posted Oct 26, 2026

You ask Copilot or Cursor for tests on a function and skim what comes back. The suite goes green, the coverage number climbs, and the PR merges. Two weeks later a bug ships through code that had tests, and nobody can point to the one that should have caught it.

The model wrote those tests by reading your code, so they check that the code still does what it did that day. Whether that was ever right is a question the model had no way to answer, and keeping its tests means keeping its answers.

Can You Trust AI-Generated Tests at All?

Not as they come out of the model. AI drafts the standard path and the boilerplate well. It’s weaker at edge cases, and it takes its idea of the right answer from the code it was asked to test.

Meta built a tool that writes tests with an LLM, and it throws most of them out before an engineer sees one. A generated test has to build, pass reliably, and add coverage. When Meta tried it on Instagram, about 4% of all the test cases it suggested cleared all three, and engineers accepted most of what made it through.

Look at what those filters check, though. None of them asks whether a test’s expected value is correct, and a test that passes on today’s code has agreed with today’s code by construction. The filter made generated tests safe to keep as regression checks. Deciding they were right stayed with the engineer who read them.

Trust whatever checked the test, never the thing that wrote it.

Four Ways AI-Generated Tests Fail, Two of Them Silently

AI-generated tests fail in four ways. Two of them hide inside a passing run and surface later as production incidents, which is the worst place to learn your coverage was decoration. The other two fail loudly for the wrong reason, and the quick fix for a loud failure can make it quiet.

• Happy-path bias: The model writes the obvious case, with valid input and a success response. It rarely tries an expired token or a permission the user shouldn’t have, and those gaps are where bugs live.
• Wrong expected values: When the model guesses an expected value wrong, the test fails even though the code is fine. In a 2025 study of AI-written tests, Copilot’s included, nearly two in three of the errors were failed assertions, which the authors read as tests that expected the wrong value.
• Tests edited to pass: When a generated test fails, a coding agent may weaken the assertion or delete the test to get back to green. TDD pioneer Kent Beck counts an agent “disabling or deleting tests” among his warning signs that it has gone off track.
• Brittle selectors: Generated UI tests bind to selectors and fixed waits the same way hand-written ones do, so a renamed class or a slow render breaks them. It’s the maintenance tax AI test automation tools were built to remove, now produced at machine speed.

None of this is a knock on the model. The output is shaped like tests because that’s what you asked for, and the shape says nothing about whether the tests check anything. Read a generated suite for what it asserts, and ignore the color.

Tautological Tests: When Passing Proves Nothing

A tautological test asserts the implementation back at itself, so it passes as long as the code does what it currently does, including when that’s wrong. It’s worse than any of the four above because it’s the most credible. The test is real and it goes green, yet it checks nothing the code didn’t already say.

It happens because the model has nowhere to learn intended behavior except the code in front of it. When researchers showed LLMs a test’s expected result next to buggy code across 24 Java projects, the models tended to side with what the program does rather than what it should do.

Here’s the shape of it. The spec says orders over $50 ship free, and the code gets the boundary wrong:

export function shippingCost(subtotal: number): number {
  return subtotal >= 50 ? 0 : 5.99; // spec: free shipping over $50
}

// Generated from the function above
test('ships free at $50', () => {
  expect(shippingCost(50)).toBe(0);
});

The model read >= and wrote a test saying a $50 order ships free. That test passes today and fails the day someone fixes the boundary. You’ve kept a test that defends the defect, and the coverage number went up while it happened.

An expected result written from the code is a scale calibrated against the package it’s supposed to weigh, and it agrees every time. When the same model wrote the code and the test, they share its blind spots, which is how teams end up with green tests on a broken app.

We Handed Three Models the Bug

We wanted to see whether models write that kind of test on their own. On 26 October 2026 we gave three free models (gpt-oss-120b, Qwen3.8-27B and Gemini 3 Flash Preview) the shippingCost function and asked each for unit tests. Its spec says orders over $50 ship free, and the code also ships an order of exactly $50 free. About half the runs had the “over $50” comment in the code and half didn’t.

All 56 test suites they wrote passed on the buggy code and failed once we fixed the boundary, comment or no comment. One suite even named its test “returns free shipping for a subtotal exactly $50 (code uses >= 50)”. It’s one function, one boundary bug and three models called through bare APIs, not Copilot or Cursor with your repo behind them, and Gemini finished 16 of its 20 runs, because its free daily quota blocked the retries of four that errored.

Handed only the code, even with the spec in a comment beside the bug, every model wrote the test the code implied. A green run from those tests says nothing about whether a $50 order should ship free.

Who Should Decide What a Test Expects

Teams don’t always agree on the right answer either. We once filed an issue that a customer didn’t think was a bug. When they took it to their team, some of their own people said they’d expected the product to work differently. If a customer’s own team can disagree about what correct looks like, a model reading the code has no chance.

Recording current behavior is exactly right for one job, and it’s the job Meta’s filter was built for. Before a refactor, a characterization test locks in how the code behaves now so any change shows up. AI drafts those quickly. Everywhere else, recorded behavior isn’t a spec.

What Mutation Testing Catches, and What It Can’t

The usual check on whether a test proves anything is mutation testing. A tool makes small changes to your code, such as flipping >= to >, and confirms some test fails for each one. Point it at a generated suite and it flags weak tests fast, like this one, which passes whatever the function returns:

test('calculates shipping', () => {
  expect(shippingCost(20)).toBeDefined();
});

Now run it against the $50 test. Flip >= to > and that test fails, so mutation testing scores it as strong. It is strong. It’s strongly asserting the wrong answer.

Every suite from our run kills that mutant too, because that mutant is the fix. Mutation testing proves a test can fail. It can’t prove the test expects the right thing, because only the spec knows that.

A 2026 study found the same gap at scale. Researchers had LLMs write code, kept the hardest bug in each task, then had an LLM write the tests. The tests often ran straight through those bugs and rarely caught more than 1 in 10 of them. As the authors put it, they “fail to assert the correct behavior.” Tests picked for a high mutation score did about as well as tests picked for coverage.

So use mutation testing to find the weak tests, and then check the strong ones against the spec.

Why “Just Review Every Test” Doesn’t Scale

Reviewing every generated line is the standard way to earn trust in AI tests. It’s correct advice, and it stops working at the moment you need it. Cursor’s own guide describes the loop as asking the agent to write tests and then verifying they’re correct. If you’re reviewing generated tests today, three checks do the most work per minute:

1. Break it: Change the line under test and rerun. A test that stays green wasn’t checking that line.
2. Check the answer: Read each expected value against the spec or the ticket, not against the code it came from.
3. Name the gaps: List the inputs the test never tries, like an empty field or a malformed date, and ask which of them can reach production.

All of it works. The catch is structural. Judging whether an expected value is right means knowing what the code should do, and that’s the same knowledge writing the test needed. The review competes with everything else an engineer could do that afternoon, and it loses, because the suite is already green.

AI also writes tests faster than people can read them, and the gap widens each time the model gets better at plausible output. Review is already behind on the code itself. In a survey Sonar published in January 2026, only 48% of developers said they always check AI-assisted code before committing it. A machine-speed pile doesn’t shrink under a human-speed audit.

One team we work with opens more pull requests than its engineers can review in a day, and it wants to merge without a human pass. It added mutation testing to keep its unit tests relevant. The one layer it had no automatic check for was the screen its users see.

Keep the Intent, Not the Generated Code

Every failure above traces to one decision, keeping a model’s guess as code your team then owns. The guess about which element to click goes stale the next time the UI moves. The guess about the expected value was copied from the code on the day the model read it. Keep the generated test and you keep both guesses.

The part worth keeping is short, and it reads in plain English. “Add a $50 item to the cart, check out, and confirm shipping is $5.99.” Whoever owns checkout can say in seconds whether that’s right. Run against the buggy code above, it fails.

Your users never see the spec, but they can tell fast when something’s wrong, and a plain-English check works from the same place. Put the two side by side and you get the line between AI for QA your team can trust at volume and AI that writes tests you then have to audit:

Read the bottom row twice. When review means reading generated code, trust drops with every test you add. When it means reading a sentence per flow, coverage can grow without the audit growing with it.

How Pie Generates Tests You Can Read

Pie generates tests too. The fair question is what it leaves you holding. There’s no test script for your team to write or maintain. What you keep is the intent in plain English, and Pie works out the clicks from the screen when the test runs.

• Autonomous discovery: Pie’s agents explore reachable, supported flows in your app like a first-time user and turn what they find into ready-to-run test cases, with no selector-bound code for your team to inherit.
• Plain-English tests: Each test is a short list of steps and expected outcomes, written as human-readable test cases that whoever owns the feature can read and edit. You can draft them from Cursor or Claude Code through Pie’s MCP server. AI writes the intent, and nobody inherits a script.
• Vision-first execution: Pie looks at the screen first to find what it needs, so a button that moves can be found where it is now. For supported tests, your team writes the steps and maintains no selectors. If a step lands somewhere it shouldn’t, a screen that doesn’t show the expected outcome can fail the run.
• Reviewed fixes: With code access, Pie Loop can draft a fix for a bug a run finds and open it as a pull request. An engineer on your team still decides what merges.

Pie doesn’t know your business rules until someone states them, and a discovered test describes the app as the agents found it. Whether a $50 order ships free is a product decision, and it stays one. What changes is where that decision lives. It’s written out where the feature’s owner can read it and fix it, instead of buried in an assertion a model lifted from the implementation.

Stop Letting the Code Grade Itself

AI will write more of your tests every quarter, and they’ll arrive green. A scale calibrated against the package was always going to agree with it.

Trust comes from the two checks a green run can’t do for you. Watch each test fail on broken code before you believe it, and take the expected result from someone who knows what the product should do.

Pie keeps that expected result in plain English and works out the clicks when the test runs, so what your team reviews is a sentence. Decide the right answer before the model does.

Dhaval Shreyas

CEO & Co-founder at Pie

13 years building mobile infrastructure at Square, Facebook, and Instacart. Now building the QA platform he wished existed the whole time. LinkedIn →