Insights

Does AI-Generated Code Have More Bugs? Yes, and You're Catching Them in the Wrong Place

AI pull requests carry about 1.7x more issues. Reviewing the diff harder cannot catch the bugs that only exist once the product is running.

Dhaval Shreyas

CEO & Co-founder at Pie

9 min read Posted Sep 14, 2026

How Many More Bugs Are We Talking About?

AI-authored pull requests carry roughly 1.7x more issues than human-written ones. The clearest measurement comes from the CodeRabbit State of AI vs Human Code Generation Report (December 2025), which analyzed 470 open-source GitHub pull requests, labelled AI or human by authorship signals, and found AI PRs averaged 10.83 issues against 6.45 for human-only PRs. The gap isn’t evenly spread, and it concentrates in the categories that hurt.

Break the number down and the shape of the problem appears. CodeRabbit found AI pull requests carried about 75% more logic and correctness issues, nearly twice as many error-handling gaps, and security issues running up to 2.74x higher, most often improper password handling and insecure object references. Veracode’s 2026 GenAI Code Security Report puts the average security pass rate of AI-generated code at 56%, so the pattern holds beyond one vendor’s data. These aren’t cosmetic. Logic errors and missing error handling are the failures that pass a quick read and break in production.

The volume point is the one teams underestimate. An Uplevel study of nearly 800 developers, with data through April 2024, found the group using Copilot introduced 41% more bugs with no significant change in efficiency metrics. More code, more bugs, same speed. That volume lands downstream, which is why testing AI-generated code is now a different job than testing what a human wrote by hand.

Why Reviewing the Diff Harder Won’t Fix It

The standard response to the 1.7x number is to review the diff harder, and it misses the bugs that matter. Every ranking article on this topic lands in the same place. Treat AI like a junior developer, add an AI code reviewer, tighten your PR checklist, run more static analysis. The advice holds. It’s just incomplete, and that incompleteness is the entire problem.

Code review, human or AI, reads the change. It inspects added and removed lines and reasons about them in isolation. That is a useful job, and tools like CodeRabbit, Greptile, and Sonar do it well.

But a reviewer looking at a diff can’t see whether the checkout flow completes end to end, whether login redirects correctly, or whether a change in one module quietly broke a feature three modules away. None of that is in the diff. It only exists when the product runs.

There’s a scaling problem stacked on top. As AI pushes more and larger commits, thorough review gets harder rather than easier, and reviewers fall back to skimming. The Stack Overflow 2025 Developer Survey found that 66% of developers name “AI solutions that are almost right, but not quite” as their biggest single frustration, 45.2% say debugging AI-generated code is more time-consuming, and distrust of AI accuracy (46%) now outweighs trust (33%).

Piling more of the fix onto the review step, the step that’s already overloaded, isn’t a strategy. It’s the definition of a bottleneck.

Three Layers in the AI Stack, and the One Nobody Automated

An AI engineering stack has three layers, and the industry automated the first two while leaving the third mostly to chance. Seeing them side by side makes the asymmetry obvious. Writing and reviewing both operate on code. Only the third operates on the product your users touch.

Layer 1 creates the problem and Layer 2 catches a slice of it, so the bugs that survive both are, by definition, the ones only Layer 3 can find. Yet Layer 3 is where teams cut corners, because testing the running product has meant either slow manual QA or brittle end-to-end scripts that break every time the UI moves.

Those economics made sense while humans wrote most of the code. They stopped making sense the moment a machine started writing a fast-growing share of it, which is why autonomous QA became a category at all.

Bugs That Cost You Money Show Up After the Merge

The expensive bugs show up after the merge, not in the pull request, and the industry’s own data now says so plainly. DORA’s 2025 State of AI-assisted Software Development report found that higher AI adoption is associated with an increase in both software delivery throughput and software delivery instability. Teams ship more and break more.

DORA names the countermeasure in the same breath. Its 2025 findings say that without robust control systems, strong automated testing and fast feedback loops among them, an increase in change volume leads to instability.

Read that finding closely, because it locates the damage precisely. AI’s cost shows up in delivery stability, which measures how often things break in production. Stability is a running-product metric, and it sits one layer below where everyone is currently looking.

New Relic’s June 2026 report, built on a survey of 200 U.S. technology leaders, puts the same gap in two numbers. Among them, 94% rate AI-generated code as higher quality than human-written code at the moment it’s reviewed, and 82% have had at least one major production failure caused by AI code in the past six months. Better in the diff, worse in production, reported by the people who own both.

You feel fast, your dashboards look busy, and your production incident rate climbs without anyone connecting the two. Coverage won’t save you either, because coverage numbers mislead in exactly the same shape. A green report tells you a line ran, not that a user flow works. Execution isn’t verification, and the diff isn’t the product.

Four Moves That Catch What Review Can’t

The move that closes this is unglamorous. Keep the fast coding tools and the AI reviewer, then add real verification of the running product on top. Concretely, the pattern looks like this.

1. Keep both upstream layers: Use Copilot or Cursor to write and an AI reviewer to catch diff-level issues. Treat both as necessary and neither as sufficient. Review is a filter, not a guarantee.
2. Verify the running product on every meaningful change: Exercise the journeys your users depend on, login and checkout and whatever generates your revenue, against a real build. Running the product is what catches the integration bugs and regressions a diff hides.
3. Make verification keep pace with the writing: Manual QA can’t run on every pull request, and more QA engineers won’t ship you faster either. Selector-based scripts break faster than AI generates the UI changes that break them, and verification slower than authoring becomes the new bottleneck and gets skipped under deadline.
4. Measure product outcomes over activity: Track escaped-defect rate and delivery stability, the DORA signal, instead of lines reviewed or tests written. Outcome metrics tell you whether the third layer is doing its job.

We built Pie for Layer 3. It’s an autonomous QA platform that tests the running product. Agents explore your app the way a user would, generate the end-to-end suite, and add cases as new flows land. There are no selectors to write or maintain, so when AI rewrites your UI the tests adapt instead of shattering. And when Pie finds a bug, Pie Loop can draft the fix as a pull request your team reviews and merges, which puts the answer back on Layer 2.

The Missing Quality Co-pilot

Every engineering team now has a coding co-pilot. Almost none have a quality co-pilot, and that asymmetry is the whole story. We poured investment into the two layers that operate on text and left the layer that operates on the product to slow manual passes and scripts that rot.

So yes, AI-generated code has more bugs, roughly 1.7x more issues per pull request. If that number sends you back to the pull request to review harder, you’re treating a symptom at the wrong altitude. Reading the diff again can’t find a broken checkout. Running the checkout can.

The teams that win the AI coding era won’t be the ones writing the most code or reviewing it the hardest. They’ll be the ones who made verifying the running product as fast and automatic as writing it. Pie lives on that layer, because it’s the only one where a broken checkout has anywhere to show itself.

Dhaval Shreyas

CEO & Co-founder at Pie

13 years building mobile infrastructure at Square, Facebook, and Instacart. Now building the QA platform he wished existed the whole time. LinkedIn →