Your Agent Writes 47% of Your Code. Who Checks the Tests It Wrote for Itself?
发布于 2026-08-28
JetBrains found agents fully generate 47% of professional developers' code. The tests come from the same agents. A look at mutation-checked verification gates, the VACUOUS vs UNVALIDATABLE split, and trust as a revocable state variable.
Your Agent Writes 47% of Your Code. Who Checks the Tests It Wrote for Itself?
JetBrains asked more than 15,000 professional developers in May-July 2026 what happened to the code they shipped last month. On average, 47% of it was fully generated by agents. Over half of those developers now write less than 20% of their code by hand, and one in five writes no code at all without AI help (JetBrains Developer Ecosystem Survey 2026).
Generation scaled. Verification did not.
The suite that used to be your evidence is now written by the same thing it is supposed to be watching.
The test that could never have failed
On August 26, a freeCodeCamp tutorial walked through a failure that should be pinned above every CI dashboard. The author had shipped a fix for a hallucination bug — a rule telling the model not to fabricate one payment provider's docs under another provider's name — and signed it off in writing as HUMAN-VERIFIED, citing a real number: zero fabricated responses out of five. Clean run. Done.
Then he re-ran it at a fixed temperature with no fixed seed. The actual clean-decline rate was 66.7% (freeCodeCamp, "How to Stop Letting AI Agents Fake Their Own Tests", 2026-08-26).
He did not lie. He confused one clean run with verification, which is a different thing and a much cheaper one.
That same confusion has a much larger version inside your test suite. When you ask an agent for a fix, it hands you a test alongside it, because that is what agents do now. Nothing in your pipeline asks the only question that matters: would this test notice if the thing it guards broke?
Plenty of LLM-written tests would not. They call the function, get a list back, assert the result is a list, and pass. It looks like coverage. The green checkmark carries no information, because red was never reachable. And the failure compounds — that test keeps passing after the behaviour it was supposed to protect breaks for a second, unrelated reason. It looks exactly like every other passing test in the suite, which is the problem.