# Most AI agents never reach production The demo is dazzling. Then the agent never ships. Survey after survey in 2025–26 finds the same cliff: almost everyone has an agent pilot, almost no one has it in production. The reason isn't the model — it's the unglamorous engineering the demo let you skip. Here's what the small minority who actually ship do differently. - **Author:** Fedor Chishchin - **Published:** 2026-06-03 - **Tags:** agents, eval, methodology - **Source:** https://fedorthinks.com/blog/agents-never-reach-production --- You've seen the demo. An agent takes a vague request, fans out across tools, writes the code, books the trip, closes the ticket — flawlessly, on stage, to applause. It looks like the future arrived early. Then, months later, you quietly notice it never shipped. The pilot is still a pilot. Nobody's actually using it. This is the most common story in AI right now, and it's worth being honest about, because the gap between "incredible demo" and "thing real people depend on" is where almost every agent dies. ## The cliff, in numbers This isn't a vibe — it's measured, repeatedly, and the numbers are brutal. MIT's NANDA initiative published [_The GenAI Divide_](https://fortune.com/2025/08/18/mit-report-95-percent-generative-ai-pilots-at-companies-failing-cfo/) in 2025 and found that **95% of corporate generative-AI pilots deliver no measurable return** — only about 5% make it to real impact. A March 2026 survey of 650 enterprise leaders found the same shape: [**78% had agent pilots, but only 14% reached production**](https://www.digitalapplied.com/blog/ai-agent-scaling-gap-march-2026-pilot-to-production). Another cut of the data: **67% saw gains in the pilot, 10% scaled it** — so roughly **90% stall** in the gap between a working proof of concept and a system anyone relies on. Whatever the exact figure, the message is identical: getting an agent to work _once, in a demo_ is easy now. Getting it to work _every time, in production_ is where the wheels come off. ## It was never the model Here's the part people get backwards. The model in your failed pilot is the same model in everyone's successful one. The frontier is shared; it's an API call. If raw model capability were the bottleneck, you'd see a few winners with secret models and everyone else losing. That's not the pattern. The pattern is the _same models_ succeeding for a few and stalling for most. MIT was blunt about where the problem actually lives: the failures trace to a **"learning gap"** — companies that can't integrate the model into real workflows, structures, and data — **not to model quality.** One 2026 analysis found that [five gaps account for **89% of scaling failures**](https://www.digitalapplied.com/blog/ai-agent-scaling-gap-march-2026-pilot-to-production): integration with existing systems, inconsistent output quality at volume, no monitoring tooling, unclear ownership, and thin domain data. Look at that list. Not one of those is "the model isn't smart enough." Every one of them is engineering and operations — the work a demo lets you skip. ## A demo is a curated best case The reason demos lie is structural, not dishonest. In a demo you control everything: you pick the input, you pick the happy path, you pick the moment. You are showing that the agent _can_ succeed — once, under conditions you chose. Production is the opposite of curated. It sends inputs you never imagined, at volume, at 3am, in the wrong format, from users actively trying to break it. And the component in the middle is a [non-deterministic guesser](/blog/grounding-is-a-constraint). An agent that's right 90% of the time is a triumphant demo and a production nightmare: at a thousand requests a day that's a hundred confident failures, every day, compounding across multi-step chains until the agent drifts off-task entirely. The demo measured "can it work?" Production measures "does it keep working on inputs I didn't choose?" — a completely different, much harder question. ## What the minority who ship actually do The teams that cross the gap aren't the ones with a cleverer prompt or a secret model. They're the ones who did the boring engineering the demo tempted everyone to skip. Concretely: - **They measure instead of vibe.** A demo is judged by how it feels; production is judged by numbers. The teams that ship have a held-out [eval set](/blog/evals-or-it-didnt-ship) and know their real success rate before users do. You cannot improve, or even trust, what you don't measure. - **They ground the model.** So that the thousandth answer isn't confidently wrong, the facts come from a deterministic source and the model only phrases them — a [constraint, not a prompt](/blog/grounding-is-a-constraint). This is the single biggest lever on "consistent output quality at volume," which was one of the five killers. - **They instrument it.** "No monitoring tooling" is on the list of failure causes for a reason. Survivors can see what their agent did, where it drifted, and what it cost — in production, not just in a notebook. - **They scope it narrow and give it an owner.** Not a god-agent that does everything in the demo, but a small agent with a defined job, living inside real systems, with someone accountable for it. "Unclear ownership" kills as many pilots as bad tech. None of that is AI wizardry. It's [the same engineering discipline](/blog/cheap-code-is-expensive) that separates software that lasts from software that demos well and collapses. The agent just makes the gap more visible, because a guesser punishes missing discipline faster than ordinary code does. ## The demo and the product are different skills Here's the uncomfortable core of it. A great demo optimizes for "look what this can do" — maximum capability, one curated shot. A production system optimizes for "this does the boring thing reliably, forever, on inputs nobody screened." Those aren't the same skill, and they're often in tension. The demo is essentially a sales artifact. The product is an engineering one. The 95% who stall built the first and assumed the second would follow. It doesn't. So if you're staring at an agent that wows in the demo and won't survive contact with real users, the missing piece almost certainly isn't a better model or a smarter prompt. It's the unglamorous part: evals, grounding, monitoring, scope, ownership — the engineering that turns "it worked once" into "it works every time." The minority who reach production aren't smarter about AI. They just didn't skip the boring 80%. That's the whole secret. The demo is the easy part. Everyone can get the demo. The product is the engineering after it — and that's the part that was never optional.