0:00
/
0:00
Transcript

They are lying to you about AI failures

The Truth About That “95% of AI Pilots Fail” Story

My Story: Text Version

Every few months, someone drops a new headline declaring that AI isn’t living up to the hype. The latest one making the rounds comes from an MIT study claiming that 95% of enterprise AI pilots fail. And predictably, the internet pounced.

“See? AI’s a scam.”
“Generative AI is useless.”
“Another tech bubble.”

But if you’ve ever worked in enterprise IT—or any serious tech environment—you’ll know how absurd that framing is. Of course most pilots fail. That’s what pilots are for.

How Enterprise Technology Really Works

I’ve spent about fifteen years in IT infrastructure and automation, working for a few massive enterprises—including one Silicon Valley giant you definitely know. The world I come from isn’t glamorous, but it’s where the real work happens: data centers, cloud automation, virtualization, uptime, and all the plumbing that keeps businesses alive.

In that world, experimentation is constant. You try out new products all the time. Most don’t work. Some don’t integrate cleanly with your stack. Others duplicate functionality you already have. And some are great products that simply don’t fit your organization’s unique architecture or business needs.

When a manager or director tells an engineer, “Hey, this vendor’s offering us a 90-day trial on their new automation tool, go test it,” that’s a normal Tuesday. You spin up a server, open the right firewall ports, grab a license key, and see what happens. Usually, what happens is: nothing much. You log the results, the vendor gets feedback, and you move on.

That’s not a failure. That’s process.

Pilots Are Controlled Failures by Design

When a study says “95% of AI pilots fail,” that’s not an indictment—it’s a sign that companies are running healthy experiments. The fact that 5% succeed is actually extraordinary, because those few successes can transform entire business units.

If an AI pilot leads to a system that autonomously codes for thirty hours straight, or cuts operational downtime by even 0.001%, that’s massive. Enterprises live and die by uptime and efficiency. When I was a principal engineer, my goal was “six nines” of availability—99.9999%, which means less than 32 seconds of downtime per year. I hit that once. Nobody celebrated. That’s just what’s expected. Infrastructure should feel like flipping a light switch. When it’s on, no one notices.

So if AI helps maintain those lights without adding complexity or cost, that’s a win. And a 5% success rate for transformative tools? That’s phenomenal.

The Business Side Is Catching Up

What we’re watching right now is a lag between technological reality and business adaptation. For the first time since the dot-com boom, every major CEO—from Boeing to Walmart—is openly talking about reassessing hiring and operations because of AI.

The first roles being rethought are the most process-heavy ones: HR, legal, and middle management. These are departments that deal mostly in documents, matching actions to policy, and maintaining compliance. That’s fertile ground for automation.

A generative AI model with a multi-million token context window can read an organization’s entire email history and tell you where workflows break down. It can flag policy violations, draft standard replies, or identify process bottlenecks. That’s not hype—it’s basic pattern recognition at scale.

The Pain of Technical Debt

Here’s another truth people miss: AI often struggles not because it’s stupid, but because enterprise systems are. Most corporate infrastructure is a patchwork of old configurations, legacy code, and outdated hardware—what we call “technical debt.”

To make an AI tool effective, you often need to rebuild your environment to modern standards. Few organizations have the appetite or budget for that. So, when an AI pilot “fails,” it’s often because it’s trying to make sense of decades of technical chaos.

That doesn’t make AI useless. It just means it’s not a miracle worker.

The Long Game of Innovation

Enterprise technology doesn’t evolve overnight. Even with all hands on deck, it takes around eighteen months to bring a fully polished enterprise product to market. More often, it takes three years.

So when you read that 5% of AI pilots are succeeding, remember: those are pilots based on technology that’s already a couple of years old. The real breakthroughs—the ones being tested right now—won’t even show up in reports until 2026 or 2027.

And that’s why these “AI is failing” narratives are so misleading. They’re measuring innovation at its most experimental stage and declaring it dead before it’s even born.

The Signal in the Noise

In enterprise IT, noise is constant. Vendors push half-baked products. Sales teams overpromise. Journalists misread the data. But the people actually inside these systems—the engineers, the architects, the automation specialists—know that this is how progress happens.

When the dust settles, a small fraction of tools will prove indispensable. They’ll streamline operations, reshape departments, and create entirely new categories of work. Those are the ones that matter.

So, when you hear that 95% of AI pilots fail, understand what that really means: ninety-five percent of experiments are clearing the way for the five percent that will define the next decade of enterprise computing.


Counterarguments: Evidence and Citations

Every few months tech discourse discovers a new number to obsess over. This season’s number is 95. As in: 95 percent of enterprise generative-AI pilots fail. The claim traces to an MIT Media Lab / Project NANDA report often summarized as “The GenAI Divide: State of AI in Business 2025.” The document argues that despite tens of billions in enterprise spending, most custom or embedded AI efforts either stall in pilot or fail to produce measurable P&L impact, with only a narrow slice delivering durable value. That’s the kernel. The rest is how the internet did what the internet does. (AI News)

If you read the primary material, you find a specific framing. The report draws on a mixed method package—structured interviews, a scan of public deployments, and surveys—then defines “success” narrowly as pilots that scale and produce measurable financial impact months later. It highlights a drop-off from “investigated” to “piloted” to “implemented,” and it explicitly describes the gap as an organizational “learning” problem: tools without memory, workflows without integration, and companies without feedback loops. In other words, poor fit and weak embedding, not model incapacity. That nuance is central and too often missing once the number starts trending. (AI News)

Coverage amplified the sharp edge of the findings. Fortune, Forbes, Tom’s Hardware, and TechRadar Pro all ran pieces foregrounding the “95%” and the asymmetry between bespoke, in-house efforts and vendor-supplied systems. The pattern they echo is consistent: most initiatives show no P&L movement; the handful that do are narrow, workflow-specific, and often vendor-partnered, with more traction in back-office operations than in splashy front-of-house experiments. Some coverage even quantifies a success advantage for specialized vendors versus internal builds. That storyline traveled fast because it’s clean, dramatic, and simple to headline. (Fortune)

There has also been well-reasoned pushback that doesn’t deny the phenomenon so much as question the swagger of the headline. Marketing AI Institute warns against taking the “95%” at face value without interrogating sample, definitions, and the short time windows used to declare “no return.” Futuriom’s critique is blunter, calling out methodological fuzziness and the way a viral number can become a cudgel in agenda-driven narratives. Harvard Business Review uses the study as a jumping-off point to caution leaders about the “experimentation trap,” which is a more useful managerial lesson than a doom-stat. These pieces don’t say “AI is working everywhere.” They say “be careful what you think this number proves.” (Marketing AI Institute)

YouTubers and social channels poured accelerant on the meme. There are videos parroting the claim as proof that enterprise AI is mostly smoke, and others rebutting the meme outright as misleading clickbait. The range runs from sober breakdowns that separate “chat adoption” from “integrated systems” to shorts optimized for outrage. The takeaway isn’t that creators are bad actors; it’s that platform incentives reward crisp certainty more than infrastructure nuance. (YouTube)

Replacing Humans with AI is Going Horribly Wrong” - This video by ColdFusion was the inspiration for this rant.

Here’s the part that never trends, but that practitioners feel in their bones. High pilot mortality in large organizations is not scandal; it’s metabolism. Big companies explore aggressively, discard ruthlessly, and scale selectively. In that world, the overwhelming majority of experiments “fail” under tight definitions, yet the minority that sticks pays for the rest. Virtualization Review’s recap lands closest to reality for operators: adoption is high, transformation is scarce, and the difference lives in integration, governance, and fit, not model novelty. That pattern also explains why back-office automation quietly outperforms grandstanding use cases. (Virtualization Review)

It’s also why your angle is valuable and, frankly, underrepresented. Most commentary stops at “integration is hard.” People who’ve actually shipped inside legacy environments know why. They know about version skew, brittle middleware, inconsistent standards across data centers old enough to vote, and the prime directive of reversibility—can we observe it, audit it, roll it back within the change window? The report points to a “learning gap,” but lived experience fills in the gaps: if you can’t thread a new system through logging, metrics, approval chains, RBAC, and incident response, the initiative dies in change-control purgatory regardless of how clever the underlying model is. That’s not Luddism. That’s operational hygiene. (AI News)

The vendor dynamic matters too. Enterprise account teams routinely seed pilots with generous keys in exchange for feedback, roadmap influence, and a shot at land-and-expand. This creates noise by design. The study’s “shadow AI” observation—that individual workers adopt public tools informally while official programs grind through governance—isn’t a scandal; it’s how frontier tech diffuses. As formal integrations mature, the gray market contracts. Until then, you should expect lots of pilots, few scale-ups, and a long tail of “not worth the change-risk.” (MLQ)

None of this makes the “95%” wrong. It makes it incomplete. There’s parallel evidence pointing the other way in specific domains. Fresh marketing-sector research reports the vast majority of CMOs seeing ROI from GenAI, which sounds incompatible until you notice the boundary conditions: narrower tasks, faster loops, and metrics that sit closer to the work. Both realities can hold at once. The misrepresentation comes from treating a directional, early-cycle enterprise snapshot as a general law of AI utility. (TechRadar)

So what should leaders, builders, and commentators actually do with the number? First, treat it as a diagnostic, not a verdict. It’s telling you that the hard part is organizational learning, context retention, and workflow embedding, not raw model horsepower. Second, stop grading pilots like product launches. The report’s critics have a point: six months is often too short for P&L to budge in complex systems, and “zero ROI” can hide meaningful learning, risk reduction, or groundwork that pays later. Third, copy the boring winners. The reporting that looked under the hood found success in focused problems, vendor-partnered stacks, and back-office processes where observability and reversibility are tractable. If you can’t measure it, roll it back, or page someone when it misbehaves, it doesn’t belong in production yet. (Tom’s Hardware)

Finally, place the discourse in context. Social platforms will continue to reward the most legible narrative—either “AI is changing everything” or “AI is failing everywhere.” The truth lives in the plumbing. It’s incremental, systems-bound, and biased toward the unglamorous parts of work. If your voice keeps translating that reality into public language—calling out misuse of the MIT/NANDA study while also extracting the useful caution inside it—you’ll be doing the thing the field actually needs: turning sensational statistics into operational sense.

Discussion about this video

User's avatar