Did you know we built a dedicated Celonis app?
Learn more
ph ph-arrow-up-right
When a new technology breaks onto the scene, especially one as hyped as AI, it tends to follow a predictable arc. Initial exuberance, bold forecasts, flashy demos. Then reality sets in.
A recent study from METR has thrown a sharp elbow into the AI narrative. According to their July 2025 report, experienced open-source developers were actually 19% slower using cutting-edge AI tools like Claude 3.5 when working on large, real-world projects. They also thought they were faster. Even expert forecasters expected a 38% speedup. All were wrong.
This isn’t a failure of the tech. It’s a failure of context. And it’s a reminder that we are now deep in what Gartner would call the Trough of Disillusionment.
The METR study offers something rare in the AI gold rush: a controlled, real-world productivity benchmark. No synthetic coding challenges. No cherry-picked autocomplete demos. Just 16 highly skilled developers trying to get real work done on their own massive codebases that are 10+ years old and have over a million lines of code.
And here’s the kicker: AI didn’t help. It made them slower.
Why? Because integrating generative AI into messy, legacy systems isn’t magic. It’s the real work of technology adoption:
These aren’t model problems. They’re process problems.
This is where the tech industry often stalls. Not because the underlying tech isn’t powerful, but because the real gains don’t come from the tech alone. They come from wrapping that tech in the full set of capabilities that make it usable, valuable, and safe inside a business.
We call this the whole product — a concept Geoffrey Moore popularized during the last tech revolution. It's all the ugly, back-end stuff that makes hot tech useful:
For generative AI in software development, that means tools that don’t just autocomplete your functions but understand your SDLC, trace changes across the org, flag risks across teams, and orchestrate what happens before and after that code hits the repo.
In other words: process intelligence and orchestration.
The METR study doesn’t tell us AI is doomed. It tells us where the work is.
This is the Slope of Enlightenment phase: the hard climb back to value. The companies that win this phase are the ones who can industrialize the magic and can turn flashy demos into dependable systems.
At Bloomfilter, we believe the AI-infused future of software development will be orchestrated, not improvised. That means understanding the actual shape of your development processes, making the right tradeoffs visible, and coordinating across the org — not just the IDE.
AI won’t replace your engineers. But it will create new failure modes that only process intelligence can detect—and new opportunities that only orchestration can unlock.
If you’re evaluating AI tools for your engineering org and not seeing the lift you expected—you’re not alone. You may not be behind. You may just be entering the part of the curve where things get real.
The trough isn’t a dead end. It’s a proving ground. And for those who can connect AI to real-world business processes, it’s the only way to reach the Plateau of Productivity.
