The Productivity Mirage
Why every AI productivity number you’ve seen is a gross figure, and what happens when you look at the net
In July 2025, a research lab called METR published the result of a randomised controlled trial on AI coding tools. The setup was rigorous. Sixteen experienced open-source developers, average five years of experience on the mature codebases they were working on, completing 246 real tasks, screens recorded, timings verified. They could use any AI tools they wanted on half the tasks. On the other half, no AI. Same people, same projects, same work.
Before the trial, the developers predicted AI would make them 24 per cent faster. After completing the tasks, they reported feeling 20 per cent faster. The actual measurement showed they were 19 per cent slower.
Forty-four points of gap between what they believed was happening and what was happening. Not because they were lying. Because they couldn’t feel it. The rework was distributed across the day, the reading of AI output, the correction of AI output, the debugging of AI output, the re-prompting when the first attempt was off. Each of those micro-moments felt productive. Stacked up, they were net-negative.
That’s the productivity mirage. You can’t see it from the inside.
The numbers you’ve been quoted
McKinsey: AI could automate 30 per cent of US work hours by 2030. GitHub’s own research: Copilot makes developers 55 per cent faster. Goldman Sachs: generative AI could raise global GDP by seven per cent. Klarna: its AI assistant handled 2.3 million customer conversations, doing the work of 700 full-time agents.
Every one of those numbers is gross. None of them is net of rework, correction, quality loss, or downstream repair.
The GitHub 55 per cent figure came from a controlled experiment where developers wrote a specific HTTP server function. The task was narrow, well-defined, and closely aligned to what Copilot had been trained on. The METR number is from real work in mature codebases where the developer has to hold the existing architecture in their head. Both numbers are real. They just measure different things.
Guess which one gets quoted in board papers.
The Next Rung is a book on how AI is quietly dismantling the middle of knowledge work, and what you can do about it before the market decides for you: pre-order it before it publishes in January.
What gross means in a P&L
Imagine you run a marketing team of ten people producing 100 pieces of content a month. You roll out an AI tool. Output doubles to 200 pieces a month. The productivity dashboard shows a 100 per cent gain. Leadership reallocates the team, lets three people go, congratulates itself.
Now count what actually happened. Of the 200 pieces, 60 needed substantial rewriting because the AI output was bland, factually off, or missed the brand voice. That’s 60 pieces at a couple of hours each, call it 120 hours of rework that didn’t exist before. Another 20 pieces got published with minor issues that nobody caught until a customer pointed them out. That’s customer trust costs, hard to measure, paid in tiny increments over months. A further 30 pieces got deployed in campaigns where the performance data came back 15 per cent below prior benchmarks, because the work was technically complete but commercially weak.
The productivity dashboard still shows a 100 per cent gain. The rework sits in a different budget line. The customer complaints sit in CX. The campaign underperformance sits in media. The three people who got let go sit on LinkedIn.
The CFO sees the headline number. The CMO’s bonus is paid against the headline number. The AI vendor’s case study uses the headline number. Nobody is looking at the rework ledger, because nobody set one up.
This isn’t a thought experiment. It’s the Klarna story.
The Klarna case, in detail
Klarna replaced around 700 customer service staff with an AI assistant in 2023. By February 2024, the AI was handling 75 per cent of customer chats, around 2.3 million conversations, in 35 languages. The company publicised cost savings of around $40 million annually. It was used as a poster child for AI-driven cost cutting. The CEO, Sebastian Siemiatkowski, spoke at conferences. Stock analysts cited it.
By spring 2025, Klarna was hiring customer service agents back. The reason, in Siemiatkowski’s own words: “We focused too much on efficiency and cost. The result was lower quality, and that’s not sustainable.”
Translated out of CEO-speak: the AI deployment saved money on the headcount line and cost money everywhere else. Customer complaints went up. Satisfaction scores went down. Complex issues didn’t get resolved. Customers who needed a human got a loop of generic AI responses. Some of them churned. The lifetime value loss from those churned customers almost certainly exceeded the saved salary budget, though nobody has published that number because nobody is forced to.
Klarna’s current model is hybrid: AI for routine, humans for escalation and complexity. This is the right model. It’s also, notably, more expensive than the original AI-only model and less profitable than just having humans who were trained properly in the first place. The rework cost of the original deployment was the entire cost of running the AI deployment, plus the customer damage, plus the cost of rebuilding the customer service function afterwards.
That cost never appeared on the productivity dashboard. The productivity dashboard still shows “75 per cent of chats handled by AI”.
The difficulty of measuring net
Rowan Jackson’s work on errors, rework, and waste (ERW) identifies the core problem. Rework doesn’t appear in any standard financial reporting structure. Companies account for cost of goods sold, cost of labour, and cost of overhead. They don’t account for cost of rework. They don’t account for cost of error. They don’t account for cost of work that was done but didn’t need to be done, or work that was done but had to be redone because it was wrong.
Rowan’s ERW estimate across industries is 20 to 45 per cent of operating expenses. In construction, the Get It Right Initiative puts avoidable error at around 21 per cent of project turnover, and total costs as high as a quarter of project spend when you count latent defects and indirect effects. These are huge numbers that don’t show up in financial statements because there’s no line for them.
AI deployments are now being layered on top of this existing invisibility. You can’t find the rework cost of a broken AI workflow in the accounts, because you couldn’t find the rework cost of the manual workflow that preceded it either. The AI vendor can claim productivity gains without ever having to reconcile them against the rework line, because the rework line doesn’t exist.
This isn’t a minor methodological quibble. It’s the main reason optimistic productivity estimates and pessimistic real-world outcomes keep diverging. You can’t manage what you don’t measure. Nobody is measuring.
The Next Rung is a book on how AI is quietly dismantling the middle of knowledge work, and what you can do about it before the market decides for you: pre-order it before it publishes in January.
What a net view looks like
A net productivity view for an AI deployment needs four things that most deployments don’t include.
First, a baseline quality score for the work before the AI rolls out. Customer-facing metrics where relevant. Error rates where available. Completion time and first-pass acceptance rate. Without this baseline, there’s no way to tell whether the AI-augmented output is better, same, or worse.
Second, a rework tracker. How many outputs needed correction after initial production? How much time did the correction take? Who did the correction? If the rework happens outside the team that deployed the AI, it needs to be traced back, not hidden in a different budget.
Third, a downstream quality check. Does the work actually do what it was meant to do, once it’s out in the world? Customer satisfaction, campaign performance, error rates in dependent processes, support ticket volume. AI that generates plausible output that doesn’t perform is worse than slower human work that performs.
Fourth, honest headcount accounting. If the AI deployment justified reducing headcount by three, but the remaining team plus the AI takes the same total hours to produce the same total net value, then no productivity gain actually occurred. The savings came from paying fewer salaries for the same amount of net work, which is a different phenomenon. It’s a distributional change, not a productivity change.
Most organisations running AI deployments right now are not doing any of this. They’re reporting gross output volume, dividing it by people on the org chart, and declaring victory.
The macro version
The same pattern plays out at the level of the whole economy. The UK Office for National Statistics, as of early 2026, has not yet detected a meaningful AI-driven productivity uplift in the national accounts. This is despite widespread adoption in the professional services sector, the spread of Copilot licences across enterprise IT estates, and the deployment of AI in financial services, healthcare, and retail.
If the gross productivity claims were real, they’d be visible at the macro level by now. The fact that they aren’t suggests one of three things is happening. Either the productivity gains are real but offset by rework and error at a scale that neutralises them; or the gains are captured by capital owners and not passed through to consumers in measurable ways; or the gains are real for a minority of users and non-existent or negative for the majority.
All three of these are plausible. All three of them mean the “AI is a productivity revolution” narrative is, at best, incomplete.
Why this matters for the Next Rung thesis
In the original Next Rung post, I argued that the three historical mechanisms for absorbing displaced workers were all cracking at once. New task creation, demand expansion, and skill transferability.
Demand expansion is the one most likely to stay intact in a standard economist’s model. Even if new tasks don’t materialise and skills don’t transfer, real productivity gains should expand the economy and create new demand somewhere. That’s the mechanism that made the Luddites wrong for two hundred years.
But demand expansion only works if the productivity gains are real and spread broadly. If gross productivity claims are hiding invisible rework costs, the gains are smaller than they look. If capital captures what remains, the gains don’t spread. If the deployment pattern is “replace the worker, absorb the rework in a different line item, keep the savings”, then the displaced worker gets neither a new job nor cheaper goods.
This is why Rowan’s ERW framework matters for the book. It’s not a side argument about quality. It’s a direct attack on the consolation prize that displacement discourse usually falls back on.
Action for anyone reading this
If you’re running an AI deployment, set up a rework ledger before you start. Measure baseline quality. Track correction time. Check downstream performance. Report net, not gross. If you can’t do this, don’t make productivity claims.
If you’re evaluating a vendor’s productivity claim, ask for the net number. If the vendor can’t produce it, assume it doesn’t exist.
If you’re writing about AI productivity, at minimum cite the METR study alongside the McKinsey forecast. Readers deserve to see both sides of the mirage.
If you’re setting corporate strategy on the assumption that AI will deliver headline productivity gains in the next three years, hedge. The Klarna reversal is not an isolated case. It’s the leading indicator.
The next piece in this series looks at why AI errors are particularly hard to catch, and what that means for quality in systems where the output volume outruns the human capacity to verify. If the productivity mirage has a specific mechanism, that’s where it lives.
With thanks to Rowan Jackson, whose ERW framework informs the financial analysis in this piece, and whose Klarna-related observations (shared separately) sharpened the case study.
Primary sources: Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity (METR, July 2025); Klarna public statements and press coverage, 2023-2025; Get It Right Initiative UK construction research; ONS productivity data to early 2026; McKinsey Global Institute, The Economic Potential of Generative AI, 2023; GitHub Next research on Copilot productivity, 2022.