Your AI ROI Is Sitting in the Review Queue
84% of developers say AI made them faster. Only 20% of companies can prove it. The saved hours are real — they are sitting in your review queue, and you can measure that today for free.
Last Thursday GitKraken published a survey of 554 developers and engineering leaders containing a finding that should unsettle anyone currently signing an AI invoice: 84 percent of developers say AI made them more productive, and only 20 percent of organisations measure productivity in any specific way. The standard reading is that engineering has a measurement problem and needs a better dashboard. I think that reading is wrong, and expensively so. The data required to answer the question is already sitting in your Git provider. When you actually look at it, the answer is not missing — it is uncomfortable. The hours are being saved. They are being spent again in the review queue, before a human ever opens the pull request.
Why this matters now
The GitKraken State of AI in Engineering report, published on 20 August, settles the adoption question and opens a harder one. 96.4 percent of teams have adopted AI coding tools. Meanwhile 39 percent of organisations have no way to measure AI's impact at all, and another 33 percent rely entirely on developers self-reporting that it helped. That is 72 percent running on belief. The same survey shows the delegation shift underneath it: in September 2025, 7.6 percent of developers said assigning whole tasks to an agent was their primary way of working; by June 2026 it was 28 percent, and 34 percent now keep agents running the entire workday.
DX's Q2 2026 report puts numbers on the other side of the ledger, drawn from more than 500 teams. AI-generated code went from 34 percent of the total in Q1 to 52 percent in Q2. Developers save an estimated four to six hours a week. Median pull request sizes have nearly doubled. The Developer Experience Index fell from 67 to 65 across four quarters, and code maintainability improved 3.8 percent while change confidence dropped 6.1 percent — engineers understand the codebase better and trust their own deploys less. Most importantly for anyone with a budget: median quarterly organisational AI spend climbed from roughly $1.5K to roughly $44K over four quarters, and the innovation ratio, the share of time spent building new things rather than maintaining old ones, stayed flat.
Both reports come from vendors who sell measurement products, which is worth stating plainly. It does not make the numbers wrong — they are the most specific public data available and the methodologies are disclosed — but it does mean the conclusion they reach for deserves independent checking. Their conclusion is that you need a platform. Read the numbers together and a different one falls out.
The bottleneck moved, and it is not where people are looking
LinearB's 2026 Software Engineering Benchmarks Report, built from more than 8.1 million pull requests across 4,800 teams in 42 countries, isolates the stage that actually broke. Agentic AI pull requests wait 5.25 times longer than unassisted work before a reviewer picks them up — over 16 hours on average, against roughly 200 minutes. They are about 2.6 times larger. And then the finding that reframes everything: once a reviewer picks them up, AI pull requests are reviewed roughly twice as fast.
Pickup time is the interval between a pull request being opened and a human starting to review it. It is the only stage in your pipeline where nothing is happening and everyone is busy.
That combination is the entire argument. If reviewers were drowning in AI-generated code, review duration would climb. It falls. What climbs is the wait before anyone begins. That is not a capacity problem, it is an avoidance problem, and LinearB names the reasons: the diff is large, the mental load of evaluating it is unpredictable, and the ambiguity about who really wrote it removes the social pull that normally makes you help a colleague ship. So the four to six hours a week your developers are genuinely saving are real, and they are being deposited straight into a queue where they earn nothing. The innovation ratio is flat because the savings never arrive anywhere they could convert.
The useful consequence is that you do not need to buy anything to see your own version of this. Three of the four numbers that matter come out of the GitHub API.
// ai-roi-ledger.ts — read-only. Computes the three numbers that tell you where
// your AI savings actually went. No vendor, no agent, no writes: it reads the
// pull request history you already have.
//
// GITHUB_TOKEN=<repo read scope> npx tsx ai-roi-ledger.ts <owner> <repo>
const OWNER = process.argv[2]
const REPO = process.argv[3]
const WINDOW_DAYS = 90
const REWORK_WINDOW_DAYS = 14
interface Pull {
number: number
created_at: string
merged_at: string | null
additions: number
deletions: number
}
async function gh<T>(path: string): Promise<T> {
const res = await fetch(`https://api.github.com${path}`, {
headers: {
authorization: `Bearer ${process.env.GITHUB_TOKEN}`,
accept: 'application/vnd.github+json',
},
})
if (!res.ok) throw new Error(`${res.status} on ${path}`)
return res.json() as Promise<T>
}
const hoursBetween = (a: string, b: string) =>
(new Date(b).getTime() - new Date(a).getTime()) / 36e5
function median(xs: number[]): number {
if (!xs.length) return 0
const s = [...xs].sort((a, b) => a - b)
const m = Math.floor(s.length / 2)
return s.length % 2 ? s[m] : (s[m - 1] + s[m]) / 2
}
async function main() {
const since = new Date(Date.now() - WINDOW_DAYS * 864e5)
const pulls = await gh<Pull[]>(
`/repos/${OWNER}/${REPO}/pulls?state=closed&per_page=100` +
`&sort=updated&direction=desc`,
)
const merged = pulls.filter(
(p) => p.merged_at && new Date(p.created_at) > since,
)
const pickups: number[] = []
const sizes: number[] = []
const touched = new Map<string, string[]>() // file -> merge timestamps
for (const pr of merged) {
// Pickup time: opened -> first human review activity. This is the number
// that moves when an agent writes the code, and the one nobody puts on a
// board. Review *duration* is the number everyone watches instead.
const reviews = await gh<{ submitted_at: string }[]>(
`/repos/${OWNER}/${REPO}/pulls/${pr.number}/reviews`,
)
const firstReview = reviews
.map((r) => r.submitted_at)
.filter(Boolean)
.sort()[0]
if (firstReview) pickups.push(hoursBetween(pr.created_at, firstReview))
sizes.push(pr.additions + pr.deletions)
const files = await gh<{ filename: string }[]>(
`/repos/${OWNER}/${REPO}/pulls/${pr.number}/files?per_page=100`,
)
for (const f of files) {
touched.set(f.filename, [
...(touched.get(f.filename) ?? []),
pr.merged_at!,
])
}
}
// Rework proxy: a file merged twice inside two weeks did not land right the
// first time. Directional, not an audit — legitimate iteration looks the
// same. Watch the trend, not the absolute value.
let reworked = 0
for (const stamps of touched.values()) {
const s = [...stamps].sort()
for (let i = 1; i < s.length; i++) {
if (hoursBetween(s[i - 1], s[i]) < REWORK_WINDOW_DAYS * 24) {
reworked++
break
}
}
}
const medianPickup = median(pickups)
const mergedPerWeek = merged.length / (WINDOW_DAYS / 7)
console.table({
'merged PRs (90d)': merged.length,
'median PR size (LOC)': Math.round(median(sizes)),
'median pickup (hours)': Number(medianPickup.toFixed(1)),
// The queue tax. Latency you add every week by shipping faster into a
// review process that did not change. Compare it to the hours your team
// says AI saved them, then decide which number to take to the board.
'queue tax (hours/week)': Math.round(medianPickup * mergedPerWeek),
'files reworked <14d (%)': Math.round((reworked / touched.size) * 100),
})
}
main()Run that against ninety days of history and you get a number in about ten minutes. The rework proxy is the weakest of the three and I would not defend it in isolation — legitimate iteration looks identical to rework at this resolution. Watch its direction over quarters rather than its absolute value. The pickup and size numbers are exact, and they are the two that matter most.
The four-number AI ROI ledger
Four numbers, in this order, answer the question a CFO is actually asking.
- Claimed savings. Hours per developer per week, self-reported. DX's benchmark is four to six. This is your hypothesis, not your answer — 33 percent of organisations stop here and call it measurement.
- Queue tax. Median pickup time multiplied by merged pull requests per week. This is latency you added, denominated in the same units as the savings, which is what makes the two comparable at all.
- Rework rate. Share of merged work touched again inside two weeks. This tells you whether the speed held after the merge, which is the question DX's falling change-confidence number raises.
- Innovation ratio. Share of merged work on new capability rather than maintenance. This is the only one that requires a label on your tickets, and it is the only one that proves the savings converted into anything.
Your AI return is the first number minus the second, discounted by the third, and it is only real if the fourth moved. Three of the four are free. If the fourth has not moved after two quarters of rising spend, you have not failed to measure — you have measured, and the answer is that the savings are being absorbed somewhere before they reach the product.
What a founder or CTO does with this
The instinct when pickup time is bad is to add reviewers. That is the expensive fix and it treats the symptom. Pickup time is a function of how intimidating the pull request is, so the cheap fixes are the ones that shrink the diff and remove the ambiguity. Cap the size of a pull request an agent is allowed to open — a hard limit in CI works better than a guideline. Require that agent-authored pull requests arrive with test evidence and a stated blast radius in the description, so the reviewer knows the mental load before opening it. Put a pickup service-level objective on the board, four working hours or better, and treat a breach as a delivery incident rather than a nudge in standup.
There is a purchasing conclusion here too. An engineering intelligence platform is a reasonable thing to buy once you know which number you are trying to move and have watched it for a quarter. Buying one first is renting a dashboard to tell you something your Git history already knows. That is the same build-versus-buy reasoning I applied to AI features: own the layer that is actually your decision, rent the layer that is genuinely commodity.
My perspective
Here is the opinion I will defend: the AI measurement gap is not a tooling gap, it is a batch-size gap wearing a tooling gap's clothes. Nobody is failing to measure because the instruments do not exist. They are failing because the honest number is embarrassing and a dashboard purchase feels like progress.
I run twelve services on Keaz with no platform team, which means I am the review queue. When I started delegating serious work to agents, my own throughput numbers looked excellent and the platform did not feel faster. The thing that changed it was not a metric — it was refusing to let an agent open a pull request larger than I would read in one sitting. Throughput on paper went down. Pickup time collapsed, because there was nothing to dread. That is the trade almost nobody makes, because generated lines of code is the number that feels like productivity and pickup time is the number that is.
This is the throughput-side version of an argument I made from the reliability side about the four capabilities a small team needs before it scales AI coding. Both arrive at the same place. AI did not create a new problem; it applied pressure to whichever part of your delivery process you had not automated, and for most teams in 2026 that part is a human deciding when to start reading.
Recommended action this quarter
Run the ledger against ninety days of history this week and write the four numbers down, because the baseline is worthless if you capture it after you start changing things. Set a pickup service-level objective and a hard pull request size cap in CI, then re-run the ledger in thirty days and compare. Add one label to your tickets separating new capability from maintenance, which is the only piece of new instrumentation this requires and the only way the fourth number ever becomes available. If your AI spend is climbing and nobody can currently answer what it returned, working out which of those four numbers your organisation is actually failing on is exactly the kind of engagement I take on as part of my fractional CTO work.
Find out what your AI spend actually returned
If your AI budget is growing and the answer to "what did it return" is a feeling rather than a number, the ledger takes an afternoon to build and the findings are usually clarifying. Book a time and we will run it against your repositories together.
Keep reading
Four Capabilities a Small Team Needs Before It Scales AI Coding
AI raised throughput and cut stability in 2026. The four delivery capabilities that let a five-person team absorb AI-generated code without an incident spike — no platform team required.
Build vs. Buy for AI Features: The Three-Layer Test
"Build vs. buy" is the wrong question for an AI feature. Split it into three layers — model, orchestration, surface — and a five-question test tells you which one to build and which to rent.
Platform Engineering Without a Platform Team
The industry turned platform engineering into a hiring decision. It isn't. The minimum viable platform I run on Keaz across twelve services — no platform team — and when to finally hire one.