Leadership has stopped asking whether your team is using AI. They’re asking what you’re delivering with it. That’s a harder question, because most of the numbers teams have been reporting, adoption rate, seats activated, prompts run, don’t actually answer it.

Adoption isn’t the same thing as impact

On a recent webinar recorded in partnership with LeadDev, Stasia Zamyshlyaeva, GitKraken’s VP of Engineering, put the problem in terms every leader has felt before: “80% of my developers are using AI, is that good enough? It’s similar to saying 80% of people have a gym membership. Does that mean they have muscle already, or not?”

A membership card doesn’t build muscle. Attendance and effort do. AI adoption works the same way: the percentage of developers with access to a tool tells you nothing about whether they’re using it well, or whether it’s changing how they work at all.

The three layers of AI ROI

Zamyshlyaeva framed AI ROI as three layers, each one a precondition for the next.

Layer 1: Culture, is behavior actually changing?

Before ROI shows up anywhere else, it shows up as depth of adoption, not headcount of adoption. A developer who ran one AI-assisted commit last month and a developer who’s restructured their entire workflow around it both count in an 80% adoption number. Only one of them is a signal.

Layer 2: System, are the metrics that matter actually moving?

This is where DORA metrics and quality signals come in. But Zamyshlyaeva was direct about their limits on their own: “an engineering team could be running 100 miles per hour in absolutely the wrong direction.”

Velocity and deploy frequency going up is not automatically good news. It’s only good news paired with quality holding steady or improving, which is why this layer has to include both, not just the metrics that make the chart look impressive.

Layer 3: Business, are customers actually better off?

The top layer is the one leadership actually cares about: are customers more satisfied, is the product better, is the business better off. Culture and system metrics are leading indicators. This is the one that closes the loop.

Don’t forget the denominator

Zamyshlyaeva also flagged something most ROI conversations skip: ROI has a denominator, and that denominator is cost. AI spend is relatively low right now compared to where it’s likely headed. Whatever gains show up across the three layers eventually get normalized against that cost, and that math will look different in a year than it does today.

Before you scale the metrics, look at who isn’t adopting

Panelist Jennifer, a freelance journalist covering developer experience, made a point worth sitting with: the developers not using AI yet are a data source, not a compliance problem. Reticence can mean the tooling isn’t accessible to everyone (visual impairments, color blindness), it can mean real fear about job security, or it can mean someone is sitting between junior and senior experience and hasn’t found where AI actually helps their specific work yet. A mandate doesn’t fix any of those. A conversation might.

The ROI you don’t want: burnout wearing productivity’s clothes

The panel didn’t stop at frameworks. Senior engineering manager Vernon raised something leadership teams chasing AI ROI can miss entirely: the same excitement that drives adoption can drive burnout. Agentic coding gives developers a fast feedback loop, ship something, see the outcome immediately, want to do it again, and that loop doesn’t naturally stop on its own.

Zamyshlyaeva pointed to a concrete, watchable signal: usage telemetry that spikes for hours on end, or commits and PRs landing at unusual hours, are worth treating as a leading indicator, the same way you’d treat a leading indicator for velocity or quality.

Measuring wellbeing without making it worse

Qualitative surveys about burnout only work if people feel safe answering them honestly, and the panel was blunt that psychological safety can’t be assumed right now. Two guardrails came up repeatedly: keep these signals at the team or org level, not tied to individual developers, and treat a metric that looks bad as a management problem to solve, not an employee problem to flag.

Metrics are the start of the conversation, not the whole story

The panel’s closing point applies to every layer above: metrics have power, and a board that’s watching them will treat them as more certain than they are. The job isn’t to make the numbers look more precise or more dramatic than what they actually show. It’s to use them as the opening of a conversation with leadership, backed by the context only your team has.

Where this framework needs a tool

A 3-layer framework is only useful if you can actually see the layers. That’s what GitKraken Insights is built for: connecting AI usage to the DORA and quality metrics that show whether adoption is turning into real impact, instead of stopping at a usage percentage. See how it works at gitkraken.com/insights.

How much is your team actually getting back from AI coding tools? Most engineering leaders can’t answer that with a number. We built an AI ROI calculator to change that: plug in your team size, AI adoption, and review cycle time, and see where the real gains (and the real bottlenecks) are hiding. Token counts and lines generated aren’t ROI. Code Flow is: how fast that AI-assisted code actually moves from written to reviewed to shipped. Try the calculator and see what your team’s Code Flow is really worth at ai-roi.gitkraken.com.

FAQ

What is AI ROI in software engineering? AI ROI measures whether AI adoption is producing real business value, not just usage. A useful framework looks at three layers: cultural adoption depth, system-level impact on metrics like DORA and code quality, and business-level outcomes like customer satisfaction, normalized against the cost of the AI tools themselves.

Do DORA metrics measure AI ROI on their own? No. DORA metrics show system-level velocity and stability, but improving velocity without holding quality steady can mean a team is moving fast in the wrong direction. DORA metrics need to be read alongside quality signals, not instead of them.

How do you know if AI adoption is actually helping your engineering team? Look past the adoption percentage to depth of use, then check whether that use is showing up in system-level metrics like velocity and quality, and finally whether it’s translating into better business or customer outcomes. All three layers should move together.

Can measuring AI impact contribute to developer burnout? It can, if usage pressure turns into an expectation that developers should be constantly more productive. Leaders should watch telemetry signals like unusual working hours as leading indicators, and keep wellbeing metrics at the team level rather than tied to individuals.