AI FinOps

The Delivery Scorecard

What does software you keep actually cost?

Count the full cost of accepted changes. See what catches defects. Understand how work moves.

A practical scorecard for engineering leaders and finance partners making sense of AI delivery.

What delivery costs.What catches defects.How work moves.

The headline measures

Cost needs context.

A lower bill is useful. Pair it with detection and production stability to understand the trade-offs behind it.

01 / Delivery cost

Cost per
accepted change

The fully-loaded cost of producing software that reaches production and stays there, divided by accepted change units.

Model + infrastructure + engineering
+ review + rework
divided byAccepted change units

CPAC brings the model bill, human effort, and rework into the same picture. Keep the five components visible so you can explain what moved.

02 / Automated detection

Machine
catch rate

The percentage of all known, confirmed issues that a machine first discovered before production.

Machine-first discoveries before productiondivided byAll issues discovered before or after production× 100

MCR credits discovery. A human can write the fix. Issues first found by people or after production count in the total, but earn no machine credit.

A hypothetical example

Who found the issue first?

One hundred distinct, confirmed issues in a declared change cohort, observed through the same cutoff.

  • 70 machine discoveries before production
  • 20 human discoveries before production
  • 10 first discoveries in production, by anyone

Machine catch rate 70 / 100 = 70%

A machine fixing a human-found issue does not earn credit. A machine finding an issue before production does, even if it later ships unresolved. Detection and prevention are different measures.

Production stability & recovery

What happens after shipping?

Read machine catch rate alongside these established DORA measures. They add the production view.

03 / Change fail rate

How often does a deployment need intervention?

Deployments requiring immediate intervention, such as a rollback or hotfix, divided by all deployments, × 100.

Count affected deployments, not individual defects. Declare the application and reporting period.

04 / Deployment rework rate

How much deployment activity is repair?

Unplanned deployments made in response to a production incident, divided by all deployments, × 100.

One failed deployment can require several repair deployments. Keep this separate from rework cost.

05 / Failed deployment recovery time

How long does recovery take?

Elapsed time to recover from a failed deployment that requires immediate intervention.

Declare the failure and recovery timestamps. Report the median and a tail percentile; no failures means N/A, not zero-time recovery.

Flow through the system

Finished work. Work still waiting.

The four Kanban flow measures help you see both. Define “started” and “finished” for your workflow, then keep those boundaries consistent.

Illustrative flow readings · not team results or targets

01 / Flow

Throughput

18items / week

How much work finishes?

Count completed work items per period. Use actual item counts, separate from CPAC’s size-normalized change units.

02 / Flow

Cycle time

3.2days · median

How long does work take?

Elapsed time from the defined start of work to its defined finish. Show the distribution, including a tail percentile such as P85.

03 / Flow

Work in progress

8unfinished items

How much is in flight?

Count items that have started but have not finished. Read it with throughput and age to understand how work is moving.

04 / Flow

Work item age

9days · oldest item

What might be getting stuck?

Elapsed time since each unfinished item started. Inspect the individual ages; an oldest-item reading is just a starting point.

Deployment and acceptance have different clocks.

Do not add CPAC’s survival window to ordinary cycle time. Report acceptance maturity separately. DORA’s change lead time runs from commit to production; deployment frequency counts deployments. Neither is the same as work-item throughput.

The counting rules

Make the numbers reproducible.

Agree on scope before reading the trend. These measures are for improving a delivery system, not ranking individuals.

Accepted means it stayed.

An accepted change reaches production and survives a defined window: by default, 30 days post-merge. A revert, defect repair, or harm-neutralizing feature flag within that window excludes the original change from the denominator; its production and rework costs still count.

Teams may choose 14 or 60–90 days. Apply the window per change and label cohorts that have not matured. Acceptance is observable survival, not proof of complete correctness.

Size-normalize consistently: a PR of 1–500 substantive changed lines is 1 unit; a PR of N lines is ⌈N / 500⌉ units. Exclude vendored and generated code, lockfiles, and bulk imports.

Credit the first discovery.

Use one record per confirmed defect, deduplicated across tools and people. Automated review, tests, scans, and evals can earn machine credit. A human starting a check or confirming its finding does not remove credit; a regression test written after human diagnosis does not create it.

Include human-found, production-found, and unknown-source issues in the denominator. Unknown attribution earns no machine credit. Exclude false positives and feature requests.

Declare the change cohort, its production boundary, and the observation cutoff. Include pre-production defects fixed before release. Later discoveries can revise MCR. With no known issues, report N/A.

Known issues are the limit of the view. MCR does not count undiscovered defects or weight severity. Show the counts and observation cutoff, and compare like-for-like scopes and observation durations. A high rate alone does not establish safety or justify less human review.

A hypothetical cost example

The model bill is one line.

A ten-engineer team measures four weeks of cost. Loaded engineering time is $150 per hour. The example cohort has completed its acceptance window.

Hypothetical four-week delivery costs
Cost componentAmount
Model / API spend$1,200
Infrastructure$400
Engineering · 120 hours$18,000
Review · 40 hours$6,000
Rework · 16 hours$2,400
Fully-loaded cost$28,000

What the team kept

42 accepted change units

36 surviving PRs: one 1,800-line PR counts as 4 units; three 600–1,000-line PRs count as 2 each; the other 32 count as 1 each.

$28,000 ÷ 42$666.67 / unit

Allocate each cost once: engineering time covers specifying, prompting, integrating, and steering; review covers validation and gating; rework covers defect repair and reversions. Infrastructure includes attributable compute, storage, observability, and tooling. Use consistent allocation assumptions across periods.

Get the tracking templates

When a headline moves

Find the work behind the number.

Keep these diagnostics beneath the scorecard. They help explain where cost and delay are coming from.

Review waiting time
Elapsed time waiting for review, separate from time spent reviewing.
Blocked time
Time work cannot advance because of a named dependency or obstacle.
Human review hours
Actual review effort, valued consistently in CPAC’s review cost.
Rework cost
Money spent repairing or reverting failed changes, included in CPAC.

Read trends within the same team and scope. No combined score, universal target, or single metric establishes business value.

The runtime companion

Running agents, too?

Cost per accepted outcome (CAPO) applies the unit-cost lens to agent work that was accepted and stayed accepted, including oversight, remediation, failed runs, and failure impact.

The term was coined by Nikhil Mungel; this site adopts it and develops the cost breakdown and survival-window approach. Read the runtime definition →

The framework behind the cost metric

Build the right thing.
Verify it. Count the cost.

Cost per accepted change was defined in The Delivery Gap (Brenn Hill, 2026) as the cost vertex of the Verification Triangle. Intent clarity and verification quality supply the context around it.

The Delivery Scorecard brings that cost view together with machine detection, established DORA measures, and Kanban flow metrics. Their names and sources stay visible. How to cite →

Verification Triangle — cost vertex highlighted Intent clarity Verification quality Cost THE VERIFICATION TRIANGLE