Measuring AI ROI in finance operations: the metrics CFOs accept

A CFO signs off on metrics tied to cash and risk: payback on the build and run cost, unit cost per processed item before and after, the change in error and rework rates, and the cost of the mistakes the system makes. Hours saved feeds those numbers; it is not one of them. Present it alone and a good finance leader will ask what actually changed on the P&L.

That question is fair, and most AI business cases fail it. A model that drafts reconciliations faster has saved nothing if the drafts still get fully re-checked by hand, or if the freed hours never turn into a headcount decision, redeployed work, or growth absorbed without hiring. The value shows up in three places, and each has a metric a controller will accept.

Payback and unit economics come first

Start with the number the business already trusts: cost per unit of work. Cost per invoice processed, per reconciliation cleared, per alert dispositioned, per credit memo drafted. Measure it fully loaded before the system exists, then measure it again after the system is in production and stable, with the AI cost included.

The “after” number is where most estimates go soft. It has to carry:

• Inference cost per item at production volume, not the demo prompt. Long documents, retries, and multi-step chains cost real money at quarter-end peaks.
• The human review the workflow still requires. If a person checks every output, you have moved effort rather than removed it. The honest figure is the review cost at your actual straight-through processing rate.
• Infrastructure, the feature store and data pipelines feeding the model, monitoring, and the engineering time to keep evals and retraining current.

With a defensible before-and-after unit cost, payback is arithmetic. Total build cost plus first-year run cost, divided by the annual saving at expected volume, gives the period. Two things make or break it. The first is straight-through processing rate: the share of items the system completes with no human touch. A jump from 40 to 70 percent STP moves the case far more than any model accuracy headline, because review labour is usually the largest line. The second is volume sensitivity. Fixed build cost amortises over throughput, so a workflow running ten thousand items a month and one running a hundred are different investments even with identical models.

Quality metrics finance will actually read

Accuracy and F1 mean nothing to a controller. Translate model quality into the language of the close and the audit.

• Exception rate and its trend. What fraction of outputs need human correction, and is it falling as the eval set grows and the model is tuned?
• Rework cost. When the system is wrong, how long does the fix take, and does a bad output cost more to repair than the manual process it replaced? A confident wrong number in a reconciliation can cost more than the whole draft saved.
• Cycle-time change on something the business already reports. Days to close, days sales outstanding, alert backlog age. Tie the system to a metric that appears in the board pack and the case survives scrutiny.
• Audit trail completeness. Every AI-assisted decision needs lineage: which inputs went in, which model version and prompt produced the output, and what a human changed. Under model-risk expectations like SR 11-7, a model you cannot document is a model you cannot use, and rebuilding that trail after the fact is its own cost.

The discipline that keeps these numbers honest is point-in-time correctness in how you evaluate. Measure quality on data as it stood when the decision was made, with no lookahead into information that only arrived later. Leakage inflates offline metrics and produces a business case that evaporates in production. Hold out a curated eval set, version it, and report against it rather than against whatever the model saw last week.

Pricing the risk: false positives and what slips through

Every finance-ops model sits on a threshold, and the threshold is an economic choice more than a technical one. Move it one way and you catch more but flood reviewers with false positives. Move it the other and you clear more automatically but let more errors through. You cannot argue ROI without pricing both sides.

The cost of false positives is concrete. Take the false-positive rate at your chosen threshold, multiply by volume, and multiply by the fully-loaded minutes an analyst spends dispositioning each one. In transaction monitoring that number is often the dominant operating cost of the whole system, larger than the models and the platform combined. Cutting it is frequently where the real return lives, which is why a false-positive budget belongs in the business case as an explicit line rather than an afterthought.

The cost of what slips through is harder but has to be estimated, because ignoring it is how you build a case that optimises the wrong threshold. For each false negative, price the expected loss: fraud paid out, a fine, a restated figure, remediation. Some of these are severe and rare, so you work with expected values and state the assumptions. A model that halves review labour while doubling missed fraud is not a win, and only a two-sided cost model shows that.

Two more risks deserve a line each. Drift erodes returns silently. A model that was well-calibrated in January can degrade by quarter-end as behaviour and data shift, so the run cost has to include monitoring and periodic recalibration, and the business case should assume maintained performance rather than launch-day performance. The second is concentration. If one automated decision path handles most of your volume, the cost of it being wrong scales with that share, which changes both the review design and the risk number.

Put together, the case reads as one sentence a CFO can act on: at this volume and this threshold, the system costs X per item all-in, clears Y percent without a human, and its errors carry an expected cost of Z, paying back the build in N months. Every term in that sentence is measurable. None of them is hours saved.