A model extracts an amount from a filing. It says the value is 4.2 million. Is that dollars or euros? Is 4.2 the figure, or did the model round a 4.23 that mattered? Was that number even in the document, or did the model fill a plausible-looking gap? On their own, none of these questions have an answer you can defend to an auditor. The model produced prose, and prose is not something a downstream finance system should ever be allowed to act on.
This is the part of finance AI that gets the least attention and causes the most damage. Everyone focuses on whether the model is smart enough. The harder problem is making its output boring enough: predictable in shape, checked before use, and safe to refuse. A number that lands in a ledger has consequences. The path it took to get there has to be one you can inspect.
Constrain the shape before you trust the value
The first move is to stop accepting free text at all. Modern models support structured output and function calling, where you hand the model a JSON schema and it returns a value that conforms to it. Instead of asking for “the revenue figure” and parsing whatever comes back, you define a function with typed arguments: an amount as a number, a currency as an enum, a period as a date, a confidence field, and a source span the model has to point at. The model fills that contract or it fails the call.
That alone removes a whole class of failures. You no longer parse “4.2 million USD” out of a sentence and hope the regex held. But schema conformance is not the same as correctness. JSON-schema-constrained output guarantees the response is well-formed. It says nothing about whether the value is right, in range, or even real. That is a separate job, and skipping it is where teams get burned.
Validate types and ranges before anything acts
Output validation runs after the model and before any system reads the value. The schema gets you well-formed data; validation decides whether you act on it. We treat these as distinct layers because they fail for different reasons and need different responses.
The checks we put between the model and the ledger:
- Type and unit: the amount is a number, the currency is one you trade, the date resolves to a real reporting period
- Range and sanity: the figure sits within bounds you can justify, so a quarter-end revenue of negative twelve, or one three orders of magnitude off the prior period, gets stopped rather than booked
- Source grounding: the value can be tied back to the span the model cited, so an extraction that points at nothing is treated as a refusal, not a result
- Cross-field consistency: the period, currency and entity agree with each other and with what the rest of the record already knows
A value that fails any of these does not get a second-best guess substituted in. It gets routed to a human or rejected. The whole point is that nothing acts on a number the system could not check.
Refusals, confidence, and not acting
Some of the time the right output is no output. The figure is not in the document, the model is guessing, or the confidence field comes back low. A finance system needs an explicit path for that case, the same way a credit decision needs a reason code rather than a shrug. We make refusal a first-class return value, with its own handling, so a low-confidence extraction queues for review instead of quietly defaulting to zero. Silent defaults are how a missing value becomes a wrong value that nobody flagged.
Then there is the boundary between the model and the systems it touches. Model calls fail, time out, and get retried, and a retry that books the same transaction twice is worse than the original failure. So the action a validated value triggers has to be idempotent: keyed on the source record and period, so replaying it is a no-op rather than a second entry. Build the idempotency key first, before you wire in the model, because retries are not an edge case here. They are the normal operating condition of anything talking to an LLM over a network.
None of this makes the model smarter. It makes the model’s output something you can account for. The model proposes a value; the schema, the validators and the idempotency layer decide whether that value is allowed to mean anything. In finance that ordering is not optional. A number you cannot check is a number you cannot book.