Output validation is the code you put between a model and a finance workflow to catch bad output before it reaches a decision or an audit trail. It works in layers: a schema checks structure, business constraints encode your domain rules, and a verification step ties every meaningful figure back to a source system. Each layer catches a different kind of error the one below it cannot see.
The failure mode that actually hurts is a confident, well-formatted answer with a wrong number in it. That answer looks finished, so it passes review by tired humans and lands in a report. A clumsy sentence never causes an incident. A clean sentence built on a figure that is off by a factor of ten does.
Most teams treat all of this as one problem and reach for JSON schema. Schema is necessary, and it is the weakest of the three layers you need. It proves the shape is right and says nothing about whether the content is true. In finance the content is the whole point, so most of the interesting work sits above the schema.
A strict schema is the floor
Start with a strict schema on every model call that feeds a system. Required fields, typed values, enums for anything categorical, format constraints on dates and identifiers. A response that does not parse never enters the pipeline. This alone removes the long tail of half-emitted objects, trailing commentary the model bolts onto valid JSON, and the occasional field it invents because the prompt drifted.
There are two ways to get the shape right, and they are not interchangeable:
- Constrained decoding forces the model to emit only tokens the grammar allows, so an enum field can physically only produce one of its members and an account ID matches its pattern at generation time. It is the right tool for fixed vocabularies and rigid formats.
- Post-hoc validation lets the model generate freely, then rejects and repairs anything that fails the schema. It is the right tool for anything long or open-ended, where a grammar would fight the model and cost you latency for little gain.
Use constrained decoding on the fields where the space of valid answers is small and known. Use validate-and-retry on the prose. Mixing them per-field inside one response is normal and usually what you want.
What schema will never do is tell you the number is correct. A model can return {"exposure": 4200000, "currency": "EUR"} that satisfies every constraint and is off by a factor of ten because it read the wrong row. Schema validation waves that through. So the floor is important and it is only the floor.
The verification layer does the real work
This is where a finance-grade pipeline earns its keep. Every value that carries meaning gets checked against something that is not the model. The pattern is the same regardless of domain:
- Ground each figure. If the model reports a balance, a covenant threshold or a counterparty exposure, that value must trace to a source record. Attach the lineage. If it cannot be traced, it does not ship.
- Reconcile across sources. When three systems report a figure three ways, the verification step surfaces the discrepancy instead of letting the model silently pick one. Reconciliation raises the conflict for a human to resolve; it never guesses.
- Recompute anything derivable. Ratios, totals, weighted averages and date arithmetic should be computed in code from the grounded inputs, not trusted from the model’s own arithmetic. Models are unreliable calculators and there is no reason to let them do sums you can do deterministically.
- Enforce point-in-time correctness. A figure attached to a reporting date must come from data as it stood on that date. This is where lookahead creeps in: a model happily pulls a restated number or a value that postdates the cutoff, and the output looks fine until an examiner asks why a Q2 memo cites a figure that only existed in Q3.
The verification layer is deterministic code sitting downstream of the model. It knows the schema is already satisfied, so it can assume structure and focus entirely on truth. When a check fails, the item routes to a human queue with the failing check and the conflicting source values attached. It does not retry silently, because a silent retry that eventually passes is how a wrong number reaches straight-through processing.
Constraints that encode your business rules
Between structure and factual grounding sits a third layer that teams skip and later regret: business constraints. These rules are not about JSON shape, and they are not about a single source value. They describe what a valid answer can be in your domain.
Examples that come up constantly:
- Cross-field consistency. A transaction flagged as domestic cannot carry a foreign settlement currency without an explanation field populated. A risk rating of low is inconsistent with an exposure above a stated ceiling.
- Bounds and sanity. A probability outside zero and one, a negative headcount, an interest rate above a plausible band. Cheap to check, and they catch the model when it confidently fabricates.
- Referential integrity. Every entity the output names must resolve to a real record. Entity resolution failures are a common and quiet source of wrong output, because the model invents a plausible counterparty name that does not exist in your book.
Encode these as explicit predicates, version them, and keep them next to the eval set. When a regulator or an internal reviewer asks why an output was accepted, the answer is the list of checks it passed, each one auditable. Under SR 11-7 that traceability is not a nicety; a model in a decision path has to be explainable and monitored, and a validation suite is a large part of how you show it.
One last discipline ties the layers together. Every failure, at any layer, is a labelled example. A schema violation, a reconciliation mismatch, a bounds breach, a human overturning an accepted answer: each one goes back into the eval set and into your drift monitoring. The validation layers do double duty. In production they gate output. Over successive quarter-ends they also tell you whether the model is improving or quietly getting worse against the cases you care about.
FAQ
Does JSON schema validation stop the model from hallucinating numbers?
No. Schema validation only proves the output is well-formed and typed correctly. A figure can be perfectly valid JSON and still be wrong, which is why you need a separate verification layer that checks values against source systems.
Is constrained decoding worth the latency cost?
For enum fields, entity IDs and fixed formats, yes, because it removes a whole class of parse failures at generation time. For long free-text like a narrative, the overhead rarely pays off and post-hoc validation is cleaner.
Where should a failed validation route in a finance workflow?
To a human queue with the specific check that failed and the source values attached, never to a silent retry that hides the failure. The failed cases are also your best material for expanding the eval set.