A demo convinces a room. It does not convince an auditor, and it should not convince you. “It looks good” is a feeling drawn from a handful of questions someone happened to ask. The moment a system touches a real decision, that feeling has to become a number you can defend, with a method behind it. LLM evaluation in finance is the discipline of producing that number.
Most teams skip it because building an eval set is slower than shipping. Then the system gives a confidently wrong answer at quarter-end, and there is no evidence to say whether that was a one-off or the third such error this month. The work below is how we avoid that.
Pick the one metric tied to the decision
Before measuring anything, decide what the system is actually for. A tool that extracts figures from filings is judged on extraction accuracy against the source document. A tool that flags reconciliation breaks is judged on recall, because a missed break costs more than a false alarm someone can dismiss in a second. A retrieval-answering assistant is judged on whether the cited passage supports the claim.
One metric, tied to the decision the output drives. Track secondary numbers if you like, but name the one you would stake the deployment on. Teams that report a basket of vague scores are usually avoiding this choice.
Build the eval set from real cases
A good eval set is labelled, and it comes from cases the system will genuinely face, not synthetic questions written to flatter it. We pull from real history: actual filings, actual reconciliation files, actual analyst queries. Each one gets a ground-truth answer a domain expert agrees with.
The hard examples matter more than the easy ones. An eval set finance teams can trust includes:
- Edge cases: restatements, amended filings, footnoted figures that contradict the headline number
- Adversarial inputs: ambiguous entity names, near-duplicate counterparties, prompts that invite a confident guess where the honest answer is “not stated”
- Known-failure cases: every production mistake, folded back in so it is tested forever after
If the set is all softballs, the resulting metric is decoration. The point is to find where the system breaks while you can still fix it cheaply.
Measure against the current process
A number means nothing in isolation. The reference point is the process the system replaces. If a junior analyst extracts these figures today with 96% accuracy, an AI model validation that shows 91% is a regression dressed as progress, however impressive the demo felt.
Baseline metrics from the existing workflow are the bar. Run the eval set through the current process, record the score, and hold the new system to it. If it cannot clear that bar, far better to learn it in week three than after six months of build.
Separate retrieval from generation
When a retrieval-augmented system gets an answer wrong, there are two suspects: it fetched the wrong context, or it reasoned badly over the right context. Collapse those into one score and you cannot tell which to fix.
So we measure them apart. Retrieval quality asks whether the correct passage was among what was returned. Generation quality asks, given the correct passage, whether the answer is faithful to it. A system can post a strong end-to-end number while hiding weak retrieval that will fail the moment the corpus shifts. Splitting the measurement surfaces that.
Keep it as a regression test and an artefact
The eval set is not a one-time gate. Every change to the prompt, the model version, the retrieval index, the chunking, runs against it before it ships. Model providers update weights and behaviour drifts; a silent regression is exactly the failure an eval suite is built to catch.
It also outlives the build. Under model-risk and audit expectations, someone will eventually ask how you know the system performs, and how you knew it before go-live. A versioned eval set, with its labels, scores against baseline, and a dated history of results, is a direct answer. It feeds model validation and conformity assessment rather than leaving you to reconstruct evidence after the fact.
None of this is glamorous. It is the difference between a number you can put in front of a regulator and a feeling you got from a demo.