Drafting SAR narratives with LLMs, safely

A suspicious activity report narrative is a legal document that law enforcement reads and can subpoena years later. The safe pattern is narrow: an investigator gathers and validates the evidence, decides the case is reportable, then the model drafts prose grounded in that evidence. A person reviews and signs. The model never files and never asserts an untraceable fact.

Most of a SAR narrative is retelling. By the time drafting starts, an investigator has already concluded the activity is reportable. The narrative just explains, in plain prose, what happened, who was involved, and why it looked suspicious. That is why an LLM fits: the reasoning is done, and what remains is disciplined writing over a fixed set of facts.

The failure mode that matters here is a confident sentence carrying a wrong number. A clumsy sentence gets fixed in review. A fluent one that says a customer wired $48,000 to three counterparties, when the transaction log says $84,000 to two, can slip past a tired reviewer and land in a federal filing. No amount of fluent writing repairs that. So the entire design points at one target: never let an unverified fact into the draft.

Ground every sentence in evidence

The narrative is generated from a structured evidence set, not from the model’s memory of the case. Before generation, the investigator’s file is resolved into discrete items: transactions with amounts and value dates, counterparty entities after entity resolution, KYC attributes, prior SARs on the same subject, and the alert or alerts that opened the case. Each item carries a stable identifier.

The model receives those items and writes the narrative referencing them. We then run a grounding check that is stricter than the generation:

• Every monetary amount, date, account number, and named entity in the draft must match a value present in the evidence set. Numbers are compared after normalization, not as strings, so “$48,000.00” and “48000” reconcile.
• Every factual claim maps to at least one evidence identifier. Sentences that assert something with no backing item are flagged and stripped before a human sees the draft.
• A complete narrative covers five elements: who, what, when, where, and why the activity is suspicious. The check confirms each is present. A draft missing the “why” is incomplete no matter how well it reads.

This is the same discipline you apply when pulling figures out of source documents in the first place. The extraction has to be point-in-time correct, and it has to carry lineage. If the transaction log was amended after the alert fired, the narrative should reflect the state of the evidence at filing, and you should be able to show which snapshot you used. Leakage of later-corrected data into a narrative describing an earlier period is a quiet way to file something wrong.

Keep the model out of the filing decision

There is a strong temptation to let the system decide whether a case warrants a SAR. Resist it. The reportability decision carries legal weight under the Bank Secrecy Act, and it belongs to a trained investigator. The LLM enters afterward, as a drafting tool.

Practically, that means the pipeline has a hard gate. A case only reaches narrative generation once an investigator has dispositioned it as reportable and locked the evidence set. The model’s output is always a draft in a review queue, never a submission. Straight-through processing is the right ambition for lower-risk automation like alert triage, but it is the wrong ambition here. Aim for a good first draft that shortens review, and keep the reviewer seat filled.

Structured output helps enforce this. Rather than free text, the model returns the narrative as sections with their supporting evidence identifiers attached, so the reviewer sees each claim next to what backs it. The reviewer edits in place, and every edit is captured. When someone changes “$48,000” to “$84,000,” you record who changed it, when, and against which evidence item, because that edit is now part of how the institution decided what to file.

Build the audit trail before you build the prompt

Regulators do not grade prose. They ask you to reconstruct decisions. So the logging schema deserves as much attention as the generation logic, and it should exist before the first prompt is written.

For each drafted narrative, persist:

• The exact evidence set passed to the model, with identifiers and the snapshot timestamps of each source.
• The prompt template version and the model version, so a draft written in March can be explained even after both have changed.
• The raw generated draft, the grounding check results, every reviewer edit with author and timestamp, and the final filed text.

Model versions and prompt templates drift. When a vendor updates a model or you revise a template, narrative style and behavior shift, and you need to know which version produced a filing that is later under review. Treat the prompt template as versioned code and pin the model. Do not let “latest” silently change what your filings look like from one quarter to the next.

Evaluation follows from the same records. Build an eval set of past cases where the filed narrative is known, and measure new drafts against it: grounding failures caught, amounts and dates that reconcile against the evidence, element coverage, and how heavily reviewers edit before signing. Edit distance is a useful proxy for draft quality over time. If reviewers start rewriting more, something upstream has drifted, whether the evidence resolution, the template, or the model, and the eval set tells you before a regulator does.

None of this makes the narrative “AI-written” in any way that matters legally. A person reviews it, a person signs it, and a person is accountable for it. The model assembles a grounded first draft and shows its work. That is the job, and it is a real one.