Using synthetic data in finance AI, carefully

Synthetic data earns a place in finance AI when the real data is too scarce or too sensitive to train or share safely. Use it to augment a rare fraud class and to build development sets outside the production boundary. Keep it out of your evaluation sets, and prove the generator does not reproduce real people.

The appeal is obvious. Confirmed fraud might be a fraction of a percent of transactions. A new sanctions typology has three known examples. A credit product launched last quarter has no defaults yet, because defaults take time to mature. In all three cases the thing you most want the model to learn is the thing you have almost none of. Generating more of it feels like the fix.

It sometimes is. But synthetic data has a habit of solving the visible problem while quietly creating one you find at quarter-end, in production, on real money. The failure is rarely that the generated records look wrong. The failure is that they look right in a way that flatters your metrics and hides a leak.

Where it actually earns its place

There are a few situations where we reach for it without much hesitation, because the alternative is worse.

• Class imbalance in fraud and financial-crime models. When the positive class is a rounding error, the model can score 99.9% accurate by calling everything legitimate. Oversampling the minority class or generating plausible fraud variants gives the training signal something to hold onto. This is data augmentation, and it belongs in the training fold only.
• Building a development environment outside the production data boundary. Engineers need something realistic to build pipelines, features, and tests against. A synthetic copy of the transaction stream lets them work without a copy of real customer data sitting in a lower environment with weaker controls. The lineage stays clean because nothing real ever left the boundary.
• Stress cases and typologies you expect but have not observed. A synthetic-identity ring that layers a thin file over six months, a structuring pattern just under a reporting threshold, a mule account that goes quiet then activates. You can encode a hypothesis about behavior and see whether the model catches it, before it costs you.
• Sharing a dataset with a vendor or across a legal boundary. A generated set, tested for leakage, can move where the real set cannot.

Notice what these have in common. Synthetic data is filling a gap in coverage or removing a real-data copy from a place it should not be. It is not standing in for the ground truth you evaluate against.

The tests that keep it honest

A generator trained on real data is a compression of that data, and compression leaks. Two records that were far apart can end up neighbors. One customer can be reconstructed closely enough to re-identify. Before we call a synthetic set usable, it goes through a fixed set of checks, and the results get written down for the audit trail.

• Near-duplicate and memorization test. For every synthetic record, find its nearest real neighbor. If the distances cluster near zero, the generator is copying rather than generating, and you are shipping real people with a synthetic label on them.
• Membership inference. Train an attacker to guess whether a given real record was in the generator’s training set, using only the synthetic output. If it can, the set is not privacy-preserving, whatever the marketing says.
• Marginal and joint fidelity. Univariate distributions are easy to match and not enough. Check the correlations that matter for the task: amount against merchant category, velocity against account age, the joint structure fraud actually lives in. A generator that gets the margins right and the joints wrong teaches the model a world that does not exist.
• Utility gap. Train the model on synthetic, test on real. Train on real, test on real. The gap between those two tells you what the synthetic data is actually worth. A small gap is a good sign. A suspiciously tiny gap usually means leakage, not brilliance.

That last point is where most teams get burned. If synthetic records derived from a customer’s real behavior end up in the same fold as that customer’s real records, your evaluation is contaminated and your metrics lie. Entity resolution has to run before the split, so that everything tracing back to one entity lands on one side of the wall. Synthetic and real, same entity, same fold.

Keeping the lineage straight

The operational discipline matters as much as the statistics. Every synthetic record carries a tag: which generator produced it, from which training snapshot, under which random seed, on which date. That tag rides through the feature store into every downstream model. When a validator or an examiner asks which of these numbers are real, you can answer without a forensic exercise.

Two rules we do not bend:

• Synthetic data never enters an evaluation or benchmark set. Eval sets measure whether the model works on reality. The moment they contain generated records, they measure whether the model agrees with the generator. Point-in-time correctness applies here too: the real held-out set has to reflect what was actually knowable at decision time, with no lookahead from labels that matured later.
• The generator is versioned and monitored like a model. It drifts. Retrain it on newer data and the synthetic distribution shifts, which quietly moves every model downstream that was augmented with it. Track that the way you track feature drift, because it is the same failure with an extra layer of indirection.

Used this way, synthetic data does honest work. It balances a fraud class the world refuses to balance for you, and it lets engineers build pipelines without holding real identities in a lower environment. What it cannot do is tell you whether the model is right. The discipline is mostly about never letting it pretend otherwise. The generator is a tool for coverage. Reality stays the judge.