Aggregating open-banking data you can build on

Aggregated open-banking data is inconsistent across providers. The same salary payment shows up with a different field layout and sign convention at each aggregator, dated to when the record was booked rather than when the money actually moved. Before any model reads it, you need a layer that maps every provider onto one canonical schema and enriches each line into something a lending decision can stand on.

PSD2 gave everyone access to accounts. It did not give anyone a shared format. Two aggregators calling the same bank return payloads that agree on almost nothing at the field level, and a third parses PDF statements into a fourth shape entirely. If you build features on whatever the first provider happened to send, you have coupled your model to that provider. The day you add a second source, or the day the first ships an API version bump, your income figure moves for reasons that have nothing to do with the borrower.

One schema, and everything maps onto it

The first decision is to define your own canonical transaction model and treat every provider as an adapter into it, never the reverse. The canonical record fixes sign convention, timezone handling, the distinction between booking date and value date, and a single notion of counterparty. Each aggregator gets a mapping layer that translates its quirks into that model, with the raw payload retained for lineage.

• Sign and direction. Some providers signal debits with a negative amount, some with a separate indicator field, some with both and they occasionally disagree. The adapter normalises to one convention and rejects records where the two signals conflict rather than guessing.
• Dates. Booking date, value date, and the timestamp the aggregator fetched the record are three different things. Point-in-time correctness depends on picking the right one, so the canonical model carries all three and downstream features declare which they use.
• Counterparty. One provider gives a clean merchant name, another gives raw bank narrative, a third gives an IBAN and nothing else. The mapping preserves what exists and marks what is missing, so enrichment knows how much work is left.
• Currency and account context. Multi-currency accounts, sub-accounts, and pots all need to resolve to the right account identity before anything nets across them.

Getting this layer right is unglamorous and it is where most of the reliability comes from. When a new provider is onboarded, you write one adapter and the rest of the pipeline does not change.

Deduplication and identity across providers

Aggregation multiplies the duplicate problem. A single provider already re-sends transactions and reissues pending entries as settled. Pull the same account through two providers and you get the transaction twice with no shared identifier between them. Entity resolution is what turns overlapping feeds into one ledger.

We resolve each transaction to a stable identity computed from amount, normalised counterparty, booking date and resolved account, then collapse matches while keeping provenance on the surviving record. Two things make this harder than a naive hash:

• The same payment can carry different descriptions and even slightly different dates across providers, so the match has to tolerate near-misses without merging genuinely distinct transactions.
• Internal transfers between an applicant’s own accounts are not income and not spending. Once accounts are resolved across providers, netting removes those transfers. Miss it and every transfer inflates both sides of the affordability picture.

Reconciliation is the check that this worked. Running balance minus summed transactions should tie out per account. When it does not, there is a gap in the feed, and a gap understates income. We reconcile before anything downstream reads the data and hold back accounts where the ledger does not balance, rather than scoring on a partial history.

Enrichment, and what it is allowed to change

Enrichment is where a raw line becomes a feature. Merchant resolution, category, whether the amount is recurring, whether a credit looks like salary or a same-day transfer from a person. A resolved merchant directory plus rules handles the high-volume, unambiguous counterparties at near-perfect precision and no per-call cost. A model handles the long tail: unrecognised merchants, free-text payments, transfers whose purpose has to be inferred from pattern.

Two constraints keep enrichment honest.

Every enriched attribute is computed as of the transaction’s booking date using only information available then. A merchant directory that was updated last week must not be used to relabel a transaction from six months ago in a way that leaks present knowledge into a past decision. This is the same lookahead discipline that governs the feature store downstream, pushed up into enrichment so the leak never enters the pipeline in the first place.

The second constraint is a false-positive budget on the categories that carry decision weight. Misreading a coffee shop barely moves a lending decision. Misreading a loan repayment as a subscription, or a gambling transaction as retail, can flip it. We set per-category precision and recall targets tied to how much each category weighs in affordability, hold out a labelled eval set that reflects the real merchant and payment mix, and track categorisation coverage per provider so a silent schema change shows up as a coverage drop.

The output of all this is a normalised, deduplicated, enriched ledger with lineage back to the raw payload for every field. That audit trail matters when a decision is challenged and you have to show exactly which record, from which provider, at which fetch time, produced a given feature. The model that reads this ledger is the easy part. The layer underneath it is what determines whether the model is reading the borrower or reading provider noise.