A lot of teams believe they have handled fairness because they deleted race, sex and age from the feature set. The field is gone, so the model cannot discriminate. That belief is where most of the real risk lives.
A model trained to predict default or willingness to pay will happily rebuild a protected characteristic from whatever proxies survive. Postcode carries it. Occupation carries it. So does the texture of someone’s transaction history, the merchants they pay, the timing of their inflows. Drop the protected attribute and a sufficiently flexible model reconstructs a close stand-in from the rest, then uses that. The influence is still in the model. You just stopped being able to see it directly. This is proxy discrimination, and dropping a field does not remove it.
So the question is not “did we exclude the attribute” but “does the model’s output vary with the attribute, by whatever path.” Answering that takes testing, not inspection of the feature list.
Two things you are actually testing for
There are two distinct failures, and they need different tests.
- Disparate impact: the model’s decisions land differently across protected groups, regardless of intent. You measure it on outcomes. For a credit decision, compare approval rates, or score distributions, across groups and look at the ratios. For pricing, compare the price or rate offered. The point-in-time discipline matters here too: you compare outcomes the model would have produced on data known at decision time, not a backfilled view.
- Proxy discrimination: the model has reconstructed the protected characteristic internally. You test it by asking how well the model’s features, or its score, predict the protected attribute you supposedly removed. If a handful of features predict group membership cleanly, you have a proxy, and you can usually name it.
Both require something many teams do not keep: the protected characteristics themselves, held to one side, used only for testing and never as model inputs. You cannot run fair lending testing on data where you threw the labels away. The attribute has to exist somewhere you can slice by it.
Build the held-out test sets, then run them on a clock
Fairness is not a one-time certificate. A model drifts, the population drifts, a new feed changes what the proxies look like. A model that passed in March can fail in September without anyone touching the code.
So the work is structural. We build held-out test sets sliced by each protected characteristic that matters for the product, sized so the slices are large enough to say something. Those sets feed a bias testing run that produces the same metrics every time: disparate-impact ratios per group, proxy-prediction scores, and the features driving them. The run goes on a schedule. Quarter-end is a natural cadence for many lenders, but the right interval is whatever matches how fast your data moves.
Then keep the results. Every run, dated, with the model version and the data vintage it ran against. This is the same audit-trail logic as data lineage: when a reviewer asks whether the model was fair on the day it made a decision, “we think so” is not an answer. A stored test result is. It also tells your own team when a fairness metric started sliding and which release moved it, the same way an eval set catches a quality regression.
Where this lands with regulators
Two regimes are worth holding in view, and neither of these is legal advice.
US fair-lending rules turn on disparate impact and on being able to explain an adverse decision. The adverse action notice and its reason codes have to reflect what the model actually did, which is hard if you never tested what the model actually did. Disparate-impact testing is the evidence that the explanation is honest.
The EU AI Act treats much credit scoring as high-risk, with expectations around data governance, documentation and conformity assessment. A scheduled fairness test with retained results is one concrete way to show the bias work was done and is ongoing, rather than asserted once at launch.
The throughline is the same in both places. Removing a field is a claim. A test you ran, on a held-out set, on a schedule, with the result written down, is the evidence. Build the evidence.