Decision engine vs rule engine: what changes when models enter lending

A rule engine runs conditions a person wrote by hand: if debt-to-income tops forty percent and the bureau score sits below 680, decline. A decision engine is the wider apparatus that reaches and records a lending outcome. Once a model sits inside it, the hard problem shifts. The logic is easy; the challenge is knowing which inputs, which model version and which rule set produced a given decision, and reconstructing all three months later.

Two different kinds of system

A rule engine is deterministic and legible. You can read the tree, trace the path an application took, and explain the outcome to anyone. That is its strength and the reason it has survived in credit for decades. The weakness is that every rule is a human hypothesis about risk, thresholds get set by committee and rarely revisited, and the engine has no memory of why a decision came out the way it did beyond the rules themselves.

A decision engine with a model in it changes what a decision even is. The output now depends on feature values computed from data that keeps moving, a model version that gets retrained, and rules that still wrap the score. The same applicant scored on Tuesday and again after a Thursday deployment can get different answers, both correct. So the engine has to carry things a rule engine never needed:

• The exact feature vector used, computed as of the decision timestamp, with no value that only became knowable afterwards
• The model version and the rule-set version, pinned to the decision id
• The lineage from each feature back to the source record and the time it was read
• An audit trail that supports replay, not just a log line saying the outcome

Miss any of these and you have a system that makes decisions you cannot later defend. In lending that is not a technical inconvenience. Under ECOA and Regulation B a declined applicant is owed specific reasons, and a reason code you cannot trace back to inputs is a reason code you cannot stand behind.

Point-in-time correctness is where migrations break

The failure mode that quietly ruins ML decisioning is lookahead. A feature that leaks future information inflates offline accuracy and collapses in production, because at decision time the value it depended on did not exist yet.

The classic example: an “average balance over the last 90 days” feature computed against the current table rather than the table as it stood at the decision. If a transaction was booked or corrected after the decision, it seeps into the feature and the model learns from information the real system never had. Training on that produces a model that looks excellent on the eval set and underperforms the day it goes live.

Getting this right means every feature is reconstructable from data that existed at the decision timestamp, and training labels come from a strictly later, non-overlapping window. A feature store that enforces point-in-time joins is what makes this hold at scale, because it stops engineers from accidentally joining to present-day values. When we migrate a lender off pure rules, this is the first thing we build and the last thing we compromise on. Reconciliation between the offline feature store and the online serving path is not optional: if the value the model sees in production differs from the value it was trained on, the score is wrong in a way monitoring will only catch after real applicants have been affected.

How we migrate legacy rules into governed decisioning

You do not rip out a working rule engine and replace it with a model. The rules encode years of policy, some of it regulatory, and most lenders cannot articulate all of it until they try to remove it. The migration is incremental and runs the old and new logic side by side long before the model touches an approval.

• Instrument the existing engine first, so every legacy decision is captured with its inputs and outcome. That log becomes the baseline you measure the model against and, with care about leakage, part of the training data.
• Separate hard policy from risk ranking. Regulatory floors, sanctions screening, exposure caps and product eligibility stay as explicit rules outside the model. The model ranks and prices within the space those rules permit.
• Run the model in shadow. It scores live traffic and records what it would have decided, while the rule engine still makes the real call. You compare distributions, disagreements and the cases where the model would have flipped an outcome, and you build the false-positive budget you are willing to spend before anything goes live.
• Move to champion-challenger only once shadow results hold. The incumbent policy stays champion; the model takes a measured share of traffic as challenger, with the split controlled and every decision replayable.
• Treat every policy change as a deployment. Rule edits and model versions go through the same review, versioning and rollback path as application code, so a threshold change on a Friday is as traceable as a code release.

Straight-through processing is the goal for the clean cases, the applications where model and rules agree with high confidence and no human needs to touch them. But straight-through only holds up when the decisions behind it can be pulled apart on demand. That is the real difference between the two systems. A rule engine explains itself because it is simple. A decision engine has to be built to explain itself, because the model inside it will not, and drift will make yesterday’s explanation wrong if nothing recorded the version that produced it.