Standing up an AI model inventory and registry

You cannot govern what you cannot list. A model inventory is the authoritative record of every model running in your institution: what it is, who owns it, what it decides, and what feeds it. A registry is the engineering store underneath, holding versioned artefacts and lineage. Together they let risk and audit answer a specific question about any decision, not a general one about your policy.

Most institutions discover the gap the same way. An examiner asks for a list of models in production, someone exports a spreadsheet, and three teams immediately point out models that are missing from it. The scoring service a data-science team stood up last quarter is not there. The extraction pipeline that reads bank statements is not there because nobody agreed it counted as a model. The vendor API someone wired into the collections workflow is not there because it was bought, not built. The spreadsheet was never wrong on purpose. It was just never the source of truth, so it drifted from reality the day after it was filled in.

What actually goes in the inventory

The inventory is a governance artefact. Its job is to let a risk officer or an auditor find a model, understand its blast radius, and reach the people and evidence attached to it. So the fields track accountability and materiality. Model internals live in the registry, one layer down.

For each model we record, at minimum:

• A stable identifier that follows the model across environments and never gets reused, so a decision logged in March can be tied back to exactly the entry that produced it
• Owner and accountable executive, named individuals rather than a team inbox, because materiality reviews and sign-offs land on a person
• Purpose and decision authority, meaning what the model outputs, whether a human reviews it, and where it sits in straight-through processing versus advisory use
• Materiality tier, which drives how much oversight the model gets; a model that auto-declines credit is not governed the same way as one that drafts internal commentary
• Inputs and data sources, including whether any feature carries point-in-time constraints, because a model fed a field that would not have been known at decision time is a leakage incident waiting to be found
• Vendor and hosting, including whether weights sit behind an API you cannot inspect and whether the provider reserves the right to update the model without notice

The materiality tier is the field that earns its place. Without it, every model looks equally important and the governance process either drowns or waves everything through. With it, you can defend where you spent your validation effort.

The registry is where the engineering lives

The inventory says a model exists and matters. The registry says what it actually is, at a level of detail an engineer can reproduce. This is the store your CI pipeline writes to and your serving layer reads from.

A useful registry entry carries the trained artefact, the exact training and evaluation datasets by hash, the feature definitions it depends on, the eval set and the scores that model version achieved, and for an LLM the versioned prompt and retrieval configuration. The point is reconstruction. If someone disputes a decision from last quarter, you should be able to name the model version, the data as it stood, the prompt, and the eval results that let it ship, without archaeology across five systems.

Lineage is the part teams most often skip and most often need. When a feature store column changes definition, you want to know every model that consumed it. When a training set is found to contain a leaked field, you want the list of models trained on it while there is still time to pull them, before a regulator finds it first. That query is only cheap if lineage was recorded at registration time rather than reconstructed later from commit history.

Metadata that makes the register worth keeping

An inventory that is only a list decays into a compliance chore nobody trusts. The metadata is what turns it into something risk and audit actually query. Three categories carry most of the weight.

Ownership and lifecycle state: is the model in development, shadow, production, or retired? A model in shadow deployment scoring alongside a champion has different monitoring expectations than one carrying live decisions, and the register should make that distinction machine-readable rather than tribal knowledge.

Monitoring and drift status: the last eval date, the current performance against realised outcomes, and whether the standing eval set is still green. This is where the inventory stops being a static filing and starts being an operational view. A model whose eval set has been red for six weeks should be visible as red in the same place you look up its owner.

Evidence links: the validation report, the model card, the approval record, the false-positive budget agreed with the business. The inventory does not need to hold these documents, but it must hold reliable pointers to them, so that finding a model and finding its audit trail are the same action.

The hard part is not designing these fields. It is keeping them true. An inventory maintained by hand is stale within a quarter. So the registration step has to sit inside the deployment path: a model that has not been registered, tiered, and given an owner does not get a serving credential. Then you reconcile the register against what is actually receiving traffic on a schedule, and you treat any model serving requests without a matching entry as an incident, not a paperwork gap. That reconciliation is the whole point. The register is only worth having if the answer to “is this everything?” is one you can defend when someone independent checks it against production.