Air-gapped AI deployments

Air-gapped AI stopped being a niche request some time ago. One industry estimate put air-gapped or fully on-premises enterprise LLM deployments at 14 to 19 percent of companies with more than 5,000 employees in the first quarter of 2026, with defence, intelligence, and government accounting for roughly 38 percent of that spend, healthcare 19 percent, regulated finance 17 percent, and legal 9 percent. That is not an edge case, it is a segment.

The reason is straightforward once you look at what an air gap actually protects. It keeps prompts, outputs, model weights, retrieval corpora, logs, and identities inside a closed boundary. For classified material, for critical infrastructure operators who cannot accept internet-borne compromise paths into operational systems, and for organisations whose source code, deal documents, or research files are the business, that boundary is the control. No contractual assurance from a model provider substitutes for it.

Where projects go wrong is in the gap between the claim and the architecture. Teams deploy an open-weight model on their own GPUs, put a firewall in front of it, and describe the result as air-gapped. Then the container image pulls from a public registry on restart, a Python package updates during a rebuild, or a component quietly reports usage to a vendor endpoint. Each of those is a routable path, and each one means the isolation claim does not survive an audit.

What isolation actually requires

An air-gapped LLM has no routable path to the public internet during normal operation. That is a stronger statement than “behind a firewall,” and it forces every runtime dependency to be local: inference, retrieval, embeddings, identity, audit logging, and the agent loop if you have one.

The immediate consequence is that hosted frontier APIs are out. Any external call breaks the enclave, which means the model choice is constrained to open-weight families you can legally and technically mirror inside the boundary. That constraint should shape the model selection process from the start rather than being discovered after a pilot on a hosted API set expectations for quality.

The less obvious consequence is that several services most teams never think about become your responsibility. DNS resolution, time synchronisation, certificate validation, license checks, authentication backends, and package management all assume connectivity in their default configurations. Internal NTP or GPS time sources and internal IAM, LDAP, and PKI are standard in mature deployments precisely because monitoring and audit records are worthless if the clock and the identity service cannot be trusted.

The transfer pipeline

Air-gapped environments replace auto-update with a controlled transfer pipeline. The stages are consistent across deployments we have seen work.

• Build in a connected staging zone. Download model weights, tokenizer and config files, serving container images, dependency bundles, security patches, evaluation datasets, and regression tests into a networked environment first.
• Verify before transfer. Check hashes, signatures, licence terms, and provenance. Many programmes add a security review and chain-of-custody logging at this point, and that record is what you show an auditor later.
• Cross the gap deliberately. Approved physical media or a one-way transfer mechanism such as a data diode, chosen by risk model rather than convenience.
• Import into internal registries. Pre-load images and packages into local artefact registries so production never reaches for a public repository, including during a restart at three in the morning.
• Re-validate after import. Confirm the model runs offline on the target GPUs, then promote through internal approval gates.

The step teams skip is rollback. A forward deployment process without an internal rollback path means that a bad model version is resolved by another full transfer cycle, which in a controlled environment is days rather than minutes. Keep the previous known-good bundle in the internal registry and rehearse reverting to it.

Licensing deserves an explicit gate here too. Offline rights and redistribution terms differ across open-weight families, and a model you cannot legally mirror into an enclave is a model you cannot use, regardless of how it benchmarks.

Offline RAG

Retrieval works offline when the whole retrieval chain lives inside the boundary. Ingestion is local: documents are scanned, chunked, embedded, and indexed on internal infrastructure. Embeddings come from a local model rather than a hosted embedding API, which is a common oversight because embedding calls feel like infrastructure rather than model usage. The vector database sits inside the same security boundary as the inference tier.

Access control is where offline RAG differs most from the connected version. Retrieval has to respect document classification and need-to-know, enforced against a local identity service, because a retrieval layer that returns any indexed chunk to any authenticated user has effectively declassified the corpus. Enforce the filter at retrieval time against user identity rather than relying on the prompt to constrain what the model discloses.

There is also no external grounding. No web search, no external citations. If the system needs current information, it comes from internal mirrors, internal knowledge bases, or a governed cross-domain ingestion process. And that ingestion process is itself a supply chain: documents, embeddings, and indexes can all be tainted if ingestion is not controlled and logged. Treat corpus ingestion with the same scrutiny you apply to container images.

Monitoring when nothing can phone home

Vendor dashboards, SaaS telemetry, and cloud-hosted observability are all unavailable, so observability has to be built locally from the start.

• Metrics, logs, traces, and audit records write to internal storage.
• Outbound telemetry is disabled in configuration, not only blocked at the network layer. Many tools retry, buffer, and fail in confusing ways when their telemetry endpoint is unreachable, so switching it off properly is both a security and a reliability measure.
• Audit logs are append-only or signed, so they hold up under forensic review.
• Health checks cover GPU memory, token throughput, queue depth, model load status, vector store latency, and crash recovery, monitored locally rather than through a vendor endpoint.

This is the layer where operator burden becomes visible. Manual transfers, re-verification, and staged promotion make air-gapped operations slower than cloud AI by design. Staffing plans that assume cloud-speed iteration inside an enclave produce frustrated teams and unpatched systems, which is a worse security outcome than the one the air gap was meant to deliver.

Common failure modes

The pitfalls repeat across programmes. Calling a network-restricted system air-gapped. Missing the hidden dependencies on DNS, time, licensing, auth, and package managers. Choosing a model that cannot be legally or technically mirrored offline. Skipping rollback planning. Leaving RAG ingestion ungoverned so the corpus becomes an unmonitored input path. Leaving telemetry enabled because it was on by default. Underestimating the operator effort and then quietly relaxing the controls to keep pace.

Every one of these is cheaper to address in design than in remediation, and all of them are visible in a single exercise.

Where to start

Run the egress test before you write the architecture. Take your intended stack, block all outbound traffic in a lab, and restart every component including the ones you are confident about. The list of failures is your true dependency inventory, and it is almost always longer than the design document says.

From there, build the two zones explicitly: a connected staging zone for downloads, scanning, signing, and validation, and an isolated production zone for inference, retrieval, identity, and logging. Between them sits one approved transfer method with a chain-of-custody record. Inside the production zone, an internal artefact registry and local security operations for audit, alerting, and incident response.

Then decide the model question early, since it constrains everything else. Pick from open-weight families whose licences permit offline mirroring, size the deployment for hardware you can maintain without vendor remote access, and validate the model against your own evaluation set inside the enclave rather than trusting results obtained on a connected system. The isolation is the point, and it only holds if every stage of the lifecycle respects it.