Air-gapped LLM inference architecture

What air-gapped actually means

An air-gapped system has no external network connectivity — no internet, no VPN, no outbound control channel, no telemetry, no heartbeat. The physical network interface is disabled or physically absent. Data enters and leaves on storage media that a human being carries across the gap. For inference infrastructure, this means the GPU nodes, the API gateway, the model registry and the identity provider all operate in complete isolation.

The stack inside the perimeter

Every component that normally phones home must be self-contained inside the air gap. Here's what that means in practice:

• Local model registry. Model weights are stored on local encrypted storage, served to GPU nodes over internal networking. No external pull, no CDN, no artifact repository outside the gap.
• Container mirror. All container images — the inference engine, the gateway, monitoring agents — are pre-loaded into a local registry that runs entirely inside the perimeter. The images are pulled once on an internet-connected staging machine, verified, and transferred.
• Local identity provider. No OAuth to an external IdP. Authentication uses a local LDAP or SAML provider, with hardware tokens for MFA where required by policy.
• Local PKI. TLS certificates are issued by an internal certificate authority. Root CA material is generated and stored inside the gap, never exported.
• Hardened logging. Audit logs write to local encrypted storage. Logs record that a request happened — timestamps, token counts, error classes — never what was in the prompt or completion.

The two-phase update workflow

Every update — new model weights, a container image, a configuration change — follows the same two-phase process. Phase one happens on the internet-connected side: assets are downloaded, checksummed, and written to transfer media. Phase two happens inside the gap: the media is mounted, every file is verified against its SHA-256 manifest, and only then are assets deployed. This is not a recommendation. In classified environments, it's a requirement.

# Phase 1 — download and checksum on internet-connected staging
  hyperinfer pull-model deepseek-v4-flash \
    --output /mnt/transfer-media/
  sha256sum /mnt/transfer-media/model.safetensors \
    > /mnt/transfer-media/checksums.txt

  # Phase 2 — verify after physical transfer across the gap
  mount /dev/sdb1 /mnt/incoming
  hyperinfer verify /mnt/incoming/model.safetensors \
    --manifest /mnt/incoming/checksums.txt
  curl -s -X POST https://inference.local/v1/admin/reload \
    -H "Authorization: Bearer $ADMIN_TOKEN"

Cryptographic controls

For US government workloads, the entire pipeline can run with a FIPS 140-3 cryptography option — from TLS termination inside the gap to the SHA-256 verification of transferred assets. Customer-managed encryption keys (BYOK) encrypt model weights at rest on local storage, so even physical access to the storage media doesn't expose the weights without the key. The root of trust lives inside the gap, and nothing cryptographic crosses the boundary.

Use cases that justify an air gap

Air-gapped inference is not a general-purpose deployment model. It's for organizations where the consequences of a data leak are existential: defense and intelligence agencies processing classified documents, critical national infrastructure operators analyzing operational plans, nuclear and energy-sector players running scenario models, and financial institutions handling payment system data under PCI DSS. These workloads share one property: the classification guide or regulatory framework says no network egress of any kind, period. VPC and on-prem deployments can't meet that bar because they maintain at least an outbound control channel.

The operational cost of the gap

Air-gapped deployments trade operational convenience for absolute perimeter guarantees. Model updates that would take minutes over the network take days over physical media. Every change requires a documented transfer procedure with chain of custody. Troubleshooting means walking into the facility, not SSH-ing from home. But for the teams that need it, those constraints are features, not bugs — they're what makes the architecture auditable. And the API inside the gap is the same OpenAI-compatible interface available in every other deployment mode, so the application team doesn't need to care what the perimeter looks like. Learn more about HyperInfer's security architecture.