PCI DSS: running LLMs on payment data

This is practical guidance, not legal advice. Work with your QSA on your specific CDE scoping and compliance obligations.

What puts an LLM provider in PCI DSS scope

PCI DSS v4.0.1 centers on the cardholder data environment (CDE) — the people, processes and technology that store, process or transmit cardholder data, plus any system connected to or that can impact the security of those systems. If full Primary Account Numbers (PANs) flow through your inference pipeline, the inference provider is processing cardholder data. At that point, every control your QSA expects — encryption at rest and in transit, access logging, segmentation, quarterly ASV scans — applies to the inference layer just as it applies to your payment application server.

Don't put PANs in your prompts

The single highest-leverage decision you can make is to keep raw PANs out of model input. Tokenization and truncation exist for exactly this reason. If you can replace a 16-digit PAN with a payment token before the prompt is assembled, the model never sees the card number, and the inference pipeline is no longer in the CDE by definition — it's handling non-sensitive tokens, not cardholder data. For use cases like transaction enrichment, fraud analysis or dispute summarization, the model rarely needs the full PAN; it needs the transaction metadata, amount, merchant category and outcome. Strip what isn't required.

There is a nuance worth flagging in 2026: some teams use embedding models to vectorize transaction descriptions for similarity search or anomaly scoring. If those descriptions contain cardholder data, the embeddings themselves can be considered in-scope storage of derived sensitive data. Feed the model clean input and the problem dissolves. This is the same minimum-necessary discipline that HIPAA workloads require, applied to a different regulatory framework.

When the inference layer stays out of scope

Even if you cannot fully de-scope the inference path — say your model genuinely needs a PAN for a specific matching operation — you can still contain the blast radius. The key is to architect the inference layer so it never stores what it processes. When request content lives only in volatile GPU memory for the duration of a single inference call and is never written to disk, logs, or an observability pipeline, there is no persistent store of cardholder data in the inference stack. That eliminates an entire category of PCI DSS controls tied to data-at-rest protection.

This is where single-tenant GPU deployment and zero data retention combine to do real compliance work. Dedicated hardware means your CDE isn't co-located with another tenant's workload. Zero retention means the inference layer holds nothing to breach. Together they let you draw a tight scope boundary around the inference path, which is what every QSA wants to see.

Encryption, segmentation and operator access

For PCI DSS, data in transit means TLS 1.2 or higher with strong ciphers. Every hop between your application and the GPU must be encrypted and mutually authenticated. If the inference gateway sits inside your VPC, reachable only over PrivateLink, that network path is inherently segmented from the internet. Key management follows PCI DSS Requirement 3: if any derived credentials or tokens touch the inference layer, they are managed under keys you control. And under Requirement 7, access to inference infrastructure is restricted to the minimum set of personnel — which, for a fully managed deployment, is often zero on the customer side.

Keeping inference out of CDE scope

• Tokenize PANs before they reach the prompt. If the model only sees a payment token, it never enters the CDE.
• Keep request content in volatile memory only. No disk writes, no log persistence, no training corpus — zero stored cardholder data.
• Deploy on single-tenant hardware. Your PCI workload runs on dedicated GPUs, segregated from every other customer.
• Segment the inference network path. Private connectivity inside your VPC, TLS 1.3 on every hop, no public internet exposure.
• Log metadata, not content. Token counts, timestamps and error classes give you operations visibility without capturing PAN data.

What your QSA will ask for

Expect your assessor to request: a network diagram showing every system that touches cardholder data and every boundary between in-scope and out-of-scope systems; evidence that the inference provider never persists PAN data (architecture diagrams, DPA language, response header verification); and attestation that the hardware is single-tenant with no shared memory or scheduler adjacency. Having these artifacts ready before the assessment begins is the difference between a clean scope discussion and a drawn-out one. For the broader compliance picture across frameworks, our compliance overview maps how SOC 2, HIPAA and PCI DSS controls intersect at the inference layer.