What zero data retention really means

The three things "retention" can hide

When a vendor says they don't retain your data, push on three separate questions, because they're often answered very differently:

• Storage. Is request content written to disk or a database at any point — even temporarily, even "just for debugging"?
• Logging. Do prompts and completions appear in application logs, request traces, or an observability pipeline that a support engineer could read?
• Training. Are your inputs and outputs used — now or later — to train, fine-tune or evaluate any model?

A real zero-retention posture answers no to all three, by design, not by policy. The distinction matters: a policy can change with a config flag; an architecture that never writes content to disk cannot quietly start doing so.

What genuine zero retention looks like

Architecturally, the guarantee comes from a simple invariant: request content exists only in volatile GPU memory, only for the duration of a single request. It is decrypted into memory when the request arrives, computed on, streamed back, and overwritten when the request completes. It never reaches persistent storage because there is no code path that writes it there.

Observability still works — but it operates on metadata. Latency, token counts, error rates and region are all you need to run a reliable service, and none of them contain a single token of your prompt.

The questions to ask any vendor

Before you send sensitive data to any inference provider, get written answers to these:

• Is prompt or completion content ever written to disk or a database? Under what circumstances?
• Are prompts present in any log, trace, or monitoring system? Who can access them?
• Is my data used for training, fine-tuning or evaluation — with or without an opt-out?
• Is retention a contractual term in the DPA, or just a policy on a webpage?
• Can I verify retention behavior programmatically, e.g. via response headers?
• What metadata is retained, and for how long?

Why contractual beats best-effort

A policy page is marketing. A Data Processing Agreement is enforceable. When zero retention is written into the DPA, it becomes a binding obligation with legal consequences — exactly what your compliance team needs to map to their own controls under SOC 2, HIPAA or GDPR. Ask for the DPA early; a vendor that hesitates to put retention in writing is telling you something.

The bottom line

"We don't store your data" should be the start of a conversation, not the end of one. The vendors worth trusting can explain the architecture that makes it true, put it in a contract, and give you a way to verify it on every request. Everyone else is asking you to take their word for it.