Reliability · Multi-tenant AI isolation
Keep tenants apart when a language model is in the request path
A prompt is not an access-control mechanism.
You probably have this problem if
- Tenant scoping is described in the system prompt
- Retrieval filters by metadata the model can influence
- A tool would happily accept a tenant ID supplied by model output
- You could not prove isolation to a security reviewer without saying "the prompt says so"
- Nobody has deliberately tried to make the agent cross the boundary
What you end up with
Isolation that holds even when the model behaves unexpectedly.
- Usually part of
- Agent Production Readiness Audit
- $4,000 – $6,000
- Phases
- 5 steps, detailed below with effort per step
- Domains I have done this in
- HR technologyRecruitmentLegal / contracts
Why this is the thing to fix
I built a conversational analytics platform over HR data — salaries, performance, attrition — serving multiple client organisations. During testing, a question phrased around a department name that existed in two tenants produced valid SQL that resolved that department without a tenant predicate.
Nothing leaked; it was caught in staging against synthetic data. But the lesson was not "the model hallucinated". The model did something reasonable given ambiguous input. The defect was architectural: we had put a security boundary inside a probabilistic system.
Prompt instructions are advisory. They are useful for producing better first drafts and worthless as enforcement. Everything below moves the boundary out of the prompt and into code that cannot be talked out of it.
The layered isolation checklist
Each layer assumes the ones above it may fail. Work down the list; if you cannot tick a box, that is where your exposure is. This is the design I now use by default.
- 1Tenant identity is derived from the authenticated session and never read from model output, user text, or a tool argument
- 2Retrieval is filtered by a tenant key applied server-side, not by metadata the model can influence or restate
- 3Generated queries are parsed and rewritten — the tenant predicate is injected by your code, not trusted from the model
- 4Queries touching tables outside the authorised set are rejected before execution, not filtered afterwards
- 5Database credentials are per-tenant or row-level-secured, so the connection itself cannot read across tenants
- 6Every tool re-checks authorisation server-side rather than trusting that the agent was scoped correctly
- 7Outputs are scanned for identifiers and PII that do not belong to the requesting tenant, and blocks are logged
- 8Conversation history, caches and embeddings are partitioned per tenant — a shared cache is a cross-tenant channel
- 9Traces and logs are themselves tenant-scoped, so debugging one customer does not expose another
- 10Your eval suite contains deliberate cross-tenant attempts, and they are expected to fail
The last item is the one teams skip. If nobody has tried to break isolation on purpose, you do not know whether it holds — you know that nobody has reported it.
Where isolation actually leaks
Ranked by how often I find each one, not by how exotic it sounds.
| Leak path | Why it is missed |
|---|---|
| Tenant scoping lives in the system prompt | It works in every test, because tests do not include ambiguous cross-tenant phrasing. |
| Shared embedding index with metadata filtering | Filtering is correct until a query path forgets the filter. One missed call site is a leak. |
| Shared prompt cache across tenants | Caching is treated as a performance concern, so nobody reviews it as a data boundary. |
| Tool accepts an ID from model output | The tool looks correct in isolation; the trust boundary is one layer up. |
| Error messages echo requested records | A "not found" that names the record confirms its existence to the wrong tenant. |
| Traces and eval datasets mix tenants | Isolation is designed for the request path and forgotten for the debugging path. |
Five of these six are ordinary engineering mistakes rather than AI-specific ones. That is the point — the model did not cause them, it just made them easier to make.
How I actually do it
With effort per step, so you can judge whether to hire me or hand this to someone on your team.
- 11 day
Map every path to tenant data
Retrieval, tools, caches, history, traces, eval datasets. Anything that touches tenant data is in scope, including the debugging path people forget.
- 21–2 days
Move identity out of the prompt
Tenant identity resolved from the session, passed out-of-band, and never accepted from model output anywhere.
- 32–4 days
Add the enforcement layer
Query parsing and predicate injection, server-side authorisation re-checks in tools, and scoped credentials so the connection cannot cross tenants.
- 41–2 days
Partition the incidental surfaces
Caches, embeddings, conversation history, traces. These are the ones that leak after the request path has been secured.
- 51 day
Prove it with adversarial evals
Deliberate cross-tenant attempts as permanent test cases that must fail. This is what you show a security reviewer.
What you keep
- A map of every path that touches tenant data, including caches and traces
- Tenant identity resolved from session and enforced outside the model
- An enforcement layer: query rewriting, server-side authorisation, scoped credentials
- Partitioned caches, embeddings, history and traces
- Adversarial cross-tenant eval cases that must fail, as evidence for review
- A short written design you can hand to a security reviewer or customer
Failure modes this prevents
From the failure taxonomy — each links to the causes and fixes in full.
critical · very common
The agent calls the right tool with invented arguments
The model picks a correct tool but fabricates its inputs — a plausible-looking customer ID, an out-of-range date, an enum value that was never defined.
high · very common
The answer is wrong because retrieval returned the wrong context
The model is behaving correctly given what it was handed — and what it was handed was wrong.
high · very common
It works on our test prompts and fails on real users
The team's test inputs are clean, well-formed, and written by people who know how the system works.
Questions
Can we enforce tenant isolation in the system prompt?
No. Prompt instructions are advisory — they influence output, they do not constrain it. One ambiguous input or one injection attempt and the instruction is simply not followed.
Use the prompt to get better first drafts, and enforce the boundary in code: session-derived identity, server-side filtering, query rewriting, and scoped database credentials.
Is one shared vector index safe for multiple tenants?
It can be, if the tenant filter is applied server-side on every single query path and no call site can omit it. In practice that discipline is hard to maintain as code grows, and one missed path is a leak.
Separate indexes or namespaces per tenant cost more and remove an entire class of mistake. For sensitive data I default to partitioning and treat shared-index-with-filtering as the optimisation you justify later.
Does prompt caching create a cross-tenant risk?
It can, and it is one of the most commonly missed paths, because caching gets reviewed as a performance feature rather than a data boundary.
Cache stable, tenant-independent content — system prompts, tool definitions. Never cache prefixes containing one tenant's retrieved data in a way another tenant's request could hit. Partition cache keys by tenant explicitly.
How do we prove isolation to a customer or auditor?
Show them enforcement in code plus adversarial tests that fail. A written design describing session-derived identity, query rewriting, and per-tenant credentials, alongside an eval suite containing deliberate cross-tenant attempts with recorded results.
"The prompt instructs the model not to" does not survive review, and reasonably so.
Want this done, or just want a second opinion on it?
The method above is genuinely what I do — if your team can run it themselves, run it. If you would rather it were done in a week by someone who has done it before, that is what the agent production readiness audit is for.