Decision support
Can we let an AI agent access our internal data?
Yes, and the risk is almost never the model. It is the permissions you hand it and the places its output gets stored.
The question
Our agent needs access to real customer data. Is that safe?
The short answer
It can be, and the controls are mostly ones you already know. Three things are genuinely different. First, an agent with tool access is an actuator, not a reader — anything that can influence its input can influence its actions, and untrusted text in a retrieved document counts as input. Second, retrieval defaults to insecure: a vector index has no concept of who is asking, so unless you filter by the caller’s permissions before the model sees anything, you have built a system that reads across every user. Third, the output has to live somewhere — traces, caches, eval datasets — and those stores are where cross-tenant leaks actually happen. The single most useful architectural rule: the agent executes with the caller’s permissions, never with its own service account.
Written for
- Engineering leaders being asked to sign off on an agent reaching production data
- Security reviewers who have never assessed an LLM system before
- Founders answering a customer security questionnaire
Why the answer is that
The question is usually asked as though the model is the threat — will it memorise our data, will it leak it to the provider. Those are worth a contract review and mostly end there: the major providers offer zero-retention endpoints, and if your data is already in a US cloud, sending it to a model in the same jurisdiction is not a new category of risk. Spending the security review on that is comfortable and misses what breaks.
What breaks is permissions. Almost every agent I have looked at was given a service account with broad read access, because that is the fastest way to make the demo work. The moment that agent is user-facing, every user inherits the union of everything the service account can see, and the only thing standing between user A and user B’s data is the model choosing not to mention it. That is not an access control. Retrieval has to be filtered by the caller’s identity before results reach the context window — post-filtering the model’s answer is too late, because the tokens have already been generated from data the user was not entitled to.
The second failure is that tool access makes the agent an actuator. If it can call `refund_order` or `send_email` or `update_record`, then any text that reaches its context is potentially an instruction. A support ticket, a PDF, a web page, a calendar invite — all untrusted. This is prompt injection, and there is no prompt that reliably prevents it; treating instructions in retrieved content as data rather than commands has to be enforced outside the model. In practice that means validating every tool call against what the caller is allowed to do, and gating anything irreversible on a human.
The third is the least discussed and the one I find most often: the exhaust. Traces contain full prompts, so your observability tool now holds customer data. Semantic caches key on question similarity, so a sufficiently similar question from a different tenant can return the first tenant’s answer. Eval datasets get built from production traffic and then shared with contractors. None of this is exotic — it is just that the data flows were added after the security review.
What gets reviewed versus what actually goes wrong
The left column is what security questionnaires ask about. The right is what I find when I look at a running agent.
| Concern raised in review | Real-world severity | What to check instead |
|---|---|---|
| Will the provider train on our data? | Low — contractual, solved by a zero-retention endpoint | Confirm the endpoint and retention terms once, in writing, then move on |
| Could the model memorise and leak our data? | Low for API use — you are not fine-tuning on it | Only relevant if you fine-tune. If you do, who can query the tuned model? |
| Is the traffic encrypted? | Low — it is TLS like everything else | Nothing. This is table stakes and already true |
| Does retrieval respect per-user permissions? | Critical — the most common real defect | Filter the index by caller identity before retrieval, not after generation |
| Does the agent use its own service account? | Critical — every user inherits its full access | Pass the caller’s credentials through. The agent should be able to see less than you |
| Can retrieved content issue instructions? | High — prompt injection via documents and tickets | Validate tool calls against caller entitlements outside the model |
| Where do traces and prompts get stored? | High — your observability vendor now holds customer data | Treat the trace store as a data processor. Redact before it leaves |
| Does the cache key cross tenants? | High — semantic caches return neighbours’ answers | Namespace every cache by tenant. Similarity is not authorisation |
| Who can read the eval dataset? | Medium — built from production traffic, shared widely | Synthesise or redact before it becomes a test fixture |
The pattern is that the low-severity rows are contractual and get all the attention, while the critical rows are architectural and get none. If a security review of your agent did not mention retrieval filtering, it did not review the agent.
Access-control checklist
Nine things to be able to answer yes to. Take this into the review rather than a generic cloud questionnaire.
| Control | Question to ask | Failure if absent |
|---|---|---|
| Identity passthrough | Does the agent act as the user, or as itself? | Every user gets the union of all access |
| Pre-retrieval filtering | Is the index filtered by entitlement before search? | Model generates from data the user cannot see |
| Tool authorisation | Is each tool call checked against the caller’s rights? | Injected text triggers actions the user could not take |
| Irreversible-action gating | What requires human approval before it commits? | A hallucinated argument becomes a real refund |
| Cache namespacing | Is every cache keyed by tenant? | Cross-tenant answer leakage |
| Trace redaction | What is stripped before traces leave your boundary? | Customer data in a third-party dashboard |
| Egress allowlist | Which hosts can the agent reach? | Exfiltration via a tool that fetches URLs |
| Audit trail | Can you reconstruct any single decision? | No answer for an auditor or an angry customer |
| Blast-radius limit | What is the worst thing one run can do? | Unbounded loops, unbounded spend, unbounded damage |
If you only implement two, make them identity passthrough and pre-retrieval filtering. Together they eliminate the entire class of failure where one user reads another’s data, which is the one that ends contracts.
Get the answer for your own system in four minutes
The free scorecard produces a score out of 100, a breakdown across reliability, cost and observability, and your gaps ranked by production risk with a fix and effort estimate for each. It prints to a PDF you can take into a planning meeting — which is usually more persuasive than an argument.
Run the scorecardIf you are the engineer who found this page: Written to be forwarded. If you are the engineer who found this, send the relevant one up — it is framed for the person who has to approve the work.
If you decide to do the work
The engagements this decision leads to, each with the full method and effort estimates — whether your team runs it or I do.
Questions
Are AI agents secure enough to access internal company data?
Yes, provided the agent executes with the calling user’s permissions rather than its own service account, and retrieval is filtered by entitlement before results reach the model.
The common failure is not the model leaking data — it is that the agent was given broad read access to make the prototype work, which means every user effectively inherits that access and the only barrier is the model choosing not to mention what it retrieved. That is not an access control.
Can an AI agent access data a user is not allowed to see?
It can if retrieval is not permission-filtered, and this is the most common real defect I find.
A vector index has no concept of identity. Unless you filter candidates by the caller’s entitlements before the search, the model generates its answer from documents the user was never authorised to read. Filtering the model’s output afterwards does not help, because the tokens were already produced from that data.
What is prompt injection and does it matter if our agent is internal?
Prompt injection is untrusted text reaching the agent’s context and being treated as instructions rather than data — from a support ticket, a PDF, a web page, a calendar invite.
It matters more, not less, when the agent has tool access, because tools make it an actuator rather than a reader. No prompt reliably prevents it. The control is outside the model: validate every tool call against what the calling user is actually permitted to do, and require human approval for anything irreversible.
Do we need to worry about the model provider training on our data?
Much less than most security reviews assume. The major providers offer zero-retention endpoints, and confirming the retention terms in writing is a one-time contractual task.
It is worth doing and then moving on. The attention is better spent on retrieval permissions, tool authorisation, and where your traces and caches are stored — those are where data actually crosses boundaries it should not.
Want a second opinion on the decision?
Describe your situation and I will tell you what I would do — including when that is "nothing" or "handle it internally". No charge for that, and it is genuinely how a lot of these conversations end.
Other decisions
- What an agent audit should cover
- Questions to ask before launch
- Build or buy: observability
- Metrics worth reporting upward
- Do you need outside help?
- Is an agent different from a chatbot?
- Governance, audit logs and compliance
- Do we need our own model?
- Where to put a human in the loop
- Agent cost versus hiring
- Working with the systems you already have