A retrieval-augmented assistant answers from documents it fetched first. Most access reviews ask what the assistant is allowed to open. Fewer ask what it is allowed to say about what it found.
In regulated systems, that second question is where the gaps are.
The citation title is a disclosure
Picture a compliance assistant at a bank. A branch employee asks how wire escalations are handled. The answer is accurate and harmless. The first citation reads "Case file: Hartwell Holdings, sanctions review."
Nobody showed the employee the file. They were shown its name. Filenames, matter titles and folder paths often hold client names, patient names and case types. Treat them as data, because they are.
The summary is written from whatever was retrieved
If the search pulled in passages the user could not see, the model will use them. Filtering the final answer afterwards rarely works, because the passage has already been blended into the prose.
Take a clinic's billing clerk asking about denial patterns for a procedure. The summary mentions a diagnosis. It came from a clinical note the clerk has no right to read, and the clerk never opened that note.
The cache remembers the first user
Some systems store generated summaries so the next question is cheaper. Others build the search index using one service account's view of the documents. In both cases, a later user can get an answer shaped by the first user's permissions.
An insurer's claims assistant can do this quietly. An adjuster with broad access asks about a policy line, and the summary is saved. A contractor with narrow access asks the same question and gets the saved version.
What to check
- Where is the permission filter applied? It should sit inside the search itself, before anything is retrieved, not after the answer is written.
- Do titles, filenames and snippets go through the same check as the body text?
- Are stored summaries tied to the permissions of the person who asked, or shared across users?
A test worth running
Sign in as the lowest-privilege user the tool serves. Ask a question whose best answer lives in a record that user should never see.
Then read three things. Read the answer. Read every citation, including titles. Read the refusal, if there is one.
That last one matters more than it looks. A message like "I can't show you the Hartwell file" tells the user the file exists. A safe response behaves as if the record were not there.
Permissions on the document store are only the start. The assistant has to respect them in the search, in the summary, in the citation and in the cache.
I'm Jonathan Melton. I run FusionAL, which reviews how AI agents and MCP tools are permissioned when they reach regulated systems.
Top comments (2)
The cache point is the one that bites quietly and I think it's under-reported. The moment you share a summary store or build one index under a service account's view, your permission model silently collapses to "whoever asked first." We hit this with a shared embedding index — the ACL filter was applied at query time against the answer, but the nearest-neighbor search had already ranked chunks the second user couldn't see, so even the refusal leaked timing and topic signals. The fix that actually held was filtering inside the retrieval (metadata-scoped ANN), so forbidden chunks are never candidates, plus keying any cache by the requester's permission set, not just the query string. Your last point deserves more emphasis than it gets: a refusal that names the record is still a disclosure. "I can't show you the Hartwell file" and "I found nothing relevant" are different answers to an attacker doing existence probing. Did you end up normalizing refusals to a single opaque response, or do you let some through for UX?
The refusal-message test is the sharpest one here. "I can't show you the Hartwell file" confirms the file's existence, which is the failure mode that auditing the document store alone will never catch. One related gap worth adding to the checklist: embedding indexes are rarely permission-scoped, so the nearest-neighbor ranking leaks information before generation even starts. The permission filter has to sit in the search, as you say — and the embedding store is part of the search.