The leave policy answer looked perfect. It named the right band, quoted a notice period, and sounded like something HR would have written. A team lead forwarded it into the onboarding channel. Three new hires planned around it.
The source was a draft Word file in a shared drive the agent should never have seen. The file was not published. The effective date was blank. Nobody owned it. By Friday the draft was unofficial policy.
That is the failure mode this post is about. We are not talking about an agent that moves money or updates a citizen record. Those need approval gates. This is about what the agent is allowed to know and cite: an enterprise AI knowledge platform where retrieval, provenance, and access control are the product surface, not homework after the demo.
Wrong answers become unofficial policy
Classic search returns links. People still open the document and notice the watermark. An agent returns prose. Prose travels. It gets pasted into SOPs, citizen replies, and vendor emails. If that prose was grounded in the wrong corpus, the organization did not just hallucinate. It operationalized ungoverned text.
Dump-everything RAG makes that easy. Crawl SharePoint and the network drive. Chunk. Embed. Ship a chat box. The indexer often runs with broader rights than any end user. Query-time security is “we will filter later.” Later is already too late once the model has seen the chunk.
Microsoft’s own engineering notes on SharePoint-to-RAG pipelines put it bluntly: authorization should happen before retrieval, not after, and document-level permissions must be materialized into the index rather than hoped for in the UI (ISE on SharePoint doc-level access). Azure AI Search’s query-time ACL and RBAC path is built around the same idea: permission metadata at index time, user identity on the query, protected content withheld when that identity is missing (Azure AI Search ACL/RBAC).
Stale content is the quiet twin of overexposure. A superseded circular still ranks. A revoked ACL still retrieves until the next sync. Buyers feel this as “the bot is confident about last year’s rule.”
Cite vs scrape is the product surface
Scrape means the product is a bag of text plus a fluent model. Success is measured as answer rate.
Cite means the product is a governed corpus plus retrieval that respects identity, plus answers that carry resolvable source cards. Success includes refusal when the corpus does not support the claim.
Source cards are not decoration. They are how a human checks ownership, version, and whether they are even allowed to open the underlying file. Model vendors are converging on structured citations for the same reason. Anthropic’s citations API returns pointers into provided documents rather than hoping the model invents a bibliography (Anthropic citations). OpenAI’s citation guidance tells builders to keep stable source IDs in the application, resolve locators in the UI, and validate that every citation maps to material actually supplied for that turn (OpenAI citation formatting). Google’s grounded generation path returns grounding metadata with support chunks, document ids, URIs, and claim-level support (Agent Search grounded answers).
None of that replaces your ACL story. It does set the bar: if the platform cannot show which governed artifact supported a sentence, it is still scraping with better lighting.
Governed sources: ownership, freshness, versioning
A knowledge platform needs a publish path, not only a sync path.
- Owner. A named role can dispute, retire, or supersede a document. “IT synced it” is not ownership.
- Freshness. Last synced time is necessary and not sufficient. Policy corpora also need effective dates and explicit supersedes links. Expired material should warn or refuse, not quietly win on cosine similarity.
- Versioning. Citations must resolve to the version that was retrieved, not “whatever is live on the intranet now.” Auditors will ask what the agent saw.
NIST’s AI Risk Management Framework treats provenance and attribution as part of accountable, transparent systems, and calls out training and retrieval data that drifts from its intended context as an AI-specific risk (NIST AI RMF 1.0). The Generative AI Profile pushes data provenance (source, versioning, signatures where feasible) into system inventory practice (NIST AI 600-1). That is governance language buyers already hear. Map your corpus fields to it instead of inventing a parallel vocabulary.
Curation costs real time. Dump-everything is cheaper until the first circulated wrong answer. That tradeoff is honest and should stay on the slide.
Retrieval with permissions (user and agent)
Two identities matter.
- The user asking the question. Retrieval must run with their entitlements (groups, site membership, classification). Entra object IDs beat emails. Least-privilege connectors (
Sites.Selectedand kin) beat tenant-wide read on the ingestion app (Graph selected permissions overview; ISE pattern above). - The agent purpose. A citizen-facing assistant and an internal investigations copilot should not share one undifferentiated bag, even if the same human could open both systems in other UIs. Purpose tags on corpora are product policy.
Deny-by-default: if the user token is absent, ACL-protected content stays out. Azure documents that ACL filters apply even when callers authenticate with service keys, and that omitting the user token withholds protected content. That is the posture you want in your own gateway even if you are not on Azure Search.
Filtering after the model “already used” the chunk is theater. So is expanding group membership only in the chat UI while the vector query ran as a superuser.
Graph-style retrieval (including Microsoft’s GraphRAG research stack) can help with global questions over a corpus. It does not excuse missing ACL at every hop. Summaries and community reports can launder permissions if you are careless (GraphRAG docs).
Provenance in the answer UX (used vs refused)
Show what was used. Also show that the system refused or filtered material, without leaking the titles of documents the user cannot access when that leak would itself be sensitive.
Good UX patterns:
- Inline markers that open a source card (title, owner, version or effective date, open-in-source link).
- A “sources considered” strip that distinguishes cited, retrieved-but-not-cited, and blocked-by-policy.
- An explicit abstain: “I do not have an approved source for that” beats a fluent guess.
Validate citations against the retrieval set before render. If the model invents a source id, drop it. OpenAI’s guidance on relevance and accurate representation is the right product instinct: irrelevant citations train users to ignore all of them.
Latency will rise versus anonymous top-k search. Spend that budget on ACL expansion and citation validation for policy answers. Do not spend it on prettier typing indicators while skipping provenance.
Three stores, not one bag
Keep these separate on purpose:
| Store | Role | May ground a policy answer? |
|---|---|---|
| Governed corpus | Published, owned, ACL-synced documents | Yes, with cite |
| Chat / session memory | Thread context, clarifications | No, unless a human promotes content into the corpus |
| Tool outputs | Live lookups, tickets, API receipts | As receipts for that turn; not silent re-ingest as policy |
Promoting a chat reply or a scraped ticket into the corpus without a publish step recreates the draft-Word-file problem with better tooling.
For India and government-adjacent builds, keep a light DPDP lens without turning this into a compliance essay. Purpose-tag what personal data may enter which corpus. Minimize identifiers in chunks when the answer does not need them. Keep source-use logs so you can answer “what was processed for this answer.” That supports auditability. It is not a claim of certification.
Thin reference architecture
A shape that holds up in client builds:
- Ingest from approved stores with least-privilege connectors. Resolve effective ACLs. Capture owner, classification, purpose tags, version, effective dates.
- ACL-aware index where every chunk carries document id + permission fields. Refresh on a documented SLA.
- Retrieve with user token and agent purpose filters. Deny by default. Prefer hybrid retrieval quality work (contextual chunking, BM25 + embeddings, rerank) inside the allowed set (Anthropic contextual retrieval is a useful quality reference, not an ACL substitute).
- Generate with citations using application-owned source ids. Validate before UI render.
- Audit query id, subject, agent purpose, retrieved ids, cited ids, refuse/filter reasons, model and policy versions. Empty-corpus abstentions are first-class events.
Incomplete corpus is a feature when the alternative is unofficial policy. Teach the organization to publish missing sources rather than lowering the abstain bar.
How we frame this at Bluelupin
In delivery we treat the enterprise AI knowledge platform as a product surface: who may know what, from which owned artifact, with which citation, under which purpose. RAG quality work sits inside that envelope. It is not a substitute for it.
We keep mutating actions on a separate track. Knowing the refund policy is this post. Executing the refund is the approval-gates post. Buyers who conflate the two usually under-build both.
If you are scoping a build, start with one high-stakes question type (HR policy, benefits, citizen FAQ), one source system with real ACLs, and a UI that refuses without a source card. Expand corpus coverage after the cite path is boring and trusted.