Enterprise reference design

Private enterprise RAG with enforceable document access

An Azure design for private retrieval, source permissions, controlled ingestion and recoverable indexes, with explicit identity and network boundaries.

Enterprise architectsAI platform engineersIdentity and data ownersSecurity operations

The enterprise use case

Design for the actual operating context.

Employees query sensitive enterprise documents through a private application. Answers must include only authorized evidence, and revoked access must propagate through indexes, caches and citations within an agreed interval.

Required outcomes

  • Authorize every retrieved document before model use.
  • Keep application and data ingress private while controlling necessary outbound paths.
  • Separate ingestion, querying and deployment identities.
  • Rebuild indexes and restore service without silently weakening access controls.

Enterprise architecture

Explore the design, one view at a time.

Follow the private question and retrieval path

Private entry ends at service endpoints; application authorization still governs every retrieved document.

Private enterprise RAG — Follow the private question and retrieval pathPrivate entry ends at service endpoints; application authorization still governs every retrieved document.. Official Microsoft service icons; explicit resource and network boundaries. Green numbered steps match the F-numbered flow register below the diagram. Private enterprise RAG Follow the private question and retrieval path Private entry ends at service endpoints; application authorization still governs every retrieved document. Enterprise Managed employee device Hub VNet Connectivity subscription Application subscription Managed services remain outside the customer VNet Spoke VNet Network attachments F1: Private entry. The employee reaches Azure through approved ExpressRoute or VPN connectivity; no public application ingress is required. F2: Spoke access. Hub-to-spoke routing and DNS deliver requests to the intended private endpoint. Private reachability does not grant data access. F3: Private UI. The App Service private endpoint is inbound-only. Public network access is separately disabled; outbound integration is separate. F5: User API token. The UI calls the private APIM gateway with the user access token for the retrieval API audience; APIM validates tenant, audience and scopes. F6: Validate API token. APIM forwards the token only to its intended retrieval API. The API validates authorization again and derives principals server-side. F7: Apply principal filter. The query-only managed identity calls Search over Private Link. The application applies tenant and principal filters to every exposed retrieval path. F8: Authorized context. The workload obtains its own inference token and submits only approved retrieved context to the selected model deployment. Employee client. Managed device Employee client Managed device ExpressRoute / VPN. Approved private entry ExpressRoute / VPN Approved private entry Private endpoints. Public ingress disabled Private endpoints Public ingress disabled Integration subnets. Separate outbound paths Integration subnets Separate outbound paths App Service UI. Private employee portal App Service UI Private employee portal API Management. Standard v2 gateway API Management Standard v2 gateway Retrieval API. Server-derived principals Retrieval API Server-derived principals Azure AI Search. Document permission filter Azure AI Search Document permission filter Azure OpenAI. Approved model deployment Azure OpenAI Approved model deployment Private entry 1 Spoke access 2 Private UI 3 User API token 5 Validate API token 6 Apply principal filter 7 Authorized context 8 Private connectivity is selective The shown endpoint represents inbound Private Link. App Service and APIM outbound integration, DNS and approved egress require separate configuration. Authorization remains explicit Search filters, citations and caches use server-derived permissions. Model input contains only the context that survived those checks. Reference design · Microsoft product icons used under Microsoft’s published terms · Numbered steps refer to the accompanying flow register
9 components · 7 numbered connections

Green step numbers match the F-numbered flow register below. Solid boundaries show ownership; dashed blue boundaries show networks. Logical groups are labeled explicitly. Microsoft service icons identify the components.

Separate user authority, retrieval and release

User tokens, query identities, inference identities and deployment permissions have distinct scopes.

Private enterprise RAG — Separate user authority, retrieval and releaseUser tokens, query identities, inference identities and deployment permissions have distinct scopes.. Official Microsoft service icons; explicit resource and network boundaries. Green numbered steps match the F-numbered flow register below the diagram. Private enterprise RAG Separate user authority, retrieval and release User tokens, query identities, inference identities and deployment permissions have distinct scopes. Enterprise Employee sign-in Tenant Identity authority Delivery Reviewed artifacts and private runner Application subscription Managed services; private network paths are abstracted in this view F4: User sign-in. The client obtains an Entra access token for the retrieval API. The identity service has its own approved route and is not treated as a VNet appliance. F16: Federated deployment. A narrowly scoped deployment identity releases reviewed application artifacts using private runner connectivity. Runtime identities do not receive deployment authority. F6: Validate API token. APIM forwards the token only to its intended retrieval API. The API validates authorization again and derives principals server-side. F7: Derive tenant / ACLs. The query-only managed identity calls Search over Private Link. The application applies tenant and principal filters to every exposed retrieval path. F8: Inference identity. The workload obtains its own inference token and submits only approved retrieved context to the selected model deployment. Employee client. Retrieval API audience Employee client Retrieval API audience Microsoft Entra ID. User token issuance Microsoft Entra ID User token issuance Release pipeline. Federated deployment role Release pipeline Federated deployment role API Management. Tenant, audience and scope API Management Tenant, audience and scope Retrieval API. Independent validation Retrieval API Independent validation Azure AI Search. Query-only identity Azure AI Search Query-only identity Azure OpenAI. Workload inference role Azure OpenAI Workload inference role User sign-in 4 Federated deployment 16 Validate API token 6 Derive tenant / ACLs 7 Inference identity 8 Tokens have specific audiences A workload inference token is not employee delegation. Use a supported on-behalf-of flow only where the downstream API must enforce user rights. Release is a separate authority Deployment federation and private runner reachability do not give runtime identities permission to change application code or resource policies. Reference design · Microsoft product icons used under Microsoft’s published terms · Numbered steps refer to the accompanying flow register
7 components · 5 numbered connections

Green step numbers match the F-numbered flow register below. Solid boundaries show ownership; dashed blue boundaries show networks. Logical groups are labeled explicitly. Microsoft service icons identify the components.

Publish trusted content and prove recovery

Quarantine, independent indexing, filtered evidence and versioned recovery inputs are separate responsibilities.

Private enterprise RAG — Publish trusted content and prove recoveryQuarantine, independent indexing, filtered evidence and versioned recovery inputs are separate responsibilities.. Official Microsoft service icons; explicit resource and network boundaries. Green numbered steps match the F-numbered flow register below the diagram. Private enterprise RAG Publish trusted content and prove recovery Quarantine, independent indexing, filtered evidence and versioned recovery inputs are separate responsibilities. Sources Authoritative document systems Operations Platform and data owners Application subscription Managed services; selected ingestion, telemetry and recovery paths F9: Versioned source intake. An authenticated exporter submits versioned content and permission manifests to the private ingestion path; objects remain quarantined pending validation. F10: Inspect source. The ingestion worker reads quarantined objects, checks inspection outcomes and creates bounded chunks with provenance and permissions. F12: Versioned index write. A separate writer identity publishes content and permission metadata; revocation and deletion updates also invalidate dependent caches. F11: Publish approved corpus. Validated content and manifests enter the approved versioned corpus. Permission failures and incomplete inspection block publication. F14: Preserve recovery inputs. Preserve approved source versions, manifests and index definitions using the selected backup/copy policy; the index is rebuilt rather than assumed to have native restore. F15: Rebuild and reauthorize. Owners restore networking, identities and model capacity, rebuild the index and repeat negative authorization tests before reopening access. F13: Filtered evidence. Representative telemetry flow: identity-independent request correlation, policy outcomes, counts, latency and usage; workers and gateways are instrumented separately. Source systems. Documents and ACL versions Source systems Documents and ACL versions Blob quarantine. Unapproved source objects Blob quarantine Unapproved source objects Ingestion worker. Inspect and version chunks Ingestion worker Inspect and version chunks Azure AI Search. Independent writer identity Azure AI Search Independent writer identity Approved corpus. Versioned content and ACLs Approved corpus Versioned content and ACLs Recovery corpus. Source versions and IaC Recovery corpus Source versions and IaC Recovery owners. Reauthorize before reopening Recovery owners Reauthorize before reopening Retrieval API. Correlation and outcomes Retrieval API Correlation and outcomes Azure Monitor. Filtered evidence and alerts Azure Monitor Filtered evidence and alerts Versioned source intake 9 Inspect source 10 Versioned index write 12 Publish approved corpus 11 Preserve recovery inputs 14 Rebuild and reauthorize 15 Filtered evidence 13 An index is rebuilt Retain approved sources, permission manifests and index definitions. Validate revocation, model capacity and negative access tests before reopening. Evidence is minimized Retain correlation, policy outcomes, latency and usage. Full prompts, document bodies, tokens and model responses are excluded by default. Reference design · Microsoft product icons used under Microsoft’s published terms · Numbered steps refer to the accompanying flow register
9 components · 7 numbered connections

Green step numbers match the F-numbered flow register below. Solid boundaries show ownership; dashed blue boundaries show networks. Logical groups are labeled explicitly. Microsoft service icons identify the components.

Follow the numbered connections

Flow register

Paths shown in the diagram have the following scope.

FlowFrom → toPurpose and control
F1TrafficEmployee client ExpressRoute / VPNPrivate entry

The employee reaches Azure through approved ExpressRoute or VPN connectivity; no public application ingress is required.

F2TrafficExpressRoute / VPN Private endpointsSpoke access

Hub-to-spoke routing and DNS deliver requests to the intended private endpoint. Private reachability does not grant data access.

F3TrafficPrivate endpoints App Service: employee UIPrivate UI

The App Service private endpoint is inbound-only. Public network access is separately disabled; outbound integration is separate.

F4Identity & controlEmployee client Microsoft Entra IDUser sign-in

The client obtains an Entra access token for the retrieval API. The identity service has its own approved route and is not treated as a VNet appliance.

F5TrafficApp Service: employee UI API ManagementAuthorized question

The UI calls the private APIM gateway with the user access token for the retrieval API audience; APIM validates tenant, audience and scopes.

F6TrafficAPI Management App Service: retrieval APIValidated API call

APIM forwards the token only to its intended retrieval API. The API validates authorization again and derives principals server-side.

F7TrafficApp Service: retrieval API Azure AI SearchFiltered retrieval

The query-only managed identity calls Search over Private Link. The application applies tenant and principal filters to every exposed retrieval path.

F8TrafficApp Service: retrieval API Azure OpenAIAuthorized context

The workload obtains its own inference token and submits only approved retrieved context to the selected model deployment.

F9TrafficAuthoritative sources Blob quarantineSource intake

An authenticated exporter submits versioned content and permission manifests to the private ingestion path; objects remain quarantined pending validation.

F10TrafficBlob quarantine App Service: ingestion workerInspect source

The ingestion worker reads quarantined objects, checks inspection outcomes and creates bounded chunks with provenance and permissions.

F11TrafficApp Service: ingestion worker Approved Blob corpusApproved publication

Validated content and manifests enter the approved versioned corpus. Permission failures and incomplete inspection block publication.

F12TrafficApp Service: ingestion worker Azure AI SearchVersioned index write

A separate writer identity publishes content and permission metadata; revocation and deletion updates also invalidate dependent caches.

F13MonitoringApp Service: retrieval API Monitor / Application InsightsFiltered evidence

Representative telemetry flow: identity-independent request correlation, policy outcomes, counts, latency and usage; workers and gateways are instrumented separately.

F14RecoveryApproved Blob corpus Recovery corpus and IaCRecoverable corpus

Preserve approved source versions, manifests and index definitions using the selected backup/copy policy; the index is rebuilt rather than assumed to have native restore.

F15RecoveryPlatform and data owners Recovery corpus and IaCRebuild and verify

Owners restore networking, identities and model capacity, rebuild the index and repeat negative authorization tests before reopening access.

F16Identity & controlRelease pipeline App Service: retrieval APIFederated release

A narrowly scoped deployment identity releases reviewed application artifacts using private runner connectivity. Runtime identities do not receive deployment authority.

Assumptions to confirm

  • One organizational Entra tenant and an existing Azure landing zone with separate connectivity and workload ownership.
  • Corporate clients reach the application spoke through private connectivity; a public customer portal is outside this design.
  • The baseline uses customer-managed App Service orchestration and push indexing, not a managed Foundry agent runtime.
  • Document sources provide an authoritative permission and deletion feed; unresolved permissions fail closed.
  • Region, model deployment type, quota and recovery targets are selected before procurement.

This reference design uses private App Service applications, an API Management gateway, Azure AI Search and Azure OpenAI inference. The application owns retrieval authorization. It is a proposed enterprise configuration, documentation-reviewed rather than deployed or load-tested; the private request view shows managed services outside the customer VNet, with private endpoints and integration subnets inside it.

The three views separate the private request path, identity and release authority, and content publication and recovery. Together they retain links F1-F16; repeated retrieval links provide context in the identity view. The procedures below expand those paths.

Deployment decisions and component responsibilities

Place application resources in a workload subscription governed by the landing zone. Reuse approved hub connectivity, DNS and egress services where ownership and capacity permit. Keep source systems authoritative; the search index is a derived copy, not the document system of record.

Component Selected configuration Security responsibility
Employee web UI App Service Premium v3, private endpoint, public ingress disabled Present citations and obtain tokens for the application API.
Application gateway APIM Standard v2, inbound private endpoint plus outbound VNet integration Validate tenant, audience and API scopes; impose request and token budgets.
Retrieval API Separate App Service app and managed identity Derive principal set, enforce document filters and minimize model context.
Ingestion worker Separately deployed worker with its own identity Validate source, content version, permission version and deletion events.
Search Billable Standard service, independent reader/writer roles Store chunks, vectors, source identifiers and authorization metadata.
Storage Separate quarantine and approved Blob containers or accounts Prevent unapproved objects becoming retrievable evidence.
Inference Azure OpenAI resource, approved deployment and private endpoint Receive only selected authorized context.
Shared controls Key Vault, Private Resolver, Firewall, Monitor Protect credentials, name resolution, egress and operational evidence.

App Service private endpoints cover inbound traffic; outbound VNet integration uses a different subnet. Private endpoints do not themselves disable public access, and App Service access restrictions are not evaluated on private-endpoint traffic. Include deployment-slot and SCM connectivity in the private deployment design. App Service boundaries.

For APIM, Standard v2 provides the selected combination of inbound Private Link and outbound integration. Private Link covers the gateway endpoint, not every administrative surface. Integration alone does not privatize ingress. Allocate the documented dedicated integration subnet, preserve service dependencies, and qualify region availability. The v2 gateway also requires a publicly resolvable DNS name; that requirement does not mean public data-plane access must remain enabled. Private endpoint limits, integration requirements.

Numbered request and identity flow

  1. A managed employee device reaches the private UI through ExpressRoute or VPN. Corporate DNS forwards the relevant Azure private-name resolution through the hub resolver. Entra sign-in follows its approved identity-service route; it is not assumed to be private merely because the application is private.
  2. The client obtains an access token for the retrieval API. APIM validates the token and passes it to that same API audience. The API independently validates issuer, tenant, audience, expiry and required scope; network reachability is insufficient authorization.
  3. The API derives tenant and user identifiers from validated claims and resolves authorized groups through a trusted server-side path. Missing group claims, group overage, stale membership or resolution failure must not become unrestricted search. Bound caching to the agreed revocation interval.
  4. The API creates the mandatory document filter and calls Search with its query-only managed identity. Caller-supplied principal lists, index names and filter fragments are not accepted as authority.
  5. Only authorized chunks enter the prompt. Treat their text as untrusted evidence, cap context size, and return source references through a citation endpoint that rechecks access.
  6. The API obtains a model-service token using its workload identity and invokes the selected deployment. That token represents the workload, not the employee. If a downstream API must enforce the employee’s own rights, implement a supported delegated/OBO path instead.

OBO exchanges a user token for a token intended for a downstream API; it does not convert an app-only identity into user delegation. Acquire the correct audience through MSAL rather than forwarding a token to an unrelated service. Microsoft OBO model.

For the dedicated Azure OpenAI resource selected here, grant the runtime only the inference role it needs at that resource. Cognitive Services OpenAI User can invoke deployed models without deployment-management privileges; it is not a per-user document entitlement or an isolated grant to one model deployment. Use separate resources when that administrative boundary is necessary. Foundry project roles require their own resource-type review. Azure OpenAI role scope.

Document authorization is an application invariant

The baseline uses a filterable collection of source-derived allowed principal identifiers, plus tenant, source version and permission version. Microsoft describes ordinary security filters as string comparison, not built-in identity authorization. The application must therefore apply the filter to every exposed retrieval path. A non-retrievable permission field is not a security boundary. Security-filter semantics.

Expose a deliberately narrow API. Do not offer unrestricted search, lookup, autocomplete, facets, raw vector results or direct browser access to Search. Include tenant, entitlement version and document version in cache keys; invalidate cached answers when their evidence loses authorization. A citation URL must not become an alternate download path around the original decision.

Separate query and ingestion identities. Search Index Data Reader queries content; Search Index Data Contributor writes content. Resource-management roles and indexer privileges need separate scrutiny, and per-index role assignments do not constrain indexers in the same way as direct API calls. Use separate search services when administrative isolation between collections is required. Search role boundaries.

Built-in ACL/RBAC, SharePoint ACL ingestion and sensitivity-label enforcement are documented preview alternatives. Their source, API/SDK and synchronization requirements differ; they are not an automatic property of every index. Query-time enforcement relies on indexed permission metadata and a documented user-authorization mechanism. Evaluate that design separately, including group expansion and permission freshness, before replacing the baseline. Access-control options, query enforcement qualifications.

Numbered ingestion and permission-update flow

  1. An approved source exporter submits content plus a signed or otherwise authenticated manifest containing source ID, version, hash and authoritative permissions. Write first to quarantine; the query identity cannot read this area.
  2. A constrained worker checks file type, size, malware-scanning outcome and parser limits. Failed or incomplete inspection stays quarantined. Scanning does not establish that document instructions are trustworthy.
  3. The worker creates bounded chunks, preserves provenance and attaches complete permission metadata to every chunk. Generate embeddings with a separately permitted workload identity; embedding generation also sends document content to a model service.
  4. Publish approved content and index records as a versioned release. Readers see only a complete active version. Missing permissions, orphan chunks and mismatched versions fail closed.
  5. Process permission reductions and deletions with priority. Apply a temporary deny record when the normal update pipeline cannot meet the revocation objective, then reconcile the index and caches. Measure lag from the authoritative change, not just from the most recent successful indexing run.

These publication and revocation rules are application design requirements. A successful index operation does not demonstrate source-permission fidelity.

Network, model and telemetry boundaries

Use private endpoints for Search, storage, Key Vault and the model resource, with public data ingress explicitly disabled after validation. Link the right private zones to the client and workload paths. Test DNS from the employee, API, ingestion worker and deployment runner; a successful lookup from an administrator’s laptop proves little about the runtime. Hybrid private DNS pattern.

Route applicable workload egress through the approved firewall and inventory identity, certificate, package and monitoring dependencies. Private Link does not constrain every outbound call or every managed-service tool. This design does not deploy Foundry Agent Service. If that becomes a requirement, assess inbound access, agent outbound networking and tool-specific dependencies separately. Classic hub managed-network instructions explicitly do not apply to new Foundry projects. Foundry network scope, classic applicability.

Model deployment type determines processing geography. Global, Data Zone and geography-based options are different choices; a private endpoint does not change those processing commitments. Confirm model/version availability and allocated quota in both primary and recovery locations. Deployment-type distinctions.

The telemetry flow is API/worker spans to Application Insights and operational metrics to Monitor/Log Analytics, with security signals routed to the SOC. Record correlation ID, policy version, pseudonymous principal reference, authorization outcome, chunk count, dependency latency, token usage and revocation lag. Do not collect full prompts, responses, source text, bearer tokens or URL query contents by default. Filter before export, and test exceptions and SDK instrumentation for leakage. Ingestion-time transformations act after Azure receives telemetry. Telemetry filtering.

Recovery, acceptance and operating cost

  1. Preserve approved source versions, manifests, index definitions, application releases and policy configuration in recoverable stores with named owners.
  2. Recreate the service in the approved recovery region and rebuild the index from authoritative data. Search provides no self-service native backup/restore; test the rebuild duration and validate permissions before reopening queries.
  3. Restore DNS, private endpoints, identities, role assignments and model capacity. Re-run negative authorization tests before routing users.

For Search availability, Microsoft requires two replicas for the read-only SLA and three for read/write workloads. Continuous ingestion makes the latter relevant. Replicas do not provide independent regional recovery. Search reliability and rebuilding.

Threat or failure test Release acceptance evidence
Forged group, tenant or filter input Request rejected or ignored; no unauthorized chunk returned.
Revoked source permission Direct retrieval, cached answer and citation access denied within the agreed interval.
Malicious or partially parsed document No unapproved chunk enters the active index.
DNS failure or public-endpoint attempt Explicit failure; no hidden public fallback.
Missing authorization dependency Query fails closed rather than dropping the filter.
Region recovery Timed rebuild, model-capacity check and repeated access tests pass.

Pilot with synthetic documents, then a bounded business collection. The platform team owns networking and deployment; data owners own permissions and deletion feeds; the application team owns filtering and cache invalidation; the SOC owns alert routing and evidence access. Record unresolved exceptions before expanding access.

Budget for always-on App Service/APIM capacity, Search replicas and partitions, private endpoints, firewall/DNS, model tokens, ingestion reprocessing and log retention. Sharing an approved hub reduces duplication; sharing application plans or indexes increases contention and administrative coupling. Choose those tradeoffs explicitly. This architecture supplies no cost estimate or recovery guarantee without workload sizing and measured validation.

Trace the design decisions

Microsoft architecture & product guidance

  1. Microsoft Learn - App Service private endpoints ↗
  2. Microsoft Learn - API Management inbound private endpoint ↗
  3. Microsoft Learn - API Management outbound VNet integration ↗
  4. Microsoft Learn - Azure AI Search security filter pattern ↗
  5. Microsoft Learn - Azure AI Search document access options ↗
  6. Microsoft Learn - Azure AI Search query-time permission enforcement ↗
  7. Microsoft Learn - Azure AI Search RBAC ↗
  8. Microsoft Learn - Entra on-behalf-of flow ↗
  9. Microsoft Learn - Microsoft Foundry private networking ↗
  10. Microsoft Learn - Classic Foundry hub managed networks ↗
  11. Microsoft Learn - Foundry model deployment types ↗
  12. Microsoft Learn - Azure AI Search reliability ↗
  13. Microsoft Learn - Private Resolver for on-premises workloads ↗
  14. Microsoft Learn - Filter OpenTelemetry in Application Insights ↗
  15. Microsoft Learn - Azure OpenAI resource role scope ↗

Inspect operational evidence

Supporting Sentinel queries

These are contextual hunting starters, not end-to-end validation of the architecture. Check each query's source tables, collection scope and limitations.

Query library →