Assumptions to confirm
- A single organizational tenant and an approved landing zone provide private connectivity and DNS.
- Customer-managed Container Apps host the orchestrator and execution services; managed Foundry Agent Service is an optional redesign.
- Backends expose narrow Entra-protected APIs with stable object identifiers and an idempotency or reconciliation contract.
- High-impact actions require an authorized reviewer independent of the planning agent.
- Service Bus Premium and private endpoints are accepted fixed-cost components; recovery capacity is budgeted separately.
This design treats model output as a proposed action, never as execution authority. Separate services validate, approve, dispatch and execute changes. It is a documentation-reviewed reference architecture, not a deployed implementation or a promise that every Microsoft agent product supplies these controls automatically.
The three views separate planning and inference, authorization and approval, and execution and recovery. Together they retain links F1-F16. Supporting network services apply to the corresponding private paths.
Concrete service selection and trust boundaries
Deploy customer-managed services into an internal Azure Container Apps workload-profile environment connected to the application spoke. Use APIM Standard v2 with a private gateway endpoint and outbound VNet integration. The model gateway and application APIs can share that APIM instance initially, but use separate API policies and identities; separate instances provide stronger administrative isolation.
| Component | Deployment choice | Authority deliberately withheld |
|---|---|---|
| Orchestrator | Container App, bounded model/tool iterations | No business write credential and no direct queue-send role. |
| Tool and model gateway | Private APIM APIs with fixed backend mappings | No caller-selected destination, audience or arbitrary forwarding URL. |
| Approval/policy service | Separate Container App and managed identity | Cannot silently change an already approved action. |
| Durable state | Azure SQL Database General Purpose, private endpoint | Planning identity cannot approve or alter execution status. |
| Outbox dispatcher | Container App with queue-sender identity | Does not invent actions or grant approval. |
| Execution worker | Separate Container App, ingress disabled | Can call only the assigned narrow backend operations. |
| Messaging | Service Bus Premium, private endpoint, queue and DLQ | A message’s presence does not confer business authorization. |
| Business API | Private Entra-protected App Service API | Rechecks target, operation and caller rights. |
| Supporting services | Key Vault, ACR Premium, DNS, Firewall, Monitor | Each has separate administrative and runtime permissions. |
Private Link for Service Bus requires Premium. Container Apps workload-profile environments support the selected egress routing features; legacy Consumption-only environments differ. Internal environments use a private load-balancer address. Configure application ingress to permit the required VNet clients without confusing environment-private ingress with access restricted to sibling apps only. Messaging tier, Container Apps network model.
Numbered planning and identity flow
- The enterprise client authenticates through Entra and calls the private APIM application API with a token for the intended backend. Validate tenant, audience, authorized client and required claims. The backend validates the token again; a trusted gateway address is not user authorization.
- APIM sends the bounded request to the orchestrator. The orchestrator loads the approved tool catalog and schema version. Disable tools outside the assigned business workflow, and reject unknown fields, unbounded selectors and model-supplied network locations.
- The orchestrator calls the model through a separate APIM API using its workload identity. That API applies usage limits and uses a narrowly scoped gateway identity for inference. It must not forward an inference token to a caller-selected destination.
- A tool response or retrieved document remains untrusted data. The orchestrator may propose an action, but only the policy service can turn that proposal into an eligible execution record.
APIM provides Entra token validation and managed-identity authentication to backends. Its managed-identity policy replaces the authorization header with a resource-specific token; policy authors can exercise that identity’s privileges. Restrict policy editing and review backend mappings as privileged configuration. Token validation, identity-policy limits.
Assign different managed identities to the orchestrator, approval service, dispatcher and executor. Prefer direct resource role assignments over a shared broadly privileged identity, and keep deployment permissions separate. Container Apps supports managed identities and managed-identity-based scale rules; configure the scaler separately from the worker’s application permissions. Runtime identities.
Use delegated OBO only when a backend must enforce the initiating user’s rights and the token lifecycle supports that interaction. For delayed work, choose a documented app-only authorization contract with server-validated approval context, or require fresh user authorization; do not place reusable user tokens in messages. OBO is for user delegation, not exchanging app-only identities. OBO scope.
Approval is an immutable action record
Our proposed record binds tenant, requesting user, tool version, operation, exact resource ID, canonical parameters, expected resource version, maximum effect, cost budget, expiry, policy version and idempotency key. Store a canonical action hash with the approval. A later parameter change creates a new proposal and invalidates the old approval.
The reviewer sees the target and effect in business terms, including the exact recipient, amount, object or permission change where relevant. The service checks reviewer authority and separation of duties; a natural-language statement that a manager approved is not evidence. Require fresh approval after expiry, material policy changes or changes to the resource precondition.
Low-impact operations may follow a deterministic preapproved policy, but record that policy decision explicitly. Do not label it human approval. Bound read tools too: unrestricted bulk export can have consequences even when no database row is modified.
Numbered dispatch and execution flow
- The approval service atomically writes the eligible action and an outbox record in Azure SQL. The outbox closes the gap between accepting an approval and attempting message publication; it does not create a distributed transaction with the backend.
- The dispatcher sends a compact reference and immutable action hash to Service Bus, using a stable message ID. It records dispatch progress and safely retries uncertain sends. Do not include secrets or the entire conversation.
- The worker receives with PeekLock, reads authoritative state, and rechecks expiry, policy, resource version, approval and execution status. Reject discrepancies even when the queue message is well formed.
- The worker calls a fixed private APIM tool route with its own token for the backend audience. APIM forwards that token only to the intended backend, which validates the identity, constrained app role and resource-level authorization. The backend also consumes the stable idempotency key or exposes a reliable status lookup.
- Commit the outcome and receipt to the ledger, then complete the message. If the backend may have succeeded but the response was lost, mark the action uncertain and reconcile by key. Never blindly repeat a consequential action whose outcome is unknown.
Service Bus PeekLock provides at-least-once delivery, so retries and lock loss can repeat processing. Duplicate detection filters repeated message IDs within its configured history window; it does not make a database update, payment or permission change happen exactly once. Delivery semantics, duplicate-detection boundary.
For a backend without idempotency or reliable reconciliation, require an explicit manual recovery path before enabling automated writes. Limit retry count and duration, use backoff, and stop retries after approval expiry. Dead-lettered actions require operator triage and fresh authorization checks before replay. DLQs are not automatically cleaned up, and their messages do not observe normal TTL expiration. DLQ behavior.
Private connectivity does not make tools trustworthy
Keep APIM, SQL, Service Bus, Key Vault, registry and model data paths private where supported. ACR private endpoints require Premium; private registry builds and image pulls need their own DNS and network path. Registry requirements. Disable public data access explicitly. Resolve the correct service names from the runtime, dispatcher, worker and private build runner. Enforce fixed destinations and redirect handling in application code; a permitted hostname does not authorize arbitrary operations or data upload.
Route supported Container Apps outbound traffic through the hub firewall and allow required platform dependencies deliberately. Use separate subnets for services requiring exclusive delegation. A NAT address is an egress identity, not a destination policy. Container Apps firewall integration.
Foundry Agent Service is an alternative execution host, not an assumed component here. Its public, BYO-VNet and managed-network options have different inbound/outbound behavior and tool dependencies. Reassess session isolation, tool reachability and data stores before substitution; an inbound private endpoint alone does not establish a private outbound tool path. Foundry networking choices.
Numbered telemetry and recovery flow
- Emit a correlation ID across gateway, proposal, approval, outbox, broker delivery and backend receipt. Record policy decisions, approval hashes, tool versions, result classes, latency and usage. Do not export access tokens, full prompts, confidential parameters or raw tool results by default.
- Send filtered spans to Application Insights and operational signals to Monitor/Log Analytics. Alert on approval mismatches, unexpected tool destinations, repeated lock loss, growing DLQ age, stuck outbox items and uncertain outcomes. Test filtering before export; ingestion transformations occur after collection. Telemetry controls.
- On incident declaration, stop new dispatch and fence old workers before restoring state. Preserve the ledger and backend receipts. Resume only after reconciling uncertain actions; restoring an older database must not authorize replay of already completed business effects.
- Test SQL point-in-time restore within the configured retention policy. For regional resilience, choose and qualify geo-capable backup/replication settings explicitly. Service Bus Geo-Replication copies message data and metadata; metadata Geo-Disaster Recovery does not copy queued messages. Promotion and private endpoint/DNS readiness need an owned runbook. SQL recovery, Service Bus regional options.
A model outage should stop new planning safely, without changing approved tool parameters or switching to an unapproved model. Qualify the recovery deployment, version, processing geography and quota before an incident. Bound retry budgets and honor throttling; regional model capacity is a separate dependency from restoring SQL or the queue. Model deployment choices.
Acceptance, rollout and cost ownership
| Test | Required result |
|---|---|
| Prompt requests an unregistered tool | No network call or write capability is issued. |
| Parameters change after approval | Hash/precondition mismatch blocks execution. |
| Worker crashes after backend success | Reconciliation finds the original result without repeating the effect. |
| Expired action returns from DLQ | Replay is blocked pending a new valid decision. |
| Wrong tenant, audience or backend role | Authorization fails at gateway and backend. |
| Primary-region recovery | Previous workers are fenced; restored ledger reconciles with backend receipts. |
| Sensitive synthetic input | Routine logs contain approved metadata only. |
Begin with read-only tools, then synthetic writes, then one reversible business action. The business owner defines approval thresholds and compensation; the application team owns idempotency and policy; the platform team owns identities, networking and release controls; operations owns uncertain outcomes and replay. Do not expand tool scope until these responsibilities have staffed escalation paths.
Budget for APIM and Service Bus Premium’s capacity floor, Container Apps minimum replicas, SQL compute and backups, ACR Premium, private endpoints, firewall/DNS, model usage and retained evidence. Queue-based scale-out can reduce idle worker cost but must not exceed backend throughput or reviewer capacity. Regional standby and long retention add cost. Set per-workflow budgets and measure failure/retry costs alongside successful actions before choosing reserved capacity.
Trace the design decisions
Microsoft architecture & product guidance
- Microsoft Learn - Container Apps networking ↗
- Microsoft Learn - Container Apps managed identities ↗
- Microsoft Learn - Container Apps firewall integration ↗
- Microsoft Learn - API Management private endpoints ↗
- Microsoft Learn - API Management Entra token validation ↗
- Microsoft Learn - API Management managed-identity policy ↗
- Microsoft Learn - Entra on-behalf-of flow ↗
- Microsoft Learn - Service Bus Premium ↗
- Microsoft Learn - Service Bus message loss and duplicate processing ↗
- Microsoft Learn - Service Bus duplicate detection ↗
- Microsoft Learn - Service Bus dead-letter queues ↗
- Microsoft Learn - Service Bus geo-replication ↗
- Microsoft Learn - Azure SQL automated backups ↗
- Microsoft Learn - Foundry Agent Service networking options ↗
- Microsoft Learn - Filter OpenTelemetry ↗
- Microsoft Learn - ACR private endpoints ↗
- Microsoft Learn - Model deployment types ↗