.NET: Hosted Agent Sample - Toolbox with various Auth (#5777)#6018
Open
rogerbarreto wants to merge 15 commits into
Open
.NET: Hosted Agent Sample - Toolbox with various Auth (#5777)#6018rogerbarreto wants to merge 15 commits into
rogerbarreto wants to merge 15 commits into
Conversation
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Automated Code Review
Reviewers: 4 | Confidence: 94%
✓ Correctness
The PR consistently migrates from FOUNDRY_AGENT_TOOLSET_ENDPOINT to FOUNDRY_PROJECT_ENDPOINT throughout the codebase, but two spots in newly-added files still reference the old env var name: one in the FoundryToolboxHealthCheck description string (user-facing in health probe responses) and one in the FoundryToolboxStartupStatus XML doc. The health check string is the more impactful issue since operators will see it when diagnosing readiness failures.
✓ Security Reliability
The PR makes substantive improvements to the Foundry Toolbox integration (URL derivation, auth scope, trace propagation, readiness gating). One operator-facing reliability issue found: the health check description and enum XML doc for the
NoEndpointstatus still reference the removedFOUNDRY_AGENT_TOOLSET_ENDPOINTenv var, while all production code now readsFOUNDRY_PROJECT_ENDPOINT. This would mislead operators during troubleshooting.
✓ Test Coverage
Good test coverage overall — 10 new unit tests cover the key behavioral changes (scope change, mandatory header, trace propagation, health check states, readiness endpoint mapping). One concrete gap: the NoEndpoint health check description still references the old env var name (
FOUNDRY_AGENT_TOOLSET_ENDPOINT) and the corresponding test doesn't validate the description, unlike the Pending-state test. The same stale reference appears in theFoundryToolboxStartupStatus.NoEndpointdoc comment.
✗ Design Approach
I found two design-level issues. First, the new automatic
/readinessmapping only checks for an existing route at the momentMapFoundryResponses()runs, but the repo already has a caller that adds its own/readinessroute afterward; that path will now register two GET/readinessendpoints. Second, the new Hosted-Toolbox-AuthPaths sample documents a configurable toolbox name, but the hosted manifest hardcodesauth-paths-toolbox, so the documented custom-name deployment path cannot work as written.
Flagged Issues
- The Hosted-Toolbox-AuthPaths README says the toolbox may have any name (exported as
TOLBOX_NAME), butagent.manifest.yaml:34-35and:42-43hardcodeauth-paths-toolbox, so hosted deployments with a non-default toolbox name will not resolve the configured toolbox.
Automated review by rogerbarreto's agents
Contributor
There was a problem hiding this comment.
Pull request overview
Updates the .NET Foundry hosting bridge to work with the live Foundry Toolbox MCP proxy (URL derivation + required headers/scope/trace propagation), adds readiness/health-check wiring for hosted runtime compliance, and introduces a new hosted sample demonstrating multiple toolbox authentication paths.
Changes:
Microsoft.Agents.AI.Foundry.Hosting: derive toolbox proxy URLs fromFOUNDRY_PROJECT_ENDPOINT, enforceFoundry-Features: Toolboxes=V1Preview, switch auth scope tohttps://ai.azure.com/.default, propagate W3C trace context, and gate/readinessvia health checks.- Tests: add/extend unit tests covering readiness auto-mapping, toolbox startup status/health check behavior, and bearer-handler header/scope/trace propagation.
- Samples/docs: add
Hosted-Toolbox-AuthPaths+ companion REPL client; refresh hosted toolbox documentation and cross-links.
Reviewed changes
Copilot reviewed 25 out of 25 changed files in this pull request and generated 11 comments.
Show a summary per file
| File | Description |
|---|---|
| dotnet/tests/Microsoft.Agents.AI.Foundry.Hosting.UnitTests/ServiceCollectionExtensionsTests.cs | Adds unit tests verifying /readiness auto-mapping behavior and idempotency. |
| dotnet/tests/Microsoft.Agents.AI.Foundry.Hosting.UnitTests/FoundryToolboxServiceTests.cs | Expands toolbox service tests for endpoint resolution and startup-status reporting. |
| dotnet/tests/Microsoft.Agents.AI.Foundry.Hosting.UnitTests/FoundryToolboxHealthCheckTests.cs | Adds coverage for readiness gating via toolbox startup status and failed toolbox reporting. |
| dotnet/tests/Microsoft.Agents.AI.Foundry.Hosting.UnitTests/FoundryToolboxBearerTokenHandlerTests.cs | Updates tests for new token scope, mandatory feature flag behavior, and trace propagation. |
| dotnet/tests/Microsoft.Agents.AI.Foundry.Hosting.UnitTests/FoundryProjectEndpointEnvFixture.cs | Serializes env-var-mutating tests to avoid parallel-test flakiness. |
| dotnet/src/Microsoft.Agents.AI.Foundry.Hosting/ServiceCollectionExtensions.cs | Registers health checks, auto-maps /readiness when missing, and wires toolbox health-check gating. |
| dotnet/src/Microsoft.Agents.AI.Foundry.Hosting/FoundryToolboxStartupStatus.cs | Introduces a startup-status enum used to drive readiness gating. |
| dotnet/src/Microsoft.Agents.AI.Foundry.Hosting/FoundryToolboxService.cs | Switches toolbox proxy URL derivation to /toolboxes/{name}/mcp, adds startup status + failed toolbox tracking. |
| dotnet/src/Microsoft.Agents.AI.Foundry.Hosting/FoundryToolboxOptions.cs | Updates toolbox URL doc and defaults ApiVersion to v1. |
| dotnet/src/Microsoft.Agents.AI.Foundry.Hosting/FoundryToolboxHealthCheck.cs | Adds health-check adapter for toolbox startup outcome. |
| dotnet/src/Microsoft.Agents.AI.Foundry.Hosting/FoundryToolboxBearerTokenHandler.cs | Enforces mandatory feature header, updates token scope, and propagates traceparent/tracestate/baggage. |
| dotnet/samples/04-hosting/FoundryHostedAgents/responses/Using-Samples/Hosted-Toolbox-AuthPaths-Client/Program.cs | New REPL client that handles OAuth mcp_approval_request consent loop. |
| dotnet/samples/04-hosting/FoundryHostedAgents/responses/Using-Samples/Hosted-Toolbox-AuthPaths-Client/Hosted-Toolbox-AuthPaths-Client.csproj | Project file for the new REPL client sample. |
| dotnet/samples/04-hosting/FoundryHostedAgents/responses/Hosted-Toolbox/README.md | New/updated README for toolbox usage + links to the auth-paths sample. |
| dotnet/samples/04-hosting/FoundryHostedAgents/responses/Hosted-Toolbox/Program.cs | Updates hosted toolbox sample to use FOUNDRY_PROJECT_ENDPOINT for project endpoint resolution. |
| dotnet/samples/04-hosting/FoundryHostedAgents/responses/Hosted-Toolbox-AuthPaths/README.md | New detailed walkthrough documenting five toolbox auth paths and setup steps. |
| dotnet/samples/04-hosting/FoundryHostedAgents/responses/Hosted-Toolbox-AuthPaths/Program.cs | New hosted agent sample pre-registering a toolbox and relying on hosting bridge readiness mapping. |
| dotnet/samples/04-hosting/FoundryHostedAgents/responses/Hosted-Toolbox-AuthPaths/Hosted-Toolbox-AuthPaths.csproj | Project file for the new hosted sample. |
| dotnet/samples/04-hosting/FoundryHostedAgents/responses/Hosted-Toolbox-AuthPaths/Dockerfile.contributor | Contributor Docker flow using pre-published output. |
| dotnet/samples/04-hosting/FoundryHostedAgents/responses/Hosted-Toolbox-AuthPaths/Dockerfile | Standard Dockerfile for package-consumer builds. |
| dotnet/samples/04-hosting/FoundryHostedAgents/responses/Hosted-Toolbox-AuthPaths/agent.yaml | Container agent YAML for the new hosted sample. |
| dotnet/samples/04-hosting/FoundryHostedAgents/responses/Hosted-Toolbox-AuthPaths/agent.manifest.yaml | Agent manifest for the new hosted sample. |
| dotnet/samples/04-hosting/FoundryHostedAgents/responses/Hosted-Toolbox-AuthPaths/.env.example | Example env vars for running the new hosted sample locally. |
| dotnet/samples/04-hosting/FoundryHostedAgents/responses/Hosted-McpTools/README.md | Adds “Related samples” links to toolbox samples. |
| dotnet/agent-framework-dotnet.slnx | Adds the new hosted sample and REPL client to the solution. |
…h toolbox health gating (microsoft#5777) Add a new hosted agent sample demonstrating five MCP tool authentication paths (API key, agent MI, project MI, custom OAuth, literal token) via a Foundry Toolbox. Package changes (Microsoft.Agents.AI.Foundry.Hosting): - MapFoundryResponses now auto-maps GET /readiness via MapHealthChecks, idempotent across Tier 1/2 (AgentHost, already mapped) and Tier 3 (WebApplication, gap filled). - AddFoundryResponses registers AddHealthChecks() so the pipeline is available. - AddFoundryToolboxes registers FoundryToolboxHealthCheck on the /readiness aggregate, gating readiness on pre-registered toolbox startup outcome (per spec section 3.1). - FoundryToolboxService now exposes StartupStatus and FailedToolboxNames properties. New types: - FoundryToolboxStartupStatus (public enum): Pending, Healthy, Failed, NoEndpoint. - FoundryToolboxHealthCheck (internal IHealthCheck): adapts startup status to the AspNetCore HealthChecks pipeline with failed toolbox names in result data. Tests: - 3 new tests for /readiness auto-mapping (Tier 3 default, pre-mapped skip, idempotent). - 4 new tests for FoundryToolboxHealthCheck (Pending, NoEndpoint, Failed, Healthy). - 3 enhanced FoundryToolboxServiceTests with StartupStatus assertions.
…oft#5777 Part A) Bring Microsoft.Agents.AI.Foundry.Hosting's toolbox path into compliance with tools-integration-spec.md sections 2-4, 6.3, and 9. Empirically validated against tao-foundry-prj: the previous code (reading FOUNDRY_AGENT_TOOLSET_ENDPOINT, which the platform never injects) silently registered zero tools in production. Package changes (Microsoft.Agents.AI.Foundry.Hosting): - FoundryToolboxService.StartAsync now derives the toolbox proxy base URL from the platform-injected FOUNDRY_PROJECT_ENDPOINT and constructs the per-toolbox URL as {FOUNDRY_PROJECT_ENDPOINT}/toolboxes/{name}/mcp?api-version={ApiVersion} per spec sections 2-3. The legacy FOUNDRY_AGENT_TOOLSET_ENDPOINT env var is removed outright (preview package, no production consumers). - FoundryToolboxOptions.ApiVersion default flipped to 'v1' to match spec example. - FoundryToolboxBearerTokenHandler always sends the mandatory Foundry-Features: Toolboxes=V1Preview header per spec section 2, merging any additional flags supplied via the FOUNDRY_AGENT_TOOLSET_FEATURES env var. - FoundryToolboxBearerTokenHandler token scope changed from https://cognitiveservices.azure.com/.default to https://ai.azure.com/.default per spec section 4. - FoundryToolboxBearerTokenHandler propagates W3C trace context (traceparent, tracestate, baggage) from Activity.Current per spec section 6.3. Sample changes: - Hosted-Toolbox-AuthPaths and Hosted-Toolbox Program.cs, README.md, and .env.example corrected to describe the actual env-var contract (FOUNDRY_PROJECT_ENDPOINT auto-injected; AZURE_AI_PROJECT_ENDPOINT as the local-dev fallback). Removes the misleading 'auto-injected by Foundry runtime' claims for FOUNDRY_AGENT_TOOLSET_ENDPOINT. - Hosted-Toolbox-AuthPaths/agent.manifest.yaml declares the toolbox and model dependencies under resources[] per the AgentManifest schema so azd ai agent init users get them provisioned automatically. Tests: - 4 new FoundryToolboxServiceTests covering env-var derivation, EndpointOverride precedence, trailing-slash normalization, and the existing NoEndpoint behavior under the new env var name. - 4 new FoundryToolboxBearerTokenHandlerTests covering token scope, mandatory feature header always present, header merging with override, no duplicate mandatory flag, trace context propagation from Activity.Current, and no override of caller-set traceparent. - New FoundryProjectEndpointEnvFixture xUnit collection definition serializes env-var-mutating tests across FoundryToolboxServiceTests and FoundryToolboxHealthCheckTests, preventing parallel-execution races. - FoundryToolboxHealthCheckTests adjusted for the new env var name.
…#5777 Part B) Empirically verified that any Azure Cognitive Services MCP endpoint already in the Foundry project (e.g., a Language service MCP) accepts Entra tokens and can serve Paths 2 and 3 without deploying a separate Azure MCP Server to ACA. README updates: - Step 0 rewritten: 'Identify an Entra-authenticated MCP target in your project' instead of 'Deploy Azure MCP Server to Azure Container Apps' (the original azmcp-foundry-aca-mi setup is now optional, not required). - Auth-paths matrix updated to describe AAD-based connections targeting a Cognitive Services MCP URL (e.g., Language service) instead of an ACA URL. - Step 2 connections table updated: the Entra ID category is now a single 'AAD' authType. The original 'Agent Identity' vs 'Project Managed Identity' as selectable connection sub-types is NOT exposed via the ARM control plane today; the platform selects the calling principal contextually. Both connections in the walkthrough share the same shape and target. - Added an explicit RBAC note: the agent identity AND project MI must hold the required role (typically Cognitive Services User) on the target resource; without it the MCP server returns HTTP 401 even though the connection wiring is correct. - Toolbox tool entries renamed lang_entra_agent / lang_entra_project to match the new connection names. Empirical validation supporting these changes is captured in the session plan.md (Part B addendum).
…Paths 2/3 (microsoft#5777) Updates the sample README with the verified connection shape and RBAC procedure for Microsoft Entra agent-identity and project-managed-identity MCP authentication: - Connection authType values: AgenticIdentityToken (agent identity) and ProjectManagedIdentity (project MI), both with category=RemoteTool. - Top-level audience property required; for Cognitive Services targets the value is https://cognitiveservices.azure.com. - Connections created via ARM REST (the Foundry portal wizard does not yet expose these authTypes). - RBAC grants target the project's shared agent identity blueprint principal (project.properties.agentIdentity.agentIdentityId) for Path 2 and the project's system-assigned MI (project.identity.principalId) for Path 3. - Troubleshooting table updated with the audience-mismatch symptom and the startup-cache behavior of FoundryToolboxService.
…ed-Toolbox-AuthPaths (microsoft#5777) Updates the sample to use only the new Foundry agent object model and removes the project managed identity path: - Auth-path matrix reduced to four paths: key, Entra agent identity, custom OAuth, inline authorization. Project managed identity is moved into a note describing when it applies (multiple agents sharing access) rather than as a documented sample path. - RBAC instructions reference the agent's own instance_identity.principal_id from the agent ARM resource (new agent object model) instead of the project's shared agent identity blueprint (legacy model). - Step 2 (connections) creates only the AgenticIdentityToken connection. - Step 3 (toolbox tools) lists four tool entries instead of five. - Sample prompts and troubleshooting table updated to match.
…icrosoft#5777) The sample's purpose is to enumerate every authentication path a Foundry toolbox can drive, not to pick one. Path 3 belongs alongside the other four with explicit guidance for when each path is the right choice. - Path 3 (project managed identity, authType=ProjectManagedIdentity) restored to the matrix with a 'When to pick this' column. - Step 2 (connections) provisions both lang-mcp-agent-id and lang-mcp-project-mi via ARM REST. - Step 3 (toolbox) lists five tool entries (one per path). - RBAC instructions cover both the agent's instance identity (Path 2) and the project's system-assigned MI (Path 3). - Sample prompts include all five paths. - Troubleshooting table updated accordingly.
- FoundryToolboxHealthCheck description: rename FOUNDRY_AGENT_TOOLSET_ENDPOINT
→ FOUNDRY_PROJECT_ENDPOINT (stale reference; operator-facing in /readiness body).
- FoundryToolboxStartupStatus.NoEndpoint XML doc: same rename.
- ServiceCollectionExtensions XML docs: same rename + URL shape update.
- Foundry.Hosting.IntegrationTests.TestContainer: remove explicit
app.MapGet('/readiness') — now redundant + would conflict with the
auto-mapped readiness route from MapFoundryResponses.
- Hosted-Toolbox-AuthPaths agent.manifest.yaml: parameterize TOOLBOX_NAME via
{{TOOLBOX_NAME}} template substitution and declare it under parameters with a
default of 'auth-paths-toolbox' so the README's 'use any name' guidance
actually works for hosted deployments.
…icrosoft#5777) - FoundryToolboxService.StartAsync: fall back to AZURE_AI_PROJECT_ENDPOINT when FOUNDRY_PROJECT_ENDPOINT is absent. Matches the local-dev convention used by the samples and resolves the doc/code mismatch flagged in review. - FoundryToolboxHealthCheck description updated for the fallback. - AddFoundryToolboxes: guard against duplicate health-check registration via an explicit name-uniqueness check on HealthCheckServiceOptions.Registrations. AddCheck<T>(name, ...) does not dedupe by name, so repeated AddFoundryToolboxes calls would have registered multiple instances. - FoundryToolboxOptions.EndpointOverride doc: clarify URL becomes {EndpointOverride}/toolboxes/{name}/mcp (was missing /toolboxes/ segment). - Hosted-Toolbox sample (Program.cs + README): switch FOUNDRY_TOOLBOX_NAME to TOOLBOX_NAME (the FOUNDRY_* prefix is reserved by the platform), default changed from 'my-toolset' to 'my-toolbox', terminology updated from 'Toolset' to 'Toolbox'. - FoundryToolboxServiceTests: 2 test renames to reflect what they actually assert (StartupStatus + FailedToolboxNames, not URL shape directly). - Tests adjusted to clear both env vars in NoEndpoint scenarios.
…soft#5777) Update FoundryToolboxStartupStatus.NoEndpoint XML doc to mention both FOUNDRY_PROJECT_ENDPOINT and AZURE_AI_PROJECT_ENDPOINT (the service checks both since the fallback was added). Fix test comment that claimed URL derivation validation when the test only asserts on StartupStatus and FailedToolboxNames.
…th paths The interactive OAuth identity passthrough path needs a protocol gap closed in the hosting package (the proprietary oauth_consent_request item is not representable through the OpenAI/MEAI abstractions), so it is deferred to a separate spike branch. This strips the OAuth path from the AuthPaths sample, the companion REPL client, the agent manifest, and the docs, then renumbers the inline Authorization path so the sample teaches four contiguous paths: API key via connection, Entra agent identity, Entra project managed identity, and inline Authorization (anti-pattern). Package code is unchanged; the consent infrastructure already present in main stays as baseline. Both samples build with --warnaserror and all 246 hosting unit tests pass.
…box-AuthPaths (microsoft#5777) Live validation against tao-foundry-prj showed the ProjectManagedIdentity path failing with an unresolved token audience 401, so the sample now ships three working auth paths instead of four: connection key, agent managed identity, and inline Authorization. Changes: - Remove the project managed identity path from the AuthPaths sample matrix, prerequisites, connections, toolbox table, prompts, Program.cs instructions and agent.manifest.yaml. - Delete the near duplicate Hosted-Toolbox-AuthPaths-Client project and remove it from the solution. The README now drives the agent with the shared SimpleAgent REPL via AsAIAgent(agentEndpoint). - Correct the troubleshooting note: the Foundry toolbox tools/list is all or nothing, so one bad source returns -32007, fails startup, and returns 424 for every path. Add the allowed_tools caveat that names must match the upstream server. - Mark the toolbox startup status and health check experimental under AgentsAIExperiments (MAAI001) instead of AIOpenAIResponses, and update the package NoWarn set accordingly.
rogerbarreto
force-pushed
the
issues/5777-net-hosted-agent-sample-toolbox-with-various-auth
branch
from
e243bf7 to
25457cb
Compare
…with-various-auth
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #5777.
Summary
Adds the
Hosted-Toolbox-AuthPathssample (three MCP-tool auth paths driven by a single Foundry Toolbox) and fixes the underlyingMicrosoft.Agents.AI.Foundry.Hostingtoolbox path so it works against the live Foundry runtime.Package changes —
Microsoft.Agents.AI.Foundry.HostingAligned with
tools-integration-spec.md§2–§4, §6.3, §9:FoundryToolboxServicebuilds the toolbox proxy URL from the platform-injectedFOUNDRY_PROJECT_ENDPOINTas{ep}/toolboxes/{name}/mcp?api-version={ApiVersion}. The legacyFOUNDRY_AGENT_TOOLSET_ENDPOINTenv var is removed (preview package, no production consumers).FoundryToolboxBearerTokenHandleralways sendsFoundry-Features: Toolboxes=V1Preview, merging any flags fromFOUNDRY_AGENT_TOOLSET_FEATURES.https://ai.azure.com/.default(washttps://cognitiveservices.azure.com/.default).traceparent/tracestate/baggagepropagation fromActivity.Current.MapFoundryResponsesauto-mapsGET /readiness;AddFoundryToolboxesregistersFoundryToolboxHealthCheckthat gates readiness on toolbox startup outcome. The toolbox startup status and health check are marked experimental underAgentsAIExperiments(MAAI001).New unit tests plus an xUnit serialization fixture for env-var-mutating tests. All 246 tests in
Microsoft.Agents.AI.Foundry.Hosting.UnitTestspass;dotnet format --verify-no-changesclean (CI-parity, .NET SDK 10 Docker image).Sample —
Hosted-Toolbox-AuthPathsServer-side toolbox demo where a single Foundry Toolbox bundles three MCP tools, each authenticated differently:
authTypeCustomKeysAgenticIdentityTokenauthorization(anti-pattern)The agent code is auth-agnostic by design: Foundry resolves every credential server-side when it proxies the MCP call, so the sample carries no auth logic. The educational surface lives in the toolbox configuration (provisioned in the portal) and in the README.
Files:
dotnet/samples/04-hosting/FoundryHostedAgents/responses/Hosted-Toolbox-AuthPaths/— hosted server (Program.cs, README,.env.example,agent.manifest.yaml,agent.yaml, Dockerfiles).Using-Samples/SimpleAgent/REPL viaAsAIAgent(agentEndpoint), the only supported way to consume a hosted Foundry agent.Hosted-Toolbox/README.mdandHosted-McpTools/README.md.README highlights
AgenticIdentityTokenauthType).audienceproperty requirement;https://cognitiveservices.azure.comfor Cognitive Services targets.instance_identity.principal_idgetsCognitive Services Userfor Path 2.allowed_toolscaveat: every name must match a real tool on the upstream server, otherwise that source fails enumeration.tools/listbehavior, audience mismatch, principal RBAC, and startup-cache behavior.Validation against live runtime
Deployed against
tao-foundry-prj(eastus2):github_pat___search_issues,response.completed).azure-rest-api-specsdocs returned via inline authorization (gitmcp.io).SSE streams across the validated invokes: zero HTTP 424, zero error events, zero 401/403.
Key runtime finding
The Foundry toolbox
tools/listis all-or-nothing: if any single tool source fails enumeration, the proxy returns a top-level JSON-RPC-32007error instead of a partial list. That fails toolbox startup,/readinessreturns 503, and every invoke returns HTTP 424 even for correctly configured paths. The readiness gating in this PR surfaces that failure cleanly, and the README documents it as a known limitation.Removed during review
ProjectManagedIdentity) was dropped: live validation hit an unresolved token-audience 401 from the Foundry-minted project MI token.OAuth2) and its dedicated REPL consent client were dropped: the sample focuses on server-side credential resolution, not a client-side consent loop.Out of scope
/toolsets/and thecognitiveservices.azure.comscope).