Repository navigation
[release/17.0] aspire agent init telemetry fixes - #22231
Open
Ella Hathaway (ellahathaway) wants to merge 3 commits into
Open
Ella Hathaway (ellahathaway) wants to merge 3 commits into
Ella Hathaway (ellahathaway) wants to merge 3 commits into
Conversation
## Description Fixed #20014 `aspire agent init` now registers telemetry hooks per selected agent client instead of per detected environment. The install message names each client by its display name, so the Copilot App and Copilot CLI no longer appear as separate entries. ```text Installed Aspire agent telemetry hooks for: GitHub Copilot, Claude Code. ``` Hook locations: - **GitHub Copilot**: `<COPILOT_HOME>\hooks\aspire-telemetry.json`. One registration covers the Copilot App, the Copilot CLI, and VS Code. - **Claude Code**: a `PostToolUse` hook in `<CLAUDE_CONFIG_DIR>\settings.json`. - **OpenCode**: no hooks, because it has no verified hook schema. When hooks install: - Only for global scope, which is the existing rule because hooks write to user-level locations. Project scope is handled later in the stack. - Only for selected clients that have the `aspire@aspire-skills` plugin or the Aspire MCP server configured, either from a previous run or from this one. Existing Aspire setup refreshes the hook even when both the skills and MCP choices are skipped. The hooks only report on that plugin's skills and Aspire MCP tools, so third-party skills alone produce no events, and `ASPIRE_CLI_TELEMETRY_OPTOUT` still disables reporting. Re-running `aspire agent init` refreshes Aspire's hook and preserves unrelated user hooks and settings. Validation: `dotnet test --project tests\Aspire.Cli.Tests\Aspire.Cli.Tests.csproj --no-launch-profile -- --filter-class "*AgentInit*" --filter-class "*NewCommand*" --filter-class "*InitCommand*" --filter-class "*McpInit*" --filter-class "*TelemetryHook*" --filter-class "*AgentTelemetry*" --filter-not-trait "quarantined=true" --filter-not-trait "outerloop=true"` passed (1257 run; 3 flaky failures passed on rerun). Fixes #20014 ## Checklist - Is this feature complete? - [ ] Yes. Ready to ship. - [x] No. Follow-up changes expected. - Are you including unit tests for the changes and scenario tests if relevant? - [x] Yes - [ ] No - Did you add public API? - [ ] Yes - If yes, did you have an API Review for it? - [ ] Yes - [ ] No - Did you add `<remarks />` and `<code />` elements on your triple slash comments? - [ ] Yes - [ ] No - [x] No - Does the change make any security assumptions or guarantees? - [ ] Yes - If yes, have you done a threat model and had a security review? - [ ] Yes - [ ] No - [x] No --------- Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
## Description Telemetry hooks now use tool-name matchers, so agent clients start the aspire CLI only for tools the classifier tracks. - **Copilot**: `CopilotAgentClient.TelemetryHookMatcher` adds a `matcher` for `skill` and the Aspire MCP tools. Copilot matchers see only the tool name, so there is no `Read` matcher; one would start the CLI for every file read in every repository. - **Claude Code**: `ClaudeCodeAgentClient` registers three `PostToolUse` groups: `Skill` (every skill, since Claude hook `if` rules don't match Skill input yet, anthropics/claude-code#95819, so the CLI classifier drops non-Aspire skills), the Aspire MCP tools, and `Read` limited to Aspire skill directories by an `if` rule. - Each matcher lives with its agent client. `AgentTelemetryHook.McpMatcher` is built from the classifier's MCP tool prefixes, so the matchers cannot drift from the classifier. - The spec documents the matchers. Clients that ignore matchers still work because the classifier stays the authoritative filter. **IMPORTANT: Copilot limitation** This doesn't fully solve the invocation problem for Copilot. Copilot hook matchers only see the tool name, and Copilot has no equivalent of Claude Code's `if` rules for filtering on tool input. So for Copilot: - Every `skill` call starts the aspire CLI, including non-Aspire skills. The classifier drops those events, but the process still launches. - File reads can't be scoped to Aspire skill directories, so the Copilot hook doesn't match `view` at all. Clients that honor the matcher don't report `reference_file_read` events. - Copilot CLI versions earlier than 1.0.63 ignore `matcher` and still start the CLI for every tool. Fully scoping Copilot needs input-based filtering in Copilot hooks. Validation: `dotnet test --project tests\Aspire.Cli.Tests\Aspire.Cli.Tests.csproj --no-launch-profile -- --filter-class "*.AgentInit*" --filter-class "*TelemetryHook*" --filter-class "*AgentTelemetry*" --filter-not-trait "quarantined=true" --filter-not-trait "outerloop=true"` passed (449/449). The `Aspire.Cli.EndToEnd.Tests` build succeeds, but the E2E tests were not run. Fixes #20480 ## Checklist - Is this feature complete? - [ ] Yes. Ready to ship. - [x] No. Follow-up changes expected. - Are you including unit tests for the changes and scenario tests if relevant? - [x] Yes - [ ] No - Did you add public API? - [ ] Yes - If yes, did you have an API Review for it? - [ ] Yes - [ ] No - Did you add `<remarks />` and `<code />` elements on your triple slash comments? - [ ] Yes - [ ] No - [x] No - Does the change make any security assumptions or guarantees? - [ ] Yes - If yes, have you done a threat model and had a security review? - [ ] Yes - [ ] No - [x] No --------- Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
## Description Stack 3/3, on #22084. `aspire agent init --scope project` now installs telemetry hooks too. The hooks are written at user scope (`~/.copilot/hooks` and `~/.claude/settings.json`) no matter which scope was requested. Skills and MCP configuration still follow the requested scope, and #22078's per-client gating still decides which clients get hooks. Because the hooks live in user-level configuration, they apply in every repository where that client runs, not only the one where `aspire agent init` was run. The PostToolUse matcher from #22084 makes the client launch `aspire` only for matching tool calls, and only Aspire skill, MCP tool, and reference-file usage is reported. Users can opt out with `ASPIRE_CLI_TELEMETRY_OPTOUT=true`. The install message now says the hooks were written to user-level configuration. ### Why we can't install project-level hooks today Every hook launches a separate `aspire` process, so it must fire exactly once per event. Today a project-level hook would make it fire more than once (see #20480): - **Copilot runs hooks from every source and doesn't dedupe them.** It loads the repo's `.github/hooks/*.json`, `~/.copilot/hooks`, `.github/copilot/settings(.local).json`, `~/.copilot/settings.json`, and plugins, then runs all of them. A project hook in `.github/hooks` plus the user hook that global init installs would launch `aspire` twice for each event. Claude Code dedupes identical handlers across its settings files, so this is a Copilot problem. - **Copilot also runs Claude's project hooks.** Copilot reads the repo's `.claude/settings(.local).json` and runs those hooks too, ignoring Claude-only fields such as `if`. A Claude project hook would fire in Copilot sessions on top of Copilot's own hook, even if the user set up only Claude for the project. - **Install time can't avoid the duplicates.** Project hook files are committed and shared, but user hooks belong to each machine. We can't delete a user hook to make room for a project hook, because it also serves the user's other repos. Skipping the project hook when a user hook exists only helps the developer who ran init, not teammates who clone the repo with their own user hooks. - **Runtime dedupe is heavy.** Each invocation is an independent process with no shared event ID. Deduping would mean hook processes coordinating through shared state on every tool call, which is the hot path. Installing hooks only at user scope gives exactly one hook per client on each machine. We can revisit project hooks if Copilot starts deduping identical hook commands or applying precedence across hook sources. Validation: `dotnet test --project tests\Aspire.Cli.Tests\Aspire.Cli.Tests.csproj --no-launch-profile -- --filter-class "*.AgentInit*" --filter-class "*.TelemetryHook*" --filter-class "*.AgentTelemetry*" --filter-not-trait "quarantined=true" --filter-not-trait "outerloop=true"` passed (464/464). The E2E tests were not run. ## Checklist - Is this feature complete? - [x] Yes. Ready to ship. - [ ] No. Follow-up changes expected. - Are you including unit tests for the changes and scenario tests if relevant? - [x] Yes - [ ] No - Did you add public API? - [ ] Yes - If yes, did you have an API Review for it? - [ ] Yes - [ ] No - Did you add `<remarks />` and `<code />` elements on your triple slash comments? - [ ] Yes - [ ] No - [x] No - Does the change make any security assumptions or guarantees? - [ ] Yes - If yes, have you done a threat model and had a security review? - [ ] Yes - [ ] No - [x] No --------- Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: b39716fb-e8ff-45f3-a048-a710d4f3052e
Ella Hathaway (ellahathaway)
requested review from
James Newton-King (JamesNK),
David Fowler (davidfowl) and
Mitch Denny (mitchdenny)
as code owners
November 13, 2026 00:09
Contributor
|
🚀 Dogfood this PR with:
curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 22231Or
iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 22231" |
Contributor
Tests selector3 / 101 PR test projects · 4 PR jobs, from 36 changed files. Selected PR test projects (3 / 101)
Selected PR jobs (4)
How these were chosen — grouped by what changed📦 affected project 🔧 🧪 🧪 🧪 🧪 🧪 🧪 🧪 🧪 🧪 🧪 🧪 Job reasons
Selection computed for commit |
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Hook installation or refresh is skipped for existing Aspire configurations in .vscode/mcp.json or .github/copilot/mcp.json under the specified project-scope options, leaving outdated all-tool hooks unchanged.
1 open issue
1 file not reviewed
src/Aspire.Cli/Resources/AgentCommandStrings.Designer.cs: Generated file
What changed in this PR
Backports #22078, #22084, and #22093 to release/17.0, updating how the Aspire CLI installs and filters agent telemetry hooks.
Changes:
- Register hooks for selected clients with existing or newly configured Aspire skills or MCP setup.
- Add tool matchers and install hooks at user scope even during project-scoped initialization.
- Update regression coverage, installation messages, localization sources, and telemetry documentation.
| File | Description |
|---|---|
tests/Aspire.Cli.Tests/TestServices/TestTelemetryHookConfigurator.cs |
Track selected clients and configurable hook results. |
tests/Aspire.Cli.Tests/Snapshots/AgentInitClientSelectionTests.AgentInit_InstallationLocations_ShowSelectedAssetsAndSharedConsumers_scope=project_installSkills=True_configureMcp=True.verified.txt |
Expect user-level hook notice for project setup. |
tests/Aspire.Cli.Tests/Snapshots/AgentInitClientSelectionTests.AgentInit_InstallationLocations_ShowSelectedAssetsAndSharedConsumers_scope=project_installSkills=False_configureMcp=True.verified.txt |
Expect hook notice for project MCP setup. |
tests/Aspire.Cli.Tests/Snapshots/AgentInitClientSelectionTests.AgentInit_InstallationLocations_ShowSelectedAssetsAndSharedConsumers_scope=global_installSkills=True_configureMcp=True.verified.txt |
Expect hook notice for global setup. |
tests/Aspire.Cli.Tests/Snapshots/AgentInitClientSelectionTests.AgentInit_InstallationLocations_ShowSelectedAssetsAndSharedConsumers_scope=global_installSkills=False_configureMcp=True.verified.txt |
Expect hook notice for global MCP setup. |
tests/Aspire.Cli.Tests/Commands/NewCommandTests.cs |
Verify hook client selection during project creation. |
tests/Aspire.Cli.Tests/Commands/AgentInitCommandTests.cs |
Cover per-client hook installation and failure messages. |
tests/Aspire.Cli.Tests/Commands/AgentInitClientSelectionTests.cs |
Cover scope, eligibility, and existing-setup detection. |
tests/Aspire.Cli.Tests/Agents/TelemetryHookConfiguratorTests.cs |
Verify scoped hooks, migration, and configuration preservation. |
tests/Aspire.Cli.Tests/Agents/AgentTelemetryHookTests.cs |
Test tool matchers and skill-path filtering. |
tests/Aspire.Cli.EndToEnd.Tests/AgentCommandTests.cs |
Check user-level hooks after project initialization. |
src/Aspire.Cli/Resources/xlf/AgentCommandStrings.zh-Hant.xlf |
Update source text; mark translation for review. |
src/Aspire.Cli/Resources/xlf/AgentCommandStrings.zh-Hans.xlf |
Update source text; mark translation for review. |
src/Aspire.Cli/Resources/xlf/AgentCommandStrings.tr.xlf |
Update source text; mark translation for review. |
src/Aspire.Cli/Resources/xlf/AgentCommandStrings.ru.xlf |
Update source text; mark translation for review. |
src/Aspire.Cli/Resources/xlf/AgentCommandStrings.pt-BR.xlf |
Update source text; mark translation for review. |
src/Aspire.Cli/Resources/xlf/AgentCommandStrings.pl.xlf |
Update source text; mark translation for review. |
src/Aspire.Cli/Resources/xlf/AgentCommandStrings.ko.xlf |
Update source text; mark translation for review. |
src/Aspire.Cli/Resources/xlf/AgentCommandStrings.ja.xlf |
Update source text; mark translation for review. |
src/Aspire.Cli/Resources/xlf/AgentCommandStrings.it.xlf |
Update source text; mark translation for review. |
src/Aspire.Cli/Resources/xlf/AgentCommandStrings.fr.xlf |
Update source text; mark translation for review. |
src/Aspire.Cli/Resources/xlf/AgentCommandStrings.es.xlf |
Update source text; mark translation for review. |
src/Aspire.Cli/Resources/xlf/AgentCommandStrings.de.xlf |
Update source text; mark translation for review. |
src/Aspire.Cli/Resources/xlf/AgentCommandStrings.cs.xlf |
Update source text; mark translation for review. |
src/Aspire.Cli/Resources/AgentCommandStrings.resx |
Clarify that hooks use user-level configuration. |
src/Aspire.Cli/Resources/AgentCommandStrings.Designer.cs |
Synchronize the installation-message documentation. |
src/Aspire.Cli/Commands/AgentInitCommand.cs |
Gate hook installation on selected clients’ Aspire setup. |
src/Aspire.Cli/Agents/OpenCode/OpenCodeAgentClient.cs |
Explicitly declare telemetry hooks unsupported. |
src/Aspire.Cli/Agents/McpConfigFileHelper.cs |
Reject duplicate properties and replace non-object server entries. |
src/Aspire.Cli/Agents/IAgentClient.cs |
Define per-client hook support and forwarded MCP paths. |
src/Aspire.Cli/Agents/Hooks/TelemetryHookConfigurator.cs |
Configure eligible clients and return individual results. |
src/Aspire.Cli/Agents/Hooks/ITelemetryHookConfigurator.cs |
Replace environment-based results with client statuses. |
src/Aspire.Cli/Agents/Hooks/AgentTelemetryHook.cs |
Share MCP matcher prefixes and extract client attribution logic. |
src/Aspire.Cli/Agents/Copilot/CopilotAgentClient.cs |
Add tool filtering and VS Code user-profile MCP discovery. |
src/Aspire.Cli/Agents/ClaudeCode/ClaudeCodeAgentClient.cs |
Register separate skill, MCP, and filtered-read hook groups. |
src/Aspire.Cli/Agents/AspireSkills/AspireSkillsRemoteConfiguration.cs |
Treat null plugin settings as unregistered. |
docs/specs/aspire-product-telemetry.md |
Document eligibility, user scope, and matcher limitations. |
Review effort: Balanced
4 of 14 tasks
Contributor
|
Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Backports of the PRs from https://github.com/microsoft/aspire/stack/22118:
Checklist
<remarks />and<code />elements on your triple slash comments?