Add telemetry for databricks aitools install#5862
Merged
Merged
Conversation
parthban-db
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
parthban-db
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
Collaborator
Integration test reportCommit: 733659b
8 interesting tests: 4 RECOVERED, 4 SKIP
Top 10 slowest tests (at least 2 minutes):
|
parthban-db
force-pushed
the
parthban-db/stack/telemetry
branch
from
cd71e38 to
f2bf35b
Compare
parthban-db
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
parthban-db
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
parthban-db
force-pushed
the
parthban-db/stack/telemetry
branch
from
f2bf35b to
b477ac6
Compare
parthban-db
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
parthban-db
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
parthban-db
changed the title
databricks aitools install
parthban-db
commented
Jul 8, 2026
| @@ -13,7 +14,7 @@ func TestLegacySkillsInstallDelegatesToInstall(t *testing.T) { | |||
| setupTestAgents(t) | |||
| calls := setupInstallMock(t) | |||
|
|
|||
| ctx := cmdio.MockDiscard(t.Context()) | |||
| ctx := telemetry.WithNewLogger(cmdio.MockDiscard(t.Context())) | |||
Contributor
Author
There was a problem hiding this comment.
We need the logger for the telemetry because otherwise, it will panic. A logger is always present in the production path through the root command.
parthban-db
force-pushed
the
parthban-db/stack/telemetry
branch
from
b477ac6 to
9b38779
Compare
parthban-db
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
parthban-db
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
parthban-db
force-pushed
the
parthban-db/stack/telemetry
branch
from
9b38779 to
733659b
Compare
parthban-db
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
parthban-db
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
parthban-db
requested review from
mihaimitrea-db and
renaudhartert-db
and removed request for
renaudhartert-db
mihaimitrea-db
approved these changes
Jul 13, 2026
mihaimitrea-db
left a comment
mihaimitrea-db
left a comment
Contributor
There was a problem hiding this comment.
Looks good, except for comment.
Comment on lines
106
to
109
| if pathFlag != "" { | ||
| _, err := installer.DumpSkillsToPath(ctx, src, pathFlag, opts) | ||
| return err | ||
| } |
Contributor
There was a problem hiding this comment.
Do we want to log the case where people "install" the skills by just dumping them at a specific path as well? If yes, then this will be missed by the current placement of defer logInstallEvent:
// --path is a dumb dump: no agents, no scope, no state.
if pathFlag != "" {
defer logInstallEvent(ctx, plan, installOpts{
Scope: opts.Scope,
Experimental: opts.IncludeExperimental,
})
_, err := installer.DumpSkillsToPath(ctx, src, pathFlag, opts)
return err
}Maybe add a call to defer in the if block before DumpSkillsToPath or find a place for the current call to cover both while not logging an install on failure.
Contributor
Author
There was a problem hiding this comment.
Yeah, it's weird to have this. I will address this as a follow-up.
Collaborator
Integration test reportCommit: a676b0d
17 interesting tests: 9 flaky, 3 KNOWN, 2 FAIL, 2 SKIP, 1 RECOVERED
Top 50 slowest tests (at least 2 minutes):
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🥞 Stacked PR
Use this link to review incremental changes.
Changes
Emit a structured
AitoolsInstallEventtelemetry event on every run ofdatabricks aitools install(and its deprecatedexperimental aitools skills installalias, which delegates to the same command). The event records three closed-set, non-PII dimensions: the agents targeted (agents, from the fixed agent registry), the install scope (scope, project or global), and whether experimental skills were requested (experimental).aitools installis otherwise a fully local command (skills are fetched from GitHub), so it never resolves auth on its own.tryConfigureAuthresolves auth config best-effort into the context sotelemetry.Uploadat command exit has somewhere to send. A user with no configured profile still installs offline and simply emits nothing.Why
aitools installhad no usage signal. This lets us measure adoption of Databricks skills across coding agents — how many installs, for which agents, at which scope, and how often the experimental skills are pulled in.Tests
Unit tests.