diff --git a/.grok/rules/zavet.md b/.grok/rules/zavet.md index f94d6e1..cd27aee 100644 --- a/.grok/rules/zavet.md +++ b/.grok/rules/zavet.md @@ -49,6 +49,12 @@ agent context at session start. Keep it short and non-negotiable. - Nothing enters `repo_dirs` unless the directory demonstrably belongs to the repo it is filed under, and `register_repo_dir` stays I/O-free. See DIRASH-0027. +- Telemetry ships only the closed `TelemetryEvent` enum — never argv, paths, + repo names, git identity, or error text. Plaintext repo refs are hashed + daemon-side with the per-install salt; the analytics id is never derived + from the device key; the flush gate is consent, never `cloud_link`. + Changing what ships changes `TELEMETRY_DISCLOSURE` and `docs/TELEMETRY.md` + in the same commit. See DIRASH-0033. ### Recorded decisions (read the file before changing guarded code; ask /zavet:why) @@ -84,6 +90,8 @@ agent context at session start. Keep it short and non-negotiable. - DIRASH-0030 — Full-content knowledge sync is opted into by its own prompt, never implied by linking (active) - DIRASH-0031 — One backoff ladder lives in dira_core; callers own their attempt budget (active) - DIRASH-0032 — A record's first-sight triple is repaired as a unit, from recorded facts (active) +- DIRASH-0033 — Telemetry is opt-out, anonymous by construction, and rides its own unsigned channel (active) +- DIRASH-0034 — Repo-visibility probing sends the plaintext ref only to the forge's own public API, anonymously (active) ### Living specs (.zavet/specs/ — keep current while you work) @@ -95,6 +103,7 @@ agent context at session start. Keep it short and non-negotiable. - harness-sources — Harness sources and hook ingestion (session, high) - knowledge-sync — Knowledge sync — the consent-gated second channel (session, medium) - onboarding — Onboarding — dira onboard and the installer handoff (session, high) +- telemetry — Telemetry — anonymous product analytics (session, high) Capture bar: record non-obvious choices a future reader could not reconstruct — micro-decisions as commit trailers (Why:/Rejected:/Constraint:/Refs:), structural ones via /zavet:decide. Spec maintenance (do this as part of normal work, no command needed): when implementing or changing a feature, update its covering spec in .zavet/specs/ — or create one from .zavet/.spec-template.md (origin: session) for substantial new features — reference the decisions involved, and add a `Spec: ` trailer to the commit. diff --git a/.zavet/INDEX.md b/.zavet/INDEX.md index ec1eef0..9a32238 100644 --- a/.zavet/INDEX.md +++ b/.zavet/INDEX.md @@ -39,6 +39,8 @@ a handful of documents. The decisions block below is regenerated by - **DIRASH-0030** — Full-content knowledge sync is opted into by its own prompt, never implied by linking (active) - **DIRASH-0031** — One backoff ladder lives in dira_core; callers own their attempt budget (active) - **DIRASH-0032** — A record's first-sight triple is repaired as a unit, from recorded facts (active) +- **DIRASH-0033** — Telemetry is opt-out, anonymous by construction, and rides its own unsigned channel (active) +- **DIRASH-0034** — Repo-visibility probing sends the plaintext ref only to the forge's own public API, anonymously (active) ## Specs @@ -55,6 +57,7 @@ the block by hand. - **harness-sources** — Harness sources and hook ingestion (session, high, 2026-08-09) - **knowledge-sync** — Knowledge sync — the consent-gated second channel (session, medium, 2026-08-11) - **onboarding** — Onboarding — dira onboard and the installer handoff (session, high, 2026-08-13) +- **telemetry** — Telemetry — anonymous product analytics (session, high, 2026-08-25) ## See also diff --git a/.zavet/RULES.md b/.zavet/RULES.md index 6760185..529256b 100644 --- a/.zavet/RULES.md +++ b/.zavet/RULES.md @@ -45,3 +45,9 @@ agent context at session start. Keep it short and non-negotiable. - Nothing enters `repo_dirs` unless the directory demonstrably belongs to the repo it is filed under, and `register_repo_dir` stays I/O-free. See DIRASH-0027. +- Telemetry ships only the closed `TelemetryEvent` enum — never argv, paths, + repo names, git identity, or error text. Plaintext repo refs are hashed + daemon-side with the per-install salt; the analytics id is never derived + from the device key; the flush gate is consent, never `cloud_link`. + Changing what ships changes `TELEMETRY_DISCLOSURE` and `docs/TELEMETRY.md` + in the same commit. See DIRASH-0033. diff --git a/.zavet/decisions/DIRASH-0033-telemetry-is-opt-out-anonymous-and-rides-its-own-unsigned-channel.md b/.zavet/decisions/DIRASH-0033-telemetry-is-opt-out-anonymous-and-rides-its-own-unsigned-channel.md new file mode 100644 index 0000000..bff9340 --- /dev/null +++ b/.zavet/decisions/DIRASH-0033-telemetry-is-opt-out-anonymous-and-rides-its-own-unsigned-channel.md @@ -0,0 +1,73 @@ +--- +id: DIRASH-0033 +title: Telemetry is opt-out, anonymous by construction, and rides its own unsigned channel +status: active +guards: + - cli/core/src/telemetry/ + - cli/dira/src/telemetry.rs + - cli/dirad/src/telemetry_sync.rs +checks: + - every wire variant carries exactly its declared keys :: cargo test -p dira-core --lib telemetry::event + - the disclosure names what ships :: cargo test -p dira --bin dira the_telemetry +origin: session +verified: false +--- + +## Decision + +Product analytics is **on by default** (opt-out), disclosed by its own prompt +in onboarding and by a one-time first-run notice, and disabled by any of: +`telemetry.enabled = false`, `DIRA_TELEMETRY_ENABLED=0`, `DO_NOT_TRACK=1`, +`CI`, or a dev build. What ships is the closed `TelemetryEvent` enum and +nothing else: command name (top-level only), duration, success plus a closed +error-kind taxonomy, host class, visibility, and a salted repo hash. Never +argv, paths, repo names, git identity, or error text. + +Identity is a **random install ULID plus a random 32-byte salt**, minted +lazily in `meta`, never derived from the Ed25519 device key. The repo hash is +HMAC-SHA256 keyed by that per-install salt, computed **daemon-side**: the +canonical `host/owner/repo` ref crosses only the local control socket, and +plaintext repo identity is never persisted in the queue nor sent to the +network. Batches are **unsigned** and flush to the cloud's `/api/v1/pulse` +gated on `cloud_url` + consent — explicitly never on device linkage. + +## Why + +**Opt-out with honest disclosure** is the only consent model that yields data +representative enough to steer a pricing strategy, and it stays honest the +same way DIRASH-0030 does: a named disclosure constant shown on every path, +pinned by a wording test to the fields that actually ship, so the promise and +the payload cannot drift apart silently. + +**The identity split is the load-bearing part.** Reusing the device key (or +anything derived from it) as an analytics id would let the analytics store +correlate back to the signing identity, and would break the moment a key +rotates. A per-install salt keyed into the repo hash means the same repo +hashes differently on every install: we can count distinct repos per install +and split public from private, but no cross-install correlation of repos is +possible even with our own database in hand — which is what lets the word +"anonymous" in the disclosure be true rather than aspirational. + +**Unsigned, and consent-gated rather than link-gated**, because the entire +point is hearing from installs that never linked. Requiring the envelope +would silence exactly the population whose conversion we want to understand, +and telemetry is not trust-critical: the cloud treats it as untrusted input +behind a server-side allowlist regardless of what we sign. + +**One final `consent_recorded(enabled=false)`** is allowed through when the +knob is turned off — the opt-out rate is itself the signal that keeps this +feature honest — but every other kill switch (env, DO_NOT_TRACK, CI, dev +build) suppresses even that. + +## Rejected + +- Authoring the wire types in `/contract` — telemetry is best-effort and + versioned independently (`v: 1`); riding the drift-gated contract would + couple every taxonomy tweak to a contract release and the cloud vendoring + dance. +- A global (unsalted) repo hash — would let public repos be dictionary- + reversed and private repos be correlated across installs; "pseudonymous" + is not what the disclosure says. +- Emitting from the CLI process directly — D-0006's rule generalizes: no + network on the foreground path. The CLI's only telemetry I/O is a + 150ms-budgeted local-socket fire-and-forget. diff --git a/.zavet/decisions/DIRASH-0034-repo-visibility-probes-the-forges-own-public-api-anonymously.md b/.zavet/decisions/DIRASH-0034-repo-visibility-probes-the-forges-own-public-api-anonymously.md new file mode 100644 index 0000000..1851bdc --- /dev/null +++ b/.zavet/decisions/DIRASH-0034-repo-visibility-probes-the-forges-own-public-api-anonymously.md @@ -0,0 +1,60 @@ +--- +id: DIRASH-0034 +title: Repo-visibility probing sends the plaintext ref only to the forge's own public API, anonymously +status: active +guards: + - cli/dirad/src/repo_visibility.rs +checks: + - visibility mapping, caching, and probe bounding hold :: cargo test -p dirad --lib repo_visibility +origin: recorded +verified: true +--- + +## Decision + +WP3 resolves a GitHub/GitLab remote's Public/Private visibility with a cache-first, +anonymous `GET` to that forge's own public API (`api.github.com` / +`gitlab.com/api/v4`), carrying the plaintext `owner/repo` path. Bitbucket and +self-hosted remotes get `Unknown` with no request. This is the one place beyond +the local control socket (DIRASH-0033) the plaintext canonical ref travels to. + +## Why + +DIRASH-0033 guards "plaintext repo identity ... never sent to the network" against +Dira's own cloud ingest — that boundary is unchanged: `/api/v1/pulse` still only +ever receives `repo_hash`, `host_class`, and the resolved visibility string, never +the ref. The forge probe is a different network and a different question: the +forge already hosts the repo and already knows it exists, so asking it "is this +public?" discloses nothing to it that it doesn't already have. The request carries +no auth header, no cookie, and no install/device identifier — only a generic +`dirad/` UA required by GitHub's API — so the forge cannot correlate the +request back to this install even if it wanted to. Visibility is a materially +useful segmentation signal for the pricing-strategy goal DIRASH-0033 already cites. + +## Rejected + +- Always reporting `Unknown` (never probing) — keeps the DIRASH-0033 boundary + literally untouched, but throws away a real segmentation signal for an exposure + that is, at most, "the forge learns someone anonymous asked about a repo it + already hosts" — not a meaningful privacy cost. +- Routing the probe through Dira's cloud (cloud resolves visibility server-side) — + rejected as WP3 scope creep; would need the cloud to hold forge credentials and + widen the trusted-cloud surface for a lookup the daemon can do statelessly. + +## Agent directives + +- Never add an `Authorization`/cookie header, or any install/device identifier, to + a request built in `repo_visibility.rs`. +- Never persist the plaintext canonical ref from this module — only cache by the + salted `repo_hash` (`VisibilityCache`), matching DIRASH-0033's key discipline. +- Bitbucket/self-hosted must stay probe-free (`Unknown`, no request) unless a + later decision adds a probe for them explicitly. + +## Verification + +`cargo test -p dirad --lib repo_visibility` covers the status→visibility mapping, +TTL choice (short for rate-limit/error, long for a confident or never-probed +answer), cache eviction/expiry, in-flight probe bounding, and the +`ingest`-integration "unknown first, real answer once warm" behavior. Whether the +request itself carries no auth/cookie headers is not separately asserted by a +test — a human reviewing `repo_visibility.rs`'s `request_visibility` is the check. diff --git a/.zavet/specs/telemetry.md b/.zavet/specs/telemetry.md new file mode 100644 index 0000000..c26f32d --- /dev/null +++ b/.zavet/specs/telemetry.md @@ -0,0 +1,80 @@ +--- +title: Telemetry — anonymous product analytics +version: 1 +origin: session +verified: false +confidence: high +date: 2026-08-25 +paths: + - cli/core/src/telemetry/ + - cli/dira/src/telemetry.rs + - cli/dirad/src/telemetry_sync.rs + - cli/dirad/src/repo_visibility.rs +decisions: [DIRASH-0033, DIRASH-0034, DIRASH-0030, DIRASH-0031, D-0006, D-0011, D-0020] +--- + +## Overview + +Anonymous, opt-out product analytics: which commands run, how long they take, +and coarse facts about the repos they run in, flushed through the daemon to +the Dira cloud's `/api/v1/pulse` proxy and forwarded server-side to PostHog +Cloud EU. The user-facing disclosure is `docs/TELEMETRY.md`; the structural +rules are DIRASH-0033. + +## Pipeline + +1. **Emit (CLI, `cli/dira/src/telemetry.rs`).** The thin `main()` times the + dispatched command and calls `record_command`, which passes + `TelemetryGate` (knob, `DIRA_TELEMETRY_ENABLED`, `DO_NOT_TRACK`, `CI`, + dev build — any one suppresses), classifies failure into the closed + `ErrorKind`, resolves the cwd's canonical repo ref via `explain_project`, + and fire-and-forgets `Request::IngestTelemetry` over the control socket + under a 150ms total budget. The CLI process never does network I/O for + telemetry (D-0006 generalized). Some paths exit the process directly and + record nothing — listed on `run()`'s doc comment. +2. **Ingest (daemon, `telemetry_sync::ingest`).** Re-checks consent, mints or + loads the install id + salt (`meta`), hashes the canonical ref + (HMAC-SHA256, per-install salt) with visibility from the probe cache + (`Unknown` on a cold cache; the probe fills it for later events), and + appends the finished wire JSON to the `telemetry_events` queue. Plaintext + repo identity is never stored. +3. **Flush (daemon, `telemetry_sync::run`).** knowledge_sync-shaped loop: + 5s debounce, jittered 300s backstop, chunks of 200 over `(cursor, until]`, + POST `{cloud_url}/api/v1/pulse` on the shared TLS-pinned client (D-0011), + cursor advances per accepted chunk on its own 2xx (D-0020), backoff via + the shared ladder (DIRASH-0031). Gate is `cloud_url` + consent — never + device linkage. 400 advances past the poison chunk loudly (rows kept); + 404 is a quiet endpoint-missing skip; 413/429/5xx/network are transient. + Health lands in `META_TELEMETRY_HEALTH`. +4. **Visibility probe (`repo_visibility.rs`).** Unauthenticated GET to the + provider that already hosts the repo (github.com / gitlab.com only), + 200→public, 404→private, else unknown; cached 24h keyed by the salted + hash, short-TTL on rate-limit/error; never blocks ingestion; no tokens. + +## Consent surfaces + +- Onboarding step (DIRASH-0030 shape): `TELEMETRY_DISCLOSURE` shown on every + path, confirm defaults to on, decline writes `telemetry.enabled = false`, + `--telemetry ` skips the prompt. A wording test pins the + disclosure to the shipped fields. +- First-run notice: once, stderr, tty-only, marker file in the config dir. +- `dira config set telemetry.enabled on|off`, `DIRA_TELEMETRY_ENABLED`, + `DO_NOT_TRACK`. Consent transitions emit `cli_consent_recorded`; turning + the knob off is the one event allowed through on the disable transition. + +## Identity + +`telemetry_install_id` (ULID) + `telemetry_salt` (32 random bytes) in `meta`, +independent of the device key. `dira device link` sends the install id in the +claim body (fetched from the daemon, gate-checked, never blocking the link); +the cloud performs the PostHog alias at claim time. `Store::nuke` clears the +queue, cursor, and health keys. + +## Invariants worth re-checking after changes + +- Changing what ships requires updating `TELEMETRY_DISCLOSURE`, + `docs/TELEMETRY.md`, and the cloud allowlist in the same change set. +- The wire enum's per-variant no-stray-field tests are the drift guard; a new + wire field without a taxonomy decision should fail review. +- The batch is versioned `v: 1`; the cloud 400s unknown majors and the daemon + skips past such batches — bump deliberately. diff --git a/AGENTS.md b/AGENTS.md index f5ca4c2..eb81d8d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -56,6 +56,12 @@ agent context at session start. Keep it short and non-negotiable. - Nothing enters `repo_dirs` unless the directory demonstrably belongs to the repo it is filed under, and `register_repo_dir` stays I/O-free. See DIRASH-0027. +- Telemetry ships only the closed `TelemetryEvent` enum — never argv, paths, + repo names, git identity, or error text. Plaintext repo refs are hashed + daemon-side with the per-install salt; the analytics id is never derived + from the device key; the flush gate is consent, never `cloud_link`. + Changing what ships changes `TELEMETRY_DISCLOSURE` and `docs/TELEMETRY.md` + in the same commit. See DIRASH-0033. ### Recorded decisions (read the file before changing guarded code; ask /zavet:why) @@ -91,6 +97,8 @@ agent context at session start. Keep it short and non-negotiable. - DIRASH-0030 — Full-content knowledge sync is opted into by its own prompt, never implied by linking (active) - DIRASH-0031 — One backoff ladder lives in dira_core; callers own their attempt budget (active) - DIRASH-0032 — A record's first-sight triple is repaired as a unit, from recorded facts (active) +- DIRASH-0033 — Telemetry is opt-out, anonymous by construction, and rides its own unsigned channel (active) +- DIRASH-0034 — Repo-visibility probing sends the plaintext ref only to the forge's own public API, anonymously (active) ### Living specs (.zavet/specs/ — keep current while you work) @@ -102,6 +110,7 @@ agent context at session start. Keep it short and non-negotiable. - harness-sources — Harness sources and hook ingestion (session, high) - knowledge-sync — Knowledge sync — the consent-gated second channel (session, medium) - onboarding — Onboarding — dira onboard and the installer handoff (session, high) +- telemetry — Telemetry — anonymous product analytics (session, high) Capture bar: record non-obvious choices a future reader could not reconstruct — micro-decisions as commit trailers (Why:/Rejected:/Constraint:/Refs:), structural ones via /zavet:decide. Spec maintenance (do this as part of normal work, no command needed): when implementing or changing a feature, update its covering spec in .zavet/specs/ — or create one from .zavet/.spec-template.md (origin: session) for substantial new features — reference the decisions involved, and add a `Spec: ` trailer to the commit. diff --git a/Cargo.lock b/Cargo.lock index bea3eeb..06aa2e2 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1042,12 +1042,15 @@ dependencies = [ "ed25519-dalek", "figment", "getrandom 0.4.3", + "hex", + "hmac 0.13.0", "keyring", "keyring-core", "proptest", "serde", "serde_jcs", "serde_json", + "sha2 0.11.0", "sqlx", "tempfile", "thiserror 2.0.20", diff --git a/Cargo.toml b/Cargo.toml index 1f8e798..8ca6aef 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -86,6 +86,11 @@ ed25519-dalek = { version = "3", features = ["rand_core"] } # Note: sha2 0.11 (digest 0.11) dropped the hasher's `io::Write` impl, so # `verify_sha256` streams the file through `update()` instead of `io::copy`. sha2 = "0.11" +# Telemetry's repo-hash HMAC-SHA256 (salted, so a canonical remote never round-trips +# without the per-install salt). 0.13 is the line that resolves against `digest 0.11` +# — the same line `sha2 0.11` and `ed25519-dalek 3` already pull in — so this adds no +# second `digest` major to the tree. +hmac = "0.13" # Hex-encode/decode the sha256 digest `dira update` compares against the # published `.sha256` asset. Already resolved transitively (sqlx-macros-core); # promoted to a direct dep rather than added fresh. diff --git a/README.md b/README.md index 732f0b2..c760236 100644 --- a/README.md +++ b/README.md @@ -27,7 +27,8 @@ effective-dated policy. /dirad resident daemon (tokio): ingress (loopback HTTP + UDS), accounting, store /dira thin CLI client over the daemon's Unix domain socket /sources per-harness hook normalization (claude_code, …) -/docs docs/install.md (installer reference), docs/zavet.md (knowledge module) +/docs docs/install.md (installer reference), docs/zavet.md (knowledge module), + docs/TELEMETRY.md (what's collected and how to turn it off) install.sh curl | sh installer for dira + dirad (see docs/install.md) mise.toml toolchain pins (rust, just) justfile task runner @@ -141,6 +142,19 @@ DIRA_CLOUD_URL=http://localhost:3000 dira device link --code LOCALDEV1 # per-i dira config set cloud_url http://localhost:3000 # persistent ``` +## Telemetry + +Dira collects anonymous product-usage analytics, on by default — never repo names, git +identity, file paths, command arguments, or error text. `dira onboard` asks about it +explicitly, on its own terms. Turn it off anytime: + +```sh +dira config set telemetry.enabled false +``` + +`DIRA_TELEMETRY_ENABLED=0` and `DO_NOT_TRACK=1` also work; dev builds and CI never send. +See [docs/TELEMETRY.md](docs/TELEMETRY.md) for exactly what is (and is never) collected. + ## Contract The wire schema is authored once in Rust (`/contract`) because the daemon is the producer. diff --git a/cli/core/Cargo.toml b/cli/core/Cargo.toml index c33ba45..3be440b 100644 --- a/cli/core/Cargo.toml +++ b/cli/core/Cargo.toml @@ -32,6 +32,9 @@ keyring.workspace = true # `keyring` installs the platform store; entries are built against whatever store # is registered, which is `keyring-core`'s job (see `identity::keychain_entry`). keyring-core.workspace = true +sha2.workspace = true +hmac.workspace = true +hex.workspace = true [dev-dependencies] # `figment::Jail` (env-isolated config tests) is gated behind its `test` feature. diff --git a/cli/core/migrations/0006_telemetry.sql b/cli/core/migrations/0006_telemetry.sql new file mode 100644 index 0000000..e9d7f03 --- /dev/null +++ b/cli/core/migrations/0006_telemetry.sql @@ -0,0 +1,18 @@ +-- The local telemetry queue: an append-only, PII-free buffer of anonymous +-- usage events awaiting sync to the cloud ingest endpoint. +-- +-- `props_json` is the already-scrubbed JSON of a `TelemetryEventWire` (see +-- `dira_core::telemetry::wire`) — the closed `TelemetryEvent` set is the only +-- producer, so nothing upstream of `Store::insert_telemetry_event` can hand +-- this table a prompt, a file path, or free text. `id` is a ULID (monotonic), +-- giving the sync channel the same id-cursor idiom every other channel here +-- uses (`events`, `zavet_guard_events`, ...). +CREATE TABLE IF NOT EXISTS telemetry_events ( + id TEXT PRIMARY KEY, -- ULID, monotonic + created_at TEXT NOT NULL, -- RFC 3339 UTC + name TEXT NOT NULL, -- e.g. cli_command_executed + props_json TEXT NOT NULL -- serialized TelemetryEventWire +); + +CREATE INDEX IF NOT EXISTS idx_telemetry_events_created_at + ON telemetry_events (created_at); diff --git a/cli/core/src/config.rs b/cli/core/src/config.rs index bfbad0f..31b8f46 100644 --- a/cli/core/src/config.rs +++ b/cli/core/src/config.rs @@ -145,6 +145,31 @@ impl Default for UpdateKnobs { } } +/// Anonymous usage-telemetry knob (`[telemetry]` in `config.toml`). +/// +/// Governs the local `telemetry_events` queue and its sync to the cloud ingest +/// endpoint (see `dira_core::telemetry`): `false` stops new events from being +/// queued and stops the daemon from flushing whatever is already queued. It +/// does not touch attestation sync (D-0001) or the knowledge channel +/// (`SyncKnobs::knowledge`) — those have their own, separately-gated consent. +/// Env override: `DIRA_TELEMETRY__ENABLED=true|false`. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] +pub struct TelemetryKnobs { + /// Whether anonymous usage telemetry may be collected and synced at all. + #[serde(default = "default_true")] + pub enabled: bool, +} + +impl Default for TelemetryKnobs { + fn default() -> Self { + // Not `#[derive(Default)]`, for the same reason as `UpdateKnobs`: that + // would give `enabled: false` (bool's zero value), contradicting the + // field's own serde default — telemetry is on out of the box, same as + // every pre-WP1 config that lacks a `[telemetry]` table entirely. + Self { enabled: true } + } +} + /// Daemon + CLI configuration. #[derive(Debug, Clone, Serialize, Deserialize)] pub struct Config { @@ -265,6 +290,9 @@ pub struct Config { /// Passive-update-check knobs (`[update]` table; see [`UpdateKnobs`]). #[serde(default)] pub update: UpdateKnobs, + /// Anonymous usage-telemetry knob (`[telemetry]` table; see [`TelemetryKnobs`]). + #[serde(default)] + pub telemetry: TelemetryKnobs, } /// Where the control socket lives by default. @@ -387,6 +415,7 @@ impl Default for Config { modules: Modules::default(), sync: SyncKnobs::default(), update: UpdateKnobs::default(), + telemetry: TelemetryKnobs::default(), } } } @@ -1042,6 +1071,44 @@ mod tests { }); } + #[test] + fn telemetry_enabled_defaults_to_true() { + // Absent [telemetry] table (every pre-WP1 config) must resolve to + // telemetry-on, matching TelemetryKnobs::default(). + assert!(Config::default().telemetry.enabled); + let c: Config = Figment::from(Serialized::defaults(Config::default())) + .merge(Toml::string("idle_seconds = 120")) + .extract() + .unwrap(); + assert!(c.telemetry.enabled); + } + + #[test] + #[allow(clippy::result_large_err)] + fn telemetry_enabled_layers_from_toml_and_env() { + let c: Config = Figment::from(Serialized::defaults(Config::default())) + .merge(Toml::string("[telemetry]\nenabled = false")) + .extract() + .unwrap(); + assert!(!c.telemetry.enabled); + + // Env wins over toml, via the same DIRA_ prefix + `__`->`.` mapping + // Config::load installs (DIRA_TELEMETRY__ENABLED -> telemetry.enabled). + figment::Jail::expect_with(|jail| { + jail.set_env("DIRA_TELEMETRY__ENABLED", "false"); + let c: Config = Figment::from(Serialized::defaults(Config::default())) + .merge(Toml::string("[telemetry]\nenabled = true")) + .merge( + Env::prefixed("DIRA_") + .map(|k| k.as_str().to_lowercase().replace("__", ".").into()), + ) + .extract() + .unwrap(); + assert!(!c.telemetry.enabled); + Ok(()) + }); + } + #[cfg(unix)] #[test] fn default_socket_path_is_unchanged_on_unix() { diff --git a/cli/core/src/lib.rs b/cli/core/src/lib.rs index ff2a556..61c7ceb 100644 --- a/cli/core/src/lib.rs +++ b/cli/core/src/lib.rs @@ -6,6 +6,7 @@ //! - [`store`]: the append-only SQLite store. //! - [`project`]: working-dir → canonical repo + identity resolution. //! - [`config`]: layered configuration. +//! - [`telemetry`]: anonymous usage-telemetry foundation (identity, event model, wire types). pub mod accounting; pub mod config; @@ -18,6 +19,7 @@ pub mod report; pub mod signing; pub mod store; pub mod sync; +pub mod telemetry; pub mod tokens; pub mod zavet; diff --git a/cli/core/src/protocol.rs b/cli/core/src/protocol.rs index 88cdba0..207e02e 100644 --- a/cli/core/src/protocol.rs +++ b/cli/core/src/protocol.rs @@ -5,6 +5,7 @@ //! many bytes of JSON. One request, one response, per connection. use crate::report::Report; +use crate::telemetry::wire::TelemetryEventWire; use dira_contract::{Harness, SessionKind}; use serde::{Deserialize, Serialize}; @@ -66,6 +67,34 @@ pub enum Request { /// the repo from the payload's `cwd` and never trusts a caller-supplied /// repo identity. IngestZavet { payload: serde_json::Value }, + /// Enqueue one anonymous usage-telemetry event onto the daemon's local + /// queue (`dira_core::telemetry`, WP1/WP2), from a CLI-side + /// [`crate::telemetry::event::TelemetryEvent`] already flattened to its + /// wire shape. The daemon re-checks `[telemetry] enabled` itself (belt and + /// braces — a CLI/daemon version skew must never let a disabled knob leak + /// events through), stamps `id`/`created_at`, and nudges the telemetry + /// sync task. Fire-and-forget, like `IngestZavet`: acked with + /// [`Response::Ok`] whether or not telemetry is enabled. + /// + /// `repo_canonical` is the CLI-resolved canonical repo ref (e.g. + /// `github.com/acme/api`) for a `CommandExecuted` event run inside a + /// repo, or `None` otherwise. It crosses only this local socket, never + /// the network: the daemon salt-hashes it via + /// `dira_core::telemetry::repo_facts` and fills `event`'s `repo_*` + /// fields before enqueuing, so the plaintext ref itself is never + /// persisted or shipped. `#[serde(default)]` so an older `dira` + /// (pre-dating this field) still deserializes against a newer daemon. + IngestTelemetry { + event: TelemetryEventWire, + #[serde(default)] + repo_canonical: Option, + }, + /// Fetch this install's telemetry id (minting it on first use), for + /// `dira device link` to attach as best-effort context on the claim. + /// Never gated on `[telemetry] enabled` here — the CLI itself already + /// skips this request when its own gate denies emission, and an id with + /// no events behind it identifies nothing on its own. + TelemetryInstallId, /// Zavet activation + capture health for a repo (resolved from `repo`, or /// `cwd`, or the daemon's own cwd — same ladder as `Start`). ZavetStatus { @@ -357,6 +386,9 @@ pub enum Response { /// Same new-variant skew posture as `ZavetSpec`, and harmless here: only a /// CLI new enough to send `CaptureProbe` can ever receive this. CaptureProbe(Box), + /// `TelemetryInstallId`: this daemon's telemetry install id, minted on + /// first use — see [`Request::TelemetryInstallId`]. + TelemetryInstallId { install_id: String }, } /// A live or recent session as shown by `status` / `sessions`. diff --git a/cli/core/src/store.rs b/cli/core/src/store.rs index 24f7de1..4a0a3f1 100644 --- a/cli/core/src/store.rs +++ b/cli/core/src/store.rs @@ -470,6 +470,13 @@ impl Store { sqlx::query("DELETE FROM meta WHERE key LIKE 'token_fp:%'") .execute(&mut *tx) .await?; + // The telemetry queue (WP1/WP2) is a stat too — wipe it so a nuked slate + // reports no backlog rather than re-sending pre-nuke events under a + // blanked cursor (which would otherwise look like a burst of brand-new + // activity to the cloud). + sqlx::query("DELETE FROM telemetry_events") + .execute(&mut *tx) + .await?; // Reset the sync cursors in the same transaction so they can't point past // the wiped tables. We blank them rather than delete the keys to keep reads // simple. @@ -480,6 +487,9 @@ impl Store { // For `token_usage` that would re-create the exact defect // `META_TOKEN_CURSOR` exists to fix, since `nuke` also clears the // `token_offset:%` watermarks above and thus guarantees a full re-import. + // Same reasoning applies to `telemetry_events`' ULID cursor: the table + // above is now empty, so a stale cursor would silently skip every event + // re-queued after the nuke. for key in [ crate::sync::META_SYNC_CURSOR, crate::sync::META_ARTIFACTS_CURSOR, @@ -490,6 +500,8 @@ impl Store { crate::sync::knowledge::META_KNOWLEDGE_SPEC_CURSOR, crate::sync::knowledge::META_KNOWLEDGE_TRAILER_CURSOR, crate::sync::knowledge::META_KNOWLEDGE_GUARD_CURSOR, + crate::telemetry::META_TELEMETRY_CURSOR, + crate::telemetry::META_TELEMETRY_HEALTH, ] { sqlx::query( "INSERT INTO meta (key, value) VALUES (?1, '') @@ -2178,6 +2190,108 @@ impl Store { specs_total: s.get::("n") as u64, }) } + + // ---- telemetry (WP1) ---- + + /// Hard cap on the local telemetry queue. An install with no `cloud_url` + /// configured (or a linked one whose flush task has been broken for a + /// long time) never runs `delete_telemetry_events_through` at all — WP2's + /// sync task is the only pruner — so without a cap this table would grow + /// without bound for the lifetime of the install. The queue exists to + /// bridge a short outage, not to serve as an unbounded local log; well + /// above anything a healthy flush cadence would ever let accumulate, so + /// it only ever bites the pathological case. + pub(crate) const TELEMETRY_QUEUE_CAP: i64 = 5000; + + /// Append one telemetry event to the local queue. `id` is a caller-minted + /// ULID (monotonic), matching every other channel's id-cursor idiom. + /// + /// Trims the tail to [`Self::TELEMETRY_QUEUE_CAP`] after every insert — + /// see its doc comment for why. Oldest-first by `id` (ULIDs sort + /// lexically by mint time), and folded into the same statement as the + /// trim decision so a burst of concurrent inserts can't race a separate + /// read-then-delete into over- or under-trimming. + pub async fn insert_telemetry_event( + &self, + id: &str, + created_at: &str, + name: &str, + props_json: &str, + ) -> Result<(), Error> { + sqlx::query( + "INSERT INTO telemetry_events (id, created_at, name, props_json) + VALUES (?1, ?2, ?3, ?4)", + ) + .bind(id) + .bind(created_at) + .bind(name) + .bind(props_json) + .execute(&self.pool) + .await?; + sqlx::query( + "DELETE FROM telemetry_events WHERE id NOT IN \ + (SELECT id FROM telemetry_events ORDER BY id DESC LIMIT ?1)", + ) + .bind(Self::TELEMETRY_QUEUE_CAP) + .execute(&self.pool) + .await?; + Ok(()) + } + + /// The largest `telemetry_events.id`, or `None` when the queue is empty — + /// the snapshot upper bound for a sync window (`until`), same role as + /// [`Self::max_event_id`]. + pub async fn telemetry_max_event_id(&self) -> Result, Error> { + let row = sqlx::query("SELECT MAX(id) AS m FROM telemetry_events") + .fetch_one(&self.pool) + .await?; + Ok(row.get::, _>("m")) + } + + /// Load telemetry events in the id window `(cursor, until]`, ordered by id — + /// the un-synced telemetry backlog for a flush. `cursor = None` means "from + /// the beginning". Mirrors [`Self::events_between`]'s id-cursor idiom. + pub async fn telemetry_events_since( + &self, + cursor: Option<&str>, + until: &str, + limit: i64, + ) -> Result, Error> { + let rows = match cursor { + Some(c) => { + sqlx::query( + "SELECT id, created_at, name, props_json FROM telemetry_events \ + WHERE id > ?1 AND id <= ?2 ORDER BY id ASC LIMIT ?3", + ) + .bind(c) + .bind(until) + .bind(limit) + .fetch_all(&self.pool) + .await? + } + None => { + sqlx::query( + "SELECT id, created_at, name, props_json FROM telemetry_events \ + WHERE id <= ?1 ORDER BY id ASC LIMIT ?2", + ) + .bind(until) + .bind(limit) + .fetch_all(&self.pool) + .await? + } + }; + Ok(rows.iter().map(row_to_telemetry_event).collect()) + } + + /// Delete telemetry events with id ≤ `id` — retention/pruning for rows + /// already acked by the cloud. Returns the number of rows removed. + pub async fn delete_telemetry_events_through(&self, id: &str) -> Result { + let res = sqlx::query("DELETE FROM telemetry_events WHERE id <= ?1") + .bind(id) + .execute(&self.pool) + .await?; + Ok(res.rows_affected()) + } } /// Tighten `path` (and its SQLite `-wal`/`-shm` sidecars) to owner-only `0600` @@ -2310,6 +2424,20 @@ pub struct ZavetSpecCapture { pub content_hash: Option, } +/// One stored telemetry event, as read from `telemetry_events`. Mirrors the +/// columns 1:1 (see migration `0006_telemetry.sql`). +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct TelemetryEventRow { + /// ULID, monotonic — also the sync-window cursor value. + pub id: String, + /// RFC 3339 timestamp string, stored verbatim. + pub created_at: String, + /// Wire event name, e.g. `cli_command_executed`. + pub name: String, + /// Serialized `TelemetryEventWire` JSON. + pub props_json: String, +} + /// A stored decision row plus its guard globs. #[derive(Debug, Clone, Default, PartialEq, Eq)] pub struct ZavetDecisionRow { @@ -2620,6 +2748,15 @@ fn row_to_token_row(row: &sqlx::sqlite::SqliteRow) -> Result { }) } +fn row_to_telemetry_event(row: &sqlx::sqlite::SqliteRow) -> TelemetryEventRow { + TelemetryEventRow { + id: row.get("id"), + created_at: row.get("created_at"), + name: row.get("name"), + props_json: row.get("props_json"), + } +} + fn row_to_event(row: &sqlx::sqlite::SqliteRow) -> Result { let at: String = row.get("at"); let harness: String = row.get("harness"); @@ -3069,6 +3206,54 @@ mod tests { ); } + #[tokio::test] + async fn nuke_clears_the_telemetry_queue_and_its_cursor_and_health() { + let store = Store::open_in_memory().await.unwrap(); + store + .insert_telemetry_event("01A", "2026-01-01T00:00:00Z", "cli_daemon_started", "{}") + .await + .unwrap(); + store + .meta_set(crate::telemetry::META_TELEMETRY_CURSOR, "01A") + .await + .unwrap(); + store + .meta_set( + crate::telemetry::META_TELEMETRY_HEALTH, + r#"{"consecutiveFailures":3}"#, + ) + .await + .unwrap(); + + store.nuke().await.unwrap(); + + assert_eq!(store.telemetry_max_event_id().await.unwrap(), None); + assert_eq!( + store + .telemetry_events_since(None, "01ZZZZZZZZZZZZZZZZZZZZZZZZ", 100) + .await + .unwrap(), + vec![] + ); + assert_eq!( + store + .meta_get(crate::telemetry::META_TELEMETRY_CURSOR) + .await + .unwrap() + .as_deref(), + Some(""), + "a stale cursor over an emptied queue would skip every re-queued event" + ); + assert_eq!( + store + .meta_get(crate::telemetry::META_TELEMETRY_HEALTH) + .await + .unwrap() + .as_deref(), + Some(""), + ); + } + #[tokio::test] async fn artifacts_capture_is_idempotent_and_windowed() { let store = Store::open_in_memory().await.unwrap(); @@ -4085,4 +4270,135 @@ mod tests { assert_eq!(totals.cache_create, 0); assert_eq!(totals.est_cost_usd, 0.0); } + + // ---- telemetry (WP1) ---- + + #[tokio::test] + async fn telemetry_events_since_empty_queue_is_empty() { + let store = Store::open_in_memory().await.unwrap(); + assert_eq!(store.telemetry_max_event_id().await.unwrap(), None); + let rows = store + .telemetry_events_since(None, "01ZZZZZZZZZZZZZZZZZZZZZZZZ", 100) + .await + .unwrap(); + assert!(rows.is_empty()); + } + + #[tokio::test] + async fn telemetry_insert_and_read_back_round_trips() { + let store = Store::open_in_memory().await.unwrap(); + store + .insert_telemetry_event("01A", "2026-01-01T00:00:00Z", "cli_daemon_started", "{}") + .await + .unwrap(); + store + .insert_telemetry_event( + "01B", + "2026-01-01T00:00:01Z", + "cli_command_executed", + r#"{"command":"status"}"#, + ) + .await + .unwrap(); + + assert_eq!( + store.telemetry_max_event_id().await.unwrap(), + Some("01B".to_string()) + ); + + let rows = store + .telemetry_events_since(None, "01B", 100) + .await + .unwrap(); + assert_eq!(rows.len(), 2); + assert_eq!(rows[0].id, "01A"); + assert_eq!(rows[0].name, "cli_daemon_started"); + assert_eq!(rows[1].id, "01B"); + assert_eq!(rows[1].props_json, r#"{"command":"status"}"#); + } + + #[tokio::test] + async fn telemetry_events_since_respects_cursor_and_until_window() { + let store = Store::open_in_memory().await.unwrap(); + for id in ["01A", "01B", "01C", "01D"] { + store + .insert_telemetry_event(id, "2026-01-01T00:00:00Z", "cli_daemon_started", "{}") + .await + .unwrap(); + } + // (01A, 01C] -> 01B, 01C. + let rows = store + .telemetry_events_since(Some("01A"), "01C", 100) + .await + .unwrap(); + let ids: Vec<&str> = rows.iter().map(|r| r.id.as_str()).collect(); + assert_eq!(ids, vec!["01B", "01C"]); + } + + #[tokio::test] + async fn telemetry_events_since_respects_limit() { + let store = Store::open_in_memory().await.unwrap(); + for id in ["01A", "01B", "01C"] { + store + .insert_telemetry_event(id, "2026-01-01T00:00:00Z", "cli_daemon_started", "{}") + .await + .unwrap(); + } + let rows = store.telemetry_events_since(None, "01C", 2).await.unwrap(); + let ids: Vec<&str> = rows.iter().map(|r| r.id.as_str()).collect(); + assert_eq!(ids, vec!["01A", "01B"]); + } + + #[tokio::test] + async fn delete_telemetry_events_through_prunes_only_up_to_id() { + let store = Store::open_in_memory().await.unwrap(); + for id in ["01A", "01B", "01C"] { + store + .insert_telemetry_event(id, "2026-01-01T00:00:00Z", "cli_daemon_started", "{}") + .await + .unwrap(); + } + let deleted = store.delete_telemetry_events_through("01B").await.unwrap(); + assert_eq!(deleted, 2); + let remaining = store + .telemetry_events_since(None, "01C", 100) + .await + .unwrap(); + let ids: Vec<&str> = remaining.iter().map(|r| r.id.as_str()).collect(); + assert_eq!(ids, vec!["01C"]); + } + + /// A no-cloud-linked install (or one whose flush is stuck) never runs + /// `delete_telemetry_events_through` — nothing else prunes this table — + /// so `insert_telemetry_event` itself must cap it, oldest-first. + #[tokio::test] + async fn insert_telemetry_event_caps_the_queue_and_evicts_the_oldest() { + let store = Store::open_in_memory().await.unwrap(); + let cap = Store::TELEMETRY_QUEUE_CAP; + let total = cap + 5; + for i in 0..total { + let id = format!("{i:010}"); // zero-padded so lexical order == insertion order + store + .insert_telemetry_event(&id, "2026-01-01T00:00:00Z", "cli_daemon_started", "{}") + .await + .unwrap(); + } + + let until = store.telemetry_max_event_id().await.unwrap().unwrap(); + let rows = store + .telemetry_events_since(None, &until, total + 100) + .await + .unwrap(); + assert_eq!( + rows.len(), + cap as usize, + "the queue must never exceed the cap" + ); + // The oldest 5 ids (0000000000..0000000004) were evicted; the newest + // `cap` rows survive. + let oldest_surviving: i64 = rows[0].id.parse().unwrap(); + assert_eq!(oldest_surviving, total - cap); + let newest_surviving: i64 = rows[rows.len() - 1].id.parse().unwrap(); + assert_eq!(newest_surviving, total - 1); + } } diff --git a/cli/core/src/telemetry/event.rs b/cli/core/src/telemetry/event.rs new file mode 100644 index 0000000..9e8d848 --- /dev/null +++ b/cli/core/src/telemetry/event.rs @@ -0,0 +1,292 @@ +//! The telemetry event model — the closed interface other crates code against. +//! +//! `dira`/`dirad` construct a [`TelemetryEvent`], never a [`super::wire::TelemetryEventWire`] +//! directly, so the set of things that can ever be reported is fixed here and +//! reviewable in one place. There is deliberately no `DeviceLinkAlias` event: +//! the cloud performs that alias at device-claim time, from data it already has. + +use super::repo_facts::RepoFacts; +use super::wire::TelemetryEventWire; + +/// The closed set of events dira may report. +#[derive(Debug, Clone)] +pub enum TelemetryEvent { + /// A CLI (or daemon-served) command finished running. + CommandExecuted { + /// The command name, e.g. `"status"`, `"config"`. Always the + /// top-level command name — never a sub-action (`config set` reports + /// as `"config"`) — and never raw argv: no path, no flag value, no + /// user text. + command: &'static str, + duration_ms: u64, + success: bool, + error_kind: Option, + repo: Option, + }, + /// `dirad` finished starting up. + DaemonStarted, + /// `dirad` is shutting down after `uptime_secs` of wall-clock life. + DaemonStopped { uptime_secs: u64 }, + /// The telemetry consent knob changed (including its initial default). + ConsentRecorded { + enabled: bool, + source: ConsentSource, + }, +} + +/// A coarse failure classification for [`TelemetryEvent::CommandExecuted`]. +/// Never the error's message or any value it carried — just which of a fixed +/// set of failure shapes occurred. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum ErrorKind { + DaemonUnreachable, + DaemonError, + InvalidInput, + IoError, + Timeout, + Internal, +} + +impl ErrorKind { + /// The lowercase snake_case wire spelling. + pub fn as_str(&self) -> &'static str { + match self { + ErrorKind::DaemonUnreachable => "daemon_unreachable", + ErrorKind::DaemonError => "daemon_error", + ErrorKind::InvalidInput => "invalid_input", + ErrorKind::IoError => "io_error", + ErrorKind::Timeout => "timeout", + ErrorKind::Internal => "internal", + } + } +} + +/// How a [`TelemetryEvent::ConsentRecorded`] came about. +/// +/// Deliberately closed to the ways the knob can actually change: there is no +/// `Env` variant because the environment kill switches (`DO_NOT_TRACK`, +/// `DIRA_TELEMETRY_ENABLED=0`) trip `TelemetryGate::hard_disabled` before +/// `record_consent` ever runs, so a production caller can never construct +/// this from an env override — and no `Default` variant, because the knob's +/// initial default is never itself an event (see `docs/TELEMETRY.md`: +/// accepting the default writes nothing and reports nothing). +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum ConsentSource { + /// The interactive onboarding prompt. + Prompt, + /// A non-interactive `--yes`-style flag. + YesFlag, + /// `dira config set telemetry.enabled ...`. + ConfigSet, +} + +impl ConsentSource { + /// The lowercase snake_case wire spelling. + pub fn as_str(&self) -> &'static str { + match self { + ConsentSource::Prompt => "prompt", + ConsentSource::YesFlag => "yes_flag", + ConsentSource::ConfigSet => "config_set", + } + } +} + +impl TelemetryEvent { + /// The wire event name, e.g. `"cli_command_executed"`. + pub fn name(&self) -> &'static str { + match self { + TelemetryEvent::CommandExecuted { .. } => "cli_command_executed", + TelemetryEvent::DaemonStarted => "cli_daemon_started", + TelemetryEvent::DaemonStopped { .. } => "cli_daemon_stopped", + TelemetryEvent::ConsentRecorded { .. } => "cli_consent_recorded", + } + } + + /// Flatten into the wire shape, stamping `timestamp_rfc3339` and the + /// running `dira_version` that every event carries regardless of variant. + pub fn into_wire(self, timestamp_rfc3339: String, dira_version: &str) -> TelemetryEventWire { + let mut wire = TelemetryEventWire::base(self.name(), timestamp_rfc3339, dira_version); + match self { + TelemetryEvent::CommandExecuted { + command, + duration_ms, + success, + error_kind, + repo, + } => { + wire.command = Some(command.to_string()); + wire.duration_ms = Some(duration_ms); + wire.success = Some(success); + wire.error_kind = error_kind.map(|k| k.as_str().to_string()); + if let Some(facts) = repo { + wire.repo_host_class = Some(facts.host_class.as_str().to_string()); + wire.repo_visibility = Some(facts.visibility.as_str().to_string()); + wire.repo_hash = Some(facts.repo_hash); + } + } + TelemetryEvent::DaemonStarted => {} + TelemetryEvent::DaemonStopped { uptime_secs } => { + wire.duration_ms = Some(uptime_secs.saturating_mul(1000)); + } + TelemetryEvent::ConsentRecorded { enabled, source } => { + wire.telemetry_enabled = Some(enabled); + wire.consent_source = Some(source.as_str().to_string()); + } + } + wire + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::telemetry::repo_facts::{RepoHostClass, RepoVisibility}; + use std::collections::BTreeSet; + + const TS: &str = "2026-01-01T00:00:00Z"; + const VERSION: &str = "0.5.1"; + + fn keys(wire: &TelemetryEventWire) -> BTreeSet { + let json = serde_json::to_value(wire).unwrap(); + json.as_object().unwrap().keys().cloned().collect() + } + + const BASE_KEYS: &[&str] = &["event", "timestamp", "diraVersion", "os", "arch"]; + + fn expected(extra: &[&str]) -> BTreeSet { + BASE_KEYS + .iter() + .chain(extra) + .map(|s| s.to_string()) + .collect() + } + + #[test] + fn command_executed_without_repo_carries_no_repo_keys() { + let ev = TelemetryEvent::CommandExecuted { + command: "status", + duration_ms: 42, + success: true, + error_kind: None, + repo: None, + }; + let wire = ev.into_wire(TS.into(), VERSION); + assert_eq!(keys(&wire), expected(&["command", "durationMs", "success"])); + } + + #[test] + fn command_executed_with_error_and_repo_carries_exactly_those_keys() { + let ev = TelemetryEvent::CommandExecuted { + command: "sync", + duration_ms: 7, + success: false, + error_kind: Some(ErrorKind::DaemonUnreachable), + repo: Some(RepoFacts { + host_class: RepoHostClass::GitHub, + visibility: RepoVisibility::Unknown, + repo_hash: "deadbeef".into(), + }), + }; + let wire = ev.into_wire(TS.into(), VERSION); + assert_eq!( + keys(&wire), + expected(&[ + "command", + "durationMs", + "success", + "errorKind", + "repoHostClass", + "repoVisibility", + "repoHash", + ]) + ); + assert_eq!(wire.error_kind.as_deref(), Some("daemon_unreachable")); + assert_eq!(wire.repo_host_class.as_deref(), Some("github")); + assert_eq!(wire.repo_visibility.as_deref(), Some("unknown")); + assert_eq!(wire.repo_hash.as_deref(), Some("deadbeef")); + } + + #[test] + fn daemon_started_carries_only_base_keys() { + let wire = TelemetryEvent::DaemonStarted.into_wire(TS.into(), VERSION); + assert_eq!(keys(&wire), expected(&[])); + assert_eq!(wire.event, "cli_daemon_started"); + } + + #[test] + fn daemon_stopped_carries_duration_ms_as_uptime() { + let wire = TelemetryEvent::DaemonStopped { uptime_secs: 90 }.into_wire(TS.into(), VERSION); + assert_eq!(keys(&wire), expected(&["durationMs"])); + assert_eq!(wire.duration_ms, Some(90_000)); + } + + #[test] + fn consent_recorded_carries_enabled_and_source() { + let wire = TelemetryEvent::ConsentRecorded { + enabled: false, + source: ConsentSource::YesFlag, + } + .into_wire(TS.into(), VERSION); + assert_eq!( + keys(&wire), + expected(&["telemetryEnabled", "consentSource"]) + ); + assert_eq!(wire.telemetry_enabled, Some(false)); + assert_eq!(wire.consent_source.as_deref(), Some("yes_flag")); + } + + #[test] + fn event_names_match_the_wire_taxonomy() { + assert_eq!( + TelemetryEvent::CommandExecuted { + command: "x", + duration_ms: 0, + success: true, + error_kind: None, + repo: None, + } + .name(), + "cli_command_executed" + ); + assert_eq!(TelemetryEvent::DaemonStarted.name(), "cli_daemon_started"); + assert_eq!( + TelemetryEvent::DaemonStopped { uptime_secs: 0 }.name(), + "cli_daemon_stopped" + ); + assert_eq!( + TelemetryEvent::ConsentRecorded { + enabled: true, + source: ConsentSource::Prompt + } + .name(), + "cli_consent_recorded" + ); + } + + #[test] + fn error_kind_as_str_is_lowercase_snake() { + let cases = [ + (ErrorKind::DaemonUnreachable, "daemon_unreachable"), + (ErrorKind::DaemonError, "daemon_error"), + (ErrorKind::InvalidInput, "invalid_input"), + (ErrorKind::IoError, "io_error"), + (ErrorKind::Timeout, "timeout"), + (ErrorKind::Internal, "internal"), + ]; + for (kind, want) in cases { + assert_eq!(kind.as_str(), want); + } + } + + #[test] + fn consent_source_as_str_is_lowercase_snake() { + let cases = [ + (ConsentSource::Prompt, "prompt"), + (ConsentSource::YesFlag, "yes_flag"), + (ConsentSource::ConfigSet, "config_set"), + ]; + for (source, want) in cases { + assert_eq!(source.as_str(), want); + } + } +} diff --git a/cli/core/src/telemetry/identity.rs b/cli/core/src/telemetry/identity.rs new file mode 100644 index 0000000..fbdbf97 --- /dev/null +++ b/cli/core/src/telemetry/identity.rs @@ -0,0 +1,124 @@ +//! Telemetry install identity: a stable anonymous id and a per-install salt, +//! both persisted in the local store's `meta` table. +//! +//! Distinct from [`crate::identity`] (the device's Ed25519 keypair, used for +//! signed attestation batches): `install_id` identifies a telemetry stream, not +//! a device, and `salt` never leaves the machine — it exists only so +//! [`crate::telemetry::repo_facts::compute`] can hash a repo remote without the +//! hash round-tripping to the plain remote. Neither value is derived from the +//! device key, so disabling telemetry and re-enabling it later does not +//! resurrect a hash computed before the gap (a fresh `dira telemetry reset`, +//! if ever added, would mint both anew). + +use crate::store::Store; +use crate::Error; +use base64::engine::general_purpose::STANDARD as B64; +use base64::Engine; +use getrandom::rand_core::{Rng, UnwrapErr}; + +/// `meta` key holding the telemetry install id (a ULID, minted once). +pub const META_TELEMETRY_INSTALL_ID: &str = "telemetry_install_id"; +/// `meta` key holding the standard-base64 32-byte telemetry salt. +pub const META_TELEMETRY_SALT: &str = "telemetry_salt"; + +/// A device's telemetry identity: the id events are tagged with, and the salt +/// [`crate::telemetry::repo_facts::compute`] HMACs canonical repo remotes with. +/// +/// `Clone` so a caller can cache one instance (e.g. `dirad`'s +/// `AppState::telemetry_identity`, behind a `tokio::sync::OnceCell`) and hand +/// out cheap copies instead of every caller re-reading the store. +#[derive(Clone)] +pub struct TelemetryIdentity { + pub install_id: String, + pub salt: [u8; 32], +} + +/// Load the telemetry identity, minting + persisting whatever is missing on +/// first use. Idempotent: repeated calls against the same store return +/// identical values, mirroring [`crate::identity::load_or_create_unlinked`]. +/// +/// Not itself race-free under concurrent callers on a fresh store: two calls +/// that both race the initial "nothing in `meta` yet" read could each mint +/// and persist a *different* salt, the later write winning silently, and +/// each call's returned identity would keep reflecting the salt it minted +/// rather than the one that ended up persisted. `dirad` never calls this +/// directly on a live daemon for exactly that reason — see +/// `dirad::state::AppState::telemetry_identity`, which serializes every +/// caller through a single `OnceCell` so this runs at most once per daemon +/// lifetime. +pub async fn load_or_mint(store: &Store) -> Result { + let install_id = match store.meta_get(META_TELEMETRY_INSTALL_ID).await? { + Some(id) => id, + None => { + let id = ulid::Ulid::generate().to_string(); + store.meta_set(META_TELEMETRY_INSTALL_ID, &id).await?; + id + } + }; + let salt = match store.meta_get(META_TELEMETRY_SALT).await? { + Some(encoded) => decode_salt(&encoded)?, + None => { + let bytes = random_salt(); + store + .meta_set(META_TELEMETRY_SALT, &B64.encode(bytes)) + .await?; + bytes + } + }; + Ok(TelemetryIdentity { install_id, salt }) +} + +/// 32 random bytes from the OS CSPRNG. Mirrors [`crate::signing::DeviceKey::generate`]'s +/// use of `getrandom`'s infallible `SysRng` — the lightest source already in the +/// tree, so no new randomness dependency is needed for a salt this small. +fn random_salt() -> [u8; 32] { + let mut rng = UnwrapErr(getrandom::SysRng); + let mut buf = [0u8; 32]; + rng.fill_bytes(&mut buf); + buf +} + +fn decode_salt(encoded: &str) -> Result<[u8; 32], Error> { + let bytes = B64 + .decode(encoded.trim()) + .map_err(|e| Error::Decode(format!("bad telemetry salt base64: {e}")))?; + bytes + .try_into() + .map_err(|_| Error::Decode("telemetry salt must be 32 bytes".into())) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[tokio::test] + async fn mints_an_id_and_salt_on_first_use() { + let store = Store::open_in_memory().await.unwrap(); + let identity = load_or_mint(&store).await.unwrap(); + assert!(!identity.install_id.is_empty()); + assert_ne!(identity.salt, [0u8; 32]); + } + + #[tokio::test] + async fn is_idempotent_across_calls() { + let store = Store::open_in_memory().await.unwrap(); + let first = load_or_mint(&store).await.unwrap(); + let second = load_or_mint(&store).await.unwrap(); + assert_eq!(first.install_id, second.install_id); + assert_eq!(first.salt, second.salt); + } + + #[tokio::test] + async fn persists_across_separate_loads() { + let store = Store::open_in_memory().await.unwrap(); + let minted = load_or_mint(&store).await.unwrap(); + let reloaded_id = store + .meta_get(META_TELEMETRY_INSTALL_ID) + .await + .unwrap() + .unwrap(); + assert_eq!(minted.install_id, reloaded_id); + let reloaded_salt = store.meta_get(META_TELEMETRY_SALT).await.unwrap().unwrap(); + assert_eq!(B64.decode(reloaded_salt).unwrap(), minted.salt.to_vec()); + } +} diff --git a/cli/core/src/telemetry/mod.rs b/cli/core/src/telemetry/mod.rs new file mode 100644 index 0000000..2d47ee3 --- /dev/null +++ b/cli/core/src/telemetry/mod.rs @@ -0,0 +1,33 @@ +//! Telemetry foundation (WP1): an anonymous, opt-out usage channel gated by +//! [`crate::config::TelemetryKnobs`]. +//! +//! - [`identity`]: the per-install id + salt the rest of this module is keyed on. +//! - [`repo_facts`]: pure classification of a canonical repo remote (no I/O — the +//! caller supplies the salt and any visibility it already knows). +//! - [`event`]: the closed set of events other crates are allowed to emit. +//! - [`wire`]: the serde-only batch shape POSTed to the cloud ingest endpoint. +//! +//! The local queue (`telemetry_events`, migration `0006_telemetry.sql`) is +//! append-only and PII-free by construction: every column is either an id, a +//! timestamp, an event name from the closed [`event::TelemetryEvent`] set, or a +//! JSON blob of the wire-shaped, already-scrubbed properties in +//! [`wire::TelemetryEventWire`]. Nothing upstream of `Store::insert_telemetry_event` +//! ever hands it a prompt, a file path, or free text. + +pub mod event; +pub mod identity; +pub mod repo_facts; +pub mod wire; + +/// `meta` key: highest `telemetry_events.id` (ULID) confirmed-synced. Same +/// id-cursor idiom as `sync::META_TOKEN_CURSOR` / the knowledge channel's +/// cursors — see [`crate::store::Store::telemetry_events_since`]. +pub const META_TELEMETRY_CURSOR: &str = "telemetry_cursor_event_id"; +/// `meta` key: the telemetry channel's health snapshot (same shape discipline +/// as `sync::META_SYNC_HEALTH`). +pub const META_TELEMETRY_HEALTH: &str = "telemetry_sync_health"; +/// Filename, under the XDG config dir (beside `config.toml`), marking that the +/// one-time "telemetry is on by default, here's how to turn it off" notice has +/// already been shown on this machine. Presence alone is the signal — the file +/// carries no content. +pub const NOTICE_MARKER_FILE: &str = ".telemetry-notice-shown"; diff --git a/cli/core/src/telemetry/repo_facts.rs b/cli/core/src/telemetry/repo_facts.rs new file mode 100644 index 0000000..942ffc4 --- /dev/null +++ b/cli/core/src/telemetry/repo_facts.rs @@ -0,0 +1,156 @@ +//! Pure repo-fact classification for telemetry. No I/O and no store access — +//! callers supply the canonical remote, the install salt, and any visibility +//! they already resolved; this module only classifies and hashes. + +use hmac::{Hmac, KeyInit, Mac}; +use sha2::Sha256; + +type HmacSha256 = Hmac; + +/// Which forge a canonical remote belongs to. Never carries owner/repo — the +/// point is to say "this is a GitHub project" without saying which one. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum RepoHostClass { + GitHub, + GitLab, + Bitbucket, + SelfHosted, +} + +impl RepoHostClass { + /// The lowercase wire spelling. + pub fn as_str(&self) -> &'static str { + match self { + RepoHostClass::GitHub => "github", + RepoHostClass::GitLab => "gitlab", + RepoHostClass::Bitbucket => "bitbucket", + RepoHostClass::SelfHosted => "self_hosted", + } + } +} + +/// Repo visibility, when the caller can determine it. `Unknown` (not a default +/// guess of `Private`) is deliberate: WP1 has no visibility source of its own, +/// so every caller until a later work package resolves this passes `Unknown` +/// rather than a fabricated answer. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum RepoVisibility { + Public, + Private, + Unknown, +} + +impl RepoVisibility { + /// The lowercase wire spelling. + pub fn as_str(&self) -> &'static str { + match self { + RepoVisibility::Public => "public", + RepoVisibility::Private => "private", + RepoVisibility::Unknown => "unknown", + } + } +} + +/// The repo facts attached to a `CommandExecuted` event: enough to segment +/// usage by forge and visibility, plus a salted hash that lets the SAME repo +/// be recognized across events from one install without identifying which +/// repo it is. +#[derive(Debug, Clone)] +pub struct RepoFacts { + pub host_class: RepoHostClass, + pub visibility: RepoVisibility, + pub repo_hash: String, +} + +/// Classify a canonical remote's host. `canonical` is +/// [`crate::project::canonicalize_remote`]'s output — lowercase `host/owner/repo` +/// — so the host is always its first `/`-separated segment. +pub fn classify_host(canonical: &str) -> RepoHostClass { + match canonical.split('/').next().unwrap_or("") { + "github.com" => RepoHostClass::GitHub, + "gitlab.com" => RepoHostClass::GitLab, + "bitbucket.org" => RepoHostClass::Bitbucket, + _ => RepoHostClass::SelfHosted, + } +} + +/// Derive [`RepoFacts`] for a canonical remote: the host class, the +/// caller-supplied visibility, and `repo_hash = hex(HMAC-SHA256(salt, canonical))`. +/// +/// Keying the hash on `salt` (not a bare digest of `canonical`) is the whole +/// point: two installs hashing the same repo produce unrelated hashes, and +/// the hash cannot be reversed to the plain remote without the salt, which +/// never leaves the machine (see [`super::identity`]). +pub fn compute(canonical: &str, salt: &[u8; 32], visibility: RepoVisibility) -> RepoFacts { + // `new_from_slice` only fails for MACs with a fixed key length; HMAC accepts + // any key length (short keys are zero-padded, long ones pre-hashed), so a + // 32-byte salt can never trip this. + let mut mac = HmacSha256::new_from_slice(salt).expect("HMAC-SHA256 accepts any key length"); + mac.update(canonical.as_bytes()); + let repo_hash = hex::encode(mac.finalize().into_bytes()); + RepoFacts { + host_class: classify_host(canonical), + visibility, + repo_hash, + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn classifies_known_forges() { + let cases = [ + ("github.com/acme/api", RepoHostClass::GitHub), + ("gitlab.com/acme/api", RepoHostClass::GitLab), + ("bitbucket.org/acme/api", RepoHostClass::Bitbucket), + ("git.acme.internal/acme/api", RepoHostClass::SelfHosted), + ("", RepoHostClass::SelfHosted), + ]; + for (canonical, want) in cases { + assert_eq!(classify_host(canonical), want, "canonical={canonical}"); + } + } + + #[test] + fn as_str_is_lowercase_snake() { + assert_eq!(RepoHostClass::GitHub.as_str(), "github"); + assert_eq!(RepoHostClass::GitLab.as_str(), "gitlab"); + assert_eq!(RepoHostClass::Bitbucket.as_str(), "bitbucket"); + assert_eq!(RepoHostClass::SelfHosted.as_str(), "self_hosted"); + assert_eq!(RepoVisibility::Public.as_str(), "public"); + assert_eq!(RepoVisibility::Private.as_str(), "private"); + assert_eq!(RepoVisibility::Unknown.as_str(), "unknown"); + } + + #[test] + fn same_canonical_and_salt_is_deterministic() { + let salt = [7u8; 32]; + let a = compute("github.com/acme/api", &salt, RepoVisibility::Unknown); + let b = compute("github.com/acme/api", &salt, RepoVisibility::Unknown); + assert_eq!(a.repo_hash, b.repo_hash); + } + + #[test] + fn different_salt_diverges() { + let a = compute("github.com/acme/api", &[1u8; 32], RepoVisibility::Unknown); + let b = compute("github.com/acme/api", &[2u8; 32], RepoVisibility::Unknown); + assert_ne!(a.repo_hash, b.repo_hash); + } + + #[test] + fn different_repo_diverges_under_the_same_salt() { + let salt = [9u8; 32]; + let a = compute("github.com/acme/api", &salt, RepoVisibility::Unknown); + let b = compute("github.com/acme/other", &salt, RepoVisibility::Unknown); + assert_ne!(a.repo_hash, b.repo_hash); + } + + #[test] + fn repo_hash_is_hex() { + let facts = compute("github.com/acme/api", &[0u8; 32], RepoVisibility::Unknown); + assert_eq!(facts.repo_hash.len(), 64); // SHA-256 -> 32 bytes -> 64 hex chars + assert!(facts.repo_hash.chars().all(|c| c.is_ascii_hexdigit())); + } +} diff --git a/cli/core/src/telemetry/wire.rs b/cli/core/src/telemetry/wire.rs new file mode 100644 index 0000000..d858603 --- /dev/null +++ b/cli/core/src/telemetry/wire.rs @@ -0,0 +1,132 @@ +//! Wire types for the telemetry batch shipped to the cloud ingest endpoint. +//! +//! Deliberately **not** part of `/contract`: unlike an attestation, a telemetry +//! batch is never signed and never verified byte-for-byte across languages, so +//! it carries no `schemars` derive and no JSON Schema is generated for it. Serde +//! only. + +use serde::{Deserialize, Serialize}; +use sha2::{Digest, Sha256}; + +/// One flattened telemetry event, ready to serialize. +/// +/// The five base fields (`event`, `timestamp`, `dira_version`, `os`, `arch`) are +/// always present; everything else is `Option` + `skip_serializing_if`, so a +/// given event's JSON carries only the keys its variant actually populated — +/// see the round-trip tests in `event.rs`, which assert exactly that. +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +pub struct TelemetryEventWire { + pub event: String, + pub timestamp: String, + pub dira_version: String, + pub os: String, + pub arch: String, + + #[serde(default, skip_serializing_if = "Option::is_none")] + pub command: Option, + /// Milliseconds. Doubles as `CommandExecuted`'s wall time and + /// `DaemonStopped`'s uptime — both are "how long did this run" measured on + /// the same clock, so one field carries both rather than adding a second + /// duration-shaped column for a single variant. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub duration_ms: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub success: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub error_kind: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub repo_host_class: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub repo_visibility: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub repo_hash: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub telemetry_enabled: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub consent_source: Option, +} + +impl TelemetryEventWire { + /// The base shape every event carries; [`super::event::TelemetryEvent::into_wire`] + /// fills in whichever optional fields its variant populates. + pub(crate) fn base(event: &'static str, timestamp: String, dira_version: &str) -> Self { + Self { + event: event.to_string(), + timestamp, + dira_version: dira_version.to_string(), + os: std::env::consts::OS.to_string(), + arch: std::env::consts::ARCH.to_string(), + command: None, + duration_ms: None, + success: None, + error_kind: None, + repo_host_class: None, + repo_visibility: None, + repo_hash: None, + telemetry_enabled: None, + consent_source: None, + } + } +} + +/// A batch of telemetry events POSTed to the cloud ingest endpoint in one call. +#[derive(Debug, Clone, Serialize, Deserialize)] +#[serde(rename_all = "camelCase")] +pub struct TelemetryBatch { + /// Wire format version; `1` for this work package. + pub v: u32, + pub batch_id: String, + pub install_id: String, + pub generated_at: String, + pub events: Vec, +} + +/// A deterministic batch id for idempotent resend (D-0020): hex SHA-256 of +/// `"{install_id}:{first_id}:{last_id}"`. Re-sending the same cursor window +/// after a dropped ack produces the same id, so the cloud can de-dupe on it +/// instead of double-counting a retried POST. +pub fn batch_id(install_id: &str, first_id: &str, last_id: &str) -> String { + let mut hasher = Sha256::new(); + hasher.update(install_id.as_bytes()); + hasher.update(b":"); + hasher.update(first_id.as_bytes()); + hasher.update(b":"); + hasher.update(last_id.as_bytes()); + hex::encode(hasher.finalize()) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn batch_id_is_deterministic() { + let a = batch_id("inst1", "01A", "01Z"); + let b = batch_id("inst1", "01A", "01Z"); + assert_eq!(a, b); + } + + #[test] + fn batch_id_changes_with_any_input() { + let base = batch_id("inst1", "01A", "01Z"); + assert_ne!(base, batch_id("inst2", "01A", "01Z")); + assert_ne!(base, batch_id("inst1", "01B", "01Z")); + assert_ne!(base, batch_id("inst1", "01A", "01Y")); + } + + #[test] + fn base_wire_carries_only_the_five_required_fields() { + let wire = + TelemetryEventWire::base("cli_daemon_started", "2026-01-01T00:00:00Z".into(), "0.5.1"); + let json = serde_json::to_value(&wire).unwrap(); + let obj = json.as_object().unwrap(); + let keys: std::collections::BTreeSet<_> = obj.keys().cloned().collect(); + let expected: std::collections::BTreeSet<_> = + ["event", "timestamp", "diraVersion", "os", "arch"] + .into_iter() + .map(String::from) + .collect(); + assert_eq!(keys, expected); + } +} diff --git a/cli/dira/src/config_cmd.rs b/cli/dira/src/config_cmd.rs index 553d0aa..2f4056e 100644 --- a/cli/dira/src/config_cmd.rs +++ b/cli/dira/src/config_cmd.rs @@ -132,12 +132,38 @@ const KNOBS: &[Knob] = &[ kind: Kind::Enum(&["on", "off"]), help: "passive update-available notice after status/version/daemon status: on, off", }, + // `Config.telemetry.enabled` is a plain `bool` (see + // `dira_core::config::TelemetryKnobs`), exactly like `update.check` above — + // same bespoke `parse_and_validate` arm, same `get()` rendering, for the + // same reason (no `Deserialize`/`Serialize` impl maps "on"/"off" to a bare + // `bool` for free). + Knob { + key: "telemetry.enabled", + kind: Kind::Enum(&["on", "off"]), + help: "anonymous usage telemetry; off disables collection and sync entirely: on, off", + }, ]; fn knob(key: &str) -> Option<&'static Knob> { KNOBS.iter().find(|k| k.key == key) } +/// Whether `key`/`raw` is a `telemetry.enabled` set that `set()` would accept, +/// and if so, the resolved bool — for the caller to fire a +/// `cli_consent_recorded` telemetry event after a successful set. `None` for +/// any other key or an unparseable value (already rejected by `set` itself +/// before this would ever be consulted). +pub(crate) fn telemetry_enabled_value(key: &str, raw: &str) -> Option { + if key != "telemetry.enabled" { + return None; + } + match raw.trim().to_ascii_lowercase().as_str() { + "on" => Some(true), + "off" => Some(false), + _ => None, + } +} + /// The "Settable keys" help block, rendered from [`KNOBS`] so `dira config set /// --help` can never drift from what [`set`] actually validates: adding a knob /// to the table updates the help (and the error message) in one place. @@ -213,12 +239,13 @@ fn render_json_scalar(v: &serde_json::Value) -> String { } /// Like [`render_json_scalar`], but lets a dotted-key lookup render in the -/// vocabulary its `set` counterpart accepts. Only `update.check` needs this: -/// its backing field is a plain `bool` (see `UpdateKnobs`), but `set` speaks -/// "on"/"off" like the enum knobs do, so `get` echoes the same words rather -/// than leaking the wire representation. +/// vocabulary its `set` counterpart accepts. `update.check` and +/// `telemetry.enabled` need this: their backing fields are plain `bool`s (see +/// `UpdateKnobs`/`TelemetryKnobs`), but `set` speaks "on"/"off" like the enum +/// knobs do, so `get` echoes the same words rather than leaking the wire +/// representation. fn render_scalar(key: &str, v: &serde_json::Value) -> String { - if key == "update.check" { + if key == "update.check" || key == "telemetry.enabled" { if let Some(b) = v.as_bool() { return if b { "on" } else { "off" }.to_string(); } @@ -348,6 +375,13 @@ fn parse_and_validate(config: &Config, knob: &Knob, raw: &str) -> Result Ok(value(false)), _ => bail!("update.check must be one of: on, off (got `{raw}`)"), }, + // Same "on"/"off" -> real `bool` bridge as `update.check`, onto + // `Config.telemetry.enabled`. + "telemetry.enabled" => match raw.trim().to_ascii_lowercase().as_str() { + "on" => Ok(value(true)), + "off" => Ok(value(false)), + _ => bail!("telemetry.enabled must be one of: on, off (got `{raw}`)"), + }, key => match knob.kind { Kind::U64 => { let n: u64 = raw @@ -439,6 +473,20 @@ mod tests { } } + #[test] + fn telemetry_enabled_value_resolves_on_off_and_nothing_else() { + assert_eq!( + telemetry_enabled_value("telemetry.enabled", "on"), + Some(true) + ); + assert_eq!( + telemetry_enabled_value("telemetry.enabled", "OFF"), + Some(false) + ); + assert_eq!(telemetry_enabled_value("telemetry.enabled", "maybe"), None); + assert_eq!(telemetry_enabled_value("idle_seconds", "on"), None); + } + #[test] fn rejects_unknown_key() { let err = set(&cfg(), "socket_path", "/tmp/x.sock").unwrap_err(); @@ -641,6 +689,47 @@ mod tests { assert_eq!(render_scalar("update.check", got), "off"); } + #[test] + fn telemetry_enabled_accepts_only_on_off() { + let knob = knob("telemetry.enabled").unwrap(); + assert!(parse_and_validate(&cfg(), knob, "on").is_ok()); + assert!(parse_and_validate(&cfg(), knob, "OFF").is_ok()); // case-folded + assert!(parse_and_validate(&cfg(), knob, "true").is_err()); + assert!(parse_and_validate(&cfg(), knob, "maybe").is_err()); + } + + #[test] + fn telemetry_enabled_writes_a_real_toml_bool_not_a_string() { + let knob = knob("telemetry.enabled").unwrap(); + let item = parse_and_validate(&cfg(), knob, "off").unwrap(); + assert_eq!(item.as_bool(), Some(false)); + let item = parse_and_validate(&cfg(), knob, "on").unwrap(); + assert_eq!(item.as_bool(), Some(true)); + } + + #[test] + fn telemetry_enabled_round_trips_through_set_and_get() { + let original = "idle_seconds = 300\n"; + let mut doc: DocumentMut = original.parse().unwrap(); + let item = parse_and_validate(&cfg(), knob("telemetry.enabled").unwrap(), "off").unwrap(); + assign(&mut doc, "telemetry.enabled", item); + let out = doc.to_string(); + assert!(out.contains("[telemetry]")); + assert!(out.contains("enabled = false")); + + use figment::providers::Format as _; + let resolved: Config = + figment::Figment::from(figment::providers::Serialized::defaults(Config::default())) + .merge(figment::providers::Toml::string(&out)) + .extract() + .unwrap(); + assert!(!resolved.telemetry.enabled); + + let v = serde_json::to_value(&resolved).unwrap(); + let got = v.get("telemetry").and_then(|t| t.get("enabled")).unwrap(); + assert_eq!(render_scalar("telemetry.enabled", got), "off"); + } + #[test] fn dotted_key_writes_a_nested_table_and_preserves_the_rest() { let original = "# my config\nidle_seconds = 300\n"; diff --git a/cli/dira/src/device.rs b/cli/dira/src/device.rs index 2d998ab..484f52a 100644 --- a/cli/dira/src/device.rs +++ b/cli/dira/src/device.rs @@ -91,8 +91,21 @@ pub async fn link(config: &Config, code: Option, label: Option) // device stays unlinked until the cloud hands back an authoritative id. let client_nonce = Ulid::generate().to_string(); + // Best-effort telemetry install id, attached purely as context for the + // "device linked" alias the cloud performs at claim time — never required: + // a link must not fail, or even slow down noticeably, because telemetry + // couldn't be reached. Skipped outright when the CLI's own gate would + // refuse to emit anyway (CI, a dev build, DO_NOT_TRACK, ...). + let install_id = fetch_install_id(config).await; + let url = format!("{}/api/v1/devices/claim", base.trim_end_matches('/')); - let body = claim_request_body(&code, &key.public_base64(), label.as_deref(), &client_nonce); + let body = claim_request_body( + &code, + &key.public_base64(), + label.as_deref(), + &client_nonce, + install_id.as_deref(), + ); let client = reqwest::Client::new(); let resp = client @@ -128,20 +141,56 @@ pub async fn link(config: &Config, code: Option, label: Option) /// without a network or store. Carries the link `code`, our `ed25519Pubkey`, an /// optional `label`, and a `clientNonce` for idempotency — but **never** a /// client-chosen `deviceId`: the cloud assigns the identity (see the module docs). +/// +/// `install_id`, when present, is this machine's telemetry install id — pure +/// best-effort context for the cloud's own device/telemetry alias at claim +/// time (see `docs/TELEMETRY.md`'s "linking a device" section), never a +/// second identity the client asserts. fn claim_request_body( code: &str, pubkey_b64: &str, label: Option<&str>, client_nonce: &str, + install_id: Option<&str>, ) -> serde_json::Value { serde_json::json!({ "code": code, "ed25519Pubkey": pubkey_b64, "label": label, "clientNonce": client_nonce, + "installId": install_id, }) } +/// Best-effort fetch of this daemon's telemetry install id, for +/// [`claim_request_body`]'s `installId`. Skipped outright when the CLI's own +/// telemetry gate would refuse to emit (CI, a dev build, `DO_NOT_TRACK`, +/// `DIRA_TELEMETRY_ENABLED=0`, or the knob off) — no point asking the daemon +/// for an id this process would never report anyway. Any failure — no +/// daemon, a timeout, an unexpected response — collapses to `None`: a device +/// link must never fail, or even meaningfully slow down, because telemetry +/// couldn't be reached. +async fn fetch_install_id(config: &Config) -> Option { + if !crate::telemetry::TelemetryGate::from_process(config).allows_emission() { + return None; + } + let resp = tokio::time::timeout( + crate::telemetry::TOTAL_BUDGET, + client::send_with_budget( + &config.socket_path, + &Request::TelemetryInstallId, + crate::telemetry::CONNECT_BUDGET, + ), + ) + .await + .ok()? + .ok()?; + match resp { + Response::TelemetryInstallId { install_id } if !install_id.is_empty() => Some(install_id), + _ => None, + } +} + /// Extract the cloud-assigned `deviceId` from a successful claim response body. /// `None` when the body is missing/blank the field — the caller treats that as a /// failed link rather than inventing an id. @@ -808,11 +857,18 @@ mod tests { fn claim_body_carries_no_client_chosen_device_id() { // The claim request must NOT smuggle a client-chosen device id under any // name. Identity is server-assigned; we only send a clearly-named nonce. - let body = claim_request_body("CODE-123", "PUBKEYb64", Some("laptop"), "01NONCE"); + let body = claim_request_body( + "CODE-123", + "PUBKEYb64", + Some("laptop"), + "01NONCE", + Some("01INSTALLID"), + ); assert_eq!(body["code"], "CODE-123"); assert_eq!(body["ed25519Pubkey"], "PUBKEYb64"); assert_eq!(body["label"], "laptop"); assert_eq!(body["clientNonce"], "01NONCE"); + assert_eq!(body["installId"], "01INSTALLID"); // No `deviceId` field at all — the cloud owns the id. assert!( body.get("deviceId").is_none(), @@ -822,11 +878,20 @@ mod tests { #[test] fn claim_body_omits_label_as_null_when_absent() { - let body = claim_request_body("CODE", "PK", None, "01NONCE"); + let body = claim_request_body("CODE", "PK", None, "01NONCE", None); assert!(body["label"].is_null()); assert!(body.get("deviceId").is_none()); } + /// A link must never fail — or even carry a required field — because + /// telemetry couldn't be reached: `installId` is optional and null when + /// there is none to attach. + #[test] + fn claim_body_omits_install_id_as_null_when_absent() { + let body = claim_request_body("CODE", "PK", None, "01NONCE", None); + assert!(body["installId"].is_null()); + } + #[test] fn device_id_comes_only_from_the_cloud_response() { // The server-assigned id is taken verbatim from the response. diff --git a/cli/dira/src/main.rs b/cli/dira/src/main.rs index 4f4ad6b..32dfeb2 100644 --- a/cli/dira/src/main.rs +++ b/cli/dira/src/main.rs @@ -12,6 +12,7 @@ mod hook_health; mod init; mod onboard; mod render; +mod telemetry; #[cfg(test)] mod test_support; mod theme; @@ -139,6 +140,9 @@ Examples: /// Knowledge sync tier: off, metadata, or full. Skips the consent prompt. #[arg(long, value_name = "TIER")] knowledge: Option, + /// Anonymous telemetry: on or off. Skips the consent prompt. + #[arg(long, value_name = "ON|OFF")] + telemetry: Option, }, /// Today's summary: engaged / agent / compute + the unbilled value. #[command( @@ -945,13 +949,100 @@ fn zavet_status_is_cwd_scoped(command: &Command) -> bool { ) } +/// This `dira`'s top-level command name, for telemetry's `command` field — +/// coarse, never a sub-action (`Config { action: Set { .. } }` is `"config"`, +/// not `"config set"`), and never raw argv. A non-wildcard match so adding a +/// new top-level [`Command`] variant fails the build here rather than +/// silently going unreported. +fn command_name(command: &Command) -> &'static str { + match command { + Command::Onboard { .. } => "onboard", + Command::Status { .. } => "status", + Command::Watch { .. } => "watch", + Command::Start { .. } => "start", + Command::Stop { .. } => "stop", + Command::Sessions => "sessions", + Command::Log { .. } => "log", + Command::Report { .. } => "report", + Command::Init { .. } => "init", + Command::Daemon { .. } => "daemon", + Command::Config { .. } => "config", + Command::Device { .. } => "device", + Command::Hook { .. } => "hook", + Command::Nuke { .. } => "nuke", + Command::Completions { .. } => "completions", + Command::Version => "version", + Command::Doctor { .. } => "doctor", + Command::Update { .. } => "update", + Command::Zavet { .. } => "zavet", + } +} + #[tokio::main] async fn main() -> Result<()> { // Before anything prints: legacy Windows consoles need to be told to // interpret ANSI escapes, or every painted line leaks its SGR bytes. theme::enable_ansi(); let cli = Cli::parse(); - let config = Config::load().map_err(|e| anyhow::anyhow!("config: {e}"))?; + let config = match Config::load() { + Ok(c) => c, + Err(e) => return Err(anyhow::anyhow!("config: {e}")), + }; + // Best-effort and read-only (see its own doc): never in CI, a dev build, + // a disabled knob, or a non-interactive session, and never twice. + telemetry::maybe_show_first_run_notice(&config); + let name = command_name(&cli.command); + let started = std::time::Instant::now(); + let result = run(cli, config.clone()).await; + // Every exit-path inside `run` that calls `std::process::exit` directly + // (skewed-daemon hints, `dira doctor`'s own exit code, `dira nuke`'s two + // direct exits) still bypasses this recording entirely — see the doc + // comment on `run` for the full list. That is accepted: those paths + // terminate the process before control ever returns here. + // + // The generic daemon-request path's former `std::process::exit(1)` is + // NOT on that list any more: it now returns `Err(SilentExit)` instead, so + // this call sees (and reports) it like any other failure. `SilentExit` + // carries no message of its own — the failure was already printed by + // `render::print_with`/`print_json` before `run` returned — so once + // telemetry has been recorded, exit silently rather than let `anyhow`'s + // default `Debug` formatting print a redundant second line. + telemetry::record_command(&config, name, started.elapsed(), &result).await; + if let Err(e) = &result { + if e.downcast_ref::().is_some() { + std::process::exit(1); + } + } + result +} + +/// The former body of `main`. Every early `return` below is unchanged from +/// when it lived there — this split exists solely so `main` can wrap the +/// whole run with timing + telemetry (see [`command_name`]) without +/// disturbing this function's control flow. +/// +/// A handful of paths inside here call `std::process::exit` directly rather +/// than returning a `Result`, and so bypass `main`'s telemetry recording +/// entirely: +/// - `dira daemon status` when no daemon answers (exit 1). +/// - `dira hook ` under `dira doctor --probe`, on a transport +/// failure (`HOOK_PROBE_FAILURE_EXIT`). +/// - `dira doctor` always exits via its own computed code, success or not. +/// - the generic daemon-request path's skewed-daemon hint (a version-skew +/// response gets its own message printed, then exits immediately). +/// - `dira nuke`'s two direct exits (daemon unreachable; a failed response +/// render). +/// +/// None of these are worth restructuring just to be observed: each already +/// terminates the process on its own terms, and forcing a `Result` return +/// through them would change behavior this refactor is not meant to touch. +/// +/// The generic daemon-request path's *other* failure — `if !ok { .. }` when +/// rendering the response reports failure, reached by `sessions`/`start`/ +/// `stop`/`log`/`report`/the `zavet` subactions — is deliberately NOT on this +/// list: it returns `Err(telemetry::SilentExit)` instead of exiting directly, +/// so `main` still records it before exiting (see `main`'s own doc comment). +async fn run(cli: Cli, config: Config) -> Result<()> { let cwd = std::env::current_dir() .ok() .map(|p| p.display().to_string()); @@ -965,6 +1056,7 @@ async fn main() -> Result<()> { no_zavet, harness, knowledge, + telemetry, } => { // Resolve harness aliases up front so a typo fails before any // step runs, rather than five steps in. @@ -987,6 +1079,10 @@ async fn main() -> Result<()> { .as_deref() .map(onboard::parse_knowledge) .transpose()?; + let telemetry = telemetry + .as_deref() + .map(onboard::parse_telemetry) + .transpose()?; return onboard::run( &config, onboard::Options { @@ -996,6 +1092,7 @@ async fn main() -> Result<()> { no_zavet: *no_zavet, harness: ids, knowledge, + telemetry, }, ) .await; @@ -1057,7 +1154,27 @@ async fn main() -> Result<()> { Command::Config { action } => { return match action { ConfigAction::Get { key } => config_cmd::get(&config, key.as_deref()), - ConfigAction::Set { key, value } => config_cmd::set(&config, key, value), + ConfigAction::Set { key, value } => { + let outcome = config_cmd::set(&config, key, value); + // Fire a consent-recorded event on a successful + // `telemetry.enabled` set — the one config knob whose + // change is itself a telemetry-relevant fact. `config` + // here is the process-start snapshot, so the gate is + // deliberately not re-derived from it; see + // `telemetry::record_consent`'s doc for why the write + // (on OR off) is reported regardless. + if outcome.is_ok() { + if let Some(enabled) = config_cmd::telemetry_enabled_value(key, value) { + telemetry::record_consent( + &config, + enabled, + dira_core::telemetry::event::ConsentSource::ConfigSet, + ) + .await; + } + } + outcome + } ConfigAction::Path => config_cmd::path(), }; } @@ -1370,7 +1487,13 @@ async fn main() -> Result<()> { } } if !ok { - std::process::exit(1); + // The failure is already on stderr (`print_with`/`print_json` just + // printed it) — returning `SilentExit` rather than exiting here + // directly lets it flow back through `main`'s `record_command` call + // before the process actually exits (see `main`'s and `run`'s doc + // comments), instead of skipping telemetry the way a bare + // `std::process::exit(1)` used to. + return Err(telemetry::SilentExit.into()); } Ok(()) } diff --git a/cli/dira/src/onboard/detect.rs b/cli/dira/src/onboard/detect.rs index ad1f331..793a3de 100644 --- a/cli/dira/src/onboard/detect.rs +++ b/cli/dira/src/onboard/detect.rs @@ -115,6 +115,8 @@ pub(crate) struct State { pub device_linked: bool, /// The currently resolved knowledge sync tier. pub knowledge: dira_core::config::KnowledgeSyncMode, + /// The currently resolved telemetry consent (`Config.telemetry.enabled`). + pub telemetry_enabled: bool, } impl State { @@ -245,6 +247,7 @@ pub(crate) async fn run(config: &dira_core::Config, cwd: &Path) -> State { zavet_installed: crate::zavet_install::plugin_root_offline().is_some(), device_linked: device_linked(config).await, knowledge: config.sync.knowledge, + telemetry_enabled: config.telemetry.enabled, } } @@ -382,6 +385,7 @@ mod tests { zavet_installed: false, device_linked: false, knowledge: dira_core::config::KnowledgeSyncMode::Off, + telemetry_enabled: true, }; let ids: Vec<_> = state.wirable().iter().map(|h| h.probe.id).collect(); @@ -402,6 +406,7 @@ mod tests { zavet_installed: false, device_linked: false, knowledge: dira_core::config::KnowledgeSyncMode::Off, + telemetry_enabled: true, }; assert!(base.daemon_running()); assert!(!base.supervised(), "a pidfile daemon dies with the session"); diff --git a/cli/dira/src/onboard/mod.rs b/cli/dira/src/onboard/mod.rs index fd08224..a5c53f6 100644 --- a/cli/dira/src/onboard/mod.rs +++ b/cli/dira/src/onboard/mod.rs @@ -82,6 +82,13 @@ pub(crate) struct Options { pub harness: Vec, /// The tier from `--knowledge`. `None` means ask. pub knowledge: Option, + /// The choice from an explicit `--telemetry `. `None` means ask. + /// + /// Parsed from argv by [`parse_telemetry`], called from the `Onboard` + /// arm of `main.rs`'s `run()` — the clap definition and the + /// `Options { .. }` construction both live there, alongside the + /// equivalent `--knowledge`/`parse_knowledge` wiring. + pub telemetry: Option, } impl Options { @@ -99,6 +106,13 @@ impl Options { if self.yes && self.knowledge.is_none() { self.knowledge = Some(KnowledgeSyncMode::Full); } + // Same fold, for the same reason: without it, `--print --yes` could + // only say "leave telemetry on (after asking)" for a run that will + // never ask. `true` is telemetry's own default answer, mirroring + // `full` above. + if self.yes && self.telemetry.is_none() { + self.telemetry = Some(true); + } } } @@ -168,6 +182,42 @@ pub(crate) async fn run(config: &Config, mut opts: Options) -> Result<()> { crate::config_cmd::set_quiet(config, "sync.knowledge", raw) }), )); + // After knowledge, not before: the two consent prompts read as a pair in + // the transcript, and `telemetry.enabled` is (like `sync.knowledge`) + // config the daemon reads at startup — the same restart-ordering + // reasoning applies. + // + // The consent source follows the same DIRASH-0030-flavored fold every + // other onboarding choice does: an explicit `--telemetry ` and a + // `--yes`-resolved default both arrive here as `Some(_)` (see + // `Options::resolve_defaults`) and are indistinguishable from each other + // by the time this step runs — both are "a non-interactive flag decided + // this", which is exactly what `ConsentSource::YesFlag` means. Only a + // genuinely unset `opts.telemetry` means the interactive prompt ran. + let telemetry_source = if opts.telemetry.is_some() { + dira_core::telemetry::event::ConsentSource::YesFlag + } else { + dira_core::telemetry::event::ConsentSource::Prompt + }; + results.push(( + "telemetry".into(), + steps::telemetry( + &state, + &opts, + ui.as_mut(), + &|enabled: bool| { + crate::config_cmd::set_quiet( + config, + "telemetry.enabled", + if enabled { "on" } else { "off" }, + ) + }, + |enabled: bool| async move { + crate::telemetry::record_consent(config, enabled, telemetry_source).await; + }, + ) + .await, + )); print_summary(&results); print_open_items(&state, &results); @@ -229,6 +279,13 @@ fn print_plan(state: &detect::State, opts: &Options) { None => "full (after asking)", }; println!(" · set knowledge sync to {tier}"); + + match opts.telemetry { + Some(true) => println!(" · leave anonymous telemetry on"), + Some(false) => println!(" · turn anonymous telemetry off"), + None => println!(" · leave anonymous telemetry on (after asking)"), + } + println!("\nNothing was changed."); } @@ -328,6 +385,19 @@ pub(crate) fn parse_knowledge(raw: &str) -> Result { } } +/// Parse `--telemetry`. Mirrors [`parse_knowledge`]'s shape; `main.rs`'s own +/// inline `match` used to duplicate this exact logic before the flag was +/// threaded through to [`Options::telemetry`]. +pub(crate) fn parse_telemetry(raw: &str) -> Result { + match raw.trim().to_ascii_lowercase().as_str() { + "on" => Ok(true), + "off" => Ok(false), + other => Err(anyhow::anyhow!( + "--telemetry must be one of: on, off (got `{other}`)" + )), + } +} + #[cfg(test)] mod tests { use super::*; @@ -343,6 +413,14 @@ mod tests { assert!(parse_knowledge("everything").is_err()); } + #[test] + fn telemetry_parses_on_and_off_and_rejects_others() { + assert!(parse_telemetry("on").unwrap()); + assert!(!parse_telemetry("off").unwrap()); + assert!(!parse_telemetry(" OFF ").unwrap()); + assert!(parse_telemetry("maybe").is_err()); + } + /// The four outcomes must be visually distinct without colour, because /// piped output loses it — the same reason `dira doctor` uses four shapes. #[test] @@ -395,4 +473,39 @@ mod tests { opts.resolve_defaults(); assert_eq!(opts.knowledge, None); } + + /// `--yes` must resolve telemetry to `Some(true)` up front too, so + /// `--print --yes` can state the concrete answer instead of "(after + /// asking)" for a run that will never ask. + #[test] + fn yes_resolves_telemetry_to_on_up_front() { + let mut opts = Options { + yes: true, + ..Options::default() + }; + assert_eq!(opts.telemetry, None, "unset before resolution"); + opts.resolve_defaults(); + assert_eq!(opts.telemetry, Some(true)); + } + + /// An explicit `--telemetry off` survives `--yes`: the more specific flag + /// wins, exactly like `--knowledge` does. + #[test] + fn an_explicit_telemetry_choice_is_not_overridden_by_yes() { + let mut opts = Options { + yes: true, + telemetry: Some(false), + ..Options::default() + }; + opts.resolve_defaults(); + assert_eq!(opts.telemetry, Some(false)); + } + + /// Without `--yes` telemetry stays unset, so the step asks. + #[test] + fn without_yes_telemetry_is_left_to_the_prompt() { + let mut opts = Options::default(); + opts.resolve_defaults(); + assert_eq!(opts.telemetry, None); + } } diff --git a/cli/dira/src/onboard/steps.rs b/cli/dira/src/onboard/steps.rs index edc4918..f3bd88a 100644 --- a/cli/dira/src/onboard/steps.rs +++ b/cli/dira/src/onboard/steps.rs @@ -305,6 +305,93 @@ pub(crate) fn knowledge( } } +/// The consent text for the telemetry prompt. +/// +/// Named, and asserted on by a test, for the same reason +/// [`KNOWLEDGE_DISCLOSURE`] is: this is the only consent UX for the channel, +/// so if this sentence is wrong or missing, nothing else catches it. +pub(crate) const TELEMETRY_DISCLOSURE: &str = "\ +Anonymous product analytics, on by default — its own channel, separate from +knowledge sync and billing consent. + sent command name, duration, success/failure kind; inside a repo: host + type (github/gitlab/bitbucket/self-hosted), public/private when + determinable (checked with the repo's own host — github/gitlab — + never with Dira), and a one-way salted hash of the repo identity + never repo names, git identity or email, file paths, command arguments, + error text +Tagged by a random install id, not your device key. Sent to Dira's EU +analytics (PostHog EU, via the Dira cloud); once this device is linked, +later usage may be associated with your workspace account. Silent in dev +builds and CI. +Turn off anytime: `dira config set telemetry.enabled false`, +DIRA_TELEMETRY_ENABLED=0, or DO_NOT_TRACK=1."; + +/// Step 8 — the telemetry consent. +/// +/// Last of the mutating steps: same reasoning as [`knowledge`] for running +/// after the daemon step (a restart it may have just done should not race +/// this write), and after knowledge itself so the two consent prompts read +/// as a pair in the transcript rather than being split by the zavet steps. +/// +/// `write_enabled` is injected for the same reason [`knowledge`]'s +/// `write_tier` is: `config_cmd::set_quiet` resolves its target via +/// `project_dirs()` regardless of the `Config` it is handed, so an +/// in-process unit test calling it directly would write the developer's real +/// `config.toml`. `mod::run` passes a closure over the real `set_quiet` +/// (translating the bool to the `"on"`/`"off"` spelling `telemetry.enabled` +/// expects); tests pass a recording stub. +/// +/// `record_consent` is injected for the same client-agnosticism reason: the +/// real one (bound by `mod::run`) is `telemetry::record_consent`, a fire- +/// and-forget send over the control socket, which no unit test in this +/// module may perform. It is only ever called after `write_enabled` +/// succeeds — a `cli_consent_recorded` event must report what was actually +/// persisted, never an attempted write that failed. +pub(crate) async fn telemetry( + state: &State, + opts: &Options, + ui: &mut dyn Ui, + write_enabled: &dyn Fn(bool) -> anyhow::Result, + record_consent: F, +) -> StepOutcome +where + F: FnOnce(bool) -> Fut, + Fut: std::future::Future, +{ + // Unconditional, and above the `opts.telemetry` match on purpose — the + // same DIRASH-0030 rule `knowledge` follows: every consent path + // (interactive, `--yes`, an explicit `--telemetry` flag) has to see + // exactly what is sent before this step acts, not just the one that + // stops to ask. + ui.say(TELEMETRY_DISCLOSURE); + + let want = match opts.telemetry { + Some(explicit) => explicit, + None => ui.confirm("Keep anonymous telemetry on?", true), + }; + + if state.telemetry_enabled == want { + return StepOutcome::AlreadyDone(format!( + "telemetry already {}", + if want { "on" } else { "off" } + )); + } + + match write_enabled(want) { + Ok(_) => { + record_consent(want).await; + if want { + StepOutcome::Done( + "telemetry stays on — anonymous usage analytics will be sent".into(), + ) + } else { + StepOutcome::Done("telemetry turned off — nothing will be sent".into()) + } + } + Err(e) => StepOutcome::Failed(format!("could not set telemetry.enabled: {e}")), + } +} + /// Step 5 — install the zavet plugin. pub(crate) fn zavet_plugin(state: &State, opts: &Options, ui: &mut dyn Ui) -> StepOutcome { if opts.no_zavet { @@ -451,6 +538,7 @@ mod tests { zavet_installed: false, device_linked: false, knowledge: KnowledgeSyncMode::Off, + telemetry_enabled: true, } } @@ -649,6 +737,188 @@ mod tests { } } + /// A `set_quiet`-style stand-in for `telemetry()`'s `write_enabled` + /// closure — same `RecordingWriter` shape, `bool` in place of the raw + /// TOML string, and the same DIRASH-0030 justification for why tests + /// never call `config_cmd::set_quiet` directly. + struct BoolRecorder(std::cell::RefCell>); + + impl BoolRecorder { + fn new() -> Self { + Self(std::cell::RefCell::new(Vec::new())) + } + + fn calls(&self) -> Vec { + self.0.borrow().clone() + } + + fn as_fn(&self) -> impl Fn(bool) -> anyhow::Result + '_ { + move |enabled: bool| { + self.0.borrow_mut().push(enabled); + Ok(PathBuf::from("/dev/null/recording-writer-stub")) + } + } + } + + /// The disclosure has to name the content, not just the toggle. Same + /// justification as `the_knowledge_prompt_names_what_it_sends`: this is + /// the only place the user is told what telemetry sends. + #[tokio::test] + async fn the_telemetry_prompt_names_what_it_sends() { + let mut ui = ScriptedUi::new(); + let recorder = BoolRecorder::new(); + let _ = telemetry( + &state(), + &Options::default(), + &mut ui, + &recorder.as_fn(), + |_| async {}, + ) + .await; + let t = ui.transcript(); + for phrase in [ + "command", + "public", + "private", + "hash", + "DO_NOT_TRACK", + "telemetry.enabled", + // DIRASH-0034: the disclosure must name that visibility is + // checked with the repo's own host, never with Dira — not just + // that a `public`/`private` value is sent. + "never with Dira", + ] { + assert!( + t.contains(phrase), + "consent text must mention {phrase:?}; got:\n{t}" + ); + } + } + + /// An explicit `--telemetry` answer is not re-asked. + #[tokio::test] + async fn an_explicit_telemetry_flag_skips_the_prompt() { + let mut ui = ScriptedUi::new(); + let opts = Options { + telemetry: Some(false), + ..Options::default() + }; + let recorder = BoolRecorder::new(); + let _ = telemetry(&state(), &opts, &mut ui, &recorder.as_fn(), |_| async {}).await; + assert!( + !ui.transcript().contains("Keep anonymous telemetry on?"), + "an explicit --telemetry must not re-ask" + ); + } + + /// Telemetry is on by default (`state().telemetry_enabled == true`), so + /// accepting the default answer must not write `config.toml` at all — + /// only a *change* from the effective value is worth persisting. + #[tokio::test] + async fn keeping_the_default_on_does_not_write() { + let mut ui = ScriptedUi::new(); + let recorder = BoolRecorder::new(); + let outcome = telemetry( + &state(), + &Options::default(), + &mut ui, + &recorder.as_fn(), + |_| async {}, + ) + .await; + assert!( + recorder.calls().is_empty(), + "must not write the default back" + ); + assert!( + matches!(&outcome, StepOutcome::AlreadyDone(m) if m.contains("already on")), + "got {outcome:?}" + ); + } + + /// Declining writes `false` and says so plainly — no partial "some data + /// still leaves" hedging. + #[tokio::test] + async fn declining_writes_false_and_says_nothing_will_be_sent() { + let mut ui = ScriptedUi::new().with_confirms(&[false]); + let recorder = BoolRecorder::new(); + let outcome = telemetry( + &state(), + &Options::default(), + &mut ui, + &recorder.as_fn(), + |_| async {}, + ) + .await; + assert_eq!(recorder.calls(), vec![false]); + match &outcome { + StepOutcome::Done(m) => assert!(m.contains("nothing will be sent"), "got {m}"), + other => panic!("expected Done, got {other:?}"), + } + } + + /// `record_consent` must fire exactly once, and only after a successful + /// write — never on an `AlreadyDone` no-op, and never before the write is + /// confirmed to have landed. + #[tokio::test] + async fn record_consent_fires_once_and_only_after_a_successful_write() { + let recorded = std::cell::RefCell::new(Vec::new()); + let mut ui = ScriptedUi::new().with_confirms(&[false]); + let recorder = BoolRecorder::new(); + let outcome = telemetry( + &state(), + &Options::default(), + &mut ui, + &recorder.as_fn(), + |enabled: bool| { + recorded.borrow_mut().push(enabled); + async {} + }, + ) + .await; + assert!(matches!(outcome, StepOutcome::Done(_))); + assert_eq!(recorded.into_inner(), vec![false]); + + // Unchanged (already on): neither the write nor the consent event + // fires. + let recorded_noop = std::cell::RefCell::new(Vec::new()); + let mut ui = ScriptedUi::new(); + let recorder = BoolRecorder::new(); + let outcome = telemetry( + &state(), + &Options::default(), + &mut ui, + &recorder.as_fn(), + |enabled: bool| { + recorded_noop.borrow_mut().push(enabled); + async {} + }, + ) + .await; + assert!(matches!(outcome, StepOutcome::AlreadyDone(_))); + assert!(recorded_noop.into_inner().is_empty()); + } + + /// A `--yes`-shaped run (after `Options::resolve_defaults` folds it to + /// `Some(true)`) must still show the disclosure, per the same rule the + /// knowledge step's `a_yes_shaped_run_still_shows_the_disclosure` pins. + #[tokio::test] + async fn a_yes_shaped_telemetry_run_still_shows_the_disclosure() { + let mut ui = ScriptedUi::new(); + let opts = Options { + telemetry: Some(true), + ..Options::default() + }; + let recorder = BoolRecorder::new(); + let _ = telemetry(&state(), &opts, &mut ui, &recorder.as_fn(), |_| async {}).await; + let t = ui.transcript(); + assert!(t.contains("Anonymous product analytics"), "got:\n{t}"); + assert!( + !t.contains("Keep anonymous telemetry on?"), + "an already-resolved answer must still not re-ask" + ); + } + /// Empty input means skip, and the skip must be first-class: the run /// continues and the reason names the command to run later. This is the /// only step that needs something from outside the terminal, so it is the diff --git a/cli/dira/src/render.rs b/cli/dira/src/render.rs index ee0250e..f78ff4a 100644 --- a/cli/dira/src/render.rs +++ b/cli/dira/src/render.rs @@ -280,6 +280,13 @@ pub fn print_with(resp: &Response, opts: RowOpts) -> bool { eprintln!("unexpected capture-probe response outside `dira doctor --probe`"); false } + // Fetched directly by `device::fetch_install_id`, which matches on it + // itself and never routes it through the generic printer. Reaching + // here means a request was built somewhere else by mistake. + Response::TelemetryInstallId { .. } => { + eprintln!("unexpected telemetry-install-id response outside `dira device link`"); + false + } } } diff --git a/cli/dira/src/telemetry.rs b/cli/dira/src/telemetry.rs new file mode 100644 index 0000000..f9f52a6 --- /dev/null +++ b/cli/dira/src/telemetry.rs @@ -0,0 +1,572 @@ +//! CLI-side telemetry instrumentation: the gate, the fire-and-forget senders, +//! and the first-run notice. +//! +//! This module owns exactly one thing the daemon's `telemetry_sync` does not: +//! the decision about whether an event may be emitted **from this process** +//! at all. Everything past that gate is a best-effort, budget-capped send to +//! the daemon over the local control socket — see [`send_fire_and_forget`]. +//! The daemon re-checks its own `[telemetry] enabled` knob independently +//! (`dirad`'s `telemetry_sync::ingest`), so a version skew or a stale gate +//! here can never let an event through that the daemon itself would refuse. +//! +//! See `docs/TELEMETRY.md` for the user-facing contract this implements. + +use crate::client; +use dira_core::protocol::Request; +use dira_core::telemetry::event::{ConsentSource, ErrorKind, TelemetryEvent}; +use dira_core::telemetry::NOTICE_MARKER_FILE; +use dira_core::Config; +use std::env; +use std::io::IsTerminal; +use std::time::Duration; + +/// Total wall-clock a telemetry send may spend end to end. Short and +/// unconditional: nothing about `dira`'s own responsiveness may ever depend +/// on the daemon or the network being fast, or even reachable. +/// +/// `pub(crate)` so [`crate::device::fetch_install_id`]'s best-effort socket +/// round-trip shares this exact budget instead of carrying its own copy of +/// the same two literals. +pub(crate) const TOTAL_BUDGET: Duration = Duration::from_millis(150); +/// How long the connect itself may retry a busy endpoint — strictly inside +/// [`TOTAL_BUDGET`], mirroring the hook shim's `HOOK_CONNECT_BUDGET`/ +/// `HOOK_TOTAL_BUDGET` split in `main.rs`. See [`TOTAL_BUDGET`] for why this +/// is `pub(crate)`. +pub(crate) const CONNECT_BUDGET: Duration = Duration::from_millis(100); + +/// Sentinel error for the generic daemon-response path at the bottom of +/// `main::run`'s big match: `render::print_with`/`print_json` has already +/// rendered the failure (`error: {message}` on stderr) by the time this is +/// constructed, so it carries no message of its own — printing one here would +/// repeat what the user already saw. `main`'s thin wrapper recognizes it +/// *after* `record_command` has fired and exits 1 silently instead of letting +/// `anyhow`'s default `Debug` formatting land a second, redundant line on top. +/// +/// Its whole reason to exist is that the path it replaces used to call +/// `std::process::exit(1)` directly, which returned control to the OS before +/// `main` ever got to record telemetry for the command — see +/// [`classify_error`], which maps this straight to [`ErrorKind::DaemonError`] +/// without inspecting any text. +#[derive(Debug)] +pub(crate) struct SilentExit; + +impl std::fmt::Display for SilentExit { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + write!(f, "command failed (already reported)") + } +} + +impl std::error::Error for SilentExit {} + +/// Whether telemetry may be emitted **from this process**, folding together +/// every kill switch `docs/TELEMETRY.md` documents plus the two the docs +/// promise but WP1/WP2 could not yet make true: dev builds and CI. +/// +/// Deliberately carries no TTY check — emission is not a display concern +/// (contrast [`maybe_show_first_run_notice`], which is). +#[derive(Debug, Clone, Copy, Default)] +pub(crate) struct TelemetryGate { + /// `CI` is set to anything. + ci: bool, + /// `DO_NOT_TRACK` is set and truthy — the cross-tool convention. + do_not_track: bool, + /// `DIRA_TELEMETRY_ENABLED` is set and falsy (`"0"`). + env_off: bool, + /// The `telemetry.enabled` config knob is off. + knob_off: bool, + /// The running executable looks like a `target/{release,debug}` dev + /// build (same predicate the update notice uses). + dev_build: bool, +} + +impl TelemetryGate { + pub(crate) fn from_process(config: &Config) -> Self { + TelemetryGate { + ci: env::var_os("CI").is_some(), + do_not_track: env::var_os("DO_NOT_TRACK") + .is_some_and(|v| crate::update::notice::is_truthy_env_value(&v)), + env_off: env::var_os("DIRA_TELEMETRY_ENABLED") + .is_some_and(|v| !crate::update::notice::is_truthy_env_value(&v)), + knob_off: !config.telemetry.enabled, + dev_build: crate::update::notice::is_dev_build(), + } + } + + /// Whether an ordinary event — `CommandExecuted`, or a `ConsentRecorded` + /// reporting the knob being turned ON — may be emitted. + pub(crate) fn allows_emission(&self) -> bool { + !self.hard_disabled() && !self.knob_off + } + + /// The subset of the gate that overrides even the one event allowed + /// through a disabled knob (see [`record_consent`]): the environment and + /// build-shape kill switches, never the knob itself. + fn hard_disabled(&self) -> bool { + self.ci || self.do_not_track || self.env_off || self.dev_build + } +} + +/// Classify an `anyhow::Result<()>` into the closed [`ErrorKind`] set, or +/// `None` on success. Never inspects the message for anything beyond +/// classification, and never returns or logs it — the caller must not ship +/// message text. +pub(crate) fn classify_error(result: &anyhow::Result<()>) -> Option { + let err = result.as_ref().err()?; + + // The generic daemon-response path's sentinel: by the time this exists, + // the daemon has already answered and the answer was rendered as a + // failure — see `SilentExit`'s own doc. A type check, not a string one, + // so it is exact rather than a guess from wording. + if err + .chain() + .any(|c| c.downcast_ref::().is_some()) + { + return Some(ErrorKind::DaemonError); + } + + // A transport-level `std::io::Error` anywhere in the chain: a genuine I/O + // failure (permissions, disk, a broken pipe), not an application-level + // rejection. + if err + .chain() + .any(|c| c.downcast_ref::().is_some()) + { + return Some(ErrorKind::IoError); + } + + let lower = err.to_string().to_ascii_lowercase(); + + if lower.contains("timed out") || lower.contains("timeout") { + return Some(ErrorKind::Timeout); + } + + // `client::connect_message`'s own wording for every flavor of "the + // daemon isn't answering" — recognizing the shapes that module already + // produces, never re-deriving them. + const UNREACHABLE_MARKERS: &[&str] = &[ + "daemon not running", + "could not reach dirad", + "the daemon is busy", + "access denied", + ]; + if UNREACHABLE_MARKERS.iter().any(|m| lower.contains(m)) { + return Some(ErrorKind::DaemonUnreachable); + } + + // clap/validation-shaped `bail!`s: they name what the user typed, not + // anything the daemon or transport did. Checked before the daemon-error + // markers below so a message that happens to contain both (unlikely, but + // this is the more specific — and more common — classification) wins. + const INVALID_INPUT_MARKERS: &[&str] = &[ + "must be", + "unknown ", + "is not settable", + "required", + "invalid", + ]; + if INVALID_INPUT_MARKERS.iter().any(|m| lower.contains(m)) { + return Some(ErrorKind::InvalidInput); + } + + // A narrow, closed set of wording that appears ONLY once the daemon has + // actually answered over the control socket with a `Response::Error` + // (`device::resync`'s `"resync failed: {message}"`, or the generic + // fallback's `"unexpected daemon response: {other:?}"`). Deliberately not + // a bare `.contains("failed")`: that used to swallow any `bail!` or + // wrapped subprocess/cloud-HTTP error that happens to say "failed" + // without the daemon being involved at all — e.g. zavet_install's + // "failed to run `zavet` — is `zavet` still on PATH?", or a direct + // `device link`/`rotate-key` cloud-HTTP failure, neither of which ever + // touches `dirad`'s own protocol. + const DAEMON_ERROR_MARKERS: &[&str] = &["resync failed", "unexpected daemon response"]; + if DAEMON_ERROR_MARKERS.iter().any(|m| lower.contains(m)) { + return Some(ErrorKind::DaemonError); + } + + Some(ErrorKind::Internal) +} + +fn now_rfc3339() -> String { + time::OffsetDateTime::now_utc() + .format(&time::format_description::well_known::Rfc3339) + .unwrap_or_default() +} + +fn clamp_millis(d: Duration) -> u64 { + u64::try_from(d.as_millis()).unwrap_or(u64::MAX) +} + +/// Fire one request at the daemon and forget it: any failure — connect, +/// timeout, a `Response::Error` — is silently discarded. Mirrors the budget +/// discipline `forward_stdin`/`client::send_with_budget` use for hook shims +/// (`main.rs`), applied to telemetry instead of hook payloads. +async fn send_fire_and_forget(config: &Config, req: Request) { + let _ = tokio::time::timeout( + TOTAL_BUDGET, + client::send_with_budget(&config.socket_path, &req, CONNECT_BUDGET), + ) + .await; +} + +/// Record one `cli_command_executed` event, if the gate allows it. +/// +/// `result` is the outcome of the command's own `run()`, inspected only to +/// classify success/failure and, on failure, a coarse [`ErrorKind`] — never +/// to extract or forward its message. When the current directory resolves to +/// a git repo, its canonical remote crosses the local control socket as +/// `repo_canonical` so the daemon can salt-hash it; see +/// [`dira_core::protocol::Request::IngestTelemetry`]'s doc for why that never +/// happens CLI-side. +/// +/// `config` is the copy `main` loaded BEFORE `run()` executed the command — +/// stale by the time this runs for the one command that can change +/// `telemetry.enabled` mid-run: `dira config set telemetry.enabled off`. +/// Gating on the stale copy would let that disabling command's own +/// `cli_command_executed` sneak through the gate it just closed, which +/// `docs/TELEMETRY.md` promises never happens ("no partial mode, no events +/// queued while off"). So for `name == "config"` (and only then — no other +/// command can touch this knob) this re-loads the config fresh, falling back +/// to the caller's stale copy if the reload itself errors rather than +/// dropping the event over an unrelated disk problem. +pub(crate) async fn record_command( + config: &Config, + name: &'static str, + elapsed: Duration, + result: &anyhow::Result<()>, +) { + let reloaded; + let config = if name == "config" { + reloaded = Config::load().unwrap_or_else(|_| config.clone()); + &reloaded + } else { + config + }; + + let gate = TelemetryGate::from_process(config); + if !gate.allows_emission() { + return; + } + + let error_kind = classify_error(result); + let repo_canonical = env::current_dir() + .ok() + .and_then(|cwd| dira_core::project::explain_project(&cwd).ok()); + + let event = TelemetryEvent::CommandExecuted { + command: name, + duration_ms: clamp_millis(elapsed), + success: result.is_ok(), + error_kind, + // Repo facts are filled in daemon-side from `repo_canonical` — see + // `telemetry_sync::ingest`. The CLI never computes (or has the salt + // to compute) the hash itself. + repo: None, + }; + let wire = event.into_wire(now_rfc3339(), env!("CARGO_PKG_VERSION")); + send_fire_and_forget( + config, + Request::IngestTelemetry { + event: wire, + repo_canonical, + }, + ) + .await; +} + +/// Record one `cli_consent_recorded` event — the telemetry toggle changing. +/// Never fires for the knob's own initial default (accepting it writes +/// nothing and reports nothing; see `docs/TELEMETRY.md`) — only for an +/// explicit choice, which is also why there is no `ConsentSource::Default`. +/// +/// Deliberately NOT gated on `knob_off`: the caller passes `enabled` as the +/// value *just* decided (onboarding's prompt, `--telemetry`, or `dira config +/// set telemetry.enabled`), and by the time this runs that decision — on or +/// off — is the one worth reporting once, including the disable transition +/// itself. Every other kill switch (CI, `DO_NOT_TRACK`, +/// `DIRA_TELEMETRY_ENABLED`, a dev build) still applies — see +/// [`TelemetryGate::hard_disabled`]. +pub(crate) async fn record_consent(config: &Config, enabled: bool, source: ConsentSource) { + let gate = TelemetryGate::from_process(config); + if gate.hard_disabled() { + return; + } + let event = TelemetryEvent::ConsentRecorded { enabled, source }; + let wire = event.into_wire(now_rfc3339(), env!("CARGO_PKG_VERSION")); + send_fire_and_forget( + config, + Request::IngestTelemetry { + event: wire, + repo_canonical: None, + }, + ) + .await; +} + +/// The XDG path of the first-run notice marker, or `None` if it cannot be +/// resolved (no XDG config dir — same "nothing to show" fallback the update +/// notice uses for its own cache path). +fn marker_path() -> Option { + dira_core::config::project_dirs().map(|d| d.config_dir().join(NOTICE_MARKER_FILE)) +} + +/// Pure decision for [`maybe_show_first_run_notice`]: show once a marker is +/// absent, the session looks interactive, and the gate allows emission at +/// all. Split out so the matrix is unit-testable without a real terminal or +/// filesystem. +fn should_show(marker_exists: bool, is_interactive_session: bool, gate: &TelemetryGate) -> bool { + !marker_exists && is_interactive_session && gate.allows_emission() +} + +/// Print the one-time "telemetry is on by default" disclosure to stderr, if +/// warranted, then best-effort mark it shown. Never shows in CI, a dev build, +/// a disabled knob, or a non-interactive session (piped stderr, or stdin/ +/// stdout not a TTY) — and never twice on the same machine. +pub(crate) fn maybe_show_first_run_notice(config: &Config) { + let gate = TelemetryGate::from_process(config); + // Checked before anything that touches the filesystem or a terminal: a + // disabled gate (CI, a dev build, the knob off, ...) is the common case + // on every single invocation, so it must be the cheapest possible no-op + // rather than paying for a marker-file stat and two TTY checks first. + if !gate.allows_emission() { + return; + } + let Some(path) = marker_path() else { + return; + }; + let is_interactive_session = + std::io::stderr().is_terminal() && crate::onboard::prompt::is_interactive(); + if !should_show(path.exists(), is_interactive_session, &gate) { + return; + } + + eprintln!("dira telemetry (first run):"); + eprintln!("{}", crate::onboard::steps::TELEMETRY_DISCLOSURE); + + if let Some(parent) = path.parent() { + let _ = std::fs::create_dir_all(parent); + } + let _ = std::fs::write(&path, ""); +} + +#[cfg(test)] +mod tests { + use super::*; + + fn allow_gate() -> TelemetryGate { + TelemetryGate::default() + } + + // --- TelemetryGate::allows_emission -------------------------------- + + #[test] + fn allows_emission_when_nothing_disables_it() { + assert!(allow_gate().allows_emission()); + } + + #[test] + fn ci_disables_emission() { + let gate = TelemetryGate { + ci: true, + ..allow_gate() + }; + assert!(!gate.allows_emission()); + } + + #[test] + fn do_not_track_disables_emission() { + let gate = TelemetryGate { + do_not_track: true, + ..allow_gate() + }; + assert!(!gate.allows_emission()); + } + + #[test] + fn env_off_disables_emission() { + let gate = TelemetryGate { + env_off: true, + ..allow_gate() + }; + assert!(!gate.allows_emission()); + } + + #[test] + fn knob_off_disables_emission() { + let gate = TelemetryGate { + knob_off: true, + ..allow_gate() + }; + assert!(!gate.allows_emission()); + } + + #[test] + fn dev_build_disables_emission() { + let gate = TelemetryGate { + dev_build: true, + ..allow_gate() + }; + assert!(!gate.allows_emission()); + } + + /// The one asymmetry: `knob_off` alone does not count as "hard disabled" + /// — it is what lets `record_consent` report a disable transition even + /// though ordinary emission (`allows_emission`) is still off. + #[test] + fn knob_off_alone_is_not_hard_disabled() { + let gate = TelemetryGate { + knob_off: true, + ..allow_gate() + }; + assert!(!gate.hard_disabled()); + assert!(!gate.allows_emission()); + } + + #[test] + fn every_hard_switch_also_disables_emission() { + for gate in [ + TelemetryGate { + ci: true, + ..allow_gate() + }, + TelemetryGate { + do_not_track: true, + ..allow_gate() + }, + TelemetryGate { + env_off: true, + ..allow_gate() + }, + TelemetryGate { + dev_build: true, + ..allow_gate() + }, + ] { + assert!(gate.hard_disabled()); + assert!(!gate.allows_emission()); + } + } + + #[test] + fn from_process_maps_the_config_knob() { + let off = Config { + telemetry: dira_core::config::TelemetryKnobs { enabled: false }, + ..Config::default() + }; + assert!(TelemetryGate::from_process(&off).knob_off); + let on = Config { + telemetry: dira_core::config::TelemetryKnobs { enabled: true }, + ..Config::default() + }; + assert!(!TelemetryGate::from_process(&on).knob_off); + } + + // --- classify_error --------------------------------------------------- + + #[test] + fn classify_error_table() { + let ok: anyhow::Result<()> = Ok(()); + assert_eq!(classify_error(&ok), None); + + let io: anyhow::Result<()> = Err(anyhow::Error::new(std::io::Error::new( + std::io::ErrorKind::PermissionDenied, + "denied", + ))); + assert_eq!(classify_error(&io), Some(ErrorKind::IoError)); + + let unreachable: anyhow::Result<()> = Err(anyhow::anyhow!( + "daemon not running — start it with `dira daemon start`" + )); + assert_eq!( + classify_error(&unreachable), + Some(ErrorKind::DaemonUnreachable) + ); + + let unreachable2: anyhow::Result<()> = + Err(anyhow::anyhow!("could not reach dirad: connection refused")); + assert_eq!( + classify_error(&unreachable2), + Some(ErrorKind::DaemonUnreachable) + ); + + let timeout: anyhow::Result<()> = Err(anyhow::anyhow!("timed out reaching dirad")); + assert_eq!(classify_error(&timeout), Some(ErrorKind::Timeout)); + + let daemon_err: anyhow::Result<()> = + Err(anyhow::anyhow!("resync failed: some cloud message")); + assert_eq!(classify_error(&daemon_err), Some(ErrorKind::DaemonError)); + + let invalid: anyhow::Result<()> = Err(anyhow::anyhow!( + "telemetry.enabled must be one of: on, off (got `x`)" + )); + assert_eq!(classify_error(&invalid), Some(ErrorKind::InvalidInput)); + + let internal: anyhow::Result<()> = Err(anyhow::anyhow!("something unexpected happened")); + assert_eq!(classify_error(&internal), Some(ErrorKind::Internal)); + } + + /// The regression this restructuring exists to fix: a subprocess `bail!` + /// that happens to contain "failed" — but never touched the daemon at + /// all — must not be misreported as a daemon protocol error. + #[test] + fn a_subprocess_failure_containing_failed_is_not_a_daemon_error() { + let zavet_install: anyhow::Result<()> = Err(anyhow::anyhow!( + "failed to run `zavet` — is `zavet` still on PATH?" + )); + assert_eq!(classify_error(&zavet_install), Some(ErrorKind::Internal)); + } + + /// `device::resync`'s other daemon-answered-with-an-error shape. + #[test] + fn an_unexpected_daemon_response_is_a_daemon_error() { + let err: anyhow::Result<()> = Err(anyhow::anyhow!("unexpected daemon response: Pong")); + assert_eq!(classify_error(&err), Some(ErrorKind::DaemonError)); + } + + /// `SilentExit` (the generic daemon-response path's sentinel) is + /// recognized by type, not by its `Display` text — so a message that + /// mentions neither "resync" nor "unexpected daemon response" still + /// classifies correctly. + #[test] + fn silent_exit_is_a_daemon_error() { + let err: anyhow::Result<()> = Err(SilentExit.into()); + assert_eq!(classify_error(&err), Some(ErrorKind::DaemonError)); + + // Still recognized wrapped in additional context, same as the + // `std::io::Error` chain-walk above. + let wrapped: anyhow::Result<()> = Err(anyhow::Error::new(SilentExit).context("run")); + assert_eq!(classify_error(&wrapped), Some(ErrorKind::DaemonError)); + } + + #[test] + fn classify_error_never_needs_the_message_to_build_the_kind() { + // A message containing a secret-looking token must still classify + // fine — proving nothing about the message itself is required beyond + // matching a marker; the caller (record_command) never stores it. + let err: anyhow::Result<()> = Err(anyhow::anyhow!("daemon not running: token=SECRET123")); + assert_eq!(classify_error(&err), Some(ErrorKind::DaemonUnreachable)); + } + + // --- first-run notice: should_show ------------------------------------- + + #[test] + fn should_show_only_when_unshown_interactive_and_allowed() { + let allow = allow_gate(); + assert!(should_show(false, true, &allow)); + assert!(!should_show(true, true, &allow), "already shown"); + assert!(!should_show(false, false, &allow), "not interactive"); + + let denied = TelemetryGate { + ci: true, + ..allow_gate() + }; + assert!(!should_show(false, true, &denied), "gate denies emission"); + } + + #[test] + fn should_show_never_fires_twice() { + let allow = allow_gate(); + assert!(should_show(false, true, &allow)); + // Once the marker exists, the same inputs stop showing it. + assert!(!should_show(true, true, &allow)); + } +} diff --git a/cli/dira/src/update/notice.rs b/cli/dira/src/update/notice.rs index 43a08fd..5d92f22 100644 --- a/cli/dira/src/update/notice.rs +++ b/cli/dira/src/update/notice.rs @@ -162,7 +162,7 @@ struct Env { /// lets `DIRA_NO_UPDATE_CHECK=0` explicitly mean "not disabled", distinct /// from the var being merely absent (both leave checking enabled, but for /// different reasons worth being able to state). -fn is_truthy_env_value(v: &std::ffi::OsStr) -> bool { +pub(crate) fn is_truthy_env_value(v: &std::ffi::OsStr) -> bool { v != std::ffi::OsStr::new("0") } @@ -193,7 +193,7 @@ impl Env { /// path has a `target/{release,debug}` ancestor. Nagging someone running /// straight out of `just install` (which symlinks into `target/release`) is /// pure noise. -fn is_dev_build() -> bool { +pub(crate) fn is_dev_build() -> bool { let Ok(exe) = env::current_exe() else { return false; }; diff --git a/cli/dira/tests/onboard_e2e.rs b/cli/dira/tests/onboard_e2e.rs index c63398c..0c678aa 100644 --- a/cli/dira/tests/onboard_e2e.rs +++ b/cli/dira/tests/onboard_e2e.rs @@ -192,6 +192,36 @@ fn the_knowledge_tier_is_written_to_config_toml() { assert!(text.contains("knowledge = \"full\""), "got:\n{text}"); } +/// Telemetry is on by default, and `--yes` accepts that default — so unlike +/// `--knowledge full` (which always writes something settable), a `--yes` run +/// must leave `telemetry.enabled` entirely absent from `config.toml`. Writing +/// the default back would defeat the point of `TelemetryKnobs::default()` +/// resolving to `true` for every pre-existing config that lacks the table. +#[test] +fn yes_leaves_telemetry_enabled_absent_from_config_toml() { + let tmp = tempfile::tempdir().unwrap(); + let home = tmp.path(); + std::fs::create_dir_all(home.join(".claude")).unwrap(); + + let out = run_onboard(home, &["--yes", "--no-zavet", "--knowledge", "off"]); + assert!( + out.status.success(), + "stderr: {}", + String::from_utf8_lossy(&out.stderr) + ); + + // `--knowledge off` still writes `[sync] knowledge = "off"`, so a + // config.toml does exist by this point — the assertion below is on its + // *content*, not on the file's absence. + if let Some(config) = find_config_toml(home) { + let text = std::fs::read_to_string(&config).unwrap(); + assert!( + !text.contains("[telemetry]") && !text.contains("telemetry.enabled"), + "a --yes run accepting the default must not write telemetry.enabled; got:\n{text}" + ); + } +} + /// A non-interactive run without `--yes` must not hang and must not act. CI /// invoking `dira onboard` by accident should be a no-op, not a wedged job or /// a machine that quietly grew a service. diff --git a/cli/dirad/src/control.rs b/cli/dirad/src/control.rs index 932281b..1d6b5d5 100644 --- a/cli/dirad/src/control.rs +++ b/cli/dirad/src/control.rs @@ -123,6 +123,11 @@ pub async fn dispatch(state: &AppState, req: Request) -> Response { Request::DaemonInfo => daemon_info(state), Request::ResyncCursor { from } => resync_cursor(state, from).await, Request::IngestZavet { payload } => crate::zavet::ingest(state, payload).await, + Request::IngestTelemetry { + event, + repo_canonical, + } => crate::telemetry_sync::ingest(state, event, repo_canonical).await, + Request::TelemetryInstallId => crate::telemetry_sync::install_id(state).await, Request::ZavetStatus { cwd, repo } => crate::zavet::status(state, cwd, repo).await, Request::ZavetWhy { query, cwd, repo } => crate::zavet::why(state, query, cwd, repo).await, Request::ZavetWiki { topic, cwd, repo } => { diff --git a/cli/dirad/src/lib.rs b/cli/dirad/src/lib.rs index 49fa581..8cc34f3 100644 --- a/cli/dirad/src/lib.rs +++ b/cli/dirad/src/lib.rs @@ -28,9 +28,11 @@ pub mod jitter; pub mod knowledge_sync; pub mod logfile; pub mod probe; +pub mod repo_visibility; pub mod state; pub mod supervisor; pub mod sync; +pub mod telemetry_sync; #[cfg(test)] pub(crate) mod test_support; pub mod writer; @@ -96,6 +98,9 @@ pub async fn build_state( ))); let (sync_handle, sync_rx) = sync::channel(); let (knowledge_handle, knowledge_rx) = knowledge_sync::channel(); + // No receiver returned alongside it (unlike the two channels above) — see + // `telemetry_sync::TelemetrySyncHandle`'s doc for why. + let telemetry_handle = telemetry_sync::channel(); // One pooled HTTP client for every device→cloud task. Keep-alive so repeat // POSTs (heartbeat/sync/billing) to the same cloud host reuse the connection @@ -119,6 +124,10 @@ pub async fn build_state( bearer: Arc::new(bearer), sync: sync_handle, knowledge_sync: knowledge_handle, + telemetry_sync: telemetry_handle, + telemetry_identity: Arc::new(tokio::sync::OnceCell::new()), + visibility_cache: Arc::new(crate::repo_visibility::VisibilityCache::new()), + github_api_base: Arc::from(crate::repo_visibility::GITHUB_API_BASE), device_key: Arc::new(tokio::sync::RwLock::new(None)), progress: Arc::new(ProgressTracker::default()), hydrated: Arc::new(AtomicBool::new(false)), @@ -454,6 +463,10 @@ pub async fn run() -> anyhow::Result<()> { // Knowledge sync (M2): consent-gated, no-ops until [sync] knowledge is // enabled AND the device is linked. knowledge_sync::spawn(state.clone(), knowledge_rx); + // Telemetry sync (WP2): consent-gated, no-ops until [telemetry] enabled + // AND cloud_url is set — deliberately NOT gated on device linkage, unlike + // the two tasks above (see `telemetry_sync`'s module doc). + telemetry_sync::spawn(state.clone()); // Live-presence heartbeat (ephemeral; no-ops until linked + cloud_url set). heartbeat::spawn(state.clone()); // Cloud billing-summary fetch (best-effort; no-ops until linked + cloud_url set). @@ -473,6 +486,16 @@ pub async fn run() -> anyhow::Result<()> { }); } + // Startup is complete — every background task is wired and both ingress + // surfaces are live. Fire-and-forget local telemetry event (WP2); a + // consent-disabled install queues nothing (`enqueue_local` re-checks the + // knob itself). + telemetry_sync::enqueue_local( + &state, + dira_core::telemetry::event::TelemetryEvent::DaemonStarted, + ) + .await; + serve_control(state.clone(), listener); // Block until shutdown. The accept loop runs detached in `serve_control`. @@ -485,6 +508,19 @@ pub async fn run() -> anyhow::Result<()> { // restart overlap unreadable after the fact. let teardown_started = std::time::Instant::now(); tracing::info!("shutting down"); + // WP2: queue the lifecycle event as early in teardown as possible — the + // rest of this function only gets a bounded budget (the WAL checkpoint + // below is itself time-boxed), and a local store append is cheap enough + // not to compete for it. `started_at` is the same `Instant` `DaemonInfo` + // already reports uptime from, so this event and `dira version`'s number + // for the same process never disagree. + telemetry_sync::enqueue_local( + &state, + dira_core::telemetry::event::TelemetryEvent::DaemonStopped { + uptime_secs: state.started_at.elapsed().as_secs(), + }, + ) + .await; // Graceful offline: tell the cloud this device is going offline with one // best-effort empty-sessions beat (short timeout, errors ignored) so it // doesn't wait out the presence TTL. diff --git a/cli/dirad/src/repo_visibility.rs b/cli/dirad/src/repo_visibility.rs new file mode 100644 index 0000000..17a5674 --- /dev/null +++ b/cli/dirad/src/repo_visibility.rs @@ -0,0 +1,777 @@ +//! Repo-visibility probing for telemetry (WP3): resolves whether a GitHub or +//! GitLab remote is public or private, on a best-effort, cache-first basis. +//! +//! This never sits on the telemetry ingestion hot path (see +//! [`crate::telemetry_sync::ingest`]'s integration): a cache hit answers +//! instantly, and a miss answers [`RepoVisibility::Unknown`] immediately for +//! the event being ingested right now, while a detached background probe +//! fills the cache so the NEXT event from that repo carries the real answer. +//! Nothing ever retro-updates an already-stored row. +//! +//! No auth headers, no cookies, ever — every probe request is exactly what +//! any anonymous caller of the public API would send, so it can only ever +//! reveal what that API already hands out to nobody-in-particular. + +use dira_core::telemetry::repo_facts::{RepoHostClass, RepoVisibility}; +use reqwest::header::USER_AGENT; +use std::collections::{HashMap, HashSet}; +use std::sync::{Arc, Mutex, PoisonError}; +use std::time::{Duration, Instant}; + +/// Real GitHub REST API base. Overridable in tests via [`probe`]'s +/// `base_github` parameter, which is why every call site threads it through +/// rather than hardcoding it at the request site. +pub(crate) const GITHUB_API_BASE: &str = "https://api.github.com"; +/// Real GitLab API base. Overridable in tests via [`probe`]'s `base_gitlab` +/// parameter — see [`GITHUB_API_BASE`]. +pub(crate) const GITLAB_API_BASE: &str = "https://gitlab.com"; + +/// Per-request timeout for a visibility probe. The shared `AppState::http` +/// client carries no default timeout by design (see its doc comment), so +/// every request built here sets this explicitly. Short in tests so a +/// deliberately-slow mock response can exercise the timeout path without +/// slowing the suite by the production value. +#[cfg(not(test))] +const PROBE_TIMEOUT: Duration = Duration::from_secs(2); +#[cfg(test)] +const PROBE_TIMEOUT: Duration = Duration::from_millis(150); + +/// How long a confident answer is trusted before re-probing: `Public`/ +/// `Private` from a real 200/404, and the `Unknown` we give Bitbucket/ +/// self-hosted without ever asking (there is nothing transient about "WP3 +/// has no probe for this forge"). +const LONG_TTL: Duration = Duration::from_secs(24 * 3600); +/// How long an `Unknown` from a rate limit, network error, timeout, or +/// unexpected status is trusted. Deliberately short: a 429/403 window is +/// typically minutes, not a day, and caching it as long as a confident answer +/// would freeze the wrong verdict for every event from that repo until the +/// entry expired on its own. +const SHORT_TTL: Duration = Duration::from_secs(10 * 60); + +/// Cache capacity. Small and size-capped — this memoizes the handful of +/// repos one install actually works in, not a general-purpose store. +const CACHE_CAP: usize = 256; + +/// How long to trust a resolved [`RepoVisibility`] before re-probing. +/// `Unknown` only ever comes out of a GitHub/GitLab probe when something went +/// wrong (a rate limit, an error status, a timeout, a network failure) — a +/// real 200/404 always resolves to `Public`/`Private` instead — so every +/// `Unknown` from an actual probe is transient by construction and gets +/// [`SHORT_TTL`]. `Public`/`Private` are confident answers and get +/// [`LONG_TTL`]. Bitbucket/self-hosted's synchronous `Unknown` (never +/// probed at all) is cached with [`LONG_TTL`] directly by [`resolve`], +/// bypassing this function entirely — there's nothing transient about it. +fn cache_ttl_for(visibility: RepoVisibility) -> Duration { + match visibility { + RepoVisibility::Public | RepoVisibility::Private => LONG_TTL, + RepoVisibility::Unknown => SHORT_TTL, + } +} + +struct CacheEntry { + visibility: RepoVisibility, + inserted_at: Instant, + expires_at: Instant, +} + +/// Caches resolved repo visibility, keyed by the SALTED `repo_hash` — +/// never the plaintext canonical ref, so the cache carries the same privacy +/// property as everything else in the telemetry pipeline (see +/// `dira_core::telemetry::repo_facts`). One instance lives on `AppState` for +/// the daemon's whole life. +/// +/// Bounds two independent things: +/// - **Answers**: a `repo_hash -> (visibility, expiry)` map, capped at +/// [`CACHE_CAP`] entries (oldest inserted evicted first on overflow) and +/// TTL'd per [`Self::insert`]'s caller. +/// - **In-flight probes**: a `repo_hash` set, so a burst of events for the +/// same not-yet-cached repo spawns at most one probe rather than one per +/// event — see [`Self::try_start_probe`] / [`Self::finish_probe`]. +#[derive(Default)] +pub struct VisibilityCache { + entries: Mutex>, + in_flight: Mutex>, +} + +impl VisibilityCache { + pub(crate) fn new() -> Self { + Self::default() + } + + /// A cached, still-fresh answer for `repo_hash`, or `None` on a miss + /// (never probed, or the entry expired). + pub(crate) fn get(&self, repo_hash: &str) -> Option { + let entries = self.entries.lock().unwrap_or_else(PoisonError::into_inner); + let entry = entries.get(repo_hash)?; + (entry.expires_at > Instant::now()).then_some(entry.visibility) + } + + /// Record `visibility` for `repo_hash`, valid for `ttl`. If the cache is + /// full and `repo_hash` is a genuinely new key, the oldest-inserted entry + /// is evicted first — an update to an existing key never grows the map, + /// so it never triggers eviction. + pub(crate) fn insert(&self, repo_hash: String, visibility: RepoVisibility, ttl: Duration) { + let now = Instant::now(); + let mut entries = self.entries.lock().unwrap_or_else(PoisonError::into_inner); + if !entries.contains_key(&repo_hash) && entries.len() >= CACHE_CAP { + if let Some(oldest) = entries + .iter() + .min_by_key(|(_, e)| e.inserted_at) + .map(|(k, _)| k.clone()) + { + entries.remove(&oldest); + } + } + entries.insert( + repo_hash, + CacheEntry { + visibility, + inserted_at: now, + expires_at: now + ttl, + }, + ); + } + + /// Claim the right to run a probe for `repo_hash`: `true` if no probe for + /// it is already in flight (and it is now marked as such), `false` if one + /// already is — the caller must not spawn a second. Always pair a `true` + /// with a later [`Self::finish_probe`] call once that probe completes, so + /// a future miss can probe again. + pub(crate) fn try_start_probe(&self, repo_hash: &str) -> bool { + let mut in_flight = self + .in_flight + .lock() + .unwrap_or_else(PoisonError::into_inner); + in_flight.insert(repo_hash.to_string()) + } + + /// Release the in-flight claim taken by [`Self::try_start_probe`]. + pub(crate) fn finish_probe(&self, repo_hash: &str) { + let mut in_flight = self + .in_flight + .lock() + .unwrap_or_else(PoisonError::into_inner); + in_flight.remove(repo_hash); + } +} + +/// Percent-encode one path segment per RFC 3986's unreserved set +/// (`A-Za-z0-9-._~`). GitLab's nested-group project path joins segments with +/// a literal `%2F`, so any `/`-or-otherwise-reserved byte *inside* a segment +/// must itself be encoded first or it would be indistinguishable from a +/// group separator. +fn percent_encode_segment(segment: &str) -> String { + let mut out = String::with_capacity(segment.len()); + for byte in segment.bytes() { + match byte { + b'A'..=b'Z' | b'a'..=b'z' | b'0'..=b'9' | b'-' | b'.' | b'_' | b'~' => { + out.push(byte as char) + } + _ => out.push_str(&format!("%{byte:02X}")), + } + } + out +} + +/// Everything in `canonical` after the host segment (`host/owner/repo` -> +/// `owner/repo`, `host/group/sub/repo` -> `group/sub/repo`). +fn path_after_host(canonical: &str) -> &str { + canonical.split_once('/').map_or("", |(_, rest)| rest) +} + +/// The daemon's `User-Agent` on every visibility probe. GitHub's REST API +/// rejects anonymous requests that carry none at all, and the shared +/// `AppState::http` client sets no default headers (by design — see its doc +/// comment), so every request built here sets this explicitly. +fn user_agent() -> String { + format!("dirad/{}", env!("CARGO_PKG_VERSION")) +} + +/// Probe a canonical remote's visibility against the real forge API (or, in +/// tests, `base_github`/`base_gitlab` pointed at a mock). Stateless — it +/// never touches a [`VisibilityCache`]; see [`resolve`] for the cache- and +/// in-flight-aware entry point [`crate::telemetry_sync::ingest`] actually +/// uses. +/// +/// - GitHub (`github.com/owner/repo`): `GET {base_github}/repos/{owner}/{repo}`. +/// - GitLab (`gitlab.com/owner/repo`, possibly nested — +/// `gitlab.com/group/sub/repo`): `GET +/// {base_gitlab}/api/v4/projects/{group%2Fsub%2Frepo}` — every path segment +/// after the host is percent-encoded and the segments are joined with +/// `%2F`, matching GitLab's namespaced-path project lookup. +/// - Bitbucket/self-hosted: `Unknown`, with no request at all — WP3 has no +/// probe for these forges yet. +/// +/// Status mapping (see [`cache_ttl_for`] for the TTL each maps to when +/// [`resolve`] caches it): +/// - `200` -> `Public`. +/// - `404` -> `Private`. Deliberately ambiguous: both forges also 404 a URL +/// that doesn't exist at all (a typo'd owner, or a renamed/deleted repo). +/// Treating that the same as "private" is the conservative reading — the +/// alternative (assuming `Public`) risks mislabeling a private repo public. +/// - `403`/`429` (forbidden/rate-limited) -> `Unknown`. +/// - Any other status, network error, or timeout -> `Unknown`. +/// +/// No auth headers, no cookies, ever. +pub(crate) async fn probe( + http: &reqwest::Client, + base_github: &str, + base_gitlab: &str, + host_class: RepoHostClass, + canonical: &str, +) -> RepoVisibility { + match host_class { + RepoHostClass::Bitbucket | RepoHostClass::SelfHosted => RepoVisibility::Unknown, + RepoHostClass::GitHub => probe_github(http, base_github, canonical).await, + RepoHostClass::GitLab => probe_gitlab(http, base_gitlab, canonical).await, + } +} + +async fn probe_github( + http: &reqwest::Client, + base_github: &str, + canonical: &str, +) -> RepoVisibility { + let owner_repo = path_after_host(canonical); + let url = format!("{}/repos/{owner_repo}", base_github.trim_end_matches('/')); + request_visibility(http, &url).await +} + +async fn probe_gitlab( + http: &reqwest::Client, + base_gitlab: &str, + canonical: &str, +) -> RepoVisibility { + let encoded_path = path_after_host(canonical) + .split('/') + .map(percent_encode_segment) + .collect::>() + .join("%2F"); + let url = format!( + "{}/api/v4/projects/{encoded_path}", + base_gitlab.trim_end_matches('/') + ); + request_visibility(http, &url).await +} + +async fn request_visibility(http: &reqwest::Client, url: &str) -> RepoVisibility { + let resp = http + .get(url) + .header(USER_AGENT, user_agent()) + .timeout(PROBE_TIMEOUT) + .send() + .await; + let resp = match resp { + Ok(r) => r, + Err(e) => { + tracing::debug!("repo visibility probe: request failed: {e}"); + return RepoVisibility::Unknown; + } + }; + match resp.status().as_u16() { + 200 => RepoVisibility::Public, + 404 => RepoVisibility::Private, + 403 | 429 => RepoVisibility::Unknown, + other => { + tracing::debug!("repo visibility probe: unexpected status {other}"); + RepoVisibility::Unknown + } + } +} + +/// The cache- and in-flight-aware entry point [`crate::telemetry_sync::ingest`] +/// calls. Never blocks on the network: +/// +/// - Cache hit (still fresh): returns it immediately. +/// - Bitbucket/self-hosted: answered — and cached with [`LONG_TTL`] — entirely +/// synchronously, since [`probe`] never issues a request for these forges +/// anyway; no point spawning a task for a foregone conclusion. +/// - Cache miss on GitHub/GitLab: returns [`RepoVisibility::Unknown`] for the +/// CALLER's event — the "first event says unknown" half of the documented +/// ingest behavior — and, unless a probe for this `repo_hash` is already +/// running, spawns a detached task that probes the real answer and fills +/// the cache, so the NEXT event from this repo gets the truth. A burst of +/// events for the same uncached repo before that probe resolves spawns at +/// most one probe: [`VisibilityCache::try_start_probe`] makes every event +/// after the first in the burst a no-op spawn. +/// +/// Nothing here ever retro-updates an already-stored event row — see +/// [`crate::telemetry_sync::ingest`]'s doc comment. +/// +/// `base_github`/`base_gitlab` are threaded through (rather than this +/// function hardcoding [`GITHUB_API_BASE`]/[`GITLAB_API_BASE`] itself) purely +/// for tests — every production call site passes the real constants. +pub(crate) fn resolve( + cache: &Arc, + http: &reqwest::Client, + base_github: &str, + base_gitlab: &str, + host_class: RepoHostClass, + canonical: &str, + repo_hash: &str, +) -> RepoVisibility { + if let Some(v) = cache.get(repo_hash) { + return v; + } + if matches!( + host_class, + RepoHostClass::Bitbucket | RepoHostClass::SelfHosted + ) { + cache.insert(repo_hash.to_string(), RepoVisibility::Unknown, LONG_TTL); + return RepoVisibility::Unknown; + } + if cache.try_start_probe(repo_hash) { + let cache = cache.clone(); + let http = http.clone(); + let base_github = base_github.to_string(); + let base_gitlab = base_gitlab.to_string(); + let canonical = canonical.to_string(); + let repo_hash = repo_hash.to_string(); + tokio::spawn(async move { + let visibility = probe(&http, &base_github, &base_gitlab, host_class, &canonical).await; + cache.insert(repo_hash.clone(), visibility, cache_ttl_for(visibility)); + cache.finish_probe(&repo_hash); + }); + } + RepoVisibility::Unknown +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::test_support::{MockCloud, MockResp}; + use dira_core::telemetry::event::TelemetryEvent; + + /// Poll `f` until it returns `Some`, or panic after `deadline`. Used to + /// wait on a detached background probe without a fixed sleep. + async fn poll_until(deadline: Duration, mut f: impl FnMut() -> Option) -> T { + let start = Instant::now(); + loop { + if let Some(v) = f() { + return v; + } + if start.elapsed() > deadline { + panic!("condition never became true within {deadline:?}"); + } + tokio::time::sleep(Duration::from_millis(10)).await; + } + } + + #[tokio::test] + async fn github_200_maps_to_public() { + let cloud = MockCloud::start(&["/repos/acme/api"]).await; + cloud.push("/repos/acme/api", MockResp::ok("{}")); + let http = reqwest::Client::new(); + let vis = probe( + &http, + cloud.base_url(), + "http://unused.invalid", + RepoHostClass::GitHub, + "github.com/acme/api", + ) + .await; + assert_eq!(vis, RepoVisibility::Public); + } + + #[tokio::test] + async fn github_404_maps_to_private() { + let cloud = MockCloud::start(&["/repos/acme/api"]).await; + cloud.push("/repos/acme/api", MockResp::status(404, "")); + let http = reqwest::Client::new(); + let vis = probe( + &http, + cloud.base_url(), + "http://unused.invalid", + RepoHostClass::GitHub, + "github.com/acme/api", + ) + .await; + assert_eq!(vis, RepoVisibility::Private); + } + + #[tokio::test] + async fn github_500_maps_to_unknown() { + let cloud = MockCloud::start(&["/repos/acme/api"]).await; + cloud.push("/repos/acme/api", MockResp::status(500, "boom")); + let http = reqwest::Client::new(); + let vis = probe( + &http, + cloud.base_url(), + "http://unused.invalid", + RepoHostClass::GitHub, + "github.com/acme/api", + ) + .await; + assert_eq!(vis, RepoVisibility::Unknown); + } + + #[tokio::test] + async fn github_429_maps_to_unknown() { + let cloud = MockCloud::start(&["/repos/acme/api"]).await; + cloud.push("/repos/acme/api", MockResp::status(429, "slow down")); + let http = reqwest::Client::new(); + let vis = probe( + &http, + cloud.base_url(), + "http://unused.invalid", + RepoHostClass::GitHub, + "github.com/acme/api", + ) + .await; + assert_eq!(vis, RepoVisibility::Unknown); + } + + #[tokio::test] + async fn a_slow_response_times_out_to_unknown() { + let cloud = MockCloud::start(&["/repos/acme/api"]).await; + cloud.push( + "/repos/acme/api", + MockResp::ok("{}").with_delay(Duration::from_secs(5)), + ); + let http = reqwest::Client::new(); + let started = Instant::now(); + let vis = probe( + &http, + cloud.base_url(), + "http://unused.invalid", + RepoHostClass::GitHub, + "github.com/acme/api", + ) + .await; + assert!( + started.elapsed() < Duration::from_secs(1), + "the probe's own timeout must fire well before the mock's 5s delay" + ); + assert_eq!(vis, RepoVisibility::Unknown); + } + + #[tokio::test] + async fn a_network_error_maps_to_unknown() { + let http = reqwest::Client::new(); + // Port 0 can never accept a connection — an immediate, deterministic + // network error without depending on any particular port being closed. + let vis = probe( + &http, + "http://127.0.0.1:0", + "http://unused.invalid", + RepoHostClass::GitHub, + "github.com/acme/api", + ) + .await; + assert_eq!(vis, RepoVisibility::Unknown); + } + + #[tokio::test] + async fn gitlab_200_maps_to_public() { + let cloud = MockCloud::start(&["/api/v4/projects/acme%2Fapi"]).await; + cloud.push("/api/v4/projects/acme%2Fapi", MockResp::ok("{}")); + let http = reqwest::Client::new(); + let vis = probe( + &http, + "http://unused.invalid", + cloud.base_url(), + RepoHostClass::GitLab, + "gitlab.com/acme/api", + ) + .await; + assert_eq!(vis, RepoVisibility::Public); + } + + #[tokio::test] + async fn gitlab_nested_group_path_is_percent_encoded_and_joined() { + // If the encoding/join logic ever produced the wrong path, this + // request would miss the registered route (axum's fallback 404) and + // be read back as `Private` instead of the `Public` we queued — + // failing this assertion. + let cloud = MockCloud::start(&["/api/v4/projects/group%2Fsub%2Frepo"]).await; + cloud.push("/api/v4/projects/group%2Fsub%2Frepo", MockResp::ok("{}")); + let http = reqwest::Client::new(); + let vis = probe( + &http, + "http://unused.invalid", + cloud.base_url(), + RepoHostClass::GitLab, + "gitlab.com/group/sub/repo", + ) + .await; + assert_eq!( + vis, + RepoVisibility::Public, + "a nested gitlab path must resolve to the %2F-joined project path" + ); + } + + #[tokio::test] + async fn gitlab_404_maps_to_private() { + let cloud = MockCloud::start(&["/api/v4/projects/acme%2Fapi"]).await; + cloud.push("/api/v4/projects/acme%2Fapi", MockResp::status(404, "")); + let http = reqwest::Client::new(); + let vis = probe( + &http, + "http://unused.invalid", + cloud.base_url(), + RepoHostClass::GitLab, + "gitlab.com/acme/api", + ) + .await; + assert_eq!(vis, RepoVisibility::Private); + } + + #[tokio::test] + async fn bitbucket_and_self_hosted_never_issue_a_request() { + let http = reqwest::Client::new(); + // Bases point at addresses that would error/hang if ever contacted; + // reaching the assertion at all (fast, no timeout) proves the match + // arm short-circuited before building a request. + for host_class in [RepoHostClass::Bitbucket, RepoHostClass::SelfHosted] { + let vis = probe( + &http, + "http://127.0.0.1:0", + "http://127.0.0.1:0", + host_class, + "bitbucket.org/acme/api", + ) + .await; + assert_eq!(vis, RepoVisibility::Unknown); + } + } + + #[test] + fn ttl_choice_is_short_for_unknown_and_long_for_a_confident_answer() { + assert_eq!(cache_ttl_for(RepoVisibility::Public), LONG_TTL); + assert_eq!(cache_ttl_for(RepoVisibility::Private), LONG_TTL); + assert_eq!( + cache_ttl_for(RepoVisibility::Unknown), + SHORT_TTL, + "an Unknown from an actual probe only ever means a rate limit, error, or timeout — \ + a real 200/404 always resolves to Public/Private instead — so it must never be \ + cached as long as a confident answer" + ); + } + + #[tokio::test] + async fn resolve_caches_after_the_spawned_probe_fills_it_and_a_second_lookup_hits_it() { + let cloud = MockCloud::start(&["/repos/acme/api"]).await; + cloud.push("/repos/acme/api", MockResp::ok("{}")); + let http = reqwest::Client::new(); + let cache = Arc::new(VisibilityCache::new()); + let repo_hash = "deadbeef"; + + // Miss: Unknown immediately, probe spawned in the background. + let first = resolve( + &cache, + &http, + cloud.base_url(), + "http://unused.invalid", + RepoHostClass::GitHub, + "github.com/acme/api", + repo_hash, + ); + assert_eq!(first, RepoVisibility::Unknown); + + // Wait for the detached probe to land, rather than sleeping a fixed + // amount. + let warmed = poll_until(Duration::from_secs(1), || cache.get(repo_hash)).await; + assert_eq!(warmed, RepoVisibility::Public); + assert_eq!(cloud.requests("/repos/acme/api").len(), 1); + + // Second lookup, now warm: served from the cache, no second request. + let second = resolve( + &cache, + &http, + cloud.base_url(), + "http://unused.invalid", + RepoHostClass::GitHub, + "github.com/acme/api", + repo_hash, + ); + assert_eq!(second, RepoVisibility::Public); + assert_eq!( + cloud.requests("/repos/acme/api").len(), + 1, + "a fresh cache hit must not issue a second request" + ); + } + + #[tokio::test] + async fn resolve_never_spawns_a_second_probe_while_one_is_in_flight() { + // A response slow enough that several `resolve` calls land before it + // answers, so if bounding failed we'd see more than one request. + let cloud = MockCloud::start(&["/repos/acme/api"]).await; + cloud.push( + "/repos/acme/api", + MockResp::ok("{}").with_delay(Duration::from_millis(300)), + ); + let http = reqwest::Client::new(); + let cache = Arc::new(VisibilityCache::new()); + let repo_hash = "deadbeef"; + + for _ in 0..5 { + let v = resolve( + &cache, + &http, + cloud.base_url(), + "http://unused.invalid", + RepoHostClass::GitHub, + "github.com/acme/api", + repo_hash, + ); + assert_eq!(v, RepoVisibility::Unknown); + } + + poll_until(Duration::from_secs(2), || cache.get(repo_hash)).await; + assert_eq!( + cloud.requests("/repos/acme/api").len(), + 1, + "a burst of misses for the same repo must spawn at most one probe" + ); + } + + #[tokio::test] + async fn resolve_answers_bitbucket_and_self_hosted_synchronously_with_no_request() { + let http = reqwest::Client::new(); + let cache = Arc::new(VisibilityCache::new()); + + let vis = resolve( + &cache, + &http, + "http://127.0.0.1:0", + "http://127.0.0.1:0", + RepoHostClass::Bitbucket, + "bitbucket.org/acme/api", + "deadbeef", + ); + assert_eq!(vis, RepoVisibility::Unknown); + // Cached immediately — no spawned probe needed for a forge we never ask. + assert_eq!(cache.get("deadbeef"), Some(RepoVisibility::Unknown)); + } + + #[test] + fn cache_evicts_the_oldest_entry_once_full() { + let cache = VisibilityCache::new(); + for i in 0..CACHE_CAP { + cache.insert(format!("hash-{i}"), RepoVisibility::Public, LONG_TTL); + } + assert!(cache.get("hash-0").is_some()); + + // One more insert past capacity evicts the very first one inserted. + cache.insert("hash-new".to_string(), RepoVisibility::Private, LONG_TTL); + assert!( + cache.get("hash-0").is_none(), + "the oldest entry must be evicted once the cache is full" + ); + assert_eq!(cache.get("hash-new"), Some(RepoVisibility::Private)); + } + + #[test] + fn cache_expires_entries_past_their_ttl() { + let cache = VisibilityCache::new(); + cache.insert( + "hash".to_string(), + RepoVisibility::Public, + Duration::from_millis(0), + ); + // A zero-length TTL is already expired relative to `Instant::now()` + // measured microseconds later. + std::thread::sleep(Duration::from_millis(5)); + assert_eq!(cache.get("hash"), None); + } + + /// End-to-end through `telemetry_sync::ingest`: the first event for a + /// never-before-seen repo stores `Unknown` (the cache miss path), and — + /// once the detached probe has warmed the cache — a second event for the + /// SAME repo stores the real, probed visibility. Neither the first row + /// nor the cache is ever retro-updated; only the second ingest's own + /// write differs. + #[tokio::test] + async fn ingest_reports_unknown_first_then_the_real_visibility_once_warm() { + let cloud = MockCloud::start(&["/api/v1/pulse", "/repos/acme/api"]).await; + cloud.push("/repos/acme/api", MockResp::ok("{}")); + let store = dira_core::Store::open_in_memory().await.unwrap(); + let config = dira_core::Config { + cloud_url: Some(cloud.base_url().to_string()), + telemetry: dira_core::config::TelemetryKnobs { enabled: true }, + ..Default::default() + }; + let (mut state, ..) = crate::build_state(store, config).await.unwrap(); + // `ingest` must resolve visibility against the mock, never the real + // forge — point its probe base at the same mock cloud serves. + state.github_api_base = Arc::from(cloud.base_url()); + + let make_wire = |n: u64| { + TelemetryEvent::CommandExecuted { + command: "status", + duration_ms: n, + success: true, + error_kind: None, + repo: None, + } + .into_wire("2026-01-01T00:00:00Z".into(), "0.0.0-test") + }; + + crate::telemetry_sync::ingest( + &state, + make_wire(1), + Some("github.com/acme/api".to_string()), + ) + .await; + + let after_first = state.store.telemetry_max_event_id().await.unwrap().unwrap(); + let rows = state + .store + .telemetry_events_since(None, &after_first, 10) + .await + .unwrap(); + assert_eq!(rows.len(), 1); + let first: dira_core::telemetry::wire::TelemetryEventWire = + serde_json::from_str(&rows[0].props_json).unwrap(); + assert_eq!( + first.repo_visibility.as_deref(), + Some("unknown"), + "the very first event for an unseen repo must report unknown, never guess" + ); + let repo_hash = first.repo_hash.clone().unwrap(); + + // Wait for the background probe `ingest` kicked off to warm the cache. + poll_until(Duration::from_secs(1), || { + state.visibility_cache.get(&repo_hash) + }) + .await; + + crate::telemetry_sync::ingest( + &state, + make_wire(2), + Some("github.com/acme/api".to_string()), + ) + .await; + // Two ULIDs minted in the same millisecond are not guaranteed to sort + // in insertion order, so identify the rows by their payload + // (duration_ms 1 vs 2) rather than by an id window. + let after_second = state.store.telemetry_max_event_id().await.unwrap().unwrap(); + let all_rows = state + .store + .telemetry_events_since(None, &after_second, 10) + .await + .unwrap(); + assert_eq!(all_rows.len(), 2); + let by_duration = |want: u64| { + all_rows + .iter() + .map(|r| { + serde_json::from_str::( + &r.props_json, + ) + .unwrap() + }) + .find(|w| w.duration_ms == Some(want)) + .expect("row with the expected duration_ms") + }; + assert_eq!( + by_duration(2).repo_visibility.as_deref(), + Some("public"), + "once the cache is warm, the next event for the same repo carries the real answer" + ); + // The first row is untouched — nothing retro-updates it. + assert_eq!(by_duration(1).repo_visibility.as_deref(), Some("unknown")); + } +} diff --git a/cli/dirad/src/state.rs b/cli/dirad/src/state.rs index e00e580..37e1f32 100644 --- a/cli/dirad/src/state.rs +++ b/cli/dirad/src/state.rs @@ -52,6 +52,32 @@ pub struct AppState { pub sync: SyncHandle, /// Handle to the knowledge sync task (M2; consent-gated, own cursors). pub knowledge_sync: crate::knowledge_sync::KnowledgeSyncHandle, + /// Handle to the telemetry sync task (WP2; consent-gated, unsigned + /// batches). See [`crate::telemetry_sync::TelemetrySyncHandle`] for why + /// this holds its receiver internally rather than threading it out of + /// [`crate::build_state`] alongside `sync`/`knowledge_sync`'s. + pub telemetry_sync: crate::telemetry_sync::TelemetrySyncHandle, + /// This daemon's telemetry identity (install id + salt), minted at most + /// once per daemon lifetime. See [`AppState::telemetry_identity`]. + /// + /// `Arc`-wrapped (unlike `device_key`'s `RwLock`, which is genuinely + /// mutable — see [`AppState::invalidate_device_key`]) so every clone of + /// this `AppState` shares the SAME cell: a `OnceCell` living directly on + /// the struct would give each clone its own, independent "first call" + /// and defeat the whole point of capping minting at one attempt. + pub telemetry_identity: + Arc>, + /// Cache + in-flight bookkeeping for [`crate::repo_visibility`]'s + /// GitHub/GitLab visibility probe (WP3), keyed by the salted `repo_hash` + /// so it carries the same privacy property as the telemetry pipeline + /// itself. Lives on `AppState` (rather than a lazy static) so tests can + /// construct an isolated daemon and inspect/seed it directly. + pub visibility_cache: Arc, + /// Base URL for [`crate::repo_visibility`]'s GitHub probe. Not a config + /// knob — every production daemon gets [`crate::repo_visibility::GITHUB_API_BASE`] + /// from `build_state` — this exists purely so a test can point `ingest`'s + /// whole visibility pipeline at a mock instead of the real network. + pub github_api_base: Arc, /// This device's signing key, used to sign attestation batches. Loaded /// **lazily** off the startup critical path: the key is only needed for /// sync/signing, never to answer a control request, and loading it can block @@ -167,6 +193,34 @@ impl AppState { } } + /// This daemon's telemetry identity, minted at most once per daemon + /// lifetime and shared by every caller thereafter. + /// + /// Replaces three independent `identity::load_or_mint` call sites in + /// `telemetry_sync.rs` (`ingest`, `install_id`, `flush_telemetry`), each + /// of which used to hit the store on every single event/flush — and, + /// because none of them serialized against each other, could race: two + /// concurrent first callers on a fresh store could each observe "nothing + /// minted yet" and persist a *different* salt, the later write winning + /// silently while the earlier caller's already-returned identity kept + /// disagreeing with what actually ended up on disk (see + /// [`dira_core::telemetry::identity::load_or_mint`]'s own doc). Routing + /// every caller through [`tokio::sync::OnceCell::get_or_try_init`] makes + /// the mint happen exactly once; every other concurrent caller waits on + /// that one attempt instead of racing it. + /// + /// Returns an owned clone rather than a borrow tied to the cell, so + /// callers don't hold anything across their own store/HTTP awaits — same + /// reasoning as [`Self::device_key`]. + pub async fn telemetry_identity( + &self, + ) -> Result { + self.telemetry_identity + .get_or_try_init(|| dira_core::telemetry::identity::load_or_mint(&self.store)) + .await + .cloned() + } + /// Discard the cached device key so the NEXT [`AppState::device_key`] call /// reloads it from the store (WP-B1b). Call this immediately after /// promoting a pending rotation key (`dira_core::identity::promote_pending_key`) @@ -885,6 +939,42 @@ mod tests { assert_eq!(reloaded.public_base64(), second.public_base64()); } + /// The TOCTOU this `OnceCell` exists to close: without it, two concurrent + /// FIRST callers on a fresh store could each observe "nothing minted + /// yet" and persist a different salt, the later write winning silently + /// while the earlier caller's already-returned identity disagreed with + /// what ended up on disk. `get_or_try_init` makes exactly one of these + /// two calls actually mint; the other only ever observes that result. + #[tokio::test] + async fn telemetry_identity_mints_at_most_once_under_concurrent_callers() { + let store = dira_core::Store::open_in_memory().await.unwrap(); + let config = dira_core::Config::default(); + let (state, ..) = crate::build_state(store, config).await.unwrap(); + // Both handles share the same `Arc>` (see the field's own + // doc) — cloning `AppState` must never give a caller its own cell. + let state2 = state.clone(); + + let (a, b) = tokio::join!(state.telemetry_identity(), state2.telemetry_identity()); + let a = a.expect("mint succeeds"); + let b = b.expect("mint succeeds"); + assert_eq!( + a.install_id, b.install_id, + "both concurrent callers must observe the identical minted id" + ); + assert_eq!(a.salt, b.salt, "and the identical minted salt"); + + // And what's actually on disk is exactly what both callers saw — + // neither call's return value is stale relative to the persisted + // meta row. + let persisted_id = state + .store + .meta_get(dira_core::telemetry::identity::META_TELEMETRY_INSTALL_ID) + .await + .unwrap() + .expect("minted id was persisted"); + assert_eq!(persisted_id, a.install_id); + } + const IDLE: Duration = Duration::minutes(5); fn ev(session: &str, kind: EventKind, project: Option<&str>) -> RawEvent { diff --git a/cli/dirad/src/telemetry_sync.rs b/cli/dirad/src/telemetry_sync.rs new file mode 100644 index 0000000..69af142 --- /dev/null +++ b/cli/dirad/src/telemetry_sync.rs @@ -0,0 +1,798 @@ +//! Telemetry sync task (WP2): ship the local `telemetry_events` queue +//! (`dira_core::telemetry`, WP1) to `POST {cloud_url}/api/v1/pulse`. +//! +//! A smaller, unsigned sibling of [`crate::sync`]/[`crate::knowledge_sync`]: +//! same debounce/backstop shape, same per-chunk cursor-after-2xx discipline +//! (D-0020), same shared backoff ladder — but no device key, no envelope, no +//! JCS canonicalization. A telemetry batch is anonymous by construction (see +//! `dira_core::telemetry`'s module doc): there is nothing here that needs a +//! device's identity to be trusted, only an install id that never leaves this +//! machine's own queue. +//! +//! ### The gate is deliberately weaker than sync/knowledge sync's +//! Both of those require the device to be **linked** (`identity::device_id`) +//! before they send anything. Telemetry only requires `cloud_url` to be set +//! and `[telemetry] enabled` to be true — an unlinked install (nobody has run +//! `dira device link` yet) still reports anonymous usage, because the whole +//! point of this channel is to see usage from installs that may never link at +//! all. `Store::insert_telemetry_event`'s callers (`ingest`/`enqueue_local`) +//! independently re-check the same knob, so a disabled install never even +//! queues an event; this gate is what stops an already-queued backlog (e.g. +//! from before the knob was turned off) from draining. +//! +//! ### Poison-chunk policy (400 `content_not_allowed`-shaped rejection) +//! The knowledge channel has a rich taxonomy of 4xx error codes because its +//! payload is signed and workspace-gated. Telemetry has none of that: a 400 +//! from `/api/v1/pulse` can only mean the batch itself is malformed or the +//! cloud no longer understands `v: 1` — something about THIS chunk's bytes, +//! not a transient condition that a retry would fix. Retrying it forever +//! would wedge the whole queue behind one bad chunk, silently dropping every +//! event minted after it (D-0020's per-chunk cursor exists precisely to avoid +//! one bad window blocking the rest). So a 400 is treated as **permanent-skip**: +//! log it loudly (the body may say why), advance the cursor past the chunk +//! exactly as a 2xx would, and keep draining. The chunk's rows are left in +//! place rather than deleted — cheap local debris, and worth keeping around +//! for `dira doctor`/support to inspect if the malformed-batch rate turns out +//! to matter. This is the same choice the underlying event model already +//! makes safe: `TelemetryEventWire` is a fixed, versioned shape, so "our own +//! batch is malformed" should be rare enough that losing one window of +//! anonymous counts is a fair trade against wedging the queue forever. +//! +//! 413 (payload too large) and 429/5xx (rate limit / server trouble) are +//! ordinary transient failures — retried with the shared backoff ladder, same +//! as sync/knowledge sync. A 404 means the cloud predates the endpoint, +//! mirroring knowledge sync's quiet `endpoint_missing` skip. + +use crate::state::AppState; +use crate::sync::{ + next_backoff, record_channel_health, retry_after_from_headers, transient_wait, HealthChannel, +}; +use dira_core::protocol::Response; +use dira_core::telemetry::repo_facts::{self, RepoVisibility}; +use dira_core::telemetry::wire::{batch_id, TelemetryBatch, TelemetryEventWire}; +use dira_core::telemetry::{META_TELEMETRY_CURSOR, META_TELEMETRY_HEALTH}; +use std::sync::{Arc, Mutex, PoisonError}; +use std::time::Duration as StdDuration; +use tokio::sync::mpsc; +use tokio::time::{sleep, sleep_until, Instant}; +use ulid::Ulid; + +/// Debounce window: coalesce a burst of triggers into one flush. +const DEBOUNCE: StdDuration = StdDuration::from_secs(5); +/// Backstop cadence — telemetry moves at command speed, not event speed; +/// slower than both sync (90s) and knowledge sync (120s). +const BACKSTOP: StdDuration = StdDuration::from_secs(300); +/// HTTP timeout for a single pulse chunk POST. Sized like knowledge sync's — +/// a chunk's worth of small, flat JSON events, not a full ingest batch. +const HTTP_TIMEOUT: StdDuration = StdDuration::from_secs(30); +/// Events per chunk POSTed to `/api/v1/pulse`. +const CHUNK_SIZE: i64 = 200; + +/// Handle to the telemetry sync task. Cloneable; shares the trigger channel. +/// +/// Unlike [`crate::sync::channel`]/[`crate::knowledge_sync::channel`], this +/// does NOT hand its receiver back to the caller to thread through +/// [`crate::build_state`]'s return tuple: that tuple is destructured at every +/// one of `build_state`'s many call sites (production and test), and growing +/// its arity for a single new background task would touch every one of them +/// for a change that is purely internal to this module. Instead the receiver +/// is built alongside the sender and stashed here, `Option`-wrapped so +/// [`spawn`] can `take()` it exactly once; a second `spawn` call (or one +/// before [`channel`] ran) is a wiring bug, logged rather than panicking a +/// running daemon over it. +#[derive(Clone)] +pub struct TelemetrySyncHandle { + /// Non-blocking trigger; `ingest`/`enqueue_local` `try_send(())` here + /// after a durable append. A full channel is fine — the backstop covers a + /// missed nudge. + pub trigger: mpsc::Sender<()>, + rx: Arc>>>, +} + +/// Create the trigger channel + handle before `AppState` exists (the handle +/// is a field of `AppState`), mirroring [`crate::sync::channel`]'s ordering — +/// see [`TelemetrySyncHandle`]'s doc for why the receiver travels inside the +/// handle instead of alongside it. +pub fn channel() -> TelemetrySyncHandle { + let (trigger, rx) = mpsc::channel::<()>(1); + TelemetrySyncHandle { + trigger, + rx: Arc::new(Mutex::new(Some(rx))), + } +} + +/// Spawn the background telemetry sync task, taking its receiver out of +/// `state.telemetry_sync` the one time this runs. +pub fn spawn(state: AppState) { + let rx = state + .telemetry_sync + .rx + .lock() + .unwrap_or_else(PoisonError::into_inner) + .take(); + let Some(rx) = rx else { + tracing::warn!( + "telemetry sync: spawn called with no receiver available (already spawned, or \ + `channel()` never ran) — the task will not start" + ); + return; + }; + tokio::spawn(run(state, rx)); +} + +async fn run(state: AppState, mut rx: mpsc::Receiver<()>) { + let mut backstop_at = + Instant::now() + crate::jitter::jittered(BACKSTOP, crate::jitter::DEFAULT_FRAC); + let mut backoff = StdDuration::ZERO; + let mut consecutive_failures: u32 = 0; + + loop { + tokio::select! { + recv = rx.recv() => { + if recv.is_none() { + break; + } + sleep(DEBOUNCE).await; + while rx.try_recv().is_ok() {} + } + _ = sleep_until(backstop_at) => { + backstop_at = Instant::now() + + crate::jitter::jittered(BACKSTOP, crate::jitter::DEFAULT_FRAC); + } + } + + match flush_telemetry(&state).await { + Ok(Outcome::Synced) | Ok(Outcome::Nothing) => { + backoff = StdDuration::ZERO; + consecutive_failures = 0; + record_health(&state, None, 0, 0).await; + } + Ok(Outcome::Skipped(kind)) => { + backoff = StdDuration::ZERO; + consecutive_failures = 0; + record_health(&state, Some(kind), 0, 0).await; + } + Err(err) => { + let (kind, wait) = match &err { + TError::Transient { + message, + retry_after, + } => { + let wait = transient_wait(*retry_after, backoff); + tracing::warn!( + "telemetry sync: transient failure, backing off {wait:?}: {message}" + ); + ("transient", wait) + } + TError::Fatal(e) => { + let wait = next_backoff(backoff); + tracing::warn!("telemetry sync: error, backing off {wait:?}: {e}"); + ("fatal", wait) + } + }; + backoff = wait; + consecutive_failures += 1; + record_health(&state, Some(kind), consecutive_failures, backoff.as_secs()).await; + sleep(backoff).await; + } + } + } +} + +#[cfg_attr(test, derive(Debug))] +enum Outcome { + Synced, + Nothing, + /// Not running, with the health kind saying why (`"off"`, `"skipped"`, + /// `"endpoint_missing"`). + Skipped(&'static str), +} + +#[cfg_attr(test, derive(Debug))] +enum TError { + Transient { + message: String, + retry_after: Option, + }, + Fatal(String), +} + +/// One telemetry flush. Reads `[telemetry] enabled` and `cloud_url` off +/// `state.config`, which is loaded once at daemon startup and never updated +/// in place — so, despite reading it fresh on every call, a config change +/// only takes effect on the NEXT daemon restart, exactly as +/// `docs/TELEMETRY.md` already documents. This function does no polling of +/// its own; "every call" only means every flush this already-running daemon +/// happens to perform. +/// +/// Deliberately does NOT gate on device linkage (see the module doc) — only +/// `cloud_url` + `[telemetry] enabled`. +async fn flush_telemetry(state: &AppState) -> Result { + if !state.config.telemetry.enabled { + return Ok(Outcome::Skipped("off")); + } + let Some(cloud_url) = state.config.cloud_url.clone() else { + return Ok(Outcome::Skipped("skipped")); + }; + + let until = state + .store + .telemetry_max_event_id() + .await + .map_err(|e| TError::Fatal(format!("read max telemetry id: {e}")))?; + let Some(until) = until else { + return Ok(Outcome::Nothing); // queue is empty + }; + let mut cursor = state + .store + .meta_get(META_TELEMETRY_CURSOR) + .await + .map_err(|e| TError::Fatal(format!("read cursor: {e}")))? + .filter(|s| !s.is_empty()); + if cursor.as_deref() == Some(until.as_str()) { + return Ok(Outcome::Nothing); // already caught up to the snapshot bound + } + + // Fetched at most once per flush — install id + salt never change between + // this flush's chunks. `AppState::telemetry_identity` mints at most once + // per daemon lifetime regardless, so this local `Option` only saves the + // cheap `OnceCell::get_or_try_init` fast-path call, not a store round-trip. + let mut install_id: Option = None; + let mut synced_any = false; + let mut any_poison = false; + + let url = format!("{}/api/v1/pulse", cloud_url.trim_end_matches('/')); + loop { + let rows = state + .store + .telemetry_events_since(cursor.as_deref(), &until, CHUNK_SIZE) + .await + .map_err(|e| TError::Fatal(format!("load telemetry events: {e}")))?; + if rows.is_empty() { + break; + } + let chunk_len = rows.len(); + let first_id = rows.first().expect("checked non-empty above").id.clone(); + let last_id = rows.last().expect("checked non-empty above").id.clone(); + + let events: Vec = rows + .iter() + .filter_map(|r| match serde_json::from_str(&r.props_json) { + Ok(w) => Some(w), + Err(e) => { + // A row that fails to deserialize is a local bug (it was + // serialized by this same binary in `ingest`/ + // `enqueue_local`), not a cloud-facing problem — drop it + // from the batch rather than failing the whole flush, and + // let the cursor advance past it like any other row. + tracing::error!(id = %r.id, "telemetry sync: dropping unreadable local row: {e}"); + None + } + }) + .collect(); + + if !events.is_empty() { + if install_id.is_none() { + let identity = state + .telemetry_identity() + .await + .map_err(|e| TError::Fatal(format!("load telemetry identity: {e}")))?; + install_id = Some(identity.install_id); + } + let install_id = install_id.as_deref().expect("just set above"); + let batch = TelemetryBatch { + v: 1, + batch_id: batch_id(install_id, &first_id, &last_id), + install_id: install_id.to_string(), + generated_at: crate::heartbeat::fmt_rfc3339(time::OffsetDateTime::now_utc()), + events, + }; + + match post_batch(&state.http, &url, &batch).await? { + PostOutcome::Accepted => {} + PostOutcome::EndpointMissing => return Ok(Outcome::Skipped("endpoint_missing")), + PostOutcome::Poison(body) => { + tracing::error!( + first_id = %first_id, + last_id = %last_id, + "telemetry sync: cloud rejected this batch as malformed/unsupported \ + (400) — skipping past it rather than wedging the queue behind it: {body}" + ); + any_poison = true; + } + } + } + + meta_put(state, META_TELEMETRY_CURSOR, &last_id).await?; + cursor = Some(last_id); + synced_any = true; + + if (chunk_len as i64) < CHUNK_SIZE { + break; // fewer than a full chunk — caught up to `until` + } + } + + if synced_any { + // Retention hygiene, and only after a fully clean drain: a poison + // chunk's rows are left in place (see the module doc) so a bad batch + // is inspectable locally even though the cursor has already moved + // past it. + if !any_poison { + if let Some(c) = &cursor { + if let Err(e) = state.store.delete_telemetry_events_through(c).await { + tracing::debug!("telemetry sync: prune after drain failed: {e}"); + } + } + } + Ok(Outcome::Synced) + } else { + Ok(Outcome::Nothing) + } +} + +enum PostOutcome { + Accepted, + /// 404: older cloud without the endpoint yet. + EndpointMissing, + /// 400: this chunk's batch itself is malformed/unsupported — see the + /// module doc's poison-chunk policy. Carries the response body for the + /// caller's log line. + Poison(String), +} + +async fn post_batch( + client: &reqwest::Client, + url: &str, + batch: &TelemetryBatch, +) -> Result { + let resp = client + .post(url) + .timeout(HTTP_TIMEOUT) + .json(batch) + .send() + .await + .map_err(|e| TError::Transient { + message: format!("post: {e}"), + retry_after: None, + })?; + let status = resp.status(); + let retry_after = retry_after_from_headers(resp.headers()); + + if status.is_success() { + return Ok(PostOutcome::Accepted); + } + if status == reqwest::StatusCode::NOT_FOUND { + tracing::debug!("telemetry sync: cloud has no /api/v1/pulse yet (404)"); + return Ok(PostOutcome::EndpointMissing); + } + let body = resp.text().await.unwrap_or_default(); + if status == reqwest::StatusCode::BAD_REQUEST { + return Ok(PostOutcome::Poison(body)); + } + if status.as_u16() == 413 || status.as_u16() == 429 || status.is_server_error() { + return Err(TError::Transient { + message: format!("cloud answered {status}: {body}"), + retry_after, + }); + } + Err(TError::Fatal(format!("cloud answered {status}: {body}"))) +} + +async fn meta_put(state: &AppState, key: &str, value: &str) -> Result<(), TError> { + state + .store + .meta_set(key, value) + .await + .map_err(|e| TError::Fatal(format!("store {key}: {e}"))) +} + +/// Enqueue one already-flattened wire event onto the local queue. Dispatched +/// from `Request::IngestTelemetry`, sent by `dira`'s own gated telemetry +/// client. +/// +/// Re-checks `[telemetry] enabled` daemon-side (belt and braces) rather than +/// trusting the caller's own gate: a version skew between an older `dira` and +/// a newer `dirad` (or vice versa) must never let an event slip past a +/// `false` knob just because one side's copy of the check is stale. Acks +/// [`Response::Ok`] either way — this is a fire-and-forget ingress, like +/// `IngestZavet`, so a disabled knob is a silent no-op, not an error the CLI +/// needs to see. +/// +/// `repo_canonical`, when present, crossed only the local control socket (see +/// [`dira_core::protocol::Request::IngestTelemetry`]'s doc): it is salt-hashed +/// here via [`AppState::telemetry_identity`] + [`repo_facts::compute`], and only the +/// resulting host class, visibility, and hash are written onto `event` before +/// it is queued. The plaintext ref itself is never persisted, and never sent +/// to Dira's own cloud — the one place it travels beyond this control socket +/// is [`crate::repo_visibility`]'s anonymous probe to the forge's own public +/// API (DIRASH-0034), which resolves the `visibility` computed below. +/// +/// Visibility (WP3) comes from [`crate::repo_visibility::resolve`], which is +/// cache-first and never blocks this call on the network: a cache hit answers +/// with the real, already-known visibility; a miss answers `Unknown` for THIS +/// event and kicks off a detached background probe that fills the cache for +/// next time. So a never-before-seen repo's first event always reports +/// `unknown`, and only a later event for the SAME repo (once the probe lands) +/// reports the real answer — this row is never retro-updated once written. +pub(crate) async fn ingest( + state: &AppState, + mut event: TelemetryEventWire, + repo_canonical: Option, +) -> Response { + if !state.config.telemetry.enabled { + return Response::Ok; + } + if let Some(canonical) = repo_canonical { + match state.telemetry_identity().await { + Ok(identity) => { + // Visibility is resolved separately below (it needs the + // salted hash to key the cache); `compute` here is only for + // the host class + hash, so its own `visibility` input is a + // throwaway placeholder. + let facts = + repo_facts::compute(&canonical, &identity.salt, RepoVisibility::Unknown); + let visibility = crate::repo_visibility::resolve( + &state.visibility_cache, + &state.http, + &state.github_api_base, + crate::repo_visibility::GITLAB_API_BASE, + facts.host_class, + &canonical, + &facts.repo_hash, + ); + event.repo_host_class = Some(facts.host_class.as_str().to_string()); + event.repo_visibility = Some(visibility.as_str().to_string()); + event.repo_hash = Some(facts.repo_hash); + } + Err(e) => { + tracing::debug!( + "telemetry: could not derive repo facts (identity load failed): {e}" + ); + } + } + } + if let Err(e) = store_event(state, &event).await { + tracing::debug!("telemetry: drop event that failed to queue: {e}"); + } + Response::Ok +} + +/// `Request::TelemetryInstallId`: hand back this daemon's telemetry install +/// id, minting it on first use. Never gated on `[telemetry] enabled` — see +/// the request's own doc comment for why. +pub(crate) async fn install_id(state: &AppState) -> Response { + match state.telemetry_identity().await { + Ok(identity) => Response::TelemetryInstallId { + install_id: identity.install_id, + }, + Err(e) => Response::Error { + message: format!("load telemetry identity: {e}"), + }, + } +} + +/// Enqueue one daemon-local lifecycle event (`DaemonStarted`/`DaemonStopped`), +/// stamping the timestamp and the running daemon's own version via +/// [`dira_core::telemetry::event::TelemetryEvent::into_wire`]. Goes through +/// the exact same store-then-trigger path as [`ingest`] — the only difference +/// is that the event never crossed the control socket at all, so it flattens +/// to its wire shape here instead of arriving already flattened. +/// +/// Respects the same consent gate as [`ingest`]: a disabled knob is a silent +/// no-op. +pub(crate) async fn enqueue_local( + state: &AppState, + event: dira_core::telemetry::event::TelemetryEvent, +) { + if !state.config.telemetry.enabled { + return; + } + let now = crate::heartbeat::fmt_rfc3339(time::OffsetDateTime::now_utc()); + let wire = event.into_wire(now, env!("CARGO_PKG_VERSION")); + if let Err(e) = store_event(state, &wire).await { + tracing::debug!("telemetry: drop local event that failed to queue: {e}"); + } +} + +/// Shared tail of [`ingest`]/[`enqueue_local`]: mint an id, append to the +/// local queue, and nudge the sync task. Callers have already applied the +/// consent gate. +async fn store_event(state: &AppState, event: &TelemetryEventWire) -> Result<(), dira_core::Error> { + let id = Ulid::generate().to_string(); + let created_at = crate::heartbeat::fmt_rfc3339(time::OffsetDateTime::now_utc()); + let props_json = serde_json::to_string(event) + .map_err(|e| dira_core::Error::Decode(format!("serialize telemetry event: {e}")))?; + state + .store + .insert_telemetry_event(&id, &created_at, &event.event, &props_json) + .await?; + let _ = state.telemetry_sync.trigger.try_send(()); + Ok(()) +} + +/// The telemetry channel's [`HealthChannel`] keys. +const TELEMETRY_HEALTH_CHANNEL: HealthChannel = HealthChannel { + log: "telemetry sync", + health_key: META_TELEMETRY_HEALTH, + cursor_key: META_TELEMETRY_CURSOR, + watermark_key: None, +}; + +/// Persist the telemetry channel's health snapshot to +/// [`META_TELEMETRY_HEALTH`] (see [`record_channel_health`]). +async fn record_health( + state: &AppState, + error_kind: Option<&str>, + consecutive_failures: u32, + backoff_secs: u64, +) { + record_channel_health( + state, + &TELEMETRY_HEALTH_CHANNEL, + error_kind, + consecutive_failures, + backoff_secs, + ) + .await; +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::test_support::{MockCloud, MockResp}; + use dira_core::telemetry::event::TelemetryEvent; + + async fn state_with(cloud: &MockCloud, telemetry_enabled: bool) -> AppState { + let store = dira_core::Store::open_in_memory().await.unwrap(); + let config = dira_core::Config { + cloud_url: Some(cloud.base_url().to_string()), + telemetry: dira_core::config::TelemetryKnobs { + enabled: telemetry_enabled, + }, + ..Default::default() + }; + let (state, ..) = crate::build_state(store, config).await.unwrap(); + state + } + + fn keys(body: &str) -> Vec { + let v: serde_json::Value = serde_json::from_str(body).unwrap(); + v.as_object().unwrap().keys().cloned().collect() + } + + #[tokio::test] + async fn happy_path_advances_the_cursor_and_ships_the_documented_shape() { + let cloud = MockCloud::start(&["/api/v1/pulse"]).await; + let state = state_with(&cloud, true).await; + enqueue_local(&state, TelemetryEvent::DaemonStarted).await; + + cloud.push("/api/v1/pulse", MockResp::ok(r#"{"status":"accepted"}"#)); + let out = flush_telemetry(&state).await.unwrap(); + assert!(matches!(out, Outcome::Synced)); + + let cursor = state + .store + .meta_get(META_TELEMETRY_CURSOR) + .await + .unwrap() + .unwrap(); + assert!(!cursor.is_empty(), "cursor must advance after a 2xx"); + + let reqs = cloud.requests("/api/v1/pulse"); + assert_eq!(reqs.len(), 1); + let body: serde_json::Value = serde_json::from_str(&reqs[0]).unwrap(); + assert_eq!(body["v"], 1); + assert!(body["installId"].as_str().is_some_and(|s| !s.is_empty())); + assert_eq!(body["events"][0]["event"], "cli_daemon_started"); + assert_eq!( + keys(&reqs[0]) + .into_iter() + .collect::>(), + ["v", "batchId", "installId", "generatedAt", "events"] + .into_iter() + .map(String::from) + .collect() + ); + + // Nothing new — the next flush is a no-op. + let out = flush_telemetry(&state).await.unwrap(); + assert!(matches!(out, Outcome::Nothing)); + assert_eq!(cloud.requests("/api/v1/pulse").len(), 1); + } + + #[tokio::test] + async fn a_500_does_not_advance_the_cursor() { + let cloud = MockCloud::start(&["/api/v1/pulse"]).await; + let state = state_with(&cloud, true).await; + enqueue_local(&state, TelemetryEvent::DaemonStarted).await; + + cloud.push("/api/v1/pulse", MockResp::status(500, "boom")); + let err = flush_telemetry(&state).await; + assert!(matches!(err, Err(TError::Transient { .. }))); + assert_eq!( + state.store.meta_get(META_TELEMETRY_CURSOR).await.unwrap(), + None, + "cursor must not advance on a failed POST" + ); + } + + #[tokio::test] + async fn disabled_knob_skips_without_touching_the_network() { + let cloud = MockCloud::start(&["/api/v1/pulse"]).await; + let state = state_with(&cloud, false).await; + // ingest/enqueue_local also re-check the knob — nothing should even + // be queued. + enqueue_local(&state, TelemetryEvent::DaemonStarted).await; + assert_eq!(state.store.telemetry_max_event_id().await.unwrap(), None); + + let out = flush_telemetry(&state).await.unwrap(); + assert!(matches!(out, Outcome::Skipped("off"))); + assert!(cloud.requests("/api/v1/pulse").is_empty()); + } + + #[tokio::test] + async fn no_cloud_url_skips_without_touching_the_network() { + let cloud = MockCloud::start(&["/api/v1/pulse"]).await; + let store = dira_core::Store::open_in_memory().await.unwrap(); + let config = dira_core::Config { + cloud_url: None, + ..Default::default() + }; + let (state, ..) = crate::build_state(store, config).await.unwrap(); + enqueue_local(&state, TelemetryEvent::DaemonStarted).await; + // Queuing does not require cloud_url — only the flush does. + assert!(state + .store + .telemetry_max_event_id() + .await + .unwrap() + .is_some()); + + let out = flush_telemetry(&state).await.unwrap(); + assert!(matches!(out, Outcome::Skipped("skipped"))); + assert!(cloud.requests("/api/v1/pulse").is_empty()); + } + + #[tokio::test] + async fn a_400_advances_the_cursor_past_the_poison_chunk_instead_of_wedging() { + let cloud = MockCloud::start(&["/api/v1/pulse"]).await; + let state = state_with(&cloud, true).await; + enqueue_local(&state, TelemetryEvent::DaemonStarted).await; + enqueue_local(&state, TelemetryEvent::DaemonStopped { uptime_secs: 1 }).await; + + cloud.push( + "/api/v1/pulse", + MockResp::status(400, r#"{"error":"unsupported_schema_version"}"#), + ); + let out = flush_telemetry(&state).await.unwrap(); + assert!(matches!(out, Outcome::Synced)); + + let cursor = state + .store + .meta_get(META_TELEMETRY_CURSOR) + .await + .unwrap() + .unwrap(); + assert!( + !cursor.is_empty(), + "a 400 must advance the cursor past the poison chunk, not wedge the queue" + ); + // The poison chunk's rows are kept locally (not pruned) for inspection. + assert_eq!( + state + .store + .telemetry_events_since(None, &cursor, 100) + .await + .unwrap() + .len(), + 2 + ); + + // A follow-up flush with nothing new past the poisoned window is a no-op. + let out = flush_telemetry(&state).await.unwrap(); + assert!(matches!(out, Outcome::Nothing)); + assert_eq!(cloud.requests("/api/v1/pulse").len(), 1); + } + + #[tokio::test] + async fn ingest_with_the_knob_disabled_inserts_nothing() { + let cloud = MockCloud::start(&["/api/v1/pulse"]).await; + let state = state_with(&cloud, false).await; + let wire = + TelemetryEvent::DaemonStarted.into_wire("2026-01-01T00:00:00Z".into(), "0.0.0-test"); + let resp = ingest(&state, wire, None).await; + assert!(matches!(resp, Response::Ok)); + assert_eq!(state.store.telemetry_max_event_id().await.unwrap(), None); + } + + #[tokio::test] + async fn ingest_with_the_knob_enabled_queues_and_triggers() { + let cloud = MockCloud::start(&["/api/v1/pulse"]).await; + let state = state_with(&cloud, true).await; + let wire = + TelemetryEvent::DaemonStarted.into_wire("2026-01-01T00:00:00Z".into(), "0.0.0-test"); + let resp = ingest(&state, wire, None).await; + assert!(matches!(resp, Response::Ok)); + assert!(state + .store + .telemetry_max_event_id() + .await + .unwrap() + .is_some()); + } + + /// `repo_canonical` never reaches the store verbatim: `ingest` hashes it + /// (salted) and classifies its host, and only those derived fields land + /// in the queued row's `props_json`. + #[tokio::test] + async fn ingest_with_a_repo_canonical_stores_hashed_facts_never_the_plaintext() { + let cloud = MockCloud::start(&["/api/v1/pulse"]).await; + let state = state_with(&cloud, true).await; + let wire = TelemetryEvent::CommandExecuted { + command: "status", + duration_ms: 12, + success: true, + error_kind: None, + repo: None, + } + .into_wire("2026-01-01T00:00:00Z".into(), "0.0.0-test"); + + let resp = ingest(&state, wire, Some("github.com/acme/api".to_string())).await; + assert!(matches!(resp, Response::Ok)); + + let until = state.store.telemetry_max_event_id().await.unwrap().unwrap(); + let rows = state + .store + .telemetry_events_since(None, &until, 10) + .await + .unwrap(); + assert_eq!(rows.len(), 1); + assert!( + !rows[0].props_json.contains("github.com/acme/api"), + "the plaintext canonical ref must never be persisted: {}", + rows[0].props_json + ); + let stored: TelemetryEventWire = serde_json::from_str(&rows[0].props_json).unwrap(); + assert_eq!(stored.repo_host_class.as_deref(), Some("github")); + assert_eq!(stored.repo_visibility.as_deref(), Some("unknown")); + assert!(stored.repo_hash.as_deref().is_some_and(|h| h.len() == 64)); + + // Deterministic under this install's own salt: hashing the same + // canonical again (a second ingest) reproduces the same hash — + // `repo_facts::compute` itself already pins salt-keyed determinism; + // this only pins that `ingest` actually calls it that way. + let wire2 = TelemetryEvent::CommandExecuted { + command: "status", + duration_ms: 1, + success: true, + error_kind: None, + repo: None, + } + .into_wire("2026-01-01T00:00:01Z".into(), "0.0.0-test"); + ingest(&state, wire2, Some("github.com/acme/api".to_string())).await; + let until2 = state.store.telemetry_max_event_id().await.unwrap().unwrap(); + let rows2 = state + .store + .telemetry_events_since(Some(&until), &until2, 10) + .await + .unwrap(); + let stored2: TelemetryEventWire = serde_json::from_str(&rows2[0].props_json).unwrap(); + assert_eq!(stored.repo_hash, stored2.repo_hash); + } + + #[tokio::test] + async fn install_id_mints_and_returns_the_same_id_across_calls() { + let cloud = MockCloud::start(&["/api/v1/pulse"]).await; + let state = state_with(&cloud, true).await; + let first = match install_id(&state).await { + Response::TelemetryInstallId { install_id } => install_id, + other => panic!("expected TelemetryInstallId, got {other:?}"), + }; + assert!(!first.is_empty()); + let second = match install_id(&state).await { + Response::TelemetryInstallId { install_id } => install_id, + other => panic!("expected TelemetryInstallId, got {other:?}"), + }; + assert_eq!(first, second); + } +} diff --git a/cli/dirad/src/test_support.rs b/cli/dirad/src/test_support.rs index d619f34..697b415 100644 --- a/cli/dirad/src/test_support.rs +++ b/cli/dirad/src/test_support.rs @@ -12,10 +12,11 @@ use axum::body::Bytes; use axum::http::{HeaderMap, StatusCode}; use axum::response::Response; -use axum::routing::post; +use axum::routing::any; use axum::Router; use std::collections::{HashMap, VecDeque}; use std::sync::{Arc, Mutex}; +use std::time::Duration; /// One canned HTTP response. #[derive(Clone)] @@ -23,6 +24,10 @@ pub struct MockResp { pub status: u16, pub body: String, pub headers: Vec<(String, String)>, + /// Sleep this long before answering — lets a test exercise a caller's + /// own request timeout without a real network stall (repo-visibility's + /// probe tests use this for its 2s-in-prod/short-in-test probe timeout). + pub delay: Option, } impl MockResp { @@ -31,6 +36,7 @@ impl MockResp { status: 200, body: body.into(), headers: vec![], + delay: None, } } @@ -39,6 +45,7 @@ impl MockResp { status, body: body.into(), headers: vec![], + delay: None, } } @@ -46,6 +53,13 @@ impl MockResp { self.headers.push((name.to_string(), value.to_string())); self } + + /// Answer this response only after `delay` — for exercising a caller's + /// request timeout. + pub fn with_delay(mut self, delay: Duration) -> Self { + self.delay = Some(delay); + self + } } type Queue = Arc>>; @@ -60,7 +74,10 @@ pub struct MockCloud { } impl MockCloud { - /// Start the mock server with POST handlers for each of `paths`. + /// Start the mock server with a handler for each of `paths` that answers + /// any HTTP method (`POST` for the cloud-sync channels' bodies; `GET` for + /// repo-visibility's probe requests) — the canned-response queue and + /// recorded-bodies list don't care which method arrived. pub async fn start(paths: &[&'static str]) -> Self { let mut routes = HashMap::new(); let mut recorded = HashMap::new(); @@ -72,7 +89,7 @@ impl MockCloud { recorded.insert(path, rec.clone()); router = router.route( path, - post(move |headers: HeaderMap, body: Bytes| { + any(move |headers: HeaderMap, body: Bytes| { let q = q.clone(); let rec = rec.clone(); async move { @@ -85,6 +102,9 @@ impl MockCloud { .unwrap() .pop_front() .unwrap_or_else(|| MockResp::ok("{}")); + if let Some(d) = resp.delay { + tokio::time::sleep(d).await; + } let mut builder = Response::builder().status(StatusCode::from_u16(resp.status).unwrap()); for (k, v) in &resp.headers { diff --git a/docs/TELEMETRY.md b/docs/TELEMETRY.md new file mode 100644 index 0000000..34db144 --- /dev/null +++ b/docs/TELEMETRY.md @@ -0,0 +1,124 @@ +# Telemetry + +Dira collects anonymous product-usage analytics, on by default. This document says +exactly what that means: what is collected, what never is, where it goes, and every way +to turn it off. + +This is a separate channel from **knowledge sync** (decision and spec content — see +[zavet.md](zavet.md)) and from **cloud sync** (attestation batches for billing). Turning +telemetry off touches none of those, and turning them off does not touch telemetry. + +## Why it exists + +Dira is a small team building a CLI that runs unattended on other people's machines. +Telemetry answers two questions we would otherwise have no way to answer: which commands +and harnesses are actually used, and where the CLI fails in the wild. It exists for +product sustainability and prioritization — deciding what to fix and what to build next — +not for surveillance of any individual's work. It carries no marketing or resale purpose. + +## What is sent + +Every event carries a fixed, closed set of fields — never a prompt, a free-text value, or +anything not listed below. + +| Event | Sent when | Properties | +|---|---|---| +| `cli_command_executed` | a CLI or daemon-served command finishes | `command` (e.g. `"status"`, `"config"` — the top-level name only, never a sub-action and never raw argv), `duration_ms`, `success`, `error_kind` (one of a fixed set: `daemon_unreachable`, `daemon_error`, `invalid_input`, `io_error`, `timeout`, `internal`), and — only when cwd is inside a git repository — `repo_host_class` (`github`/`gitlab`/`bitbucket`/`self_hosted`), `repo_visibility` (`public`/`private`/`unknown`), `repo_hash` | +| `cli_daemon_started` | `dirad` finishes starting up | none beyond the base fields below | +| `cli_daemon_stopped` | `dirad` shuts down | `duration_ms` (uptime in ms) | +| `cli_consent_recorded` | the telemetry toggle changes | `telemetry_enabled`, `consent_source` (`prompt`/`yes_flag`/`config_set`) | + +Every event also carries a timestamp, the running `dira`/`dirad` version, OS, and CPU +architecture. + +## What is never sent + +- Repo names, owners, or URLs — never to Dira. (Working out `repo_visibility` sends + the plaintext `owner/repo` to the repo's own host, not to Dira — see the next + section.) +- Git identity — author name or email +- File paths, file contents, or diffs +- Command arguments or flag values +- Error messages or any other free text +- Anything that identifies you as a person, as opposed to an anonymous install + +## How public/private is determined + +To resolve `repo_visibility`, the daemon asks the forge that already hosts the +repo — an unauthenticated GitHub or GitLab API lookup of `owner/repo` — because +that forge already knows the repo exists; the lookup discloses nothing to it +that it doesn't already have. The plaintext name goes only to that provider, +never to Dira's servers, and the request carries no auth token, cookie, or +install/device identifier — a generic `dirad/` user agent is the only +thing beyond the bare lookup. Any other host (Bitbucket, self-hosted) is +reported as `unknown`, with no request made at all. See DIRASH-0034 for the +full rationale. + +## The repo hash + +`repo_hash` lets the same repository be recognized across events from **one install** +without ever naming it: it is `HMAC-SHA256(per-install salt, canonical remote)`, hex +encoded. The salt is generated once per install, stored locally, and never transmitted. +Two installs hashing the same repository produce unrelated hashes, and the hash cannot be +reversed back to the repo without the salt — so repos can't be correlated across +different people's machines, and Dira cannot learn which repo it is from the hash alone. + +`repo_visibility` is only ever `public` or `private` when something in the pipeline has +actually determined it; otherwise it is `unknown` rather than a guessed default. + +## The install id + +Events are tagged with a random `install_id`, generated once per machine and stored +locally — not derived from and not linked to your device's signing key or attestation +identity. It exists only to let Dira de-duplicate and count installs, not to identify a +device across the two systems. + +**Linking a device is different.** Once you link this device to a cloud workspace +(`dira device link`), subsequent telemetry from that install may be associated with your +workspace account, the same way a signed-in product associates analytics with the account +once you sign in. This is the one place telemetry crosses from purely anonymous into +workspace-attributable, and it only happens after you take the linking action yourself. + +## Where it goes + +Telemetry is queued locally and flushed by `dirad` to Dira's cloud ingest endpoint, which +forwards it to PostHog's **EU** region. It never goes anywhere else, and it is not sold or +shared with third parties. + +## Turning it off + +Any of the following disables collection and sync entirely — no partial mode, no events +queued while off: + +```sh +dira config set telemetry.enabled false # persistent, in config.toml +DIRA_TELEMETRY_ENABLED=0 # per-invocation or exported +DO_NOT_TRACK=1 # the cross-tool convention +``` + +`dira onboard` also asks, in its own step, before any of the above — see below. + +Dev builds (`cargo build` without a release profile) and CI runs never send telemetry, +regardless of the knob's value. + +## The onboarding step + +`dira onboard` shows this disclosure — the same content as this document, in short form — +before asking, on every path: the interactive prompt, `--yes`, and an explicit +`--telemetry ` flag all see it first. The default answer is on, matching +`TelemetryKnobs::default()`. + +Accepting the default writes nothing to `config.toml` — an absent `[telemetry]` table +already means "on" for every pre-existing install, so recording the default would be +noise. Declining writes `telemetry.enabled = false` and confirms that nothing further will +be sent. + +## Checking or changing it later + +```sh +dira config get telemetry.enabled +dira config set telemetry.enabled off +``` + +Daemon-side changes take effect after a restart: `dira daemon stop` then `dira daemon +start` (or `dira daemon restart`).