Repository navigation
feat(pipeline): say whose request an OutcomeSnapshot is - #4698
JerrettDavis merged 2 commits into
Conversation
PR governanceThis PR follows the template and is marked ready for human review. |
|
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
JerrettDavis
left a comment
There was a problem hiding this comment.
Reviewed exact head 4896d5404dadcd0f7e4d4e06d37c4c3b7760158e, containing current upstream main bb8d18a56abb07ca7e811235c86ea6b626eb1f8b, and the author/Devin discussion. Outcome snapshots gain a defensive read-only label copy and the conversation/input/output-savings fields without losing ordinary serialization, copying, pickling or hashing. Identity-based extension registration avoids duplicate delivery for the same object. No correctness blocker found in this scope.
All 23 pipeline/outcome-stage tests pass. The serialization regressions fail against the preceding mapping-proxy implementation and pass on this head, addressing Devin's compatibility finding for both empty and populated labels. The outcome funnel passes the fields already resolved by the existing project/client paths; this source approval does not qualify the separate configurable attribution feature in #4699.
All upstream CI, Rust, Security, Merge Conflicts, Changelog Guard and latest governance runs succeeded; the older cancelled governance run is superseded. Ready for independent human merge review. Nothing merged.
🤖 I have created a release *beep* *boop* --- ## [0.44.0](v0.43.1...v0.44.0) (2026-11-09) ### ⚠ BREAKING CHANGES * **packaging:** keep PyTorch extras out of all ([#4067](#4067)) ### Features * **pipeline:** say whose request an OutcomeSnapshot is ([#4698](#4698)) ([a2bcbd1](a2bcbd1)) * **providers:** add Hermes agent wrap support ([#934](#934)) ([7877241](7877241)) * **proxy/model-router:** add shadow mode (HEADROOM_MODEL_ROUTER_LOG_ONLY) ([#2395](#2395)) ([861ef95](861ef95)) * **proxy:** add lossless /v1/compress mode and detect provider by wire shape ([#4617](#4617)) ([33f067a](33f067a)) * **proxy:** expose per-request cost_usd in /stats recent_requests ([#1079](#1079)) ([#1565](#1565)) ([84cedbf](84cedbf)) * **proxy:** expose shared Redis store availability in /health and /metrics ([#4578](#4578)) ([602ca65](602ca65)) * **proxy:** give turn hooks one conversation id per client conversation ([#4496](#4496)) ([94a99ed](94a99ed)) * **proxy:** report DeepSeek as its own provider in dashboard ([#2565](#2565)) ([9ad16a7](9ad16a7)) * **proxy:** show per-request /stats metadata to token-authenticated callers ([#4168](#4168)) ([9b2dec5](9b2dec5)) * **proxy:** unified multi-upstream routing for Anthropic Cloud and local oMLX ([#4597](#4597)) ([f896161](f896161)) * **savings:** carry cumulative actual output_tokens in history checkpoints ([#2921](#2921)) ([8388ff5](8388ff5)) * **wrap:** add `headroom wrap droid` for Factory Droid ([#4143](#4143)) ([bb8d18a](bb8d18a)) * **wrap:** make the Serena code-memory MCP opt-in ([#4509](#4509)) ([2649835](2649835)) ### Bug Fixes * **asgi:** stop buffering when the client disconnects (GHSA-mc2f-vg4v-2fh4) ([#4701](#4701)) ([c6d7081](c6d7081)) * **beacon:** enforce the allowlist below the top-level keys ([#4691](#4691)) ([456288d](456288d)) * **binaries:** fail closed on HEADROOM_BINARIES_ALLOW_UNVERIFIED=0/false ([#3218](#3218)) ([da6d973](da6d973)) * **cache:** record TTL observations with relative filenames ([#4711](#4711)) ([51d9dc5](51d9dc5)) * **cache:** round learned TTL upper bounds up to whole seconds ([#4710](#4710)) ([13347e3](13347e3)) * **cache:** save prefix-tracker state when the exit signal arrives ([#4715](#4715)) ([3ba4223](3ba4223)) * **ccr:** lowercase the hash in the streaming feedback recorders ([#2313](#2313)) ([3912c04](3912c04)) * **ci:** keep the npm publish token out of install and build steps ([#4690](#4690)) ([8f6366c](8f6366c)) * **cli:** resolve active proxy port for read commands ([#3522](#3522)) ([02c4e92](02c4e92)) * **content-router:** trust confident HTML detection over mixed-content heuristics ([#3627](#3627)) ([f50db67](f50db67)) * **dashboard:** lifetime metrics and refresh ([#2374](#2374)) ([eb9932a](eb9932a)) * **doctor:** recognize IPv6 loopback and case-insensitive proxy routing URLs ([#3255](#3255)) ([2b0315b](2b0315b)) * **install:** decode Windows Task Scheduler XML output before parsing ([#3969](#3969)) ([#4064](#4064)) ([b78e07d](b78e07d)) * **install:** make install status agree with doctor when the proxy is reachable ([#3072](#3072)) ([#3141](#3141)) ([05431b2](05431b2)) * **langchain:** check optional integration availability ([#4042](#4042)) ([cde584d](cde584d)) * **learn:** harden analyzer JSON parsing against malformed LLM output ([#2471](#2471)) ([12864fc](12864fc)) * **learn:** support opencode v2 session schema in learn scanner ([#4199](#4199)) ([c05466a](c05466a)) * **lossless:** fold only pure path listings, never markup ([#4716](#4716)) ([0ccd559](0ccd559)) * **memory:** gracefully degrade on context lookup and initialization errors ([#3251](#3251)) ([#4049](#4049)) ([00d4f44](00d4f44)) * **metrics:** cap savings_by_source rows and fold the rest into other ([#4257](#4257)) ([2038fb2](2038fb2)) * **packaging:** keep PyTorch extras out of all ([#4067](#4067)) ([a890521](a890521)) * **proxy:** add --no-ccr-response-handling flag to bypass response buffering ([#3082](#3082)) ([#3100](#3100)) ([ce14271](ce14271)) * **proxy:** bound the Bedrock Prometheus model label ([#4694](#4694)) ([d70a8b7](d70a8b7)) * **proxy:** drop headers named by Connection when forwarding responses ([#4674](#4674)) ([3871b36](3871b36)) * **proxy:** keep cold-turn reasoning drops sticky so the next warm turn reuses the cached prefix ([#4592](#4592)) ([3544cf5](3544cf5)) * **proxy:** keep the Bedrock credentials provider when startup resolution fails ([#4705](#4705)) ([5438f00](5438f00)) * **proxy:** refresh Bedrock AWS credentials before they expire ([#4696](#4696)) ([adc6724](adc6724)) * **proxy:** rename leaked_threads metric to timed_out_workers_total ([#3908](#3908)) ([bd164a2](bd164a2)) * **proxy:** track response.-prefixed item and content-part events in Responses SSE state ([#4695](#4695)) ([8f4122f](8f4122f)) * **router:** compress tool_result blocks in mutable window with trailing messages ([#4717](#4717)) ([0511bcd](0511bcd)) * **router:** ignore shell operators inside quoted arguments in read detection ([#4646](#4646)) ([8cdce17](8cdce17)) * **scripts:** skip ci-precheck in the pre-push hook when a push only deletes refs ([#4702](#4702)) ([cbcf8d8](cbcf8d8)) * **sdk:** keep Vercel AI SDK tool-result output kinds through compression ([#4670](#4670)) ([ce42754](ce42754)) * **search_compressor:** keep kept rows byte-for-byte and in input order ([#3982](#3982)) ([a538740](a538740)) * **smart-crusher:** keep mixed-array dict rows and exact drop count with factor_out_constants ([#4692](#4692)) ([ced2685](ced2685)) * **smart-crusher:** keep the row the question names ([#4713](#4713)) ([e6e2ebb](e6e2ebb)) * **storage:** return the latest snapshot from JSONLStorage.get ([#4675](#4675)) ([b25c45b](b25c45b)) * **tabular:** split fixed-width rows on column boundaries, not on double spaces ([#4541](#4541)) ([b80245e](b80245e)) * **tag-protector:** do not treat a slash in an unquoted attribute as self-closing ([#4681](#4681)) ([d5644cb](d5644cb)) * **tag-protector:** keep the enclosing tag open after a nested close in marker mode ([#4682](#4682)) ([385d487](385d487)) * **telemetry:** keep install ID stateless ([#3542](#3542)) ([27bba79](27bba79)) * **tokenizers:** resolve chatgpt-4o-* aliases to o200k_base ([#4672](#4672)) ([9648d2d](9648d2d)) * **tokenizers:** unwrap BatchEncoding in HuggingFaceTokenizer.count_messages ([#2447](#2447)) ([9eed58f](9eed58f)) * **vertex:** explain Vertex failures in-band, correct the model/region matrix ([#2640](#2640)) ([0ddaa56](0ddaa56)) * **wrap:** route Claude Code's Bedrock mode through Headroom ([#4316](#4316)) ([aa2093b](aa2093b)) ### Dependencies * bump rusqlite from 0.40.2 to 0.41.1 ([#4658](#4658)) ([74b7ac2](74b7ac2)) * bump the cargo-minor-patch group with 3 updates ([#4657](#4657)) ([c01cf07](c01cf07)) * bump the npm-minor-patch group across 1 directory with 2 updates ([#4650](#4650)) ([def4ef1](def4ef1)) * update click requirement from >=8.3.3 to >=8.5.0 ([#4655](#4655)) ([ea59ac3](ea59ac3)) * update sqlite-vec requirement from >=0.1.6 to >=0.1.9 ([#4654](#4654)) ([de1a1e0](de1a1e0)) * update tree-sitter requirement from <0.27,>=0.25.2 to >=0.26.0,<0.27 ([#4656](#4656)) ([910588e](910588e)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). --------- Co-authored-by: headroom-release[bot] <352661243+headroom-release[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Description
OutcomeSnapshotis the one public seam a pipeline extension gets atOUTCOME_OBSERVED, but it says nothing about whose request it measured. An extension that wants to split what it observes (an exporter, a chargeback job, a budget alert) cannot tell projects, clients or conversations apart without patching core. This adds four read-only fields and fills them from valuesemit_request_outcomealready has:attribution:{dimension: label}. For nowprojectandclient, from the same values the savings tracker books, and only when known.conversation_key: the handler's key, already onRequestOutcome.original_tokens: the pre-compression size, on the same tokenizer astokens_saved.output_tokens_saved: the output-shaping estimate this request booked.attributionis frozen into a read-onlydictsubclass (FrozenLabels) in__post_init__. The dataclass is already frozen, but a frozen field can still hold a mutable dict, so without this one extension could rewrite the labels the next one reads. Adictsubclass rather than aMappingProxyTypekeeps the snapshot hashable, picklable and safe fordataclasses.asdict,copy.deepcopyandjson.dumps, as it was before.It also adds
PipelineExtensionManager.add(). A proxy extension (headroom.proxy_extension) that also wants lifecycle events is installed after the manager exists, so it cannot be passed to the constructor. Adding the same object twice is a no-op.This is PR 1 of the split proposed in #4676 (generic attribution labels). It stands on its own, and the later PRs only widen what goes into
attribution.Type of Change
Changes Made
headroom/pipeline.py:OutcomeSnapshot.attribution,conversation_key,original_tokens,output_tokens_saved, andPipelineExtensionManager.add(). All new fields have defaults, so existing constructors are unchanged.headroom/proxy/outcome.py:emit_request_outcomefills the new fields.add()(late attach, no duplicates) intests/test_pipeline_outcome_stage.py, and end-to-end throughemit_request_outcomeintests/test_proxy/test_conversation_savings.py.Testing
pytest)ruff check .)mypy headroom)Test Output
Real Behavior Proof
OUTCOME_OBSERVEDextension reading these fields.Runtime Rollout Safety
Review Readiness
Checklist
CHANGELOG.md— it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this)Screenshots (if applicable)
N/A
Additional Notes
attributionuses a mapping rather than more scalar fields so that later dimensions (host, team, …) need no further change to this contract.