Skip to content

feat(pipeline): say whose request an OutcomeSnapshot is - #4698

Merged
JerrettDavis merged 2 commits into
headroomlabs-ai:mainfrom
mosatch:feat/outcome-snapshot-attribution
Nov 9, 2026
Merged

JerrettDavis merged 2 commits into
headroomlabs-ai:mainfrom
mosatch:feat/outcome-snapshot-attribution

Conversation

@mosatch

@mosatch mosatch commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

Description

OutcomeSnapshot is the one public seam a pipeline extension gets at OUTCOME_OBSERVED, but it says nothing about whose request it measured. An extension that wants to split what it observes (an exporter, a chargeback job, a budget alert) cannot tell projects, clients or conversations apart without patching core. This adds four read-only fields and fills them from values emit_request_outcome already has:

  • attribution: {dimension: label}. For now project and client, from the same values the savings tracker books, and only when known.
  • conversation_key: the handler's key, already on RequestOutcome.
  • original_tokens: the pre-compression size, on the same tokenizer as tokens_saved.
  • output_tokens_saved: the output-shaping estimate this request booked.

attribution is frozen into a read-only dict subclass (FrozenLabels) in __post_init__. The dataclass is already frozen, but a frozen field can still hold a mutable dict, so without this one extension could rewrite the labels the next one reads. A dict subclass rather than a MappingProxyType keeps the snapshot hashable, picklable and safe for dataclasses.asdict, copy.deepcopy and json.dumps, as it was before.

It also adds PipelineExtensionManager.add(). A proxy extension (headroom.proxy_extension) that also wants lifecycle events is installed after the manager exists, so it cannot be passed to the constructor. Adding the same object twice is a no-op.

This is PR 1 of the split proposed in #4676 (generic attribution labels). It stands on its own, and the later PRs only widen what goes into attribution.

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update
  • Performance improvement
  • Code refactoring (no functional changes)

Changes Made

  • headroom/pipeline.py: OutcomeSnapshot.attribution, conversation_key, original_tokens, output_tokens_saved, and PipelineExtensionManager.add(). All new fields have defaults, so existing constructors are unchanged.
  • headroom/proxy/outcome.py: emit_request_outcome fills the new fields.
  • Tests: the fields and their immutability, add() (late attach, no duplicates) in tests/test_pipeline_outcome_stage.py, and end-to-end through emit_request_outcome in tests/test_proxy/test_conversation_savings.py.

Testing

  • Unit tests pass (pytest)
  • Linting passes (ruff check .)
  • Type checking passes (mypy headroom)
  • New tests added for new functionality
  • Manual testing performed

Test Output

$ pytest tests/test_proxy tests/test_pipeline.py tests/test_pipeline_outcome_stage.py tests/test_5xx_accounting_all_providers.py
615 passed, 7 warnings in 62.72s          # upstream/main at bb8d18a5: 608 passed

$ ruff check <changed files> && ruff format --check <changed files>     # ruff 0.17.3
All checks passed!
4 files already formatted

$ mypy headroom
No errors in the changed files. The 12 errors reported are in ccr/mcp_server.py,
memory/mcp_server.py, memory/adapters/embedders.py and providers/model_metadata.py,
identical on upstream/main, from local mcp/library versions.

Real Behavior Proof

  • Environment: macOS, a Headroom proxy serving as a LAN gateway for three machines.
  • Exact command / steps: a superset of this change (the full RFC: one label set for request attribution (project, client, host, …) instead of one axis per dimension #4676 proposal) has run there since 2026-11-06 with an OUTCOME_OBSERVED extension reading these fields.
  • Observed result: the extension reads per-request project, client and conversation from the snapshot with no core patch.
  • Not tested: this exact subset was not deployed on its own; it is covered by the unit tests above.

Runtime Rollout Safety

  • Rollout-managed feature(s): none
  • Minimum rollout channel: n/a
  • Stable/default behavior changed: no. The fields are new and additive, and nothing reads them unless an extension is enabled.
  • Kill switch / disable path: n/a (extensions are already opt-in)
  • Unsafe override required: no
  • Qualification impact: none
  • Rollback path: revert

Review Readiness

  • I have performed a self-review
  • This PR is ready for human review

Checklist

  • My code follows the project's style guidelines
  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I did not edit CHANGELOG.md — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this)

Screenshots (if applicable)

N/A

Additional Notes

@github-actions

github-actions Bot commented Nov 8, 2026

Copy link
Copy Markdown
Contributor

PR governance

This PR follows the template and is marked ready for human review.

@github-actions github-actions Bot added the status: ready for review Pull request body is complete and the author marked it ready for human review label Nov 8, 2026

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 potential issue.

Devin Review

Comment thread headroom/pipeline.py Resolved
@codecov-commenter

codecov-commenter commented Nov 8, 2026 •

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 72.00000% with 7 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
headroom/pipeline.py 72.00% 7 Missing ⚠️

📢 Thoughts on this report? Let us know!

@JerrettDavis JerrettDavis left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed exact head 4896d5404dadcd0f7e4d4e06d37c4c3b7760158e, containing current upstream main bb8d18a56abb07ca7e811235c86ea6b626eb1f8b, and the author/Devin discussion. Outcome snapshots gain a defensive read-only label copy and the conversation/input/output-savings fields without losing ordinary serialization, copying, pickling or hashing. Identity-based extension registration avoids duplicate delivery for the same object. No correctness blocker found in this scope.

All 23 pipeline/outcome-stage tests pass. The serialization regressions fail against the preceding mapping-proxy implementation and pass on this head, addressing Devin's compatibility finding for both empty and populated labels. The outcome funnel passes the fields already resolved by the existing project/client paths; this source approval does not qualify the separate configurable attribution feature in #4699.

All upstream CI, Rust, Security, Merge Conflicts, Changelog Guard and latest governance runs succeeded; the older cancelled governance run is superseded. Ready for independent human merge review. Nothing merged.

@JerrettDavis
JerrettDavis merged commit a2bcbd1 into headroomlabs-ai:main Nov 9, 2026
45 of 47 checks passed
@headroom-release headroom-release Bot mentioned this pull request Nov 9, 2026
JerrettDavis pushed a commit that referenced this pull request Nov 10, 2026
🤖 I have created a release *beep* *boop*
---


##
[0.44.0](v0.43.1...v0.44.0)
(2026-11-09)


### ⚠ BREAKING CHANGES

* **packaging:** keep PyTorch extras out of all
([#4067](#4067))

### Features

* **pipeline:** say whose request an OutcomeSnapshot is
([#4698](#4698))
([a2bcbd1](a2bcbd1))
* **providers:** add Hermes agent wrap support
([#934](#934))
([7877241](7877241))
* **proxy/model-router:** add shadow mode
(HEADROOM_MODEL_ROUTER_LOG_ONLY)
([#2395](#2395))
([861ef95](861ef95))
* **proxy:** add lossless /v1/compress mode and detect provider by wire
shape ([#4617](#4617))
([33f067a](33f067a))
* **proxy:** expose per-request cost_usd in /stats recent_requests
([#1079](#1079))
([#1565](#1565))
([84cedbf](84cedbf))
* **proxy:** expose shared Redis store availability in /health and
/metrics
([#4578](#4578))
([602ca65](602ca65))
* **proxy:** give turn hooks one conversation id per client conversation
([#4496](#4496))
([94a99ed](94a99ed))
* **proxy:** report DeepSeek as its own provider in dashboard
([#2565](#2565))
([9ad16a7](9ad16a7))
* **proxy:** show per-request /stats metadata to token-authenticated
callers
([#4168](#4168))
([9b2dec5](9b2dec5))
* **proxy:** unified multi-upstream routing for Anthropic Cloud and
local oMLX
([#4597](#4597))
([f896161](f896161))
* **savings:** carry cumulative actual output_tokens in history
checkpoints
([#2921](#2921))
([8388ff5](8388ff5))
* **wrap:** add `headroom wrap droid` for Factory Droid
([#4143](#4143))
([bb8d18a](bb8d18a))
* **wrap:** make the Serena code-memory MCP opt-in
([#4509](#4509))
([2649835](2649835))


### Bug Fixes

* **asgi:** stop buffering when the client disconnects
(GHSA-mc2f-vg4v-2fh4)
([#4701](#4701))
([c6d7081](c6d7081))
* **beacon:** enforce the allowlist below the top-level keys
([#4691](#4691))
([456288d](456288d))
* **binaries:** fail closed on
HEADROOM_BINARIES_ALLOW_UNVERIFIED=0/false
([#3218](#3218))
([da6d973](da6d973))
* **cache:** record TTL observations with relative filenames
([#4711](#4711))
([51d9dc5](51d9dc5))
* **cache:** round learned TTL upper bounds up to whole seconds
([#4710](#4710))
([13347e3](13347e3))
* **cache:** save prefix-tracker state when the exit signal arrives
([#4715](#4715))
([3ba4223](3ba4223))
* **ccr:** lowercase the hash in the streaming feedback recorders
([#2313](#2313))
([3912c04](3912c04))
* **ci:** keep the npm publish token out of install and build steps
([#4690](#4690))
([8f6366c](8f6366c))
* **cli:** resolve active proxy port for read commands
([#3522](#3522))
([02c4e92](02c4e92))
* **content-router:** trust confident HTML detection over mixed-content
heuristics
([#3627](#3627))
([f50db67](f50db67))
* **dashboard:** lifetime metrics and refresh
([#2374](#2374))
([eb9932a](eb9932a))
* **doctor:** recognize IPv6 loopback and case-insensitive proxy routing
URLs ([#3255](#3255))
([2b0315b](2b0315b))
* **install:** decode Windows Task Scheduler XML output before parsing
([#3969](#3969))
([#4064](#4064))
([b78e07d](b78e07d))
* **install:** make install status agree with doctor when the proxy is
reachable
([#3072](#3072))
([#3141](#3141))
([05431b2](05431b2))
* **langchain:** check optional integration availability
([#4042](#4042))
([cde584d](cde584d))
* **learn:** harden analyzer JSON parsing against malformed LLM output
([#2471](#2471))
([12864fc](12864fc))
* **learn:** support opencode v2 session schema in learn scanner
([#4199](#4199))
([c05466a](c05466a))
* **lossless:** fold only pure path listings, never markup
([#4716](#4716))
([0ccd559](0ccd559))
* **memory:** gracefully degrade on context lookup and initialization
errors
([#3251](#3251))
([#4049](#4049))
([00d4f44](00d4f44))
* **metrics:** cap savings_by_source rows and fold the rest into other
([#4257](#4257))
([2038fb2](2038fb2))
* **packaging:** keep PyTorch extras out of all
([#4067](#4067))
([a890521](a890521))
* **proxy:** add --no-ccr-response-handling flag to bypass response
buffering
([#3082](#3082))
([#3100](#3100))
([ce14271](ce14271))
* **proxy:** bound the Bedrock Prometheus model label
([#4694](#4694))
([d70a8b7](d70a8b7))
* **proxy:** drop headers named by Connection when forwarding responses
([#4674](#4674))
([3871b36](3871b36))
* **proxy:** keep cold-turn reasoning drops sticky so the next warm turn
reuses the cached prefix
([#4592](#4592))
([3544cf5](3544cf5))
* **proxy:** keep the Bedrock credentials provider when startup
resolution fails
([#4705](#4705))
([5438f00](5438f00))
* **proxy:** refresh Bedrock AWS credentials before they expire
([#4696](#4696))
([adc6724](adc6724))
* **proxy:** rename leaked_threads metric to timed_out_workers_total
([#3908](#3908))
([bd164a2](bd164a2))
* **proxy:** track response.-prefixed item and content-part events in
Responses SSE state
([#4695](#4695))
([8f4122f](8f4122f))
* **router:** compress tool_result blocks in mutable window with
trailing messages
([#4717](#4717))
([0511bcd](0511bcd))
* **router:** ignore shell operators inside quoted arguments in read
detection
([#4646](#4646))
([8cdce17](8cdce17))
* **scripts:** skip ci-precheck in the pre-push hook when a push only
deletes refs
([#4702](#4702))
([cbcf8d8](cbcf8d8))
* **sdk:** keep Vercel AI SDK tool-result output kinds through
compression
([#4670](#4670))
([ce42754](ce42754))
* **search_compressor:** keep kept rows byte-for-byte and in input order
([#3982](#3982))
([a538740](a538740))
* **smart-crusher:** keep mixed-array dict rows and exact drop count
with factor_out_constants
([#4692](#4692))
([ced2685](ced2685))
* **smart-crusher:** keep the row the question names
([#4713](#4713))
([e6e2ebb](e6e2ebb))
* **storage:** return the latest snapshot from JSONLStorage.get
([#4675](#4675))
([b25c45b](b25c45b))
* **tabular:** split fixed-width rows on column boundaries, not on
double spaces
([#4541](#4541))
([b80245e](b80245e))
* **tag-protector:** do not treat a slash in an unquoted attribute as
self-closing
([#4681](#4681))
([d5644cb](d5644cb))
* **tag-protector:** keep the enclosing tag open after a nested close in
marker mode
([#4682](#4682))
([385d487](385d487))
* **telemetry:** keep install ID stateless
([#3542](#3542))
([27bba79](27bba79))
* **tokenizers:** resolve chatgpt-4o-* aliases to o200k_base
([#4672](#4672))
([9648d2d](9648d2d))
* **tokenizers:** unwrap BatchEncoding in
HuggingFaceTokenizer.count_messages
([#2447](#2447))
([9eed58f](9eed58f))
* **vertex:** explain Vertex failures in-band, correct the model/region
matrix
([#2640](#2640))
([0ddaa56](0ddaa56))
* **wrap:** route Claude Code's Bedrock mode through Headroom
([#4316](#4316))
([aa2093b](aa2093b))


### Dependencies

* bump rusqlite from 0.40.2 to 0.41.1
([#4658](#4658))
([74b7ac2](74b7ac2))
* bump the cargo-minor-patch group with 3 updates
([#4657](#4657))
([c01cf07](c01cf07))
* bump the npm-minor-patch group across 1 directory with 2 updates
([#4650](#4650))
([def4ef1](def4ef1))
* update click requirement from &gt;=8.3.3 to &gt;=8.5.0
([#4655](#4655))
([ea59ac3](ea59ac3))
* update sqlite-vec requirement from &gt;=0.1.6 to &gt;=0.1.9
([#4654](#4654))
([de1a1e0](de1a1e0))
* update tree-sitter requirement from &lt;0.27,&gt;=0.25.2 to
&gt;=0.26.0,&lt;0.27
([#4656](#4656))
([910588e](910588e))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

---------

Co-authored-by: headroom-release[bot] <352661243+headroom-release[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

status: ready for review Pull request body is complete and the author marked it ready for human review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants