Skip to content

feat(ci): add shared measurement inventory and mechanism gate - #804

Merged
lahlfors merged 1 commit into
mainfrom
feat/measurement-inventory-gate
Nov 17, 2026
Merged

lahlfors merged 1 commit into
mainfrom
feat/measurement-inventory-gate

Conversation

@lahlfors

Copy link
Copy Markdown
Collaborator

Summary

Implements the shared AST/config measurement discovery layer (POAM-2026-222 & POAM-2026-234) and wires the zero-waiver mechanism-reference and TTM signing-hold gate (POAM-2026-222):

  • scripts/measurement_inventory.py: Discovers all four repository measurement surfaces directly from source and declarative configuration — PrometheusMetric (>= 42), AlertPolicy (>= 8), LulaBridgeSpec (>= 4), and BridgeScoreSurface (>= 51) — with non-vacuous CorpusFloorError floors and zero hard-coded lists, ready for independent consumption by POAM-2026-222 and POAM-2026-234's bidirectional failure-direction gate.
  • scripts/check_mechanism_references.py: Zero-waiver CI and pre-commit gate verifying:
    1. Every cage_* Prometheus series in AlertPolicy.referenced_metrics, infra/**/*.tf PromQL/MQL/GMP descriptors, and infra/**/*.{json,yaml,yml} dashboards/relabeling manifests resolves to a discovered PrometheusMetric.
    2. Every active LulaBridgeSpec targets a real compliance-bridge route and reads fields produced by BridgeScoreSurface.
    3. Every google_monitoring_alert_policy.<label> and cage_<metric> series cited across compliance/oscal/**/*.yaml and living DOC_SCOPE documents resolves in the discovered inventory.
    4. CAGE-TTM-001 (compliance/risk_acceptance/THRESHOLD_TRACEABILITY_MATRIX.md) keeps its Evidence Withdrawal and Signing Hold notice and unsigned §5 RA-001..RA-004 / §6 Approval Block while POAM-2026-222 is Open, and fails closed if POAM-2026-222 is marked closed while any cage.<subsystem>.* un-emitted metric citation remains in §2.1 or Appendix B.
  • Doc Fixes Caught by Gate: Updates stale metric citations in docs/architecture/EVIDENCE_CHAIN.md (cage_evidence_custody_batches_total) and docs/architecture/GATEWAY_ARCHITECTURE.md (cage_ftra_parse_failures_total{failure_class}).
  • CI & Hook Wiring: Wired into .github/workflows/ci.yml (lint job), Makefile (docs-check), scripts/setup_git_hooks.sh, and tests/test_ci_gate_effectiveness.py.

Verification

  • CAGE_DEPLOYMENT_REGION=US_FED uv run pytest tests/unit/test_measurement_inventory.py tests/unit/test_check_mechanism_references.py tests/test_ci_gate_effectiveness.py -q -> 58 passed, 0 failed
  • uv run python scripts/check_mechanism_references.py --verbose -> pass (43 Prometheus metrics, 8 alert policies, 4 Lula bridge specs, 53 compliance-bridge score surfaces; 0 waivers)
  • uv run bandit -r src/ scripts/ -c pyproject.toml -ll -ii -> 145,891 lines scanned, 0 Medium / 0 High issues

Add a shared AST/config measurement discovery layer and wire the POAM-2026-222 mechanism-reference and TTM signing-hold gate:

- Add scripts/measurement_inventory.py discovering all four repository measurement surfaces directly from source and declarative config (PrometheusMetric >= 42, AlertPolicy >= 8, LulaBridgeSpec >= 4, BridgeScoreSurface >= 51) with non-vacuous CorpusFloorError floors for consumption by both POAM-2026-222 and POAM-2026-234.
- Add scripts/check_mechanism_references.py (zero waivers) verifying alert-policy/dashboard Prometheus series, active Lula compliance-bridge payload fields, OSCAL and DOC_SCOPE metric/alert citations, and CAGE-TTM-001 signing-hold / POAM-2026-222 closure invariants.
- Correct two stale documentation metric citations caught by the new gate in docs/architecture/EVIDENCE_CHAIN.md (cage_evidence_custody_batches_total) and docs/architecture/GATEWAY_ARCHITECTURE.md (cage_ftra_parse_failures_total).
- Wire check_mechanism_references.py into .github/workflows/ci.yml (lint), Makefile (docs-check), scripts/setup_git_hooks.sh, and tests/test_ci_gate_effectiveness.py.
@lahlfors
lahlfors merged commit 54d9484 into main Nov 17, 2026
49 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant