Skip to content

fix(governance)!: split OPA outages from signed policy refusals - #803

Open
lahlfors wants to merge 1 commit into
mainfrom
fix/infra-block-taxonomy
Open

lahlfors wants to merge 1 commit into
mainfrom
fix/infra-block-taxonomy

Conversation

@lahlfors

Copy link
Copy Markdown
Collaborator

Summary

Remediates Cause 1 and Consumers 1–3, 7–8, 10, and 19 of POAM-2026-233 (infrastructure failures recorded as KMS-signed policy refusals) while preserving fail-closed execution blocking across every stage:

  • OPAClient & PolicyClient (src/gateway/core/policy.py, src/gateway/governance/contracts.py): Raise PolicyEngineUnavailableError (instead of returning "DENY") across circuit-breaker open, latency-budget bankruptcy, timeout, transport error, HTTP non-2xx, and malformed or undefined Rego replies.
  • OpaStage & ViolationKind.INFRASTRUCTURE (src/gateway/governance/governor/stages/opa.py, src/gateway/governance/classification_engine.py, src/gateway/governance/governor/pipeline.py): Emit OPA_UNAVAILABLE with ViolationKind.INFRASTRUCTURE, rank HARD > INFRASTRUCTURE > RELIANCE_INELIGIBLE > ... (InfrastructureOutcome.INFRASTRUCTURE_BLOCK), and preserve Phase2Mode.SKIP, immediate pipeline stop, ReservationScope rollback, and zero seal issuance.
  • handle_infrastructure_block() (src/gateway/governance/governor/verdicts.py, src/gateway/governance/governor/governor.py, src/gateway/governance/governor/metrics.py, src/gateway/governance/governor/errors.py): Hash-chain unsigned GOVERNANCE_INFRASTRUCTURE_BLOCK operational events and increment cage_governance_infrastructure_blocks_total{dependency,code} without constructing RefusalReceipt, calling the KMS signer, incrementing cage_governance_refusals_total, calling mark_request_refused(), or dispatching auto-quarantine; raise GovernanceInfrastructureError (receipt=None).
  • POST /governance/validate-action (src/gateway/server/governance_middleware.py): Return typed HTTP 503 ({"error": "policy_engine_unavailable", "dependency": "opa", "failure_code": "OPA_UNAVAILABLE", "retry_after_seconds": 5}) with Retry-After: 5 and no proof_hash or refusal_receipt.
  • Formal Model, Conformance & Clean-Break Ledger: Align proof/model.py, packages/cage-conformance/, scripts/check_producer_consumer_keys.py, golden verdict vectors, and FA-1 in docs/BREAKING_CHANGES.md. (POAM-2026-233 remains Open for the remaining enumerated causes and consumers.)

Breaking Change

BREAKING CHANGE: PolicyClient.evaluate_policy() and OPAClient.evaluate_policy() raise PolicyEngineUnavailableError instead of returning "DENY" on OPA unavailability or malformed/undefined replies; SymbolicGovernor raises GovernanceInfrastructureError (receipt=None) and POST /governance/validate-action returns HTTP 503 (policy_engine_unavailable) instead of HTTP 403 (GovernanceViolation). Recorded as FA-1 in docs/BREAKING_CHANGES.md.

Verification

  • CAGE_DEPLOYMENT_REGION=US_FED uv run pytest tests/unit/test_opa_unavailability_fail_closed.py tests/test_gateway_core_policy.py tests/test_violation_kinds.py tests/test_opa_client.py tests/test_symbolic_governor.py tests/test_distributed_cbf_proof.py tests/test_governance_trace_conformance.py tests/test_cage_conformance.py tests/governor/ -q -> 1,040+ passed, 0 failed (plus full -m "local or unit" sweep: 11,944 passed, 0 failed)
  • uv run bandit -r src/ scripts/ -c pyproject.toml -ll -ii -> 143,705 lines scanned, 0 Medium / 0 High issues
  • uv run python scripts/check_producer_consumer_keys.py --verbose & uv run python scripts/check_doc_references.py -> pass (0 errors)

Separate policy-engine dependency unavailability from policy-evaluated refusals (POAM-2026-233 Cause 1 and Consumers 1-3, 7-8, 10, 19):

- Raise PolicyEngineUnavailableError across OPAClient circuit-breaker open, latency-budget bankruptcy, timeout, transport error, HTTP non-2xx, and malformed or undefined Rego replies instead of returning "DENY".
- Emit OPA_UNAVAILABLE with ViolationKind.INFRASTRUCTURE in OpaStage and rank HARD > INFRASTRUCTURE > RELIANCE_INELIGIBLE in ClassificationEngine (InfrastructureOutcome.INFRASTRUCTURE_BLOCK), preserving Phase2Mode.SKIP, immediate pipeline stop, ReservationScope rollback, and zero seal issuance.
- Route INFRASTRUCTURE_BLOCK to handle_infrastructure_block(), hash-chaining unsigned GOVERNANCE_INFRASTRUCTURE_BLOCK operational events and incrementing cage_governance_infrastructure_blocks_total{dependency,code} without constructing RefusalReceipt, calling the KMS signer, incrementing cage_governance_refusals_total, marking the request refused, or triggering auto-quarantine.
- Return typed HTTP 503 (policy_engine_unavailable) with Retry-After and without proof_hash or refusal_receipt on POST /governance/validate-action.
- Align proof/model.py, cage_conformance, Gate 11 producer keys, and golden/unit conformance suites.

BREAKING CHANGE: PolicyClient.evaluate_policy() and OPAClient.evaluate_policy() raise PolicyEngineUnavailableError instead of returning "DENY" on OPA unavailability or malformed/undefined replies; SymbolicGovernor raises GovernanceInfrastructureError (receipt=None) and POST /governance/validate-action returns HTTP 503 policy_engine_unavailable instead of HTTP 403 GovernanceViolation (FA-1 in docs/BREAKING_CHANGES.md).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant