feat(contracts): loop.yaml run contract — compile, fail closed, gate only tightens - #47
Merged
Conversation
…only tightens Run inputs (target, goal, lane, ten budget knobs) existed only as CLI flags: un-diffable, unreviewable, impossible to attach to the proof pack the run produced. Ships src/loopeng/contracts/ + `run --contract`, `contract check`, `contract evidence`. Three rules keep it from becoming decoration: 1. It compiles, it doesn't extend. Every key becomes a config.Budget / Lane the loop already reads — no new controller state, no knob the engine ignores. 2. An unenforced declaration is a parse error, not a no-op. A typo'd `max_iteratons:`, a `safety:` block nothing consumes, or maker-authored `evaluation.dimensions` (maker != checker laundering) all fail the parse. 3. The gate is monotonic. `require_human_confirm: false` is rejected — a caller-authored file must never hand back the bypass VerificationGate deliberately withholds. evidence.required names real ProofPack fields and is verified against the recorded run (exit 1 on a gap). Conflicting flags alongside --contract fail closed rather than silently taking precedence. Also records docs/solutions/external-eval-2026-08-12.md: the item-by-item answer to the external architecture review that prompted this, including the one proposal rejected as a regression (artifact_type as a closed enum re-centralizes the open Domain registry) and the counter-scorecard. 57 tests in tests/test_run_contract.py; suite 543 -> 600 passing, ruff clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…nt in the eval doc The reviewer of the article caught the off-by-one: the eval doc was written before the example-pinning test landed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Run inputs (target, goal, lane, ten budget knobs) existed only as CLI flags: un-diffable, unreviewable, and impossible to attach to the proof pack the run produced. This adds
src/loopeng/contracts/plusrun --contract,contract check, andcontract evidence.Three rules that keep it from becoming decoration
config.Budget/Lanethe loop already reads — no new controller state, no knob the engine ignores. One code path, two doors.max_iteratons:, asafety:block nothing consumes, or maker-authoredevaluation.dimensions(maker ≠ checker laundering) all fail the parse, and the message names what is accepted.gate.require_human_confirm: falseis rejected — a caller-authored file must never hand back the bypassVerificationGatedeliberately withholds.evidence.requirednames realProofPackfields and is verified against the recorded run (exit 1 on a gap). Conflicting flags alongside--contractfail closed rather than silently taking precedence.Origin
This is items 1 and 7 of an external architecture review that scored the repo 8.6/10.
docs/solutions/external-eval-2026-08-12.mdadjudicates all twelve proposals against the code rather than the README: 6 real gaps, 5 partial, and 1 rejected as a regression (artifact_typeas a closedLiteralwould re-centralize what the openDomainregistry decentralized). It also records the meta-finding — every claim the review made about the code cited the README — and a counter-scorecard that scores human-gate and generality higher than the review, and safety, observability and enterprise readiness lower.The best idea in the review (a false-green rate benchmark) is named as the top remaining P0.
Evidence
tests/test_run_contract.py; suite 543 → 600 passing, 2 skipped.ruff checkclean against the design-fitness budget (C901 / PLR0915).docs/examples/loop.yamlis pinned by a test, so it cannot rot.skills/loop-anything/SKILL.mdupdated in the same change.🤖 Generated with Claude Code