Skip to content

feat(contracts): loop.yaml run contract — compile, fail closed, gate only tightens - #47

Merged
wjlgatech merged 3 commits into
mainfrom
feat/run-contract
Aug 13, 2026
Merged

feat(contracts): loop.yaml run contract — compile, fail closed, gate only tightens#47
wjlgatech merged 3 commits into
mainfrom
feat/run-contract

Conversation

@wjlgatech

Copy link
Copy Markdown
Owner

What

Run inputs (target, goal, lane, ten budget knobs) existed only as CLI flags: un-diffable, unreviewable, and impossible to attach to the proof pack the run produced. This adds src/loopeng/contracts/ plus run --contract, contract check, and contract evidence.

loop-anything contract check docs/examples/loop.yaml
loop-anything run --contract docs/examples/loop.yaml
loop-anything contract evidence docs/examples/loop.yaml --run 12

Three rules that keep it from becoming decoration

  1. It compiles, it doesn't extend. Every key becomes a config.Budget / Lane the loop already reads — no new controller state, no knob the engine ignores. One code path, two doors.
  2. An unenforced declaration is a parse error, not a no-op. A typo'd max_iteratons:, a safety: block nothing consumes, or maker-authored evaluation.dimensions (maker ≠ checker laundering) all fail the parse, and the message names what is accepted.
  3. The gate is monotonic. gate.require_human_confirm: false is rejected — a caller-authored file must never hand back the bypass VerificationGate deliberately withholds.

evidence.required names real ProofPack fields and is verified against the recorded run (exit 1 on a gap). Conflicting flags alongside --contract fail closed rather than silently taking precedence.

Origin

This is items 1 and 7 of an external architecture review that scored the repo 8.6/10. docs/solutions/external-eval-2026-08-12.md adjudicates all twelve proposals against the code rather than the README: 6 real gaps, 5 partial, and 1 rejected as a regression (artifact_type as a closed Literal would re-centralize what the open Domain registry decentralized). It also records the meta-finding — every claim the review made about the code cited the README — and a counter-scorecard that scores human-gate and generality higher than the review, and safety, observability and enterprise readiness lower.

The best idea in the review (a false-green rate benchmark) is named as the top remaining P0.

Evidence

  • 57 new tests in tests/test_run_contract.py; suite 543 → 600 passing, 2 skipped.
  • ruff check clean against the design-fitness budget (C901 / PLR0915).
  • The shipped example docs/examples/loop.yaml is pinned by a test, so it cannot rot.
  • README, CHANGELOG, and skills/loop-anything/SKILL.md updated in the same change.

🤖 Generated with Claude Code

wjlgatech and others added 3 commits August 12, 2026 14:10
…only tightens

Run inputs (target, goal, lane, ten budget knobs) existed only as CLI flags:
un-diffable, unreviewable, impossible to attach to the proof pack the run
produced. Ships src/loopeng/contracts/ + `run --contract`, `contract check`,
`contract evidence`.

Three rules keep it from becoming decoration:

1. It compiles, it doesn't extend. Every key becomes a config.Budget / Lane the
   loop already reads — no new controller state, no knob the engine ignores.
2. An unenforced declaration is a parse error, not a no-op. A typo'd
   `max_iteratons:`, a `safety:` block nothing consumes, or maker-authored
   `evaluation.dimensions` (maker != checker laundering) all fail the parse.
3. The gate is monotonic. `require_human_confirm: false` is rejected — a
   caller-authored file must never hand back the bypass VerificationGate
   deliberately withholds.

evidence.required names real ProofPack fields and is verified against the
recorded run (exit 1 on a gap). Conflicting flags alongside --contract fail
closed rather than silently taking precedence.

Also records docs/solutions/external-eval-2026-08-12.md: the item-by-item answer
to the external architecture review that prompted this, including the one
proposal rejected as a regression (artifact_type as a closed enum re-centralizes
the open Domain registry) and the counter-scorecard.

57 tests in tests/test_run_contract.py; suite 543 -> 600 passing, ruff clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…nt in the eval doc

The reviewer of the article caught the off-by-one: the eval doc was written
before the example-pinning test landed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@wjlgatech
wjlgatech merged commit f0af567 into main Aug 13, 2026
3 checks passed
@wjlgatech
wjlgatech deleted the feat/run-contract branch August 13, 2026 02:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant