Skip to content

Harden Forge validation boundaries and fail-closed requirement evidence - #33

Merged
Daniele-Cangi merged 5 commits into
mainfrom
codex/validated-artifact-boundaries
Oct 9, 2026
Merged

Daniele-Cangi merged 5 commits into
mainfrom
codex/validated-artifact-boundaries

Conversation

@Daniele-Cangi

Copy link
Copy Markdown
Owner

Summary

Consolidate four separately tested checkpoints on top of 3804f1d:

  • c987f33: validate the entire artifact file set before writing it; enforce portable, confined paths and distinct files; detect changes to declared artifact bytes across validation layers; independently guard packaging and terminate on integrity violations.
  • 7195ee6: scope quality requirements to unquoted, positive prose mentions rather than borrowing requirements from literal examples or locally rejected features.
  • 1b84901: track assertion provenance through binding changes; do not promote overwritten results, future calls, comments/docstrings or unexecuted nested definitions into causal test evidence.
  • 5685ab7: distinguish an absent numeric expiration obligation from a detected but uninterpretable one. Preserve unresolved threshold/relation and requirement IDs instead of silently generating a 90-day horizon. Refs Expiration horizon silently falls back to 90 days when the requirement's wording does not match the compiler's single regex #32.

Issue #32 reproduction and correction

The five explicit examples were independently reproduced on 1b84901 using the existing deterministic _numeric_plan harness and full generation/validation. Canonical in less than 30 days compiled 30 and ran one numeric probe. The four unsupported variants generated defaults of 90, passed validation and ran no numeric probes; actual function and CLI execution flagged a 31-day expiration as within the horizon.

The correction separates broader, local numeric-expiration detection from the unchanged strict extractor. Detected hard/universal policies not precisely extracted remain unresolved (threshold_days=None, comparison_relation=None) and enter the existing material-ambiguity/numeric-unproven path. A recognized bound cannot hide an additional unrecognized policy. Generation, validation and packaging refusal are covered both locally and through four real Docker regressions in Actions.

Negation, quoted literals, unrelated retention durations and inclusive comparative coordination have paired regressions. Inclusive wording is not reinterpreted as <.

Verification at exact head 5685ab71c5f97304844b76b522e7eb26be63a3ec

  • Push Actions run: both jobs successful; 1,074 tests passed, including real Docker integration and the new unresolved-horizon cases.
  • Docker extended internal gate: 30/30 expected outcomes, status accuracy and Verified@1 1.0, false-verified rate 0.0, infeasible detection rate 1.0; zero model requests/tokens/cost.
  • Final local numeric/compiler/fidelity/quality compatibility run: 268 passed, 4 Docker skipped.
  • Ruff F/E9 and git diff --check passed.
  • Each checkpoint received an independent local review. Concrete findings were reproduced before correction; no residual blocker was reported within the reviewed scope. This is not a claim of completed GitHub Codex review or approval.

Scope and remaining limits

No new domains, retries, reverse_chunks-specific rules, best-candidate rollback changes, README edits, benchmark edits, frozen-evidence changes or blind-baseline rescoring. V8-005 is not used as new blind evidence. This internal gate is not a new blind experiment.

Expiration detection is bounded local recognition, not universal natural-language interpretation. Assertion analysis is conservative provenance evidence, not a full control-flow or object-alias proof. The 48-phrase external matrix was not fully reproduced and no real-world failure rate is claimed.

Review focus

Check concrete regressions in artifact boundary enforcement, quality polarity/literal scoping, causal assertion provenance, and unresolved numeric expiration propagation. In particular, verify false rejection controls and that unsupported numeric wording cannot silently regain a generated default or a verified package. Do not infer semantic correctness from lexical overlap or the internal benchmark alone.

No automatic merge requested.

@Daniele-Cangi

Copy link
Copy Markdown
Owner Author

@codex review

Please review the consolidated four-checkpoint diff at head 5685ab71c5f97304844b76b522e7eb26be63a3ec against main 3804f1d.

Focus on concrete correctness regressions in declared artifact file confinement/integrity across validation and packaging, unquoted positive quality-requirement scoping, causal assertion provenance under rebinding and nested scopes, and fail-closed propagation of detected but unrecognized numeric expiration obligations (issue #32). Check the paired false-rejection controls for negation, quoted literals, unrelated retention durations and inclusive comparative coordination.

The strict expiration extractor remains unchanged; detection does not prove a threshold or a comparison. Bounded prose recognition and conservative assertion analysis are not general NLP/control-flow proofs. The internal 30/30 gate is not new blind evidence. No new domains, retries, case-specific rules, rollback changes or frozen-baseline rescoring are intended. Please report reproducible actionable findings, distinguishing introduced regressions from pre-existing limitations. Review only; do not push fixes automatically.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5685ab71c5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread core/forge/expiration_contract.py Resolved
Comment thread core/forge/expiration_contract.py Resolved
Comment thread core/forge/requirement_text.py Resolved
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-09T12:45:15.030130Z 5685ab7 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Address PR #33 review findings with regression-first fixes; retain unresolved numeric semantics and list-wide prohibitions. Refs #32.
@Daniele-Cangi

Copy link
Copy Markdown
Owner Author

Review follow-up completed at 74b699570f8002bb10412fcd6bb19de38d9ddfba.

All three reported findings (two P1, one P2) were reproduced and addressed with regression-first tests:

  • Inspect additional numeric expiration policies after a recognized prefix, retaining a single originating requirement ID.
  • Preserve prohibitive exceed comparisons as unresolved numeric obligations, without introducing new threshold/relation extraction or masking outer rejections.
  • Scope determiner no to its object under positive directives while preserving intervening list-wide rejection such as without.

39 additional test cases, including three additional real Docker cases. Initial targeted run: 16 failed / 33 passed before correction. Independent local review caught a temporary without regression, corrected before commit; final focused verification: 52 passed. Final local compatibility: 304 passed / 7 Docker skipped.

Both push CI and PR CI passed 1,113 tests and the Docker internal gate 30/30, with zero model requests/tokens/cost. CodeQL is green. No README, benchmark, frozen-evidence, retry, domain or rollback changes.

The three original review threads have explicit evidence-backed replies and are marked resolved. The GitHub Codex review remains on 5685ab7; no fresh automated review or approval is claimed for 74b6995. No automatic re-review/fix loop or merge was initiated. Issue #32 remains open pending integration.

@Daniele-Cangi
Daniele-Cangi merged commit ffece8e into main Oct 9, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant