Repository navigation
Harden Forge validation boundaries and fail-closed requirement evidence - #33
Conversation
Refs #32. Keep unsupported horizons unresolved instead of silently using 90 days; preserve literal extraction, polarity and CSV CLI scope.
|
@codex review Please review the consolidated four-checkpoint diff at head Focus on concrete correctness regressions in declared artifact file confinement/integrity across validation and packaging, unquoted positive quality-requirement scoping, causal assertion provenance under rebinding and nested scopes, and fail-closed propagation of detected but unrecognized numeric expiration obligations (issue #32). Check the paired false-rejection controls for negation, quoted literals, unrelated retention durations and inclusive comparative coordination. The strict expiration extractor remains unchanged; detection does not prove a threshold or a comparison. Bounded prose recognition and conservative assertion analysis are not general NLP/control-flow proofs. The internal 30/30 gate is not new blind evidence. No new domains, retries, case-specific rules, rollback changes or frozen-baseline rescoring are intended. Please report reproducible actionable findings, distinguishing introduced regressions from pre-existing limitations. Review only; do not push fixes automatically. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5685ab71c5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Review follow-up completed at All three reported findings (two P1, one P2) were reproduced and addressed with regression-first tests:
39 additional test cases, including three additional real Docker cases. Initial targeted run: 16 failed / 33 passed before correction. Independent local review caught a temporary Both push CI and PR CI passed 1,113 tests and the Docker internal gate 30/30, with zero model requests/tokens/cost. CodeQL is green. No README, benchmark, frozen-evidence, retry, domain or rollback changes. The three original review threads have explicit evidence-backed replies and are marked resolved. The GitHub Codex review remains on |
Summary
Consolidate four separately tested checkpoints on top of
3804f1d:c987f33: validate the entire artifact file set before writing it; enforce portable, confined paths and distinct files; detect changes to declared artifact bytes across validation layers; independently guard packaging and terminate on integrity violations.7195ee6: scope quality requirements to unquoted, positive prose mentions rather than borrowing requirements from literal examples or locally rejected features.1b84901: track assertion provenance through binding changes; do not promote overwritten results, future calls, comments/docstrings or unexecuted nested definitions into causal test evidence.5685ab7: distinguish an absent numeric expiration obligation from a detected but uninterpretable one. Preserve unresolved threshold/relation and requirement IDs instead of silently generating a 90-day horizon. Refs Expiration horizon silently falls back to 90 days when the requirement's wording does not match the compiler's single regex #32.Issue #32 reproduction and correction
The five explicit examples were independently reproduced on
1b84901using the existing deterministic_numeric_planharness and full generation/validation. Canonicalin less than 30 dayscompiled 30 and ran one numeric probe. The four unsupported variants generated defaults of 90, passed validation and ran no numeric probes; actual function and CLI execution flagged a 31-day expiration as within the horizon.The correction separates broader, local numeric-expiration detection from the unchanged strict extractor. Detected hard/universal policies not precisely extracted remain unresolved (
threshold_days=None,comparison_relation=None) and enter the existing material-ambiguity/numeric-unproven path. A recognized bound cannot hide an additional unrecognized policy. Generation, validation and packaging refusal are covered both locally and through four real Docker regressions in Actions.Negation, quoted literals, unrelated retention durations and inclusive comparative coordination have paired regressions. Inclusive wording is not reinterpreted as
<.Verification at exact head
5685ab71c5f97304844b76b522e7eb26be63a3ecgit diff --checkpassed.Scope and remaining limits
No new domains, retries, reverse_chunks-specific rules, best-candidate rollback changes, README edits, benchmark edits, frozen-evidence changes or blind-baseline rescoring. V8-005 is not used as new blind evidence. This internal gate is not a new blind experiment.
Expiration detection is bounded local recognition, not universal natural-language interpretation. Assertion analysis is conservative provenance evidence, not a full control-flow or object-alias proof. The 48-phrase external matrix was not fully reproduced and no real-world failure rate is claimed.
Review focus
Check concrete regressions in artifact boundary enforcement, quality polarity/literal scoping, causal assertion provenance, and unresolved numeric expiration propagation. In particular, verify false rejection controls and that unsupported numeric wording cannot silently regain a generated default or a verified package. Do not infer semantic correctness from lexical overlap or the internal benchmark alone.
No automatic merge requested.