deep-review: add convergent round thresholds - #28
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c8f783aeb2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| Reproducer confirmation strengthens the evidence side of the threshold but | ||
| does not bypass the multiplier for a new finding. An unresolved blocker from an |
There was a problem hiding this comment.
On round 2 or later, a new bug whose static evidence does not yet clear the multiplier is downgraded to SUGGESTION/NOTE by step 5, but Phase 4 of SKILL.md launches reproducers only for findings already classified as BLOCKING. Such a candidate can therefore never obtain the confirmation that this paragraph says strengthens its evidence, allowing a reproducible bug to remain non-blocking and the review to approve. Reproduce eligible reproducer_needed candidates before final threshold classification, or introduce a provisional-candidate state.
Useful? React with 👍 / 👎.
| After collecting this context, read | ||
| [iterative-reviews.md](iterative-reviews.md) to determine the review round, | ||
| threshold multiplier, carried blockers, and new candidates. For a local branch |
There was a problem hiding this comment.
For GitLab MRs, the context collected before this newly required round calculation excludes verdicts produced by the skill itself: setup.md line 73 searches for Generated by /...deep-review, while SKILL.md line 190 mandates the footer Generated by the deep-review skill; the literal footer does not match that regex. Consequently repeated GitLab reviews are treated as round 1 and previously blocking findings are not carried forward. Update the GitLab filter to recognize the actual footer, retaining any legacy marker if needed.
Useful? React with 👍 / 👎.
| or repost of one verdict does not create another round. If the user explicitly | ||
| supplies a round, use it. Without review history or explicit round context, use | ||
| round 1. |
There was a problem hiding this comment.
The fallback for local branches depends on the user explicitly supplying a round, but Step 1.1 and the skill's argument table define no round option; moreover, a bare integer is parsed as a PR identifier. Thus a user cannot reliably invoke a local second-round review through the documented command interface, and it silently remains at round 1. Add a documented, validated option such as --round N and include it in argument parsing.
Useful? React with 👍 / 👎.
| For a PR/MR, count completed deep-review panel verdicts from earlier | ||
| invocations; the current invocation is the next round. A retry, duplicate post, | ||
| or repost of one verdict does not create another round. If the user explicitly |
There was a problem hiding this comment.
On GitHub, the setup query treats every PR comment containing the public footer text as a prior panel verdict, without checking its author or validating the verdict structure. Any participant who quotes or copies that footer can therefore inflate the count used here; several such comments can push the review to a very high multiplier and cause legitimate new blockers to be downgraded. Count only comments created by the expected reviewer identity and matching a complete verdict, or store a verifiable invocation identifier.
Useful? React with 👍 / 👎.
1 participant
Summary
Validation
uvx skillsaw==0.18.0 lint plugins/reviews/skills/deep-review— A+, zero findingsmake lint— A+, zero errors or warningsValidator note
The generic skill-creator
quick_validate.pyrejects the existingargument-hintfrontmatter key. The same failure reproduces unchanged onorigin/main; the repository-pinned skillsaw validation passes.