From c8f783aeb2ddd9b53fae82b6a3b5125ded1635b9 Mon Sep 17 00:00:00 2001 From: Stephen Benjamin Date: Tue, 1 Sep 2026 12:14:27 -0400 Subject: [PATCH] feat(deep-review): add convergent round thresholds --- plugins/reviews/skills/deep-review/SKILL.md | 40 ++++++++----- .../skills/deep-review/references/dispatch.md | 12 ++++ .../references/iterative-reviews.md | 56 +++++++++++++++++++ .../skills/deep-review/references/setup.md | 8 ++- 4 files changed, 102 insertions(+), 14 deletions(-) create mode 100644 plugins/reviews/skills/deep-review/references/iterative-reviews.md diff --git a/plugins/reviews/skills/deep-review/SKILL.md b/plugins/reviews/skills/deep-review/SKILL.md index 45b06a7..1ff5837 100644 --- a/plugins/reviews/skills/deep-review/SKILL.md +++ b/plugins/reviews/skills/deep-review/SKILL.md @@ -112,19 +112,26 @@ branch." If a PR/MR exists, also fetch its description for context. -#### Step 1.4: Detect prior reviews (PR only) +#### Step 1.4: Detect prior reviews and determine the round Follow the GitHub/GitLab commands in [references/setup.md](references/setup.md). Pass prior `Generated by /deep-review` findings to specialists and the -arbiter so resolved items and regressions are handled. +arbiter so resolved items and regressions are handled. Then apply +[the iterative review convergence policy](references/iterative-reviews.md) +to determine the round and its blocking-threshold multiplier. Round 1 uses +the baseline threshold; later rounds use 2x, 4x, 8x, and continue doubling. +Unresolved blockers from earlier rounds carry forward without requalifying +against the higher threshold. ### Phase 2 — Dispatch Specialists Each specialist prompt lives under [references/specialists/](references/specialists/). Dispatch rules, JSON schema, parallel/serial modes, and external reviewers are in -[references/dispatch.md](references/dispatch.md). +[references/dispatch.md](references/dispatch.md). Give every specialist the +review round, multiplier, prior findings, and the distinction between carried +blockers and new candidates. | Specialist | Prompt | |------------|--------| @@ -164,15 +171,19 @@ Perform synthesis directly in the main agent (not a sub-agent). 1. **Deduplicate** — merge duplicates, keep strongest reproducer 2. **Filter noise** — remove false positives, style nitpicks, speculative findings, and issues already addressed in the branch -3. **Resolve conflicts** — corroboration strengthens; adversarial - concerns are blocking unless concretely refuted +3. **Resolve conflicts** — corroboration strengthens. Unresolved adversarial + blockers carry forward; new adversarial concerns must meet the current + round's threshold 4. **Assign disposition** — APPROVE (no BLOCKING), REQUEST_CHANGES (BLOCKING findings), or NEEDS_DISCUSSION (needs author input). Biases: security over ergonomics, consistency over elegance, - reproduced bugs are always BLOCKING, do not manufacture findings + reproduced bugs that meet the applicable round threshold are BLOCKING, + and unresolved reproduced blockers remain BLOCKING. Do not manufacture + findings 5. **Prioritize** — reproduced security bugs > reproduced functional bugs > unreproduced > architecture > style/docs -6. **Emit verdict** — use collapsible `
` blocks for +6. **Emit verdict** — state the review round and multiplier, then use + collapsible `
` blocks for specialist findings (each specialist collapsed with severity counts). Sections: Disposition, Specialist Findings, Panel Synthesis, Required Actions, Optional Follow-ups, Stats. @@ -189,12 +200,15 @@ be set from Step 1.2. ## Quality Gates -A change passes when: no unresolved functional bugs, no unrefuted -adversarial scenarios, no unmitigated vulnerabilities or supply -chain risks, sound architecture, no duplicate helpers, adequate -test coverage, documentation consistent with changes, and the -shipped text is free of review residue and generated-sounding -prose, and the panel arbiter has ratified the disposition. +At round 1, a change passes when there are no unresolved functional bugs, +unrefuted adversarial scenarios, unmitigated vulnerabilities or supply-chain +risks, structural defects, duplicate helpers, material test gaps, +documentation inconsistencies, or review residue and generated-sounding prose +in shipped text, and the panel arbiter has ratified the disposition. On later +rounds, those criteria still guide investigation, but only unresolved carried +blockers and new findings that satisfy the current round's escalating threshold +prevent approval. Keep below-threshold discoveries visible as suggestions or +notes so convergence does not hide useful feedback. ## Error Handling diff --git a/plugins/reviews/skills/deep-review/references/dispatch.md b/plugins/reviews/skills/deep-review/references/dispatch.md index f0c95c8..b12c26a 100644 --- a/plugins/reviews/skills/deep-review/references/dispatch.md +++ b/plugins/reviews/skills/deep-review/references/dispatch.md @@ -27,6 +27,10 @@ Append this schema to each specialist prompt: If no issues found, return an empty array and state what was checked. +Specialist severity rules describe the round 1 baseline. On later rounds, the +[iterative review policy](iterative-reviews.md) takes precedence for every new +BLOCKING classification; unresolved carried blockers retain their severity. + ## Prompt path resolution Resolve specialist prompts from the skill directory (repository @@ -49,6 +53,10 @@ Each sub-agent gets: - The merge base ref - The PR number or branch name being reviewed - Any prior review findings (if detected in Step 1.4) +- The current review round and threshold multiplier from the + [iterative review policy](iterative-reviews.md), including which prior + blockers carry forward and that the multiplier applies only to new BLOCKING + findings - The findings JSON schema above Sub-agents have full read access to the locally checked-out @@ -73,6 +81,10 @@ for review instructions, review through that lens, and produce findings in the same JSON format. Context from earlier specialists' file reads and findings carries over automatically. +Before the first specialist, read the +[iterative review policy](iterative-reviews.md) and apply the current round's +threshold consistently to every specialist's severity classification. + **Do NOT modify any files, and do NOT push to any remote.** Serial mode is read-only, same as parallel. diff --git a/plugins/reviews/skills/deep-review/references/iterative-reviews.md b/plugins/reviews/skills/deep-review/references/iterative-reviews.md new file mode 100644 index 0000000..64f3e6c --- /dev/null +++ b/plugins/reviews/skills/deep-review/references/iterative-reviews.md @@ -0,0 +1,56 @@ +# Iterative review convergence + +Read this reference when an earlier deep-review verdict exists for the same +PR/MR or the user identifies the current review as a later round. It governs +severity classification and final disposition in both parallel and serial +modes. + +## Determine the round and multiplier + +Round 1 uses the ordinary baseline for a blocking finding. Each successive +round doubles the threshold for a **new** blocking finding: + +| Round | New-blocker threshold | +|------:|----------------------:| +| 1 | 1x (baseline) | +| 2 | 2x | +| 3 | 4x | +| 4 | 8x | +| N | `2^(N-1)`x | + +For a PR/MR, count completed deep-review panel verdicts from earlier +invocations; the current invocation is the next round. A retry, duplicate post, +or repost of one verdict does not create another round. If the user explicitly +supplies a round, use it. Without review history or explicit round context, use +round 1. + +## Classify carried and new findings + +1. Inventory prior BLOCKING findings by their underlying defect, not by title, + line number, or wording, which may change between revisions. +2. Keep every unresolved prior blocker BLOCKING. It does not need to satisfy the + current multiplier again. Require evidence before treating it as resolved. +3. Treat a finding as new when no earlier verdict classified the underlying + defect as BLOCKING. Promoting an earlier suggestion or note is also a new + blocker. A defect shown resolved in an intervening verdict and later + reintroduced is a new regression. +4. Apply the multiplier only when deciding whether a new finding is BLOCKING. + The multiplier raises the combined bar for evidence, confidence, likelihood + in supported use, and consequence of merging. It is a comparative decision + rule, not an arithmetic bug score: at 2x a candidate must be materially more + compelling than an ordinary blocker; at 4x it needs high confidence and + severe merge impact; at 8x and beyond only increasingly exceptional, + well-supported risks should block. +5. Report a new finding that misses the current blocking threshold as a + SUGGESTION or NOTE. Do not suppress it, and do not let it determine the + disposition. + +Reproducer confirmation strengthens the evidence side of the threshold but +does not bypass the multiplier for a new finding. An unresolved blocker from an +earlier round continues to block even if no new reproduction is needed. + +## Arbiter output + +State the round and multiplier in the disposition. Separate carried blockers +from new findings, and say whether each new BLOCKING finding cleared the +current threshold. This makes convergence decisions auditable across rounds. diff --git a/plugins/reviews/skills/deep-review/references/setup.md b/plugins/reviews/skills/deep-review/references/setup.md index bbecb97..892f3b8 100644 --- a/plugins/reviews/skills/deep-review/references/setup.md +++ b/plugins/reviews/skills/deep-review/references/setup.md @@ -59,7 +59,7 @@ MERGE_BASE=$(git merge-base "$BASE_REMOTE/$BASE_BRANCH" HEAD) If no base ref can be determined, error and exit. -## Step 1.4: Detect prior reviews (PR/MR only) +## Step 1.4: Detect prior reviews and determine the round **GitHub:** ```bash @@ -80,3 +80,9 @@ to all specialists and the arbiter as context. Specialists should: - Flag prior findings that remain unresolved - Avoid re-raising issues that were already noted and resolved - Call out any regressions — issues that were fixed but reappeared + +After collecting this context, read +[iterative-reviews.md](iterative-reviews.md) to determine the review round, +threshold multiplier, carried blockers, and new candidates. For a local branch +without review history, use explicit round context supplied by the user; if +none was supplied, treat the invocation as round 1.