Skip to content

Promotion quality: stale pins survive on inline snapshots, empty roster re-pin under wham#310, and new-failing fixtures don't gate #13

Description

@amis92

Three connected findings from the live Beat-3 promotion demo (amis92/wh40k-11e#8, from issue amis92/wh40k-11e#6):

  1. Stale pins survive for inline-sourced snapshots. SpecRePinner.RewriteWithPins strips from the SpecEmitter observed-values marker — but a verbatim inline spec has no marker, so the append-only fallback keeps the original (now-wrong) selection-level pins alongside the new block. Promoted fixture pinned pts=60 after the data was fixed to 55. Fix direction: strip ALL assertion-only steps (RePin already identifies them when building replaySteps), not just the marker block.

  2. Roster-level re-pin is empty under Roster-level costs drop cost types only reachable via entryLinks wham#310. The re-pin uses RosterState.Costs, which is empty for wh40k-11e data (entryLink-only cost types dropped from roster totals). The appended block was costs: with nothing under it. Selection-level re-pins are blocked by the positional comparer (Task 7 decision). Until wham#310 lands, promotion on affected repos produces assertion-free (or stale) fixtures — consider failing promotion loudly when the re-pin would be empty.

  3. The PR gate doesn't catch new-failing fixtures. Diff mode evaluates head's fixtures against BOTH data sides, so a brand-new failing fixture classifies still-failing (not broke) and --fail-on-broke stays green — a bad promotion can merge silently. Consider gating new fixtures that fail on head (fail-on-new-failing or fold into fail-on-broke).

Also from the same demo run (kit requirements, fold into #8 if preferred): promotion PR creation requires the repo setting 'Allow GitHub Actions to create and approve pull requests' (or the pr_token secret) — document in executable-bug-reports.md install steps.

🤖 Generated with Claude Code

https://claude.ai/code/session_01KcYK8hmTmXi8eLWpg9LGKJ

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions