Skip to content

fix(shader): keep s_cselect_b64 of two masks as a lane bit and fold constant masks - #1640

Merged
adriadam10 merged 1 commit into
boykopovar:mainfrom
morten-holm:pr/shader-select-mask
Oct 10, 2026
Merged

adriadam10 merged 1 commit into
boykopovar:mainfrom
morten-holm:pr/shader-select-mask

Conversation

@morten-holm

Copy link
Copy Markdown
Contributor

What

s_cselect_b64 sd, exec, 0 (and the same form with vcc or all ones) gives a mask that is the same for every lane. The translation stored it as a plain register pair, so a later read of the pair went through LaneId. Metal has no lane number in a vertex shader, and MoltenVK rejects such a shader ("subgroup capability 61 is unsupported for shader stage 1"), so the draw is skipped.

  • s_cselect_b64 with two mask sources now stores the lane bit scc ? source1 : source2 and marks the pair as a mask, the same way v_cmp results are kept. The sources are read before the destination is written.
  • A mask of all zero or all one reads as a constant, with no lane number.
  • The constant folder folds ((W >> (x & 31)) & 1) != 0 to a constant when W is all zero or all one. Some masks only become constant after folding.

The translation is exact. Nothing changes when the lanes differ.

Tested

  • macOS 27 (Apple M2), x86-64 under Rosetta, MoltenVK: one retail title, 30 s run. Before, 8 draws were skipped with "subgroup capability 61 ... stage 1". After, none, and the picture is shown.
  • New test agc_constant_bit_test: fails without the fold (all-zero and all-one cases) and passes with it.
  • This branch does not build libc on macOS (link.h), so I ran the test and the existing masked-select, control-flow, shader-disk-cache and DPP-write tests on a macOS port tree that carries the same shader code: all pass. The four changed shader files compile on this branch.
  • Not tested: Linux and Windows (CI), and the scalarSelectMask64 translation by a unit test; it was only checked end to end on the title above.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed in TechnicalDebt
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source (none added)
  • Generic behaviour, not specific to one title
  • Depends on: none
  • AI-assisted: yes

🤖 Generated with Claude Code

@klosixgaming-arch klosixgaming-arch left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Well-structured fix for the s_cselect_b64 mask handling issue on Metal/MoltenVK. The three-part approach is clean: (1) scalarSelectMask64 properly stores lane bits and marks pairs as masks, (2) foldConstantBitTest eliminates LaneId usage for constant all-zero/all-one masks, and (3) threadBit recognizes constant masks upfront. The test specifically validates the folding cases that were failing before. Good end-to-end testing on a retail title showing the MoltenVK error is resolved.

@morten-holm morten-holm mentioned this pull request Oct 8, 2026
@github-actions github-actions Bot added build shader core/shader agc libSceAgc* needs rebase Conflicts with main, rebase needed labels Oct 8, 2026
@morten-holm
morten-holm force-pushed the pr/shader-select-mask branch from e575bba to af97509 Compare October 9, 2026 04:28
@github-actions github-actions Bot added needs rebase Conflicts with main, rebase needed and removed needs rebase Conflicts with main, rebase needed labels Oct 9, 2026
@morten-holm
morten-holm force-pushed the pr/shader-select-mask branch from af97509 to 8dc1a82 Compare October 9, 2026 15:45
@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 9, 2026
@popcoder565

Copy link
Copy Markdown
Contributor

Native Windows run, since no Windows results were posted. Setup: Windows 10 (19045), RTX 3060 Laptop (NVIDIA 616.92), MinGW-w64 GCC 15.2.0 (WinLibs r7), Release. Head 8dc1a828, merged onto main 8b393b6 in one build with 31 other PRs that had no Windows run.

No title was run.

🤖 Addressed by Claude Code

@adriadam10 adriadam10 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked: build + ctest on Linux are green, and agc_constant_bit_test fails when the non-test changes are reverted, so the folder part is covered. The translation looks right to me (sources read before the destination is written, same mask-tag pattern as v_cmp).

One thing needed before merging:

  • scalarSelectMask64 (ControlFlowInstructions.cpp:247) and the constant paths in readMask/threadBit (ExecMask.cpp:21, :41) are not covered by any test; the PR description says so too. Please add a test (execution, or IR-level next to agc_masked_select_tests) that pins values: scc = 0 and 1, a non-constant source (lanes differ), destination equal to one of the sources, and a wave64 case where the two halves differ.

@morten-holm
morten-holm force-pushed the pr/shader-select-mask branch from 11cbb4c to 1fc9679 Compare October 10, 2026 06:56
@morten-holm

Copy link
Copy Markdown
Contributor Author

The requested test is in: agc_scalar_select_mask (tests/ScalarSelectMask.cpp, registered next to agc_constant_bit_test). It translates small wave64 vertex programs, runs the IR over every lane and pins the lane values, with scc set and clear, for:

  • a source that differs per lane (vcc and exec, in both orders),
  • the two wave64 halves differing (vcc holds the lanes below 40),
  • a destination that is also a source (s[4:5], and vcc),
  • constant sources (exec, zero, -1) and literal sources, including an all-ones literal in wave32 and in wave64.

The constant cases also check that the select keeps both sources as IR constants, which is how threadBit's constant path is covered.

I removed the two constant returns in readMask in the same commit. With them gone the results are identical, because threadBit folds the same constants, so no test can tell them apart and they only added code.

To check that the test can fail, I changed the fix four ways: with s_cselect_b64 reverted it fails (the pair is not marked as a mask), with the two sources swapped it fails on lane values, with the threadBit constant fold disabled it fails on the wave32 case, and without the readMask returns it still passes.

I ran the test on macOS by linking the recompiler objects without libc (main's libc does not build there), so the CI run is its first one through CMake.

@adriadam10 could you take another look?

@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 10, 2026
…onstant masks

A select of exec and zero gives a mask that is the same for every lane. Reading
it back went through a lane number, which Metal does not have in a vertex
shader, so MoltenVK rejected the shader. The select now stores the lane bit and
marks the pair as a mask, a constant mask word reads as a constant, and the
constant folder removes the bit test on an all-zero or all-one word.

agc_scalar_select_mask runs the translated IR over a wave and pins the lane
values for scc set and clear, a source that differs per lane, the two wave64
halves differing, a destination that is also a source, and constant and
literal sources.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@morten-holm
morten-holm force-pushed the pr/shader-select-mask branch from 1fc9679 to 93b0f48 Compare October 10, 2026 08:15
@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 10, 2026

@adriadam10 adriadam10 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked on Linux (build + ctest): agc_constant_bit_test and agc_scalar_select_mask pass, and both fail with the non-test changes reverted. Measured s_cselect_b64 on hardware (gfx1035) with vcc/exec/0/-1/literal/SGPR-pair sources and scc set and clear: the result is scc ? source1 : source2 and a literal is zero-extended, which matches what the tests pin. Thanks for adding the translation test; merging.

@adriadam10
adriadam10 merged commit 4254212 into boykopovar:main Oct 10, 2026
14 checks passed
@morten-holm
morten-holm deleted the pr/shader-select-mask branch October 10, 2026 12:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agc libSceAgc* build shader core/shader

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants