feat(shader): give reads the masked-select pass keeps the whole pixel ancillary word by AlxcBTW · Pull Request #39 · AlxcBTW/AnyPS5

AlxcBTW · 2026-10-11T05:44:04Z

What

Follow-up to boykopovar#3279, stacked on it: the five commits below this one are boykopovar#3279's. Refs boykopovar#2548. It replaces #38 with only the whole-word commit: no environment switch and no trace.

A select the masked-lane walk can't drop still stops the recompile on boykopovar#3279. Palworld's 0x1b40e5c000 is one (Rohan03x on boykopovar#2548: the kept select is reached through a ReadFirstLane, which reads other lanes), and so are Minecraft Dungeons II's 0x190095e900 and 0x190095fc00. With this commit every read the walk keeps gets the whole word:

prim type at bits 0-1: 0 point, 1 line, 2 triangle, 3 rectangle, from the draw (RasterPrimitiveOf);
sample index at 8-11 (SampleId, always 0 here);
render target index at 16-28 (Layer);
every other bit 0.

The layout is from the RDNA 3/3.5 ISA guides. The GCN register guides agree for layers below 2048, Mesa reads sample and layer from those bits, and AMD's LLPC reads the prim type from bits 1:0 with the same encoding. Nobody has checked it on gfx1013; TechnicalDebt.md says so.

Where the primitive class comes from (owner's call)

This PR: a pipeline specialization constant, the way dual-source blending does it (feat(agc): implement dual-source blending from the MRT1 export boykopovar/AnyPS5#1492). The class is a request field. It is folded into the specialized module and is part of the result memo, the draw key and the serialized request (version 15 → 16). The class costs nothing per pixel: it is a constant in the module. Only shaders that read it get a module per class: ShaderInfo records the read (disk cache format 25), and a shader that reads only the sample or layer field keeps one module. The draw sets the class only when the pixel shader loads the ancillary VGPR, so other draws keep their draw-cache entries.
Run-time per-draw data: needs a layout change, so not done. Nothing the fragment shader receives today carries the class. Its push data is its own compact shader-data block (user SGPRs, buffer offsets, image metadata: IrBindingLayout in DescriptorBinding.hpp), and no fragment builtin can tell a rect list (drawn as tessellated triangles) from a triangle. The smallest run-time variant adds one dword to that block in shaders that read the class, and patches it per draw from Graphics::State the way ShaderResources patches guest buffer offsets (pushByte for push constants, dataByte for the ShaderData buffer). The module key, result memo, draw key and request format could then stay as on main. But the shader-data layout changes (IrBindingLayout, disk cache format), the block spills to the ShaderData buffer one dword sooner, and every pixel wave loads the class instead of using a constant.
Specialization when the pipeline is created (not built). The driver could pass constant 8233 through VkSpecializationInfo from Graphics::State, whose pipeline key already holds the topology and rect-list state. The request format, module key, result memo and draw key would then stay as on main. Prepared modules would have to leave that one constant open (buildSpecializedModule requires every SpecId today), and the GPL fragment-library key would need the class for shaders that read it.

Tested

Windows 11, RTX 5070 Laptop GPU (NVIDIA driver; the AMD iGPU is hidden from the Vulkan loader):

Full local ctest -j4: 764 of 766 pass. guest_filesystem fails the same way on a tree without this commit, and guest_sce_net timed out under load and passes on its own.
agc_driver_pixel_ancillary_word exports the word from a draw and reads back 2, 3 and 1 for the triangle, rectangle and line classes. The draw itself is a triangle and the request carries the class, as DrawDecode sets it; the test checks RasterPrimitiveOf on its own. It also compiles one whole-word shader and one that reads only the sample field (v_bfe_u32 v20, v0, 8, 4) for the three classes: the whole-word shader gets three modules of 629 SPIR-V words, and the sample-only shader shares one module of 616 words.
Negative checks: with the composition disabled, pixel_ancillary and agc_driver_pixel_ancillary_word fail with the old unsupported live use refusal. If the module key is keyed on ANCILLARY_ENA instead of on the read, the module check fails (sample field only: a line or rectangle draw built its own module).
tools/check_conventions.py passes.
Titles: the composition is the same as in fix(shader): compose the whole ancillary word where it is exported; image reads end a masked select; APS5_TRACE_ANCILLARY #30/Carry: pixel ancillary (#2548) #37/feat(shader): give reads the masked-select pass keeps the whole pixel ancillary word #38, where Minecraft Dungeons II (MisterbALLIN) moved on to feat(agc): depth planes sampled as 2D arrays, cubes and integer views boykopovar/AnyPS5#425 and Palworld's 0x1b40e5c000 compiled in Rohan03x's offline replay.
Kena: Bridge of Spirits (PPSA01802), run by alfalfa-star on Arch Linux, RADV (RX 9070 XT): its pixel shader 0x20b8770000 is refused on main 1e8fff6 and with fix(shader): drop masked selects proven unobserved through whole quad mode boykopovar/AnyPS5#3391 added, and compiles with this head (74,245 SPIR-V words). In game the per-frame skips of that draw are gone, with no GPU hang in two runs (2.5 and 4 min). Other draws on that frame are still skipped, so this shader's output is not checked yet.
Serialization: feat(agc): render layered draws into 3D color targets boykopovar/AnyPS5#1077/feat(agc): vertex clip and cull distances boykopovar/AnyPS5#423 also add a pixel-info field at serialization version 16. Whichever lands second moves to the next version and disk-cache bump.
CI, run 38115746038: Windows passes 766 of 766, with the GPU tests skipped for lack of a device. Linux passes 761 of 762 (lavapipe, SPIRV-Tools on), and agc_driver_pixel_ancillary_word passes there. The one failure is agc_unit_shadow, which fails on this base without the change too (agc_driver_recorder_tests: partial publish drops a shadowed unit without copying it boykopovar/AnyPS5#1429, fixed on main by fix(agc): compare host imports when device writes reach the write watch boykopovar/AnyPS5#3226).

Checklist

Based on current main plus fix(shader): prove masked lanes unread across exec phis, loops and image operations boykopovar/AnyPS5#3279; no other open PR implements the same functions
One topic per PR; follow-ups go in a new PR
No comments in code except technical debt
Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed in TechnicalDebt
No notes, investigation .md files or images added to the repository (attach them to this PR)
New third-party code is a submodule built from source
Generic behaviour, not specific to one title
Depends on: fix(shader): prove masked lanes unread across exec phis, loops and image operations boykopovar/AnyPS5#3279
AI-assisted: yes (Claude Code)

🤖 Generated with Claude Code

…d lanes unread Taxi Chaos 2 (PPSA30279) pixel shader 0x18c07bf000 is refused when it is registered, and the title stops: packed pixel ancillary input has an unsupported live use (used by SelectU32 ? ?) The shader reads the ancillary VGPR (v2) once, with v_bfe_u32 v3, v2, 16, 11, which boykopovar#1904 lowers to Layer. The next write to v2, v_rcp_f32 v2, s33, runs before any kill under the initial exec and becomes SelectU32(IEqual32(HelperInvocation, 0), rcp, ancillary). The raw word therefore survives only in helper lanes. The shader has no s_wqm, no DPP, readlane or swizzle, no stores, and only *_lz samples and gathers. Every helper-lane value that comes from v2 is dropped by a later exec select, a compare that is ANDed with exec before its ballot, or an export whose exec excludes helper lanes. MaskedSelectEliminator exists to remove selects like this one, but it could not prove the case. The kill paths save and restore exec, so the later execs are phis at block merges, and the AND chains the kills build go deeper than its eight-level limit. When it checks whether a later exec implies the write's exec, it now: - accepts a phi when the phi and each incoming value lie outside loops and every incoming value implies that exec; - pairs low and high exec words merged by phis in one block, predecessor by predecessor; - memoises results for each select and follows up to 64 levels. A cycle counts as not implied. Once the select is gone, nothing reads the ancillary word, so no bit of it is produced. A select whose false lanes can still be observed is kept and the shader is still refused. That covers a data test that leaves the word in non-helper lanes that later export, an export under the full wave, and an unmasked ballot. Related: boykopovar#2548 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

LEGO Horizon Adventures (PPSA14632) still refuses a pixel shader with packed pixel ancillary input has an unsupported live use (used by SelectU32 ? ?) although it has the form the previous commit handles: the only whole-word reader is SelectU32(IEqual32(HelperInvocation, 0), BitFieldUExtract(x, 0, 11), ancillary), the exec words are Ballot(not helper) & Ballot(true), and there are no image ops, stores or loop phis. About 10,250 IR values derive from the word, so the walk from that select stops at its fixed limit of 4096 visited values and the select is kept. With the limit raised to 8192 the shader prepares (reported on Linux with an RX 9070 XT). A walk may now take max(4096, 2 * instructions) steps. A step is a value visited for the first time or an implication evaluated for the first time. The walk only visits values derived from the select, at most one per instruction, so a walk is only cut short when the implications it evaluates cost as much as the program itself. On the 198 Taxi Chaos 2 requests no walk needs more than 0.79 * instructions steps. One walk stays linear in the program, O(instructions + operands): each value is expanded once, and with the per-walk memo each implication is evaluated once. The walk is also cheaper, so that the larger limit costs nothing: - visited values are marked in a table indexed by value id instead of a hash set, and the table, the work list and the memo are reused across walks; - blocks outside loops are looked up in a table indexed by block id instead of a hash set. This lookup ran once per use and was most of the pass's time. Time in MaskedSelectEliminator per prepare (median of 3, the pass runs twice per prepare), 54 Taxi Chaos 2 requests: shader before 40a8a5a this commit main 5edae35 0x1ac0413000 689 ms 285 ms 401 ms 0x19002bd200 102 ms 34 ms 100 ms 0x18c07bf000 191 ms 52 ms (refused) all 54 1294 ms 484 ms 919 ms 0x1ac0413000 and its three copies in the NVIDIA dump now remove 659 selects instead of 625: their walks no longer stop at 4096. No other request changes what it removes. 197 of the 198 requests prepare before and after; the other one fails the same way in both (v_readlane_b32 of lane 63 on a 32-lane host subgroup). Related: boykopovar#2548 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Kena: Bridge of Spirits (PPSA01802) refuses ten pixel shaders with packed pixel ancillary input has an unsupported live use (used by SelectU32 ? ?) at any visit limit. All ten have 30 to 33 phis on loop back edges (reported on Linux with an RX 9070 XT). Two rules stopped the proof at a loop, and the first one fires before the second is ever asked: - The walk from the select gave up at the first use inside a loop. A value derived from the select reaches a loop phi as soon as its register is written inside the loop, because the masked write becomes Select(loopExec, new, phi). A loop that leaves exec alone needs nothing more than this. - An exec that a loop narrows and carries around its back edge is a phi inside the loop, and such phis counted as not implying the write's exec. The walk now follows uses into loops. A phi inside a loop implies the write's exec when all of its incoming values do, where the phi itself counts as implying it while its back-edge values are checked. This is sound because each rule of the proof takes a lane bit of one value from a value computed earlier: a phi's value on each iteration is the incoming value computed before the edge was taken, a set lane of an And is set in both operands, a set lane of a ballot is a lane whose predicate held. If any value in the proof had a lane set where the write's exec is clear, the earliest such value would take that lane from an earlier value of the proof that is set there too, which contradicts it being the earliest. Two conditions keep this true: - The write's exec must be computed once, outside every loop. Otherwise a phi can carry a ballot of it from an earlier iteration, which is not the value the select used. Masks computed inside a loop keep the old rules. - A result may only be kept if the assumptions it used held. When a value that was assumed turns out not to be implied, every result recorded since it was assumed is dropped. Results that say "not implied" are kept; they only make the proof refuse more. A lane that is turned back on inside the loop makes the back-edge exec wider than the write's exec, so the phi is not implied and a read under it keeps the select. A lane that is turned back on but never reads the derived value does not. New tests cover a loop that carries the write with exec saved, narrowed and restored (read inside and after the loop), a loop that leaves exec alone, and the helper-only ancillary select feeding such loops. The refusing tests each fail when the matching check is removed: a revived lane that reads the write inside the loop (ignoring back-edge values), a result proven under an assumption that later fails (keeping it), and a write mask computed inside an earlier loop (allowing such masks). Two existing tests expected a loop to keep the select although every read is under the write's own exec; they now expect it to go, next to new counterparts with a wider exec. On the Taxi Chaos 2 requests, compute shaders with a loop inside a branch now lose the branch's masked writes: 0x194000b400 removes 207 selects instead of 36, 0x19002bd200 111 instead of 28, and seven more gain 5 to 43. In 0x16400cf700 the 26 new removals are writes under the branch condition tid < count; the loop's exec starts as a ballot of that condition, only loses lanes at the latch, and every store and atomic inside the loop runs under it. No proof on these requests had to drop a result. Walks now go on into loops, so the pass takes longer on loop shaders, but it stays below main. Time in MaskedSelectEliminator per prepare (median of 3, the pass runs twice per prepare), 54 Taxi Chaos 2 requests, one run: shader previous commit this commit main 5edae35 0x1ac0413000 334 ms 349 ms 424 ms 0x19002bd200 38 ms 103 ms 112 ms 0x19c016aa00 9 ms 74 ms 16 ms 0x194000b400 13 ms 31 ms 32 ms all 54 547 ms 805 ms 977 ms 0x19c016aa00 removes nothing more; its walks only end later inside its loops. The 144 requests of the NVIDIA dump go from 1283 to 1589 ms and gain removals in the copies of the same shaders. 197 of the 198 requests prepare, as before. Related: boykopovar#2548 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…xec is active An ImageRead carries a trailing exec argument like the buffer loads, but it was in neither the read-only-where-active list nor the lane-local list, so a masked select feeding its address was kept. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

…pass the masked-lane walk Following the ImageRead case, every image operation was checked against the two lists the masked-lane walk uses: accesses guarded by a trailing exec operand, and lane-local values. - ImageWrite carries its exec last like StoreBuffer, and EmitWriteOp stores only inside an if on it (bindless tables and FMASK refuse the store). It joins the exec-guarded list. - A gather with an LOD of its own (gather4_l, gather4_lz) or outside a pixel shader reads only its lane's address, like the explicit-LOD samples the walk already follows. A pixel gather without one still keeps the select, since on the GPU it takes its LOD from the quad. - ImageQueryDimensions returns the size of its lane's level of the image and is lane-local. Left out on purpose: - ImageQueryLod and implicit-LOD samples read their quad. - ImageBvhIntersectRay has a trailing exec operand, but its emitter ignores it and records faults in every lane. - Image, buffer, shared and address atomics are guarded by their exec in the emitter too, but the test from 60c6552 pins them as keeping the select, and no refused shader has needed them so far. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

… ancillary word A packed ancillary word that still reaches an export, or any other read the masked-select pass cannot drop, no longer stops the recompile. After that pass every remaining use gets the composed word: the primitive class at bits 0-1, SampleId at bits 8-11 and Layer at bits 16-28, with every other bit zero. Those lanes do export the ancillary VGPR on the console, so the word has to be right, not just present. The primitive class (0 point, 1 line, 2 triangle, 3 rectangle) comes from the draw. RasterPrimitiveOf maps rect lists to 3, line lists on the vertex path to 1 and everything else the driver accepts to 2, because its geometry and tessellation paths only output triangles. It reaches the shader as pipeline specialization constant 8233, the way dual-source blending does: a request field that is folded into the specialized module and is part of the result memo, the draw key and the serialized request (version 16). Only shaders that read the class pay for it. ShaderInfo records the read (disk cache format 25), and only those shaders put the class in their module key, so a shader that reads just the sample or layer field keeps one module for every primitive class. The draw sets the class only when the pixel shader loads the ancillary VGPR, so other draws keep the draw cache entries they have today. The VRS rates (bits 2-5) and the eye id (bit 7) are 0 because the driver supports neither VRS nor multiview. SampleId is 0 because the driver rejects MSAA and per-sample shading, and Layer is 0 while it ignores vertex layer exports. TechnicalDebt.md records the layout sources and these assumptions. Tests: - pixel_ancillary builds an export to target 0 whose two channels select between a packed value and a phi carrying the word. It checks the word for each primitive class, sample and layer, checks that the three shapes the masked-select pass keeps now compile with the composed word, and that a program whose select the pass drops reads no class. - agc_driver_pixel_ancillary_word exports the word from a draw and reads back 2, 3 and 1 for triangle, rectangle and line. A shader that reads only the sample field shares one module across the classes, and the whole-word shader gets one per class. It also checks RasterPrimitiveOf. - agc_shader_memory covers the version 16 round trip, and agc_shader_disk_cache the new flag. Both ancillary tests fail with the old refusal when the composition is disabled. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

alfalfa-star · 2026-10-11T15:50:34Z

Another title data point for this PR: Kena: Bridge of Spirits (PPSA01802), Arch Linux, RADV (RX 9070 XT, Mesa 26.2.4).

On upstream main 1e8fff6 (so with fix(shader): prove masked lanes unread across exec phis, loops and image operations boykopovar/AnyPS5#3279), plus our local Kena stack (feat(agc): depth planes sampled as 2D arrays, cubes and integer views boykopovar/AnyPS5#425, fix(libSceAgcDriver): retire a depth surface once another writer covers its memory boykopovar/AnyPS5#454, feat(agc): render layered draws into 3D color targets boykopovar/AnyPS5#1077, fix(recompiler): read NumWorkgroups in every mesh shader so RADV sizes its user data boykopovar/AnyPS5#3645), one pixel shader is still refused: packed pixel ancillary input has an unsupported live use (used by SelectU32 1065353216 ?), 936 code words, wave64. The game draws it about once per frame at the title menu (~400 skipped draws in 2.5 min).
agc_shader_replay of its full APS5_DUMP_SHADERS request: refused on that stack, refused with fix(shader): drop masked selects proven unobserved through whole quad mode boykopovar/AnyPS5#3391 added, and compiles with this PR (74,245 SPIR-V words).
In game with this PR: those skips are gone, and there was no GPU hang over two runs (2.5 and 4 min). The menu background is still wrong, but other skipped draws remain on that frame, so I can't tell yet whether this shader renders correctly.
To join it onto our stack I had to resolve conflicts with feat(agc): render layered draws into 3D color targets boykopovar/AnyPS5#1077/feat(agc): vertex clip and cull distances boykopovar/AnyPS5#423, which also add a pixel-info field at serialization version 16. I kept both fields and moved rasterPrimitive to version 18 (disk cache 28). That's local only. Your branch on main is unaffected, but whichever of these lands second needs the same bump.

I can attach the request (shader_20b8770000.req) if it helps for a test.

AlxcBTW · 2026-10-11T16:51:59Z

@alfalfa-star thanks, that's the first title run on this head. Kena's 0x20b8770000 is refused on main and with boykopovar#3391, compiles with this PR, and the per-frame skips are gone without a hang in two runs. I've added Kena (PPSA01802) to the Tested section with your caveat: other draws on that frame are still skipped, so this shader's output isn't checked yet.

Good catch on the serialization version. boykopovar#1077/boykopovar#423 and this PR both add a pixel-info field at version 16, so whichever lands second takes the next version and cache bump; that's in the description now too.

No need to attach the request: a title's shader can't go into the repo as a test fixture anyway.

AI-assisted: yes (Claude Code)

AlxcBTW · 2026-10-12T18:57:10Z

Continued upstream as boykopovar#4031.

AlxcBTW and others added 6 commits October 11, 2026 17:55

This was referenced Oct 11, 2026

fix(agc): take depth fast clears from HTILE written by any kernel boykopovar/AnyPS5#2053

Closed

fix(agc): take an HTILE fill from a value in memory as a depth fast clear boykopovar/AnyPS5#3816

Merged

This was referenced Oct 11, 2026

Shader preparation: masked-lane walk takes 150 to 400 s on Stray's large pixel shaders since fda4aa90 (#3279), and the title no longer starts boykopovar/AnyPS5#3504

Closed

Sword of the Sea (PPSA25344) working group #44

Closed

AlxcBTW closed this Oct 12, 2026

Navigation Menu

Repository navigation

feat(shader): give reads the masked-select pass keeps the whole pixel ancillary word - #39

feat(shader): give reads the masked-select pass keeps the whole pixel ancillary word#39
AlxcBTW wants to merge 6 commits into
mainfrom
carry/pixel-ancillary-word-v2

AlxcBTW commented Oct 11, 2026 •

edited

Loading

Uh oh!

alfalfa-star commented Oct 11, 2026

Uh oh!

AlxcBTW commented Oct 11, 2026

Uh oh!

AlxcBTW commented Oct 12, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

AlxcBTW commented Oct 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Where the primitive class comes from (owner's call)

Tested

Checklist

Uh oh!

alfalfa-star commented Oct 11, 2026

Uh oh!

AlxcBTW commented Oct 11, 2026

Uh oh!

AlxcBTW commented Oct 12, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

AlxcBTW commented Oct 11, 2026 •

edited

Loading