fix(shader): take a scratch pool slot with a relaxed compare-exchange

core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvScratchPool.cpp

@@ -67,7 +67,7 @@ void EmitScratchPoolAcquire(SpirvEmitterState& state) {
         FailEmit("scratch pool was not declared before function emission");
     }
     const auto device = ConstantU32(state, spv::ScopeDevice);
-    const auto acquire = ConstantU32(state, spv::MemorySemanticsAcquireMask | spv::MemorySemanticsUniformMemoryMask);
+    const auto relaxed = ConstantU32(state, spv::MemorySemanticsMaskNone);
     const auto length = state.module.AllocateId();
     state.module.AddFunction(spv::OpArrayLength, TypeU32(state), length, state.scratchPoolVariable, 0u);
     const auto slotDwords = ScratchPoolSlotDwords(state);
@@ -94,10 +94,10 @@ void EmitScratchPoolAcquire(SpirvEmitterState& state) {
         EmitLabel(state, body);
         const auto candidate = LoadU32(state, state.scratchPoolCandidateVariable);
         const auto observed = state.module.AllocateId();
-        state.module.AddFunction(spv::OpAtomicLoad, TypeU32(state), observed, PoolWord(state, candidate), device, ConstantU32(state, spv::MemorySemanticsMaskNone));
+        state.module.AddFunction(spv::OpAtomicLoad, TypeU32(state), observed, PoolWord(state, candidate), device, relaxed);
         EmitIfCondition(state, EmitCompareU32Constant(state, spv::OpIEqual, observed, 0u), [&] {
             const auto previous = state.module.AllocateId();
-            state.module.AddFunction(spv::OpAtomicCompareExchange, TypeU32(state), previous, PoolWord(state, candidate), device, acquire, acquire, ConstantU32(state, 1u), ConstantU32(state, 0u));
+            state.module.AddFunction(spv::OpAtomicCompareExchange, TypeU32(state), previous, PoolWord(state, candidate), device, relaxed, relaxed, ConstantU32(state, 1u), ConstantU32(state, 0u));
             state.module.AddFunction(spv::OpStore, state.scratchPoolWonVariable, Select(state, TypeU32(state), EmitCompareU32Constant(state, spv::OpIEqual, previous, 0u), ConstantU32(state, 1u), ConstantU32(state, 0u)));
         });
         const auto acquired = EmitCompareU32Constant(state, spv::OpIEqual, LoadU32(state, state.scratchPoolWonVariable), 1u);
@@ -114,7 +114,7 @@ void EmitScratchPoolAcquire(SpirvEmitterState& state) {
         state.module.AddFunction(spv::OpAtomicStore, PoolWord(state, EmitAddU32(state, slots, slot)), device, ConstantU32(state, 0u), ConstantU32(state, InvocationsPerGroup(state)));
         state.module.AddFunction(spv::OpStore, state.scratchPoolSlotVariable, slot);
     });
-    state.module.AddFunction(spv::OpControlBarrier, ConstantU32(state, spv::ScopeWorkgroup), ConstantU32(state, spv::ScopeWorkgroup), ConstantU32(state, spv::MemorySemanticsAcquireReleaseMask | spv::MemorySemanticsUniformMemoryMask | spv::MemorySemanticsWorkgroupMemoryMask));
+    state.module.AddFunction(spv::OpControlBarrier, ConstantU32(state, spv::ScopeWorkgroup), device, ConstantU32(state, spv::MemorySemanticsAcquireReleaseMask | spv::MemorySemanticsUniformMemoryMask | spv::MemorySemanticsWorkgroupMemoryMask));
     state.scratchPoolSlot = LoadU32(state, state.scratchPoolSlotVariable);
     const auto header = EmitBinaryU32(state, spv::OpIMul, slots, ConstantU32(state, HeaderWordsPerSlot));
     const auto slotBase = EmitAddU32(state, header, EmitBinaryU32(state, spv::OpIMul, state.scratchPoolSlot, ConstantU32(state, slotDwords)));

docs/dev/debt/shader.md

@@ -249,7 +249,7 @@
 - [Descriptor web arms without a valid T#](../../../core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp): an image arm of a read split through a descriptor web (`ImageResource::optionalDescriptor`) whose words are not a T# is bound as a null image; a lane that selects it reads zeros and records an `InvalidDescriptor` BDA fault. The console would sample through those words; this is undefined and not measured. Not a T#: not a null descriptor, and a `TYPE` from 2 to 7, or a `FORMAT` outside the GFX10 image format table ([Mesa](https://gitlab.freedesktop.org/mesa/mesa/-/blob/main/src/amd/registers/gfx10-rsrc.json) `GFX10_FORMAT`: 1-77, 128-154, 156-182, 265-292). Other invalid arms, and a direct image with such words, are refused as invalid; an image table entry faults when selected (`InvalidFormat`). Such shaders bind the fault buffer, so their dispatches are not pipelined or replayed for stencil. PPSA23566 (UFC 6) has such an arm (formats 402-463).
 - [Compute scratch in device memory](../../../core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvScratchPool.cpp) (shader recompiler) - the per-lane scratch of a compute program lives in a storage buffer, not Function arrays, which NVIDIA compiles slowly when indexed dynamically (downstream probe, PPSA26344 program 0x1994ed: about 141 s to about 100 s; not upstream). The buffer is `Coherent`: without it, on an RTX 4090 (Windows), an unaligned 32-bit store (four byte read-modify-writes) loses the first byte of each pair sharing a dword (`flat_store32` private and mixed at offset 2, and `FlatApertureSources`; aligned, 8-bit and 16-bit rows pass). The wrong value is the same every run and survives a wider lane stride, SPIR-V optimization off and `Aliased`; the cause is unknown.
 - [Compute scratch runtime cost](../../../core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvScratchPool.cpp) (shader recompiler) - scratch accesses reach global memory: with 448 scratch dwords per lane on an RTX 4090 and the default 64 MiB pool, a dispatch takes about 2.4 times as long as with Function arrays (256 workgroups 3.8 against 1.56 ms, 32768 389 against 183 ms), with or without `Coherent`. The cost in a game frame is unmeasured, and no AMD or Intel GPU has been measured.
-- [Scratch pool slot acquisition](../../../core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvScratchPool.cpp) (shader recompiler) - invocation 0 of a compute workgroup scans the slot locks from `linear workgroup id modulo slot count` and spins until it wins one, reading a lock before trying to take it; with about 9 slots of 114696 bytes (`APS5_SCRATCH_POOL_MIB=1`) a dispatch of 32768 workgroups takes 1308 ms against 395 ms with 64 MiB on an RTX 4090, so too few slots serialize the workgroups; every invocation decrements the slot counter at each exit and the last frees the lock. A workgroup that holds a slot must never wait for one that does not: a guest with such a dependency between the workgroups of one dispatch would hang once there are more workgroups than slots, which the hardware does not guarantee to run either. The emitter fails the compile when a compute `OpReturn` lacks the release, and the driver guarantees at least one slot.
+- [Scratch pool slot acquisition](../../../core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvScratchPool.cpp) (shader recompiler) - invocation 0 of a compute workgroup scans the slot locks from `linear workgroup id modulo slot count` and spins until it wins one, reading a lock before trying to take it; with about 9 slots of 114696 bytes (`APS5_SCRATCH_POOL_MIB=1`) a dispatch of 32768 workgroups takes 1308 ms against 395 ms with 64 MiB on an RTX 4090, so too few slots serialize the workgroups; every invocation decrements the slot counter at each exit and the last frees the lock. A workgroup that holds a slot must never wait for one that does not: a guest with such a dependency between the workgroups of one dispatch would hang once there are more workgroups than slots, which the hardware does not guarantee to run either. The emitter fails the compile when a compute `OpReturn` lacks the release, and the driver guarantees at least one slot. The compare-exchange that takes a lock is relaxed and the workgroup barrier after the scan acquires at device scope instead: AMD's Windows driver compiles an acquire on a storage buffer to `s_barrier`, and inside invocation 0's divergent scan that barrier paired with the barrier the other waves of the workgroup reached after the scan, which then read the slot from workgroup memory before invocation 0 stored it and lost every scratch store of those waves (`agc_driver_scratch_pool`, wave64 isolation, driver 32.0.21043.5001 on a Radeon 0x13c0, RDNA 2; the same failure is reported on a 0x164e). The release store of the last invocation keeps its release semantics in a divergent branch at the end of the program, where that driver's `s_barrier` was not seen to pair with anything; only that test was measured.
 
 - [NGG host subgroup layout](../../../core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp) maps wave64 mesh programs to 32- or 64-lane host subgroups; narrower host subgroups have no verified wave64 mapping. The layered-volume mesh execution regression requires those supported widths. Vertex execution and mesh compilation remain separate checks; compile-only coverage does not establish execution on narrower hosts.
 - [T#s with an unset format field](../../../core/shader/recompiler/Optimization/src/ResourceMaterializer.cpp) (shader recompiler) - a directly bound sampled read that is neither written nor atomic decodes like a null descriptor when its TYPE is an image type the decode implements and its format field names no format (0 or 0x1ff: the zero-format and 0xffffffff-filled unbound slots PPSA02721 samples), in the compile-time decode and the materialization; a reserved or non-image TYPE keeps its rejection however its format reads, as do storage and atomic images with an unset format, a TYPE the driver does not decode, and an image-table entry. What the console returns is not measured; the null decode follows the all-zero descriptor rule. The driver's [IsNullSampledTextureResource](../../../core/libs/prx/libSceAgcDriver/Graphics/src/GuestTextureResource.cpp) applies the same rule to sampled consumers alone; storage bindings and written-buffer tracking keep the base-and-type null rule.