Repository navigation
AGC: each 32-bit BDA store emits five write lookups, so agc_driver_flat_store compiles to 134K/269K GEN instructions and takes ~150 s cold on Intel #1256
Description
agc_driver_flat_store runs a 45-dword program and takes about 150 s on an Intel iGPU, almost all of it creating its compute pipelines, so it exceeds the 120 s ctest timeout whenever the driver's shader cache is cold. It isn't a hang: the same run takes 0.8 s once the driver has cached the pipelines. The size comes from the BDA store emission.
Measured with APS5_PIPELINE_STATS=1 on main at 4764e59, Windows 11:
| Device | Program | Size of the compiled shader | Cold pipeline creation (whole test) | Warm |
|---|---|---|---|---|
| Intel Raptor Lake-S, driver 32.0.101.6790 | wave32 | 134,185 GEN instructions, 10,437 basic blocks, 83 loops | ~150 s | 0.8 s |
| Intel Raptor Lake-S, driver 32.0.101.6790 | wave64 | 268,907 GEN instructions, 20,870 basic blocks, 166 loops | ||
| RTX 4070 Laptop, driver 32.0.15.9597 | wave32 / wave64 | 490,496 / 930,048 bytes of binary | 27 s | 3.2 s |
The cold timings were taken with ANYPS5_NO_SHADER_CACHE=1, plus a fresh APS5_LOOP_GUARD value to miss the driver caches. The APS5_PIPELINE_STATS run without the guard also missed the Intel cache and took 150 s.
Where the size comes from
After translation the program has 15 StoreAddressU32, one StoreAddressU16, one StoreAddressU8 and one LoadAddressU32 (APS5_DUMP_IR).
For each 32-bit store, EmitBdaStore (SpirvBdaRead.cpp) emits:
- the aligned
EmitBdaStoreAt; - under the unaligned branch, four byte
EmitBdaStoreAt.
Each of those calls get_bda_write_pointer and note_bda_write:
get_bda_write_pointer(DefineBdaLookup) contains a loop.note_bda_writeunrollsWrittenPageProbes = 8atomic compare-exchange probes.
For this program that is 15 × 5 + 2 + 1 = 78 write lookups. The Intel wave32 executable reports 83 loops, which suggests the driver inlines the calls.
Reads already avoid the same expansion: EmitBdaDwordReads probes the span once instead of four lookups per dword (the comment there cites a Bink pass).
Possible directions (no PR, because the order in which faults are recorded has to stay exactly as it is):
- Probe the 4-byte span of a 32-bit store once, and take the byte path only when the probe rejects it, as the reads do.
- Keep the unaligned byte path in a loop, so each store emits one byte lookup instead of four.
Found with AI assistance (Claude Code); the numbers above were measured on the hardware listed.
Activity
hashaam1217 commentedon Oct 8, 2026
Linux numbers for this issue and the open fixes. Intel Iris Xe (Alder Lake GT2), Mesa 25.2.8 ANV, kernel 7.0, each PR applied to 2cc5735. Cold runs with
ANYPS5_NO_SHADER_CACHE=1 MESA_SHADER_CACHE_DISABLE=true APS5_PIPELINE_STATS=1 APS5_PROFILE_DRAW=1, one test at a time in a cgroup limited to 7 GiB. Peak RSS is from GNUtime. On Linux the cost is memory as well as time: on main, ANV's compile offlat_storereaches 7.04 GiB.agc_driver_flat_store(no spills in any column):The other store-heavy tests follow the same pattern. The table gives GEN instructions per pipeline; both tests fail on this device for the subgroup reason in #1185:
global_store_addtidglobal_vcc_baseWhy #1282 loses #1265's gain on Mesa. #1282 contains #1265 and #1280, which measure about the same, so the difference comes from its last commit (e63cc30). That commit adds a 4-byte probe whose fallback
write_bda_dword_bytes(andnote_bda_write) is markedDontInline. Mesa inlines every function in a Vulkan shader regardless:vk_spirv_to_nir(vk_pipeline.cL180), which runsnir_inline_functions(vk_nir.cL167).radv_shader.cL440).driver_functions(nir_functions.cL224-225). The SPIR-VDontInlinebit is stored asdont_inline(vtn_cfg.cL328), and the pass does not check it.So on Linux the shared fallback is copied back into every store site. #1282's SPIR-V is smaller than main's, but the compiled shader is not, and the same applies to AMD GPUs on RADV.
With #1185. #1185 makes ANV compile these at SIMD32, where main's shaders spill heavily (numbers in my comment there;
flat_storeandglobal_vcc_basewere killed at a 5 GiB limit). With #1185 and #1265 together, all three pass:flat_storeglobal_store_addtidglobal_vcc_baseThe full Intel suite with both has 14 failures, against 16 with #1185 alone:
flat_storeandglobal_vcc_basenow pass. The remaining 14 are:shaderFloat64orVK_KHR_shader_clock;pixel_lds;vop1_float_unary(fix(shader): compute V_FRACT_F32 from the exact fraction of |x| #1250);wave32_subgroup(its wide run, see fix(agc): require 32-wide compute subgroups on devices whose default is 32 #1185).NVIDIA RTX 3060 (595.91.07): full suite 402/402 with main, #1265, #1282, and #1185 + #1265.
kr4y4 commentedon Oct 8, 2026
Windows numbers for the same three PRs, to go with the Linux ones above. Windows 11, Intel Raptor Lake-S (driver 32.0.101.6790) and RTX 4070 Laptop (driver 32.0.15.9597). All three heads are on main c5e1018: #1265 at 2e60f55, #1280 at 1dca7b4, #1282 at e63cc30. Settings:
ANYPS5_NO_SHADER_CACHE=1 APS5_PIPELINE_STATS=1, one test at a time.The Intel driver cache can't be cleared here because the directory is locked. The PR variants emit new SPIR-V, so their times are cold compiles. main's
flat_storewas served from the driver cache in this run (1.8 s); its cold time, 150 s, is the one in the description.agc_driver_flat_store, wave32 / wave64 pipelines:Intel GEN instructions, other store-heavy tests (wave32 / wave64;
global_vcc_basehas two programs):global_store_addtidglobal_vcc_baseThe Windows Intel driver behaves like ANV:
DontInlinefallback doesn't survive, and its shaders come out about twice the size of perf(recompiler): loop the BDA byte stores and written-page probes #1265's.flat_storetakes 195 s cold with perf(shader): probe 32-bit BDA stores and share the write fallback #1282, over the 120 s ctest timeout.flat_storebinary is smallest with perf(shader): probe 32-bit BDA stores and share the write fallback #1282: 68,608 bytes, against 77,696 with perf(recompiler): loop the BDA byte stores and written-page probes #1265.So of the three, #1265 is the one that shrinks the compiled shader on both Intel drivers. All variants pass the three tests on the Intel GPU here, and
flat_storeon NVIDIA.Measured with AI assistance (Claude Code) on the hardware listed.
klosixgaming-arch commentedon Oct 8, 2026
This is a significant performance issue with BDA store emission. Based on my review of recent PRs related to the shader recompiler (including #1523 for memory amplification and quadratic scan fixes), I can provide some context:
The analysis showing that each 32-bit BDA store emits five write lookups is compelling. The comparison to how reads already avoid this expansion (with
EmitBdaDwordReadsprobing the span once) suggests a clear path forward for fixing this issue.Some observations based on similar issues I've seen:
I'd suggest prioritizing this fix, as it affects compilation time for any program that uses BDA stores, which is likely many games.
LeandroLP23 commentedon Oct 8, 2026
AMD numbers for the same tests and the current heads of the three PRs: #1265 at d3cdffa, #1280 at 1dca7b4 and #1282 at fe567b5 (the commit that drops
DontInlinefromnote_bda_write), each merged intomain1081602. Radeon 860M (RDNA 3.5), Windows driver 32.0.13058.8001, Windows 11, MinGW-w64 GCC 15.2.0, Release. SettingsANYPS5_NO_SHADER_CACHE=1 APS5_PIPELINE_STATS=1, one test at a time.The AMD driver caches pipelines per executable, so each run used a fresh copy of the test binary under a new name: 0.2 s warm, the times below are cold. The AMD driver reports no instruction count through
pipelineExecutableInfo, only registers and scratch; as a rough size measure, the table gives the driver's cache file for the run, which grows in 64 KiB steps. Wall time and peak working set are for the whole test, median of 3 cold runs.flat_storeglobal_store_addtidglobal_vcc_baseNo scratch in any pipeline. So the AMD Windows driver compiles all four variants in seconds and well under 1 GiB; the BDA store size matters on Intel (and Mesa), not here. The only visible difference is
flat_store's cache file, half the size with #1282, in the same direction as NVIDIA's binary sizes above.Full
cteston this GPU: main, #1265 and #1280 pass 444/445 (the failure is #825). #1282 also failsagc_shader_memory; details in #1282.Measured with AI assistance (Claude Code) on the hardware listed.
oneandonlydean commentedon Oct 10, 2026
Fixed by #1265: the unaligned byte path of a BDA store is now a loop with one write lookup, and the written-page probes are a loop instead of eight unrolled CAS probes (#1280 also loads the slot before the CAS). On the Intel numbers above that takes
flat_storefrom ~150 s to ~42 s cold, under the ctest timeout. Reopen if it still times out on current main.