agc_driver_bda_device times out in vkWaitForFences since the BDA anchor lookup (#5859, d422c760c)

agc_driver_bda_device fails on main with

AGC graphics: vkWaitForFences graphics: Vulkan result 2

(VK_TIMEOUT from the 5 s fence wait in CommandBatch::Wait; the test takes 6.6-7.2 s instead of 0.3 s).

Environment: Windows 11 Pro 26220, NVIDIA GeForce RTX 4070, driver 596.21, Ryzen 7 5700X, WinLibs GCC 15.2.0 r7, Release, ctest -R '^agc_driver_bda_device$' -j1.

Bisect over 3e3e8b9..a62c724:

3e3e8b9: pass 3/3
385ed71 (merge of fix(shader): use 64-bit buffer atomics for device-memory LDS #7368): pass 3/3
385ed71 + d422c76 cherry-picked ("perf(shader): look up the BDA range of a user-data pointer once at program entry", perf(shader): look up the BDA range of a user-data pointer once at program entry #5859): fail, 3 of 3 separate runs
644e384 (merge of perf(shader): look up the BDA range of a user-data pointer once at program entry #5859) and a62c724: fail

So #7368 is not involved; the first bad commit is d422c76. It adds RunBdaAnchorTests to tests/BdaExecution.cpp and emits the anchor range search at program entry (EmitBdaAnchors, with OpGroupNonUniformElect / OpGroupNonUniformBroadcastFirst of a 64-bit vector in EmitElectedBdaRange). One of the anchored dispatches never completes on this GPU. I have not isolated which compare() case hangs.

Found during cross-validation of #7445, where the test fails identically with the PR's sources reverted, so it is unrelated to that PR.

Written with Claude Code assistance (bisect by a subagent, reviewed in this session).

This failure is now directly investigated in open PR #7571. It reproduces the mixed mapped/below-table/past-end subgroup case, adds it to agc_driver_bda_device, and changes the lookup helpers to return through one merge path. The PR reports 3/3 passes on NVIDIA RTX 4070 Ti/Linux with driver 595.99.02, while the Windows RTX 5080/610.62 control passes with both the original lookup and the change. This is strong coverage for the reported Linux failure, but it does not establish the behavior on other driver versions.

Third data point — this reproduces on RADV too, as an assertion rather than a timeout, so it is not NVIDIA-specific:

Linux, CachyOS, kernel 7.2.9, GCC 16.2.1, Release, AMD RX 9070 XT (RADV Mesa 26.2.4)
main @ 39ff29696: agc_driver_bda_device fails 2/2 with an anchored BDA read differs from the same read through the lookups at anchor 0x140733498786049 + 0 + (-1), 16 bits, 1 dwords — a wrong-result assertion, ~0.4 s, no hang
644e38480~1 (before the perf(shader): look up the BDA range of a user-data pointer once at program entry #5859 merge): passes 1/1
644e38480 (the merge): fails — same assertion

So on AMD the same commit produces a deterministic wrong value from the anchored-read lookup rather than a fence timeout, which points at the anchor-lookup change itself rather than a driver hang. The assertion's shape — an anchored read differing from the same read through the lookups at a negative dword offset — may be the more direct repro for whoever takes the fix: the value is computably wrong before any fence is involved.

@blawesom

@blawesom thanks, that's a useful data point. The anchor in that message is printed as 0x followed by a decimal number: 140733498786049 is 0x7fff12340101, the test's guest + 0x100. That's where the 0x10-byte words range begins. So the first batch's lane 0 (low 0) reads 16 bits at guest + 0xff: one unmapped byte, then the first byte of the range. That address is outside what the anchor holds, so the anchored program takes the lookup path for that lane, while other lanes of the same subgroup load directly. The 8-bit read at -1 (one byte, fully unmapped) passes before it.

#7571 changes the lookup functions that this path calls (one exit instead of three divergent returns). Could you run agc_driver_bda_device at its head 8e7d887 on the RX 9070 XT? I can't run RADV here, and lavapipe skips this part of the test (it crashes in it when the skip is removed, both on main and on #7571).

AI-assisted: yes (Claude Code)

A data point from AMD's proprietary Windows driver: agc_driver_bda_device passes here with #5859 in.

Radeon 860M (RDNA 3.5, driver 32.0.13058.8001), Windows 11 26300, WinLibs GCC 15.2.0 r7, Release, BUILD_TESTING=ON. main c3591adf9 (includes the #5859 merge 644e38480), with seven unrelated open PRs merged that don't touch the BDA code: agc_driver_bda_device passes, 3 runs of 3, in 0.6 to 0.7 s each. So the hang (RTX 4070, Windows) and the wrong result (RADV) don't reproduce on this RDNA 3.5 GPU and driver.

AI-assisted

Thanks for testing this on the Radeon 860M. That is a useful control: it shows the reported timeout/wrong-result behavior does not reproduce on this RDNA 3.5 Windows setup, while the Linux NVIDIA and RADV reports remain. It does not settle the driver/architecture boundary, so I would keep the conclusions scoped to the tested combinations. The independent RADV wrong-value report also means this is not only a fence-timeout symptom.

Windows data from two more GPUs: agc_driver_bda_device passes on an Intel UHD 770 and an AMD RX 9070 XT, with no fence timeout.

Windows 11 Pro 26300, WinLibs GCC 15.2.0 r7, Release, every execution test built, ANYPS5_NO_SHADER_CACHE=1, serial CTest. Intel UHD Graphics 770 (8086:a780), driver 32.0.101.7088. AMD Radeon RX 9070 XT (RDNA 4, 1002:7550), driver 32.0.31041.1004, with VkCache emptied first. Unchanged main f3059421f, which includes d422c76 (#5859).

`agc_driver_bda_device`, ×3	UHD 770	RX 9070 XT
`main` `f3059421f`	3 of 3 pass: 14.46 s, 0.95 s, 0.97 s	3 of 3 pass: 13.61 s, 0.96 s, 0.94 s

The long first run of each set is pipeline compilation: the shader cache is disabled and VkCache starts empty. It passes even though it takes longer than the 5 s fence wait, so that time is not spent waiting on a fence here. In the parallel full CTest run of the same build it also passed, in 12.6 s on the UHD 770 and 23.2 s on the RX 9070 XT.

AI-assisted: yes (Claude Code)

Windows RTX 4080 data point: on this GPU, main before #7571 loses the device in agc_driver_bda_device, and #7571 (merged since) fixes it.

Windows 11 Pro 26200, NVIDIA GeForce RTX 4080 (driver 617.42), AMD Ryzen 7 5800X, WinLibs GCC 15.2.0 r7, Ninja, Release, bundled dependencies, BUILD_TESTING=ON, ANYPS5_NO_SHADER_CACHE=1.

Build	`agc_driver_bda_device`
main `f78675ae4` (before #7571)	fails 11/11 (full suite and 10 serial runs): `AGC graphics: vkWaitForFences graphics: Vulkan result -4`
main `f78675ae4` + #7571 head `8e7d8874c`	passes 12/12 (targeted, full suite and 10 serial runs), about 1.0 s each serial run
main `f51724078` (before #5859), same machine, 2026-10-17	passed, 0.6 to 0.8 s

So on the RTX 4080 the symptom was VK_ERROR_DEVICE_LOST (-4) rather than the RTX 4070's VK_TIMEOUT (2). With #7571, agc_driver_bda_anchor_reads and agc_driver_bda_span_reads pass too, and the full ctest -j 8 is 1628/1628 (1627/1628 on f78675ae4 alone).

AI-assisted: yes (runs done with Claude Code).

Ampere on Windows: main fails, the fix passes — first Windows/30-series datapoint I can find in this thread.

Environment: Windows 11 (build 22635.5305); GeForce RTX 3070 (GA104, driver 32.0.16.1088, subgroup size 32); Ryzen 5 5600X; WinLibs GCC 15.2.0 r7; Release, all execution tests built; ANYPS5_REQUIRE_VULKAN=1; serial ctest -R '^agc_driver_bda_device$' --output-on-failure; shader disk cache at its default (no ANYPS5_NO_SHADER_CACHE).

Before the fix — main 2eb4de8e1 (with only the test-only change from #7824 on top, agc untouched): fails 3/3, each run ending at the 5 s fence wait:

1/1 Test #424: agc_driver_bda_device ............***Failed    5.34 sec
1/1 Test #424: agc_driver_bda_device ............***Failed    5.05 sec
1/1 Test #424: agc_driver_bda_device ............***Failed    5.16 sec
AGC graphics: vkWaitForFences graphics: Vulkan result -4   # same error in all three runs

Every attempt today failed — 10 runs across two main revisions (4 on a 6b8228b2b build, 6 on this one), 0 passes. Note this machine reports -4 (VK_ERROR_DEVICE_LOST) where the opening report shows 2 (VK_TIMEOUT): the same hang, different terminal result — and the same -4 that the newer Windows runs in #7571 show.

With the fix — PR #7571 head 8e7d8874c8ee078aa0d932a70d0eb92cff853f33 (now merged): passes 5/5.

1/1 Test #379: agc_driver_bda_device ............   Passed    8.18 sec    # first run, cold pipeline cache
1/1 Test #379: agc_driver_bda_device ............   Passed    1.17 sec    # warm

The other warm runs passed the same way, with no Vulkan result line at all (one measured 1.17 s). Same machine, same build options, nothing else changed between the two trees.

One last gap I can see: the original RTX 4070 2 (VK_TIMEOUT) variant hasn't been re-posted against the merged fix — worth a rerun on current main if anyone has that card.

AI-assisted: yes.

Follow-up after the merge, as proposed: pulled current main (@ 36f60df9e, which includes #7571), rebuilt, and agc_driver_bda_device passes 3/3 on the same RTX 3070 machine as above — 1.72 s / 1.18 s / 1.51 s, no Vulkan result line in any run. The merged tree behaves the same as the PR head here.

AI-assisted: yes.

@Neithennn

@Neithennn @Rajkamalcode @savvaniss thanks for the runs. Together with the earlier ones, they settle what #5859 did and what #7571 fixed:

NVIDIA fails on main before fix(shader): leave the BDA lookup functions through one return #7571 and passes with it. The RTX 4070 and 4070 Ti reported earlier time out at the fence (2). On Windows, the RTX 4050 Laptop, 3070 and 4080 lose the device instead (-4).
Rajkamalcode's run on merged main 36f60df9e passes 3 of 3, so the merge behaves like the PR head.
On Windows, Intel UHD 770 and AMD RX 9070 XT pass on main even before fix(shader): leave the BDA lookup functions through one return #7571. They never failed.

@screamm this looks fixed on main to me. Could you close it, unless it still fails for you?

AI-assisted: yes (Claude Code)

Closing as fixed by #7571 (merged 2026-10-19, 64e2a810a). On NVIDIA, agc_driver_bda_device failed on main before it and passes with it: RTX 4080 (Neithennn, 11 of 11 fail, then 12 of 12 pass), RTX 3070 (Rajkamalcode, 3 of 3 fail, then 5 of 5 pass and 3 of 3 on merged main 36f60df9e), RTX 4070 on 616.92 and 617.42, RTX 4090 and RTX 4050 Laptop (#7571 thread), and RTX 4070 Ti on Linux.
Rechecked on the reporting host (RTX 4070, driver 596.21, Windows 11 Pro 26220) on main 1d4164552: 3 of 3 pass, no fence timeout (21-84 s per run, with six builds and a GPU pass sharing the host at the time).
Intel UHD 770, AMD RX 9070 XT and Radeon 860M on Windows never failed.
The RADV assertion blawesom reported has no post-fix run in this thread yet, so it is not covered by this close.
Written with Claude Code assistance.

Navigation Menu

Repository navigation

agc_driver_bda_device times out in vkWaitForFences since the BDA anchor lookup (#5859, d422c760c) #7584

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

agc_driver_bda_device times out in vkWaitForFences since the BDA anchor lookup (#5859, d422c760c) #7584

Description

Activity

MarcZX10 commented on Oct 18, 2026

blawesom commented on Oct 18, 2026

milescoviello commented on Oct 18, 2026

LeandroLP23 commented on Oct 18, 2026

MarcZX10 commented on Oct 18, 2026

savvaniss commented on Oct 18, 2026

Neithennn commented on Oct 19, 2026

Rajkamalcode commented on Oct 19, 2026

Rajkamalcode commented on Oct 19, 2026

milescoviello commented on Oct 19, 2026

screamm commented on Oct 20, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions