Skip to content

fix(shader): keep faulting invocations in their wave and mask their later writes - #6416

Open
Celegans12 wants to merge 3 commits into
boykopovar:mainfrom
Celegans12:fix/two-lane-wave64-faults
Open

Celegans12 wants to merge 3 commits into
boykopovar:mainfrom
Celegans12:fix/two-lane-wave64-faults

Conversation

@Celegans12

@Celegans12 Celegans12 commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

What

Stopping a faulting invocation can leave wave ballots unable to converge and stop its healthy partner lane. Eligible non-pixel programs with two lanes per invocation, ballots or shuffles now keep voting with the failed read's zero and mask that lane's later writes, atomics and exports. Existing spanned-wave, barrier and mesh exclusions remain.

Continuing exposed a guarded-SRT reporting regression: the second word of a null pointer replaced the first fault record. Adopt ThatsLeo's first-fault latch from #7922, adapted so masked lanes count as continuing. The existing guarded-pointer runtime oracle stays unchanged, and its host checks require compare-from-zero latches without unconditional overwrites. No spanned-wave policy from #7922 is included.

Masking adds work even without faults. Astro Bot's reported startup delay remains unexplained; no performance claim. The barrier-safe extension and subgroup-specific skip remain separate in #7392 and #6751.

Tested

Based on main e6f5730. Windows 11, MinGW-w64 GCC 15.2.0: Release, -DBUILD_TESTING=ON, SPIR-V Tools ON. Before repair, main passes agc_driver_guarded_null_pointer; the merged PR fails 3/3 at address4 instead of0, PC0x24. One diagnostic run confirms the overwrite.

Repaired head 771c8bdae4: 13/13 focused processes pass: host shader-memory; NVIDIA guarded-pointer x3, faulting-wave, GPU-buffer-format, BDA-device and span-reads; the same five GPU tests once on Windows lavapipe. All GPU runs require Vulkan, disable application shader caching and load validation 1.4.363; zero skips, timeouts or validation diagnostics. Lavapipe subgroup8 does not establish paired wave64-on32 behavior. Test registration, conventions and source review pass.

Earlier baseline e2df06e661 plus the faulting-wave fixture fails on main because partner lane37 stays untouched. Not tested: current-head Linux, AMD/Intel hardware, console behavior or titles.

On the merge with main 98346b748c, the wave invocation case initially failed on the hardware-spanning wave path. The follow-up keeps those lanes active but masks writes after a fault, and guards byte loads when the lane continues. Six selected fault, guarded-load, wave-decode and cache tests passed twice on Windows with an NVIDIA RTX 3080 Laptop GPU and Vulkan validation enabled. The updated emitter, BDA read and shader-cache sources also passed Linux C++ syntax and layout-guard compilation. No Linux GPU run was performed.

Checklist

  • Based on current main; the first-fault latch from perf(shader): span single-wave wave64 compute by default after a uniformity check #7922 is adopted with attribution
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: none
  • Unblocks: PPSA26344 Ghost of Yōtei (PPSA26344) #675
  • GPUs tested: NVIDIA GeForce RTX 3080 Laptop GPU, driver 610.62; llvmpipe (Mesa 26.2.4); Windows 11
  • AI-assisted: yes

@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request Conflicting files (lines on main) Same exports Same new files
PR 7395 core/shader/recompiler/ShaderDiskCache.hpp L16
PR 8939 core/shader/recompiler/ShaderDiskCache.hpp L16
PR 9457 core/shader/recompiler/ShaderDiskCache.hpp L16
PR 9633 core/shader/recompiler/ShaderDiskCache.hpp L16
PR 9749 core/shader/recompiler/ShaderDiskCache.hpp L16
PR 9790 core/shader/recompiler/ShaderDiskCache.hpp L16
PR 9969 core/shader/recompiler/ShaderDiskCache.hpp L16
PR 10489 core/shader/recompiler/ShaderDiskCache.hpp L16
PR 11551 core/shader/recompiler/ShaderDiskCache.hpp L16
PR 14636 core/shader/recompiler/ShaderDiskCache.hpp L16
PR 14871 core/shader/recompiler/ShaderDiskCache.hpp L16

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 5021 core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterState.hpp
core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp
core/shader/recompiler/SpirvBackend/src/SpirvFlowEmitter.cpp
core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp
PR 5371 core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp
PR 5925 core/shader/recompiler/ShaderDiskCache.hpp
PR 6235 core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp
PR 6280 core/shader/recompiler/ShaderDiskCache.hpp
PR 6412 core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp
core/shader/recompiler/ShaderDiskCache.hpp
PR 6488 core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp
core/shader/recompiler/ShaderDiskCache.hpp
core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp
PR 6858 core/shader/recompiler/ShaderDiskCache.hpp
PR 6924 core/shader/recompiler/ShaderDiskCache.hpp
PR 7085 core/shader/recompiler/ShaderDiskCache.hpp
PR 7223 core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp
core/shader/recompiler/ShaderDiskCache.hpp
PR 7410 core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp
core/shader/recompiler/SpirvBackend/src/SpirvFlowEmitter.cpp
PR 7642 core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp
core/shader/recompiler/ShaderDiskCache.hpp
core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterState.hpp
core/shader/recompiler/SpirvBackend/src/SpirvBdaFault.cpp
core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp
core/shader/recompiler/SpirvBackend/src/SpirvFlowEmitter.cpp
core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp
PR 7684 core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp
core/shader/recompiler/ShaderDiskCache.hpp
PR 8004 core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp
core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterState.hpp
core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp
core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp
PR 9020 core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp
core/shader/recompiler/ShaderDiskCache.hpp
PR 9335 core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp
core/shader/recompiler/ShaderDiskCache.hpp
PR 9356 core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp
core/shader/recompiler/ShaderDiskCache.hpp
core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterState.hpp
core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp
PR 9387 core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp
core/shader/recompiler/ShaderDiskCache.hpp
core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterState.hpp
core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp
PR 9852 core/shader/recompiler/ShaderDiskCache.hpp
PR 9889 core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp
core/shader/recompiler/ShaderDiskCache.hpp
PR 9897 core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp
core/shader/recompiler/ShaderDiskCache.hpp
PR 10049 core/shader/recompiler/ShaderDiskCache.hpp
PR 10233 core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp
PR 10359 core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp
PR 10757 core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp
PR 10867 core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp
core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp
PR 11351 core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp
core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterState.hpp
core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp
PR 11483 core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp
core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterState.hpp
core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp
PR 12324 core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp
PR 12548 core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp
core/shader/recompiler/SpirvBackend/src/SpirvFlowEmitter.cpp
PR 12828 core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp
PR 12944 core/shader/recompiler/ShaderDiskCache.hpp
core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp
core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp
PR 14032 core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp
core/shader/recompiler/ShaderDiskCache.hpp
PR 14217 core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterState.hpp
core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp
PR 14218 core/shader/recompiler/SpirvBackend/include/SpirvBackend/SpirvEmitterState.hpp
core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp
core/shader/recompiler/SpirvBackend/src/SpirvFlowEmitter.cpp
core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp
PR 14561 core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp
PR 14853 core/shader/recompiler/ShaderDiskCache.hpp
PR 14944 core/libs/prx/libSceAgcDriver/tests/ShaderMemory.cpp
core/shader/recompiler/ShaderDiskCache.hpp

@Pipodi

Pipodi commented Oct 15, 2026

Copy link
Copy Markdown
Contributor

@Celegans12 a data point from Ghost of Yōtei (PPSA26344): RTX 4090, driver 617.42, Windows 11. This PR on main 0dab493 + #5815 #5781 #6227 #5405 #6402 #6174, with APS5_SPANNED_WAVE=1 (#5405) and a cold shader cache (our run163).

In our base, this PR conflicted in SpirvEmitter.cpp with a local change that kept spanned programs out of the BDA stop. I took this PR's version and counted spannedWave as a wave operation (waveOperations = ... || state.spannedWave), so spanned programs take the write-masking path.

It still loses the device, at ~1,460 s (frame ~817):

  • nvlddmkm event 153, a TDR (TdrDelay 10 s);
  • vkQueueSubmit recorder: Vulkan result -4;
  • 42 device faults with an unknown instruction pointer at 0x309250c60-0x3092511c0.

That's the same signature as our earlier spanned runs without this PR.

The non-spanned build of 0x80001e5f00 (839,036 words) dispatched fine. The spanned pipeline (838,306 words) was created about 5 minutes before the loss, but the log has no dispatch line for it. After it came several minutes of cold compiles of new 0x801d* programs, and the loss hit while 0x801d29eb00 was being created. So I can't tell whether that program or another spanned one hung. With spanned waves off, the same base runs well past that point.

If you have a suspect, I can rerun with more logging.

@Celegans12

Copy link
Copy Markdown
Contributor Author

A cost I measured tonight that reviewers should know about: on NVIDIA, the write mask roughly doubles pipeline compile time for PPSA26344's giant two-lane compute programs.

Compiled side by side on the same machine, compute 0x801cdd2700 (about 1.75M SPIR-V words) takes 700 s with the earlier "reads continue, nothing masked" version and 1,411 s with this PR. The module is only about 0.5% bigger: roughly 90 more branches and about 550 more stores and loads of the per-lane faulted flag. So the extra time looks like NVIDIA's compiler handling a flag that stays live across the whole program, not the extra code itself.

For now our game branch keeps the older version, so our precompiles don't take twice as long. I'll look at making the mask cheaper, for example only guarding writes that a fault can reach, or keeping the flag in a form the compiler handles better, and post numbers here. If anyone has a preferred shape for this, I'm happy to hear it before I start.

Details
  • Windows 11, RTX 3080 Laptop GPU (driver 610.62). agc_shader_replay --pipeline-only into an empty VkPipelineCache and an empty NVIDIA store, both builds compiling concurrently.
  • Request: PPSA26344 compute 0x801cdd2700. 1,751,533 SPIR-V words without the mask, 1,760,989 with it.
  • A smaller program (0x806eddec00, 589k words) doesn't take the flag path and compiles in 18 s either way, with an identical pipeline fingerprint.

@Dextroid17

Copy link
Copy Markdown
Contributor

Tested on Linux with lavapipe, the CI device: llvmpipe (Mesa 26.2.4, LLVM 23.1.1), CachyOS, GCC 16.2.1, CMake 4.4.3, default build type, full build with BUILD_TESTING=ON. Each run used a clean environment (env -i with HOME, PATH, VK_ICD_FILENAMES and GALLIUM_OVERRIDE_CPU_CAPS=avx).

This PR (b5a0c1ff7 merged into main 44df21632, no conflicts):

  • agc_driver_faulting_wave_invocation and agc_driver_gpu_buffer_format: pass, 3 runs out of 3.
  • With this PR's 5 source files reverted to main and its tests kept, both still pass, 3 runs out of 3. So on lavapipe the new test passes without the fix.

The reason, from the code: lavapipe reports 8-wide subgroups (minSubgroupSize = maxSubgroupSize = 8). Two lanes per invocation only happens when the host subgroup is 32 (SpirvEmitter.cpp:265, laneCount is 2 only when hostSubgroupSize == 32u). The write mask itself is still enabled there, since the execz branch makes the program need ballots. But the failures the test checks (the execz loop that never ends, and the partner lane 37 stopping with lane 5) need two lanes per invocation. So the description's "on main it fails" holds on 32-wide subgroups but not on lavapipe, and CI would stay green if the fix regressed. Wave32Subgroup.cpp:73 skips with a reason when the subgroup does not fit. Something similar here, for example:

if (device->Target().subgroupSize != 32u) {
    std::printf("skipped, the device runs wave64 programs on %u-wide subgroups\n", device->Target().subgroupSize);
    return VulkanTestSkipped;
}

would make lavapipe report a device-limited skip (tools/check_skips.py accepts reasons that mention subgroups) instead of a pass. Only a suggestion; whether a 64-wide device should run it is your call.

This may be useful for the mask rework you mention: on lavapipe the test passes even with the fix reverted, so it can't show whether a cheaper mask still handles the two-lane cases. That can only be checked on a 32-wide device, such as your RTX 3080 Laptop GPU.

Merged together with #6402 (on main 44df21632, no conflicts): full ctest: 2 of 1269 fail, 57 skipped (the same skips as main). The two failures are not from either PR:

  • agc_driver_mesh_index_cache segfaults on main too (and 5 runs out of 5 on the merge).
  • agc_frame_dump_io (from fix(agc): disable optional BMP dumps after host I/O failures #5800) also fails on plain main, in 13 of 200 runs on an idle machine: closed pipe pixel write was accepted. Reading the test, its 128×128×4 = 65,536 pixel bytes are exactly Linux's default pipe buffer, so if the reader drains the 54-byte header before the pixel write starts, the empty pipe takes the whole write and it succeeds before the reader closes. Not measured beyond that.

Apart from those, the failed and skipped lists match main's full run.

Not tested: NVIDIA, AMD, Intel, PPSA26344.

AI-assisted: yes

@Amoenus

Amoenus commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Barrier-program follow-up: #6689.

I reproduced later guest writes after an unmapped BDA read plus s_barrier on otherwise unchanged main. Main plus your current head b5a0c1ff79eb52bd52fea0ba6d0434e072190b99 still reproduces it in 5/5 runs on each of an RTX 5080 and AMD Radeon iGPU (Windows). bdaFaultMasksWrites currently requires bdaStopsInvocations, which excludes the barrier programs.

Proposed follow-up reuses your per-lane mask, keeps the invocations at their barriers, and carries the checked BDA helper's fault result back to the correct lane half. All-fault and mixed valid/faulting lane tests pass 5/5 per GPU. The full Windows build and focused checks passed, but broader regression/static-analysis gates remain incomplete; the current checkpoint and remaining checks are recorded in #6689. This prototype is deferred while we pivot to PR maintenance.

Proposed ordering is #6416 first, then this compute follow-up. If you prefer to incorporate it here, the focused commit is available to take. No competing upstream PR is open.

AI-assisted: yes.

@MarcZX10

Copy link
Copy Markdown
Contributor

While preparing the barrier follow-up, I checked the narrow-subgroup case raised in the review. The current agc_driver_faulting_wave_invocation passes on llvmpipe with subgroup size 8, and the review reports it also passes with the implementation reverted there; that device does not exercise the two-lane EXEC case the test is meant to cover. A device-limitation skip when subgroup size is not 32 would make that coverage status explicit. The follow-up mixed-lane control had the same assumption: on subgroup 8 it reported 63 mismatches. I have a local test-only guard for that mixed control on subgroup != 32; the two focused CTests then pass on llvmpipe (with the host CPU capability guard temporarily bypassed). The 32-lane mixed case still needs a suitable device, as your Windows NVIDIA/AMD runs provide.

@MarcZX10

Copy link
Copy Markdown
Contributor

I opened stacked follow-up #6751 for the test-only subgroup guard discussed here. The agc_driver_faulting_wave_invocation case now reports a device-limited skip unless the host subgroup is exactly 32 lanes, because narrower/wider subgroups do not exercise the two-lanes-per-invocation wave64 case. #6751 carries your current PR commits unchanged and depends on #6416; Linux target build and conventions passed locally. The focused CTest cannot reach Vulkan on my Xeon because it lacks the required CPU instruction sets.

@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 16, 2026
@oneandonlydean oneandonlydean self-assigned this Oct 17, 2026
@blawesom

Copy link
Copy Markdown
Contributor

Rebase handoff — rebased onto current main (authorship unchanged). One conflict in SpirvEmitter.cpp: main added !state.spannedWave to the bdaStopsInvocations condition while this PR adds the bdaFaultMasksWrites computation after it — the merged version keeps main's condition and layers your fault-masking computation.

Verified on Linux (CachyOS, kernel 7.2.9, GCC 16.2.1, Ninja, Release, BUILD_TESTING=ON, AMD RX 9070 XT / RADV):

To pick it up:

git fetch https://github.com/blawesom/AnyPS5 rebase/6416
git checkout <your branch>
git merge --ff-only FETCH_HEAD

Head is 48482f629.

@oneandonlydean

oneandonlydean commented Oct 18, 2026 •

Copy link
Copy Markdown
Collaborator

Thanks. Main has moved under this PR: since bd6ed97, bdaStopsInvocations is false in spanned waves (a single-wave wave64 compute program on a 32-lane host with full subgroups, 80e3d90/7a6cc717863). Because bdaFaultMasksWrites requires bdaStopsInvocations and StopBdaInvocationIf returns early without it, a straight rebase turns the masking off for every single-wave program, including the test here, which I'd expect to fail on an NVIDIA or full-subgroup AMD device after the rebase. Since masking keeps invocations running, could it apply in spanned waves too (record the fault in StopBdaInvocationIf before the !bdaStopsInvocations return, and gate masking on the fault kinds rather than on stopping), with the test covering a spanned single wave and a two-wave workgroup? Pipodi still sees spanned wave64 dispatches hang in PPSA26344, so a run of the rebased head there would tell us whether this is the remaining cause.

@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 18, 2026
@oneandonlydean

oneandonlydean commented Oct 18, 2026 •

Copy link
Copy Markdown
Collaborator

Correction to my earlier note: spanned waves are opt-in on main (APS5_SPANNED_WAVE, VulkanDevice.cpp:1784), so by default this PR's test runs the two-lane path with masking on and passes, and it fails with masking forced off, so the rebase onto main is fine as is. I pushed it to this branch (9c76817) and started a full CI run. The gap I described applies only with APS5_SPANNED_WAVE=1: there bdaStopsInvocations is false, masking is off, and the test fails with "lane 5 stored after its read faulted". Not a blocker for this PR; as a follow-up, could the masking also cover spanned waves (record the fault before the !bdaStopsInvocations return) and the test run once with APS5_SPANNED_WAVE=1? That may be what still hangs PPSA26344's spanned runs.

@jotitadev jotitadev left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apple GPU / MoltenVK run. Apple's subgroup size is exactly 32, the case agc_driver_faulting_wave_invocation needs (see #6751): red without the fix, green with it, and no other change.

Machine: Apple M2 Max, macOS 27.2, x86-64 build under Rosetta. Head 9c76817, PR diff applied cleanly. Driver: MoltenVK 1.4.2 (Vulkan SDK 1.4.363.0), subgroup=32.

Tested: the tree is main a373e85 + mugurc's macOS series (ci/macos-series-2) + the PRs needed to build on macOS (#6667, #6668, #7524, #6969, and two files of #7332), not plain main (it doesn't build on macOS yet). Base on that tree: 40 ^agc failures, the same known failures in every run below unless noted.

  • agc_driver_faulting_wave_invocation with the PR: passes 3 of 3.
  • The test alone on the base tree: fails with lane 0 left the EXEC loop after 100 passes, expected 4 (the loop stops at 100).
  • Full ^agc suite with the PR: 40 of 728 fail, the same 40 as the base.

About the spanned-wave gap @oneandonlydean described: it can't occur on MoltenVK today. With APS5_SPANNED_WAVE=1 the driver logs spans two full 32-lane subgroups: 0, because MoltenVK doesn't give full compute subgroups, so wave64 always takes the two-lane path here and the masking stays on.

GPUs tested: Apple M2 Max (MoltenVK 1.4.2)

Written with AI assistance (Claude Code). I ran the builds and tests above on my machine.

@Celegans12

Copy link
Copy Markdown
Contributor Author

@oneandonlydean, thanks for doing the rebase and for sorting out the spanned-wave question. Yes, I'll take the spanned-wave masking as a follow-up PR once this one is in. The plan is what you describe: record the fault in StopBdaInvocationIf before the !bdaStopsInvocations return, gate the masking on the fault kinds rather than on stopping, and run the test once more with APS5_SPANNED_WAVE=1, for a single spanned wave and for a two-wave workgroup. @jotitadev's run shows the gap can't happen on MoltenVK, so the follow-up is about NVIDIA and full-subgroup AMD.

About Yōtei: our runs do use APS5_SPANNED_WAVE=1, but they don't carry this PR's masking yet, because of the compile-time cost I mentioned earlier. So I can't tell yet whether this is what still hangs the spanned runs. I'll try it in a game run with the follow-up once the machine is free.

hritikutekar added a commit to hritikutekar/AnyPS5 that referenced this pull request Oct 20, 2026
…x for named wave-operation programs (boykopovar#6416 masks elsewhere)
@TechLevelZero

Copy link
Copy Markdown
Contributor

✅ NVIDIA GPU test: passed

PR head 9c76817 merged onto main f6fddd6 · tested 2026-10-30 04:40 UTC · 25 min

GPU NVIDIA RTX 2000 Ada Generation, driver 595.99.02, Vulkan 1.4.329
Host Ubuntu 26.04.1 LTS, kernel 7.0.0-38-generic, container ubuntu:24.04
Logs build and test logs for this run
Build ✅ passed (Release, -DANYPS5_ENABLE_SPIRV_TOOLS=ON)
Tests ✅ 2,263 passed · 3 failed (0 new, 2 also fail on main, 1 unstable on main) · 41 skipped (ANYPS5_REQUIRE_VULKAN=1)
Test setup CPU-only tests without a GPU; GPU tests (and any CPU-only test that failed or skipped without one) once on the GPU with VK_LAYER_KHRONOS_validation and ANYPS5_NO_SHADER_CACHE=1; main ran the same way
Test selection 2,307 of 2,950 tests run: the ones this change can reach, chosen by tools/select_tests.py from main's build graph
Vulkan validation ✅ no new validation IDs vs main (863 tests run on the GPU)
Slow tests (>30 s, GPU tests timed with validation layers) none
Unstable on main (1, these also failed in recent main runs on this machine; not counted against this PR)
agc_driver_submit_progress: failed in 5 of the last 7 main runs
Failing on main too (2, not counted against this PR)
guest_mmap_executable
guest_socket_options_multicast
Passed on retry (1, failed in the parallel run, passed when re-run with a 360 s timeout; not counted against this PR)
agc_driver_job_manager_compute_capacity: Failed in the parallel run; also passed only on retry in 77 other runs in the last 7 days
Validated but not compared (1, not in the main baseline's validation pass)
agc_driver_faulting_wave_invocation
Validation IDs already present on main (4 IDs in 390 tests)
VUID-VkDeviceCreateInfo-pNext-pNext (390 tests): agc_compute_pipeline_replay, agc_driver, agc_driver_address_passes, agc_driver_address_passes_draw_transitions, agc_driver_address_passes_targets_split, agc_driver_alias_copy_label, agc_driver_array_layer_target_cpu, agc_driver_bitfield_mask, and 382 more
VUID-vkCmdCopyImage-srcImage-01548 (11 tests): agc_driver_depth_clear_hle, agc_driver_depth_plane_buffer_memory, agc_driver_depth_plane_cpu_write_after_memo, agc_driver_depth_plane_cpu_write_after_page_unwatch, agc_driver_depth_plane_cpu_write_after_partial_unwatch, agc_driver_depth_plane_partial_store, agc_driver_depth_plane_partial_store_unimported, agc_driver_depth_storage_dispatch, and 3 more
VUID-vkCmdCopyImage-srcImage-01551 (11 tests): agc_driver_depth_clear_hle, agc_driver_depth_plane_buffer_memory, agc_driver_depth_plane_cpu_write_after_memo, agc_driver_depth_plane_cpu_write_after_page_unwatch, agc_driver_depth_plane_cpu_write_after_partial_unwatch, agc_driver_depth_plane_partial_store, agc_driver_depth_plane_partial_store_unimported, agc_driver_depth_storage_dispatch, and 3 more
VUID-VkPipelineColorBlendStateCreateInfo-pAttachments-00605 (2 tests): agc_driver_prepared_exports, agc_driver_srgb8_color_target
Skipped (41)
agc_driver_bda_reads_shadowed_results: skipped, the device reports watched host memory written when it imports or reads it
agc_driver_bda_reads_shadowed_results_per_build: skipped, the device reports watched host memory written when it imports or reads it
agc_driver_msaa_dcc_compared_import_2: skipped, the device reports host imports that keep write protection, so the write-watch probe compares none of them
agc_driver_msaa_dcc_compared_import_4: skipped, the device reports host imports that keep write protection, so the write-watch probe compares none of them
agc_driver_msaa_dcc_compared_import_8: skipped, the device reports host imports that keep write protection, so the write-watch probe compares none of them
agc_driver_sample_order_grid_reference: skipped, the device does not keep depth drawn at custom sample locations at those locations, even after a transition that names them: the composite of uniform renders holds the guest's depth in 0 of 3005 sloped samples, so no path can be compared with it bit for bit
agc_driver_sparse_dma_buf_import: skipped, host imports are unavailable: /dev/udmabuf is not accessible
agc_driver_stencil_ref_export: skipped, the device lacks VK_EXT_shader_stencil_export
agc_driver_vertex_elected_atomic_max: skipped, the device runs vertex programs in subgroups, so they keep their cross-lane reductions
agc_host_import_sparse: skipped, host imports are unavailable: /dev/udmabuf is not accessible
agc_import_settle_initial_dcc: skipped, the device has no deferred settle of host imports: imports under 1 MiB wait for one only on Windows
agc_unit_shadow_completion_store: skipped, the device reports watched host memory written when it imports or reads it
guest_http_uri_locale: 
guest_net_ctl: skipped, the host has no IPv4 address
guest_video_out_color_space: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_hdr: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_high_frame_rate: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_output_mode: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_resolution_status: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_wqhd_detection: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
video_out_flip_controls: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_cpuflip: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_cpuflip_in_order: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_eop: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_eop_aftergpu: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_eop_main: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_fresh: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or plat
Reproduce
git fetch origin 9c768176f53bc372e8620c863481aad6bc1d0343 f6fddd60d826831f70f7c62e25423610ee742bcc
git checkout f6fddd60d826831f70f7c62e25423610ee742bcc && git merge --no-ff 9c768176f53bc372e8620c863481aad6bc1d0343
git submodule update --init --recursive
cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=ON -DANYPS5_ENABLE_SPIRV_TOOLS=ON
cmake --build build && cmake --build build --target libs
VK_DRIVER_FILES=/nonexistent/icd.json ctest --test-dir build -E '^(agc|shader|recompiler|videoout|video_out|guest_video_out|vulkan)|gpu' --timeout 120 --parallel 32 --output-on-failure
ANYPS5_REQUIRE_VULKAN=1 ANYPS5_NO_SHADER_CACHE=1 VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation \
  ctest --test-dir build -R '^(agc|shader|recompiler|videoout|video_out|guest_video_out|vulkan)|gpu' --timeout 120 --parallel 4 --output-on-failure
# failures are then re-run alone: --timeout 360 --parallel 2; new validation IDs: the test alone, twice, --parallel 1

Automated build and test on a contributor's NVIDIA machine; it does not replace upstream CI. A new comment is posted for each new push. Full logs.

@fogydes

fogydes commented Oct 30, 2026

Copy link
Copy Markdown
Contributor

stale+rebase

1. Freshness Envelope

  • Head SHA: 9c768176f53bc372e8620c863481aad6bc1d0343
  • Merge-base SHA: 6efb7261b5ca879c26083ad36bad92c2a9293a94
  • Behind origin/main: 12,359 commits
  • Merge-tree status: exit code 1 (conflict against origin/main in core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp)

2. Findings

  1. [verified] Merge conflict requires rebase: git merge-tree --write-tree origin/main 9c768176f53bc372e8620c863481aad6bc1d0343 exits with code 1 due to a conflict in core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp.
  2. [verified] Conventions clean against merge-base: python tools/check_conventions.py --base 6efb7261b5ca879c26083ad36bad92c2a9293a94 in an isolated scratch worktree reported 0 findings.
  3. [read-only] Non-conventional merge commit: Commit 9c768176f5 (Merge branch 'main' into fix/two-lane-wave64-faults) introduces a merge commit; AGENTS.md requires all commit subjects to follow Conventional Commits. A linear rebase will resolve this cleanly.
  4. [read-only] Missing checklist line: The PR description omits the required Fixes: line from .github/pull_request_template.md.
  5. [read-only] Missing Vulkan validation layer comparison: AGENTS.md requires shader recompiler PRs to report test runs with ANYPS5_REQUIRE_VULKAN=1, ANYPS5_NO_SHADER_CACHE=1, and VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation, confirming no new VUID- validation errors are introduced.
  6. [author-reported] Reported Windows 11 / RTX 3080 Laptop GPU and Linux llvmpipe test suite runs and agc_driver_faulting_wave_invocation verification.

3. Cross-PR Ledger

@Celegans12

Copy link
Copy Markdown
Contributor Author

I've rebased this onto main e2df06e as one conventional commit, preserving main's spanned-wave exclusion and mesh-window initialization. The fault-mask patch is otherwise unchanged, and the missing Fixes: none checklist item is now filled in.

All four focused tests pass on the RTX 3080 Laptop GPU. With main's runtime code and only the new fixture added, the three existing controls pass but the fixture fails because partner lane 37 stays untouched (0xdeadbeef) instead of storing its loaded 2. Both runs loaded Vulkan validation with shader caching disabled and produced zero VUIDs. Windows lavapipe also passed all four, though its 8-wide subgroups do not exercise the paired wave64-on-32 path.

The Astro startup delay is still open: masking adds a flag load and branch around qualifying writes even without faults, so a warm cache and no pipeline misses don't rule out shader execution cost. The missing texture-budget message is conditional reporting and doesn't establish skipped budget work. I haven't run titles on this rebased head; paired main/slow/fast startup logs with the exact settings and revisions, or a reduced reproduction, are still needed to explain the delay.

@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 31, 2026
@TechLevelZero

TechLevelZero commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

❌ NVIDIA GPU test: failed

PR head 8e6e734 merged onto main 3741ca9 · tested 2026-11-01 09:56 UTC · 36 min

GPU NVIDIA RTX 2000 Ada Generation, driver 595.99.02, Vulkan 1.4.329
Host Ubuntu 26.04.1 LTS, kernel 7.0.0-38-generic, container ubuntu:24.04
Logs build and test logs for this run
Build ✅ passed (Release, -DANYPS5_ENABLE_SPIRV_TOOLS=ON)
Tests ❌ 2,424 passed · 6 failed (1 new: 1 pass on main with the PR's tests, 3 also fail on main, 2 unstable on main) · 44 skipped (ANYPS5_REQUIRE_VULKAN=1)
Test setup CPU-only tests without a GPU; GPU tests (and any CPU-only test that failed or skipped without one) once on the GPU with VK_LAYER_KHRONOS_validation and ANYPS5_NO_SHADER_CACHE=1; main ran the same way
Test selection 2,474 of 3,121 tests run: the ones this change can reach, chosen by tools/select_tests.py from main's build graph
Vulkan validation ✅ no new validation IDs vs main (930 tests run on the GPU)
Slow tests (>30 s, GPU tests timed with validation layers) ⚠️ agc_driver_pipeline_journal 45.8 s, ⚠️ agc_driver_graphics_maps_fallback 31.2 s

Failing tests

This PR changes test files, so each failing test was also built and run on main with only the PR's test changes applied. ✅ there means the failure comes from the PR's other changes; ❌ means the test fails on main too, so this run does not show the PR caused it.

Test With this PR Same test on main 3741ca9 (main + this PR's test changes only)
agc_driver_guarded_null_pointer ❌ failed ✅ passed
agc_driver_guarded_null_pointer: Failed (also failed when re-run alone), assertion and diagnostics
ctest: Failed

[write-watch] host imports resolve write protection: no (0 of 4 scratch pages written after a GPU read, 0 after a GPU write, 4 after the import); imported ranges stay watched
AGC graphics: guarded pointer: a pointer read through a null pointer, used only by a skipped read, did not fault at its own pc:
Unstable on main (2, these also failed in recent main runs on this machine; not counted against this PR)
agc_submit_progress: failed in 6 of the last 10 main runs
agc_submit_progress_no_timeline: failed in 8 of the last 10 main runs
Failing on main too (3, not counted against this PR)
agc_driver_image_table
guest_mmap_executable
guest_socket_options_multicast
Validated but not compared (1, not in the main baseline's validation pass)
agc_driver_faulting_wave_invocation
Validation IDs already present on main (5 IDs in 480 tests)
VUID-VkDeviceCreateInfo-pNext-pNext (480 tests): agc_driver, agc_driver_address_passes, agc_driver_alias_copy_label, agc_driver_alpha_to_coverage, agc_driver_array_layer_target, agc_driver_bc_block_copy, agc_driver_bda_anchor_reads, agc_driver_bda_span_reads, and 472 more
VUID-vkCmdCopyImage-srcImage-01548 (7 tests): agc_driver_depth_clear_hle, agc_driver_depth_plane_buffer_memory, agc_driver_depth_plane_partial_store, agc_driver_depth_plane_sampling, agc_driver_depth_storage_dispatch, agc_driver_depth_surface_from_storage, agc_driver_dispatch_move_depth
VUID-vkCmdCopyImage-srcImage-01551 (7 tests): agc_driver_depth_clear_hle, agc_driver_depth_plane_buffer_memory, agc_driver_depth_plane_partial_store, agc_driver_depth_plane_sampling, agc_driver_depth_storage_dispatch, agc_driver_depth_surface_from_storage, agc_driver_dispatch_move_depth
VUID-VkPipelineColorBlendStateCreateInfo-pAttachments-00605 (1 test): agc_driver_prepared_exports
VUID-VkShaderModuleCreateInfo-pCode-08737 (1 test): agc_driver_image_sample_derivatives
Skipped (44)
agc_driver_bda_reads_shadowed_results: skipped, the device reports watched host memory written when it imports or reads it
agc_driver_bda_reads_shadowed_results_per_build: skipped, the device reports watched host memory written when it imports or reads it
agc_driver_sample_order_grid_reference: skipped, the device does not keep depth drawn at custom sample locations at those locations, even after a transition that names them: the composite of uniform renders holds the guest's depth in 0 of 3005 sloped samples, so no path can be compared with it bit for bit
agc_host_import_sparse: skipped, host imports are unavailable: /dev/udmabuf is not accessible
agc_import_settle_initial_dcc: skipped, the device has no deferred settle of host imports: imports under 1 MiB wait for one only on Windows
agc_unit_shadow_completion_store: skipped, the device reports watched host memory written when it imports or reads it
guest_http_uri_locale: 
guest_net_ctl: skipped, the host has no IPv4 address
guest_video_out_color_space: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_hdr: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_high_frame_rate: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_output_mode: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_resolution_status: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_wqhd_detection: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
video_out_flip_aftergpu: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_backtoback: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_controls: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_cpuflip: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_cpuflip_in_order: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_dcc: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_deferred: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_eop: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_eop_aftergpu: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_eop_main: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_fresh: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_f
Reproduce
git fetch origin 8e6e73474f8d272f71c8d1235a193b1463afca26 3741ca9327246d0df81638726a986cef862b6db5
git checkout 3741ca9327246d0df81638726a986cef862b6db5 && git merge --no-ff 8e6e73474f8d272f71c8d1235a193b1463afca26
git submodule update --init --recursive
cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=ON -DANYPS5_ENABLE_SPIRV_TOOLS=ON
cmake --build build && cmake --build build --target libs
VK_DRIVER_FILES=/nonexistent/icd.json ctest --test-dir build -E '^(agc|shader|recompiler|videoout|video_out|guest_video_out|vulkan)|gpu' --timeout 120 --parallel 32 --output-on-failure
ANYPS5_REQUIRE_VULKAN=1 ANYPS5_NO_SHADER_CACHE=1 VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation \
  ctest --test-dir build -R '^(agc|shader|recompiler|videoout|video_out|guest_video_out|vulkan)|gpu' --timeout 120 --parallel 4 --output-on-failure
# failures are then re-run alone: --timeout 360 --parallel 2; new validation IDs: the test alone, twice, --parallel 1
# the failing tests, on this merge and on main:
ANYPS5_REQUIRE_VULKAN=1 ANYPS5_NO_SHADER_CACHE=1 VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation \
  ctest --test-dir build -R '^(agc_driver_guarded_null_pointer)$' --parallel 1 -V

Automated build and test on a contributor's NVIDIA machine; it does not replace upstream CI. A new comment is posted for each new push. Full logs.

@cookie42073-cmd

Copy link
Copy Markdown
Contributor

Hey @Celegans12, I checked the rebased 8e6e73474f8d272f71c8d1235a193b1463afca26 against main 6a691f69ee44888635bed90833a0ab20ed9889ab on two Windows GPUs, then checked how it layers with #7392.

Windows 11 Pro 26300, WinLibs GCC 15.2.0 r7, Release, bundled SPIRV-Tools:

Source RTX 4070 SUPER, 621.01 / 32.0.16.2101 Intel UHD 770, 32.0.101.7088
Main, existing controls 9 passed 8 passed, mesh test skipped
Updated #6416 10 passed 9 passed, same mesh test skipped
Then #7392, original tests 7 passed, 3 failed 6 passed, 3 failed, same skip
With the two test corrections, and after exact restoration 10 passed 9 passed, same skip

Both GPUs report native subgroup 32. Intel reports mesh=0, so agc_driver_mesh_vector_window_waves is not covered there; it passes on NVIDIA.

With main's runtime and only your exact new fixture added, agc_driver_faulting_wave_invocation fails on both GPUs because lane 37 stays 0xdeadbeef instead of storing 2; the other three controls pass. The updated prerequisite fixes that fixture without changing the existing controls.

For #7392 I applied only its barrier-write and test-gating commits over the updated prerequisite, preserving authorship and main's mesh initialization. No second copy of the old prerequisite. The same helper assertion and direct/draw span-report failures remain until the two supplied test corrections are applied. Those three corrected tests also pass three fresh-process runs per GPU, 18 additional executions.

Turning only barrierCompute off makes all three regression tests fail on each GPU. All 24 fault reports remain visible while 16 controls write incorrect output per GPU. Exact production bytes were then restored, rebuilt, and the final sets above passed. This retains #7392's published masking expression and native-32 mixed-case gate; it does not settle the separate spanned-wave follow-up.

All seven source arms passed the full default Windows build (FFmpeg reused its pinned prebuilt cache). Focused runs required Vulkan, disabled the application shader cache, excluded implicit layers and actually loaded Khronos validation. No VUID/SYNC errors or test timeouts were observed. Stale installed-layer manifest lookups still emit loader discovery errors. The full CTest suite was not run.

The refreshed test-only commits are bf83e36e658e56eccbbb88fd599912893e0afb00 and e511f0dec1573c03316a8711936755d0fe380dc3; their patch IDs match the previous corrections. Blazing Cook authorship and the existing implementation authors are preserved. No maintainer branch was changed and no new PR was opened.

This is focused correctness evidence, not an Astro startup, title-unblock or performance result. AMD/native64, Linux and console behavior were not tested here. AI-assisted, with independent source and raw-evidence review.

AnyPS5-6416-7392-verified-stack-20261031.zip

ZIP SHA256: 89020f6dce1837edb4e2926079f6dfdf2ab730ba8c3529176d1acad493421579

…ater writes

Keep faulted wave participants voting instead of returning, while masking
their later writes, atomics and exports. This also preserves the healthy
partner of a two-lane invocation. Retain existing eligibility exclusions.

Latch the first guarded SRT fault so later words cannot replace it.
Strengthen host assertions; retain the runtime address and PC oracle.

Based on ThatsLeo/AnyPS5@b9362d8
core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvMemoryInstructions.cpp
(author: ThatsLeo). Adapt the actual-stopping predicate for masked lanes.
No spanned-wave policy from boykopovar#7922 is included.

AI-assisted: yes
@Celegans12
Celegans12 force-pushed the fix/two-lane-wave64-faults branch from 8e6e734 to 4414d78 Compare November 1, 2026 11:25
@github-actions github-actions Bot added the review queue Next in the maintainer review queue: checks pass, no open change request, no conflict label Nov 1, 2026
@TechLevelZero

Copy link
Copy Markdown
Contributor

✅ NVIDIA GPU test: passed

PR head 4414d78 merged onto main e6f5730 · tested 2026-11-01 16:47 UTC · 22 min

GPU NVIDIA RTX 2000 Ada Generation, driver 595.99.02, Vulkan 1.4.329
Host Ubuntu 26.04.1 LTS, kernel 7.0.0-38-generic, container ubuntu:24.04
Logs build and test logs for this run
Build ✅ passed (Release, -DANYPS5_ENABLE_SPIRV_TOOLS=ON)
Tests ✅ 2,431 passed · 6 failed (0 new, 3 also fail on main, 3 unstable on main) · 45 skipped (ANYPS5_REQUIRE_VULKAN=1)
Test setup CPU-only tests without a GPU; GPU tests (and any CPU-only test that failed or skipped without one) once on the GPU with VK_LAYER_KHRONOS_validation and ANYPS5_NO_SHADER_CACHE=1; main ran the same way
Test selection 2,482 of 3,148 tests run: the ones this change can reach, chosen by tools/select_tests.py from main's build graph
Vulkan validation ✅ no new validation IDs vs main (932 tests run on the GPU)
Slow tests (>30 s, GPU tests timed with validation layers) ⚠️ agc_driver_pipeline_journal 120.6 s, ⚠️ agc_driver_lookup_log_fits_build 48.9 s, ⚠️ agc_driver_graphics_maps_fallback 30.9 s, ⚠️ agc_driver_graphics 30.3 s
Unstable on main (3, these also failed in recent main runs on this machine; not counted against this PR)
agc_driver_pipeline_journal: failed in 1 of the last 10 main runs
agc_submit_progress: failed in 6 of the last 10 main runs
agc_submit_progress_no_timeline: failed in 8 of the last 10 main runs
Failing on main too (3, not counted against this PR)
agc_driver_image_table
guest_mmap_executable
guest_socket_options_multicast
Validated but not compared (1, not in the main baseline's validation pass)
agc_driver_faulting_wave_invocation
Validation IDs already present on main (5 IDs in 481 tests)
VUID-VkDeviceCreateInfo-pNext-pNext (481 tests): agc_driver, agc_driver_address_passes, agc_driver_alias_copy_label, agc_driver_alpha_to_coverage, agc_driver_array_layer_target, agc_driver_bc_block_copy, agc_driver_bda_anchor_reads, agc_driver_bda_span_reads, and 473 more
VUID-vkCmdCopyImage-srcImage-01548 (7 tests): agc_driver_depth_clear_hle, agc_driver_depth_plane_buffer_memory, agc_driver_depth_plane_partial_store, agc_driver_depth_plane_sampling, agc_driver_depth_storage_dispatch, agc_driver_depth_surface_from_storage, agc_driver_dispatch_move_depth
VUID-vkCmdCopyImage-srcImage-01551 (7 tests): agc_driver_depth_clear_hle, agc_driver_depth_plane_buffer_memory, agc_driver_depth_plane_partial_store, agc_driver_depth_plane_sampling, agc_driver_depth_storage_dispatch, agc_driver_depth_surface_from_storage, agc_driver_dispatch_move_depth
VUID-VkPipelineColorBlendStateCreateInfo-pAttachments-00605 (1 test): agc_driver_prepared_exports
VUID-VkShaderModuleCreateInfo-pCode-08737 (1 test): agc_driver_image_sample_derivatives
Skipped (45)
agc_driver_bda_reads_shadowed_results: skipped, the device reports watched host memory written when it imports or reads it
agc_driver_bda_reads_shadowed_results_per_build: skipped, the device reports watched host memory written when it imports or reads it
agc_driver_sample_order_grid_reference: skipped, the device does not keep depth drawn at custom sample locations at those locations, even after a transition that names them: the composite of uniform renders holds the guest's depth in 0 of 3005 sloped samples, so no path can be compared with it bit for bit
agc_host_import_sparse: skipped, host imports are unavailable: /dev/udmabuf is not accessible
agc_import_settle_initial_dcc: skipped, the device has no deferred settle of host imports: imports under 1 MiB wait for one only on Windows
agc_unit_shadow_completion_store: skipped, the device reports watched host memory written when it imports or reads it
guest_http_uri_locale: 
guest_net_ctl: skipped, the host has no IPv4 address
guest_video_out_color_space: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_hdr: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_high_frame_rate: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_output_mode: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_resolution_status: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_wqhd_detection: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
video_out_flip_aftergpu: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_backtoback: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_controls: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_cpuflip: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_cpuflip_in_order: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_dcc: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_deferred: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_eop: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_eop_aftergpu: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_eop_main: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_fps_limit: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_o
Reproduce
git fetch origin 4414d7880e564dc52832227045157136b3b16527 e6f57307fa7f47165d7fc1bee945abdb2f6816e7
git checkout e6f57307fa7f47165d7fc1bee945abdb2f6816e7 && git merge --no-ff 4414d7880e564dc52832227045157136b3b16527
git submodule update --init --recursive
cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=ON -DANYPS5_ENABLE_SPIRV_TOOLS=ON
cmake --build build && cmake --build build --target libs
VK_DRIVER_FILES=/nonexistent/icd.json ctest --test-dir build -E '^(agc|shader|recompiler|videoout|video_out|guest_video_out|vulkan)|gpu' --timeout 120 --parallel 32 --output-on-failure
ANYPS5_REQUIRE_VULKAN=1 ANYPS5_NO_SHADER_CACHE=1 VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation \
  ctest --test-dir build -R '^(agc|shader|recompiler|videoout|video_out|guest_video_out|vulkan)|gpu' --timeout 120 --parallel 4 --output-on-failure
# failures are then re-run alone: --timeout 360 --parallel 2; new validation IDs: the test alone, twice, --parallel 1

Automated build and test on a contributor's NVIDIA machine; it does not replace upstream CI. A new comment is posted for each new push. Full logs.

@github-actions github-actions Bot added needs rebase Conflicts with main, rebase needed and removed review queue Next in the maintainer review queue: checks pass, no open change request, no conflict labels Nov 2, 2026
Keep spanning wave invocations active for subgroup operations while masking writes after a fault. Guard byte loads when fault handling records a lane fault without returning from the invocation.
@Celegans12

Copy link
Copy Markdown
Contributor Author

@cookie42073-cmd @TechLevelZero I merged current main and found that the faulting-wave test failed when main selected a hardware-spanning wave: the faulted lane continued and performed its later store. The follow-up keeps those lanes participating while masking their writes, and guards the byte-load path when recording a fault doesn't return from the invocation.

With shader caching disabled and Vulkan validation enabled, the wave case, guarded-pointer cases, guarded-load checks, wave decoding and shader-cache test each passed twice on Windows with an NVIDIA RTX 3080 Laptop GPU. The updated emitter, BDA read and shader-cache sources also compile on Linux with the layout guards enabled. Could you rerun the updated head when convenient?

@github-actions github-actions Bot added review queue Next in the maintainer review queue: checks pass, no open change request, no conflict and removed needs rebase Conflicts with main, rebase needed labels Nov 2, 2026
@TechLevelZero

Copy link
Copy Markdown
Contributor

✅ NVIDIA GPU test: passed

PR head 19eddec merged onto main 1d91f29 · tested 2026-11-02 19:54 UTC · 23 min

GPU NVIDIA RTX 2000 Ada Generation, driver 595.99.02, Vulkan 1.4.329
Host Ubuntu 26.04.1 LTS, kernel 7.0.0-38-generic, container ubuntu:24.04
Logs build and test logs for this run
Build ✅ passed (Release, -DANYPS5_ENABLE_SPIRV_TOOLS=ON)
Tests ✅ 2,618 passed · 6 failed (0 new, 3 also fail on main, 3 unstable on main) · 44 skipped (ANYPS5_REQUIRE_VULKAN=1)
Test setup CPU-only tests without a GPU; GPU tests (and any CPU-only test that failed or skipped without one) once on the GPU with VK_LAYER_KHRONOS_validation and ANYPS5_NO_SHADER_CACHE=1; main ran the same way
Test selection 2,668 of 3,544 tests run: the ones this change can reach, chosen by tools/select_tests.py from main's build graph
Vulkan validation ✅ no new validation IDs vs main (983 tests run on the GPU)
Slow tests (>30 s, GPU tests timed with validation layers) ⚠️ agc_driver_pipeline_journal 120.1 s, ⚠️ agc_driver_graphics_maps_fallback 31.7 s
Unstable on main (3, these also failed in recent main runs on this machine; not counted against this PR)
agc_driver_pipeline_journal: failed in 3 of the last 10 main runs
agc_submit_progress: failed in 8 of the last 10 main runs
agc_submit_progress_no_timeline: failed in 7 of the last 10 main runs
Failing on main too (3, not counted against this PR)
agc_driver_image_table
guest_mmap_executable
guest_socket_options_multicast
Validated but not compared (1, not in the main baseline's validation pass)
agc_driver_faulting_wave_invocation
Validation IDs already present on main (4 IDs in 512 tests)
VUID-VkDeviceCreateInfo-pNext-pNext (512 tests): agc_driver, agc_driver_address_passes, agc_driver_alias_copy_label, agc_driver_alpha_to_coverage, agc_driver_array_layer_target, agc_driver_bc_block_copy, agc_driver_bda_anchor_reads, agc_driver_bda_span_reads, and 504 more
VUID-vkCmdCopyImage-srcImage-01548 (7 tests): agc_driver_depth_clear_hle, agc_driver_depth_plane_buffer_memory, agc_driver_depth_plane_partial_store, agc_driver_depth_plane_sampling, agc_driver_depth_storage_dispatch, agc_driver_depth_surface_from_storage, agc_driver_dispatch_move_depth
VUID-vkCmdCopyImage-srcImage-01551 (7 tests): agc_driver_depth_clear_hle, agc_driver_depth_plane_buffer_memory, agc_driver_depth_plane_partial_store, agc_driver_depth_plane_sampling, agc_driver_depth_storage_dispatch, agc_driver_depth_surface_from_storage, agc_driver_dispatch_move_depth
VUID-VkPipelineColorBlendStateCreateInfo-pAttachments-00605 (1 test): agc_driver_prepared_exports
Skipped (44)
agc_driver_bda_reads_shadowed_results: skipped, the device reports watched host memory written when it imports or reads it
agc_driver_bda_reads_shadowed_results_per_build: skipped, the device reports watched host memory written when it imports or reads it
agc_driver_sample_order_grid_reference: skipped, the device does not keep depth drawn at custom sample locations at those locations, even after a transition that names them: the composite of uniform renders holds the guest's depth in 0 of 3005 sloped samples, so no path can be compared with it bit for bit
agc_host_import_sparse: skipped, host imports are unavailable: /dev/udmabuf is not accessible
agc_import_settle_initial_dcc: skipped, the device has no deferred settle of host imports: imports under 1 MiB wait for one only on Windows
agc_unit_shadow_completion_store: skipped, the device reports watched host memory written when it imports or reads it
guest_http_uri_locale: 
guest_video_out_color_space: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_hdr: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_high_frame_rate: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_output_mode: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_resolution_status: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_wqhd_detection: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
video_out_flip_aftergpu: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_backtoback: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_controls: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_cpuflip: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_cpuflip_in_order: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_dcc: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_deferred: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_eop: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_eop_aftergpu: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_eop_main: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_fps_limit: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_fresh: skipped, no display or Vulkan device: 
Reproduce
git fetch origin 19eddecba1143143bb4c5b750f51632a4cf2eef8 1d91f2966d0eef82a021628a7aa8e78e615ac807
git checkout 1d91f2966d0eef82a021628a7aa8e78e615ac807 && git merge --no-ff 19eddecba1143143bb4c5b750f51632a4cf2eef8
git submodule update --init --recursive
cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=ON -DANYPS5_ENABLE_SPIRV_TOOLS=ON
cmake --build build && cmake --build build --target libs
VK_DRIVER_FILES=/nonexistent/icd.json ctest --test-dir build -E '^(agc|shader|recompiler|videoout|video_out|guest_video_out|vulkan)|gpu' --timeout 120 --parallel 32 --output-on-failure
ANYPS5_REQUIRE_VULKAN=1 ANYPS5_NO_SHADER_CACHE=1 VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation \
  ctest --test-dir build -R '^(agc|shader|recompiler|videoout|video_out|guest_video_out|vulkan)|gpu' --timeout 120 --parallel 4 --output-on-failure
# failures are then re-run alone: --timeout 360 --parallel 2; new validation IDs: the test alone, twice, --parallel 1

Automated build and test on a contributor's NVIDIA machine; it does not replace upstream CI. A new comment is posted for each new push. Full logs.

@XairRrR XairRrR left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed — approved

The merge with main is clean and builds. The focused tests pass with no new validation IDs, and the updated guarded-pointer host check fails without the production change. This machine's lavapipe cannot run the wave64 two-lane path or the spanned-wave path, so for those paths I rely on the NVIDIA runs that other people reported.

What I ran

Head 19eddecba1143143bb4c5b750f51632a4cf2eef8, main 9090972b7e91fd266819cbd77a8b058e177b5682, merged tree 8d69bdea97d7ba1a41e27e654c4ef880fb16bf9c (--no-ff). Machine: Debian 13 VM, 2× Xeon Gold 6136, GCC 14.2.0, Release, Ninja, lavapipe (Mesa 26.1.6, LLVM 19, subgroup 8).

git diff --check 9090972b7e91fd266819cbd77a8b058e177b5682...19eddecba1143143bb4c5b750f51632a4cf2eef8
python3 tools/check_conventions.py --base 9090972b7e91fd266819cbd77a8b058e177b5682
env ANYPS5_REQUIRE_VULKAN=1 GALLIUM_OVERRIDE_CPU_CAPS=avx ctest --test-dir build -j8 --test-load 32 --output-on-failure \
  -R "^(agc_driver_faulting_wave_invocation|agc_driver_guarded_null_pointer|agc_shader_memory|agc_driver_bda_device|agc_driver_bda_span_reads|agc_driver_bda_span_reads_draw_pipeline|agc_driver_bda_anchor_reads|agc_driver_gpu_buffer_format)$"
env ANYPS5_REQUIRE_VULKAN=1 ANYPS5_NO_SHADER_CACHE=1 build/tests/agc_driver_faulting_wave_invocation_tests
# negative control: the PR's 8 production files restored from main, then
ninja -C build -j32 -l32 agc_shader_memory_tests agc_driver_guarded_null_pointer_tests agc_driver_faulting_wave_invocation_tests
ctest --test-dir build --output-on-failure -R "^(agc_driver_faulting_wave_invocation|agc_driver_guarded_null_pointer|agc_shader_memory)$"
# merged files restored from 8d69bdea, rebuilt, same ctest again; git status --porcelain empty

Results

Check Result
git diff --check clean
check_conventions.py All conventions checks passed
Configure / build / libs (script) ok
Selected tests (script) 2033 ran: 0 new failures, 0 timeouts. 7 known main failures (agc_driver_division, denormal-mode tests, unwrap_self, vector_constant_factor_tininess). agc_driver_runtime_formatted_store passed on rerun
Skips 111, all with device-limitation reasons (check_skips.py)
Vulkan validation vs main (script) 758 tests, layer loaded in 757, missing in none, 0 tests with new VUIDs
Focused tests (8) all pass. agc_shader_memory 8.7 s, every other test under 3.4 s
New test registration picked up automatically by the tests/execution/*.cpp glob as agc_driver_faulting_wave_invocation
New test on lavapipe
Selected GPU name=llvmpipe (LLVM 19.1.7, 256 bits) ... subgroup=8
single-wave wave64 compute spans two full 32-lane subgroups: 0
[gpu] deferred write-back failed: BDA access failed: address=0x... instruction=0x10 bytes=1 stage=4 reason=1
faulting wave invocation tests passed

Negative control

I restored the 8 production files from main and kept the PR's tests: ShaderDiskCache.hpp, SpirvBda.hpp, SpirvEmitterState.hpp, SpirvBdaFault.cpp, SpirvBdaRead.cpp, SpirvEmitter.cpp, SpirvFlowEmitter.cpp and SpirvMemoryInstructions.cpp. Then I rebuilt the three affected test executables.

  • agc_shader_memory fails as it should: guarded pointer: a continuing guarded read must latch its first fault without overwriting the guard word or calling record_bda_fault.
  • agc_driver_faulting_wave_invocation still passes. That is expected on this device: with 8-wide subgroups, laneCount is 1 and no wave spans subgroups, so neither the two-lane path nor the spanned path is reached. This confirms with my own run what Dextroid17 and MarcZX10 reported for lavapipe. The red-without/green-with evidence for this test comes from jotitadev (MoltenVK, subgroup 32), from cookie42073-cmd (RTX 4070 SUPER, UHD 770) and from the author.
  • agc_driver_guarded_null_pointer passes in both arms on lavapipe.

After restoring the merged files and rebuilding, all three tests pass again and git status --porcelain is empty.

Findings

Blocking

None.

Non-blocking

  1. The description no longer matches the last commit. The description says "Existing spanned-wave, barrier and mesh exclusions remain". But 19eddecba1 removes !state.spannedWave from bdaStopsInvocations (core/shader/recompiler/SpirvBackend/src/SpirvEmitter.cpp:331). It then enables write masking for spanned waves through (waveOperations || state.spannedWave) (SpirvEmitter.cpp:335). Spanned waves are on by default on main for devices with full 32-lane compute subgroups: they are disabled only by APS5_NO_SPANNED_WAVE, in VulkanDevice.cpp:2098. So on NVIDIA this is now the main path that the new test exercises. This is in effect the spanned-wave follow-up that oneandonlydean asked for. Please update the "What" paragraph so reviewers know that spanned compute waves now continue with masked writes instead of being excluded. Pipodi's PPSA26344 report of a device loss with spanned waves plus masking was made on an earlier version, so a rerun with this head would be useful.
  2. The technical-debt entry for unlinked calls is now partly outdated. docs/dev/debt/shader.md:47 says an unlinked call "records the UnlinkedCall fault ... and stops the invocation, or continues after the call in programs whose faults do not stop invocations". With this PR, EmitUnlinkedCall (SpirvFlowEmitter.cpp:959) → StopBdaInvocationIf (SpirvBdaFault.cpp:132) no longer stops in masked programs. The invocation instead continues past the call with its later guest writes masked. A one-clause update of that entry would keep the debt file accurate.
  3. Lavapipe CI cannot detect a regression in the new test. I confirmed this with the negative control above; Dextroid17 and MarcZX10 had already reported it. #6751 adds the subgroup-32 skip. Mentioning that dependency here, or folding the guard in, would avoid a test that passes on CI without exercising anything.
  4. Cost without faults. The author reports roughly 2× NVIDIA pipeline compile time on very large two-lane programs, and an Astro Bot startup delay on an RTX 4090 is still unexplained. I did not measure either. The PR makes no performance claim, so this is not a rule issue, but it is worth tracking before titles with large DMA compute programs depend on it.

Not verified

  • Behaviour on a device with 32-wide subgroups: the two-lane wave64 path and the hardware-spanning path. This machine only has lavapipe with subgroup 8, so these come from third-party runs (TechLevelZero's RTX 2000 Ada run on this exact head, the author's RTX 3080 Laptop GPU run, and earlier MoltenVK, RTX 4070 SUPER and UHD 770 runs).
  • Whether every StopBdaInvocationIf caller that may now continue in a masked program guards its later accesses. The byte-load path is guarded in SpirvBdaRead.cpp:44-45. I read the runtime-buffer and overflow callers, and they look guarded or zero-valued, but I did not check all of them exhaustively.
  • Compile time and run time impact, and title behaviour (PPSA26344).

@TechLevelZero

Copy link
Copy Markdown
Contributor

✅ AMD GPU test: passed

PR head 19eddec merged onto main a4eca2a · tested 2026-11-02 23:22 UTC · 21 min

GPU AMD Radeon RX 6300 (RADV NAVI24), driver Mesa 25.2.8-0ubuntu0.24.04.4, Vulkan 1.4.318
Host Ubuntu 26.04.1 LTS, kernel 7.0.0-42-generic, container ubuntu:24.04
Logs build and test logs for this run
Build ✅ passed (Release, -DANYPS5_ENABLE_SPIRV_TOOLS=ON)
Tests ✅ 2,599 passed · 12 failed (0 new, 11 also fail on main, 1 unstable on main) · 53 skipped (ANYPS5_REQUIRE_VULKAN=1)
Test setup CPU-only tests without a GPU; GPU tests (and any CPU-only test that failed or skipped without one) once on the GPU with VK_LAYER_KHRONOS_validation and ANYPS5_NO_SHADER_CACHE=1; main ran the same way
Test selection 2,664 of 3,591 tests run: the ones this change can reach, chosen by tools/select_tests.py from main's build graph
Vulkan validation ✅ no new validation IDs vs main (968 tests run on the GPU)
Slow tests (>30 s, GPU tests timed with validation layers) ⚠️ agc_driver_pipeline_journal 120.1 s, ⚠️ agc_driver_graphics 89.4 s, ⚠️ public_headers 61.0 s, ⚠️ unwrap_self 47.4 s, ⚠️ agc_driver_image_sample_unnormalized 41.8 s
Unstable on main (1, these also failed in recent main runs on this machine; not counted against this PR)
agc_driver_pipeline_journal: failed in 4 of the last 6 main runs
Failing on main too (11, not counted against this PR)
agc_driver_guest_memory_lifetime_direct
agc_driver_image_table
agc_driver_job_manager_compute_capacity
agc_driver_native_depth_bda_clear
agc_driver_native_depth_buffer_clear
agc_driver_native_depth_clears
agc_driver_native_multisample_depth
agc_idle_publish_wait
agc_submit_progress
agc_submit_progress_no_timeline
guest_socket_options_multicast
Validated but not compared (1, not in the main baseline's validation pass)
agc_driver_faulting_wave_invocation
Validation IDs already present on main (4 IDs in 510 tests)
VUID-VkDeviceCreateInfo-pNext-pNext (510 tests): agc_driver, agc_driver_address_passes, agc_driver_alias_copy_label, agc_driver_alpha_to_coverage, agc_driver_array_layer_target, agc_driver_bc_block_copy, agc_driver_bda_anchor_reads, agc_driver_bda_span_reads, and 502 more
VUID-vkCmdCopyImage-srcImage-01548 (6 tests): agc_driver_depth_clear_hle, agc_driver_depth_plane_buffer_memory, agc_driver_depth_plane_partial_store, agc_driver_depth_storage_dispatch, agc_driver_depth_surface_from_storage, agc_driver_dispatch_move_depth
VUID-vkCmdCopyImage-srcImage-01551 (6 tests): agc_driver_depth_clear_hle, agc_driver_depth_plane_buffer_memory, agc_driver_depth_plane_partial_store, agc_driver_depth_storage_dispatch, agc_driver_depth_surface_from_storage, agc_driver_dispatch_move_depth
VUID-VkPipelineColorBlendStateCreateInfo-pAttachments-00605 (1 test): agc_driver_prepared_exports
Skipped (53)
agc_driver_bda_reads_shadowed_results: skipped, the device reports watched host memory written when it imports or reads it
agc_driver_bda_reads_shadowed_results_per_build: skipped, the device reports watched host memory written when it imports or reads it
agc_driver_copied_writer_label_order: skipped, the device reports a minStorageBufferOffsetAlignment of 4, so the dispatch binds its record at +0x1004 in place and nothing is written back; the test needs an alignment above 4
agc_driver_float16_unused_nan_payloads: skipped, the device has no native f16 conversions with RTE rounding and kept f16 denormals
agc_driver_image_table_vouched_lookups: skipped, the device reports 4 of 4 host-imported pages written after a GPU read, so a watched imported table cannot be vouched for
agc_driver_ngg_vertex_wave_reduction: skipped, s3 describes a 32-lane wave only on 32-lane subgroups, and the device has 64 lanes
agc_driver_unit_shadow: skipped, the device does not import watched host memory for unit shadows
agc_driver_unit_shadow_budget: skipped, the device does not import watched host memory for unit shadows
agc_host_import_sparse: skipped, host imports are unavailable: /dev/udmabuf is not accessible
agc_import_settle_initial_dcc: skipped, the device has no deferred settle of host imports: imports under 1 MiB wait for one only on Windows
agc_shadowed_key_stores: skipped, the device imports no watched host memory under a unit shadow
agc_small_tiled_shadow_label: skipped, host imports are unavailable: the test block is not imported and watched
agc_storage_clear_hand_over: skipped, the device imports no watched host memory for the clear-key hand-over
agc_unit_shadow: skipped, the device does not import watched host memory for unit shadows
agc_unit_shadow_completion_store: skipped, the device reports watched host memory written when it imports or reads it
guest_http_uri_locale: 
guest_video_out_color_space: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_hdr: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_high_frame_rate: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_output_mode: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_resolution_status: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_wqhd_detection: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
video_out_flip_aftergpu: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_backtoback: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_controls: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_cpuflip: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_cpuflip_in_order: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_dcc: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_deferred: skipped, no display or Vulkan device: DisplayWind
Reproduce
git fetch origin 19eddecba1143143bb4c5b750f51632a4cf2eef8 a4eca2a056bf3bbbec98ab42e9d28163e4ea9cb8
git checkout a4eca2a056bf3bbbec98ab42e9d28163e4ea9cb8 && git merge --no-ff 19eddecba1143143bb4c5b750f51632a4cf2eef8
git submodule update --init --recursive
cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=ON -DANYPS5_ENABLE_SPIRV_TOOLS=ON
cmake --build build && cmake --build build --target libs
VK_DRIVER_FILES=/nonexistent/icd.json ctest --test-dir build -E '^(agc|shader|recompiler|videoout|video_out|guest_video_out|vulkan)|gpu' --timeout 120 --parallel 12 --output-on-failure
ANYPS5_REQUIRE_VULKAN=1 ANYPS5_NO_SHADER_CACHE=1 VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation \
  ctest --test-dir build -R '^(agc|shader|recompiler|videoout|video_out|guest_video_out|vulkan)|gpu' --timeout 120 --parallel 16 --output-on-failure
# failures are then re-run alone: --timeout 360 --parallel 2; new validation IDs: the test alone, twice, --parallel 1

Automated build and test on a contributor's AMD machine; it does not replace upstream CI. A new comment is posted for each new push. Full logs.

@cookie42073-cmd

Copy link
Copy Markdown
Contributor

Hey, I reran the spanning-wave follow-up on Intel UHD 770 (32.0.101.7088) and RTX 4070 SUPER (621.01), Windows 11 Pro 26300, GCC 15.2.0 Release with SPIRV-Tools enabled. The new four-line change fixes the reproduced failure on both GPUs.

I used pinned main f6232fcf, merged the pre-follow-up 0784655d into it (24335b10 locally), then applied 19eddec. Its patch ID is unchanged and your authorship is retained. Final qualification is 2f101cbb2a90e98b26dfcc2ac60e8950d20e4f4f.

Source Spanning mode, both GPUs Nonspanning mode, both GPUs
Main + the same fixture Faulting lane 5 stores after its fault Healthy lane 37 stays untouched
Pre-follow-up, including the earlier first-fault latch Faulting lane 5 stores after its fault Pass
With 19eddec Pass Pass

The fixture differs from yours only by one metadata print after recompilation. Its guest program, faulting lane and assertions are unchanged. The emitted result reports hostSubgroupSize=32, fullSubgroups=1 in the default mode and 0 with APS5_NO_SPANNED_WAVE=1; this checks the actual emitted mode, not just the device capability.

Reverting only the four-line follow-up brings the spanning failure back on both GPUs, while all nine companion tests still pass. After restoring it and completing the full build, the ten-test matrix and the separate nonspanning run pass on each GPU: 22 final checks. The matrix covers the faulting wave, BDA span/draw-pipeline reads, guarded null/direct-root pointers and loads, shader memory, wave decoding and shader-cache checks.

All GPU runs required Vulkan, disabled the application shader cache and loaded Khronos validation. No VUID/SYNC errors or skips; the fault fixture intentionally reports its BDA fault. Full build and conventions pass. This does not qualify #7392's barrier extension or a title. AI-assisted testing and independent review.
AnyPS5-6416-spanning-2f101cbb-20261102.zip

@github-actions github-actions Bot added needs rebase Conflicts with main, rebase needed and removed review queue Next in the maintainer review queue: checks pass, no open change request, no conflict labels Nov 4, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agc libSceAgc* needs rebase Conflicts with main, rebase needed shader core/shader unblocks title Names a title ID on the Unblocks line

Projects

None yet

Development

Successfully merging this pull request may close these issues.