Skip to content

fix(shader): declare compute invocation builtins used by lowering - #14870

Open
edubox wants to merge 1 commit into
boykopovar:mainfrom
edubox:fix/shader-compute-builtins
Open

edubox wants to merge 1 commit into
boykopovar:mainfrom
edubox:fix/shader-compute-builtins

Conversation

@edubox

@edubox edubox commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

What

Compute shaders that read GlobalInvocationId or WorkgroupId could fail module emission with stage input 12 was not declared (WorkgroupId) or stage input 15 was not declared (GlobalInvocationId), even though the IR uses them. DefineInputs only added WorkgroupId when laneCount == 2 or the wave was spanned, and GlobalInvocationId was left to CollectBuiltinInputs; when a lowering pass introduces either builtin after collection, the interface variables are missing.

DefineInputs now adds only the compute invocation builtins that the IR actually reads, plus the existing lanes and LDS conditions. Unused compute builtin inputs are legal, and the lowering paths that combine local and workgroup ids find their inputs. A compute module that reads none of them keeps its previous SPIR-V and shader cache identity.

God of War Ragnarök (PPSA08330) fails on 0x1058daf400 and several 0x10527... compute programs with stage input 12; with this change those programs prepare, and the run advances to a different, unrelated blocker (a vertex shader without a position export).

Tested

Linux x86-64 (CachyOS), GCC 16.2.1, Release, NVIDIA GeForce RTX 4080 SUPER (driver 615.78.8.0):

  • agc_compute_builtins, agc_spirv_module, agc_guarded_memory_loads, agc_tessellation_lowering, agc_validate_program and agc_shader_disk_cache pass.
  • agc_driver_image_packed_horizontal_gather and agc_driver_graphics pass with ANYPS5_REQUIRE_VULKAN=1, ANYPS5_NO_SHADER_CACHE=1 and VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation on main and on the change; neither log has VUID- lines.
  • God of War Ragnarök with the flatten work from the related branch: the stage input 12/15 failures disappear from the log and the game reaches the next shader problem.

tools/check_conventions.py --base origin/main and git diff --check are clean. tools/pr_overlap.py reports only a shared file with #12619 and #12764 (no shared function; #12764 also uses these builtins).

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: none
  • Unblocks: none
  • GPUs tested: NVIDIA GeForce RTX 4080 SUPER, driver 615.78.8.0, Linux (CachyOS, x86-64)
  • AI-assisted: yes

@github-actions

github-actions Bot commented Nov 1, 2026 •

Copy link
Copy Markdown

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 6235 core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvEmitterHelpers.cpp
PR 7642 core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvEmitterHelpers.cpp
PR 9356 core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvEmitterHelpers.cpp
PR 9387 core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvEmitterHelpers.cpp
PR 10193 core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvEmitterHelpers.cpp
PR 10867 core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvEmitterHelpers.cpp
PR 12619 core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvEmitterHelpers.cpp
PR 12944 core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvEmitterHelpers.cpp
PR 13928 core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvEmitterHelpers.cpp
PR 13929 core/shader/recompiler/SpirvBackend/src/SpirvMemory/SpirvEmitterHelpers.cpp

@github-actions github-actions Bot added the shader core/shader label Nov 1, 2026

@BunnyFl BunnyFl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DefineInputs currently adds each extra builtin only under the condition that needs it (laneCount == 2 || spannedWave, ldsLock, LdsInDeviceMemory). The PR declares WorkgroupId, LocalInvocationIndex and GlobalInvocationId in every compute module instead.

  1. Which lowering adds the GetBuiltin after CollectBuiltinInputs (ShaderInfoCollector.cpp:650) ran? If it's one pass, re-collecting there, or adding the input under that pass's condition as the other cases do, would keep Info().inputs accurate. Unconditional declarations hide the next stale-info case.
  2. The change alters every compute module's SPIR-V, so RecompileResult::Identity, every compute pipeline journal entry and the disk cache entries change. Fine if intended; worth a line in the PR.
  3. There's no regression test for the failing shape. A recompile/emission test of a compute program that reaches the failing lowering (the PPSA08330 shape, without the title ID in the test name) would fail on main with stage input 12 was not declared.

Not run here.

@TechLevelZero

Copy link
Copy Markdown
Contributor

✅ NVIDIA GPU test: passed

PR head f764187 merged onto main 0dbf9fb · tested 2026-11-02 01:45 UTC · 25 min

GPU NVIDIA RTX 2000 Ada Generation, driver 595.99.02, Vulkan 1.4.329
Host Ubuntu 26.04.1 LTS, kernel 7.0.0-38-generic, container ubuntu:24.04
Logs build and test logs for this run
Build ✅ passed (Release, -DANYPS5_ENABLE_SPIRV_TOOLS=ON)
Tests ✅ 2,443 passed · 5 failed (0 new, 3 also fail on main, 2 unstable on main) · 45 skipped (ANYPS5_REQUIRE_VULKAN=1)
Test setup CPU-only tests without a GPU; GPU tests (and any CPU-only test that failed or skipped without one) once on the GPU with VK_LAYER_KHRONOS_validation and ANYPS5_NO_SHADER_CACHE=1; main ran the same way
Test selection 2,493 of 3,164 tests run: the ones this change can reach, chosen by tools/select_tests.py from main's build graph
Vulkan validation ✅ no new validation IDs vs main (931 tests run on the GPU)
Slow tests (>30 s, GPU tests timed with validation layers) ⚠️ agc_driver_pipeline_journal 120.1 s, ⚠️ agc_driver_graphics_maps_fallback 32.5 s, ⚠️ agc_driver_graphics 31.7 s
Unstable on main (2, these also failed in recent main runs on this machine; not counted against this PR)
agc_driver_pipeline_journal: failed in 1 of the last 10 main runs
agc_submit_progress_no_timeline: failed in 7 of the last 10 main runs
Failing on main too (3, not counted against this PR)
agc_driver_image_table
guest_mmap_executable
guest_socket_options_multicast
Validation IDs already present on main (5 IDs in 484 tests)
VUID-VkDeviceCreateInfo-pNext-pNext (484 tests): agc_driver, agc_driver_address_passes, agc_driver_alias_copy_label, agc_driver_alpha_to_coverage, agc_driver_array_layer_target, agc_driver_bc_block_copy, agc_driver_bda_anchor_reads, agc_driver_bda_span_reads, and 476 more
VUID-vkCmdCopyImage-srcImage-01548 (7 tests): agc_driver_depth_clear_hle, agc_driver_depth_plane_buffer_memory, agc_driver_depth_plane_partial_store, agc_driver_depth_plane_sampling, agc_driver_depth_storage_dispatch, agc_driver_depth_surface_from_storage, agc_driver_dispatch_move_depth
VUID-vkCmdCopyImage-srcImage-01551 (7 tests): agc_driver_depth_clear_hle, agc_driver_depth_plane_buffer_memory, agc_driver_depth_plane_partial_store, agc_driver_depth_plane_sampling, agc_driver_depth_storage_dispatch, agc_driver_depth_surface_from_storage, agc_driver_dispatch_move_depth
VUID-VkPipelineColorBlendStateCreateInfo-pAttachments-00605 (1 test): agc_driver_prepared_exports
VUID-VkShaderModuleCreateInfo-pCode-08737 (1 test): agc_driver_image_sample_derivatives
Skipped (45)
agc_driver_bda_reads_shadowed_results: skipped, the device reports watched host memory written when it imports or reads it
agc_driver_bda_reads_shadowed_results_per_build: skipped, the device reports watched host memory written when it imports or reads it
agc_driver_sample_order_grid_reference: skipped, the device does not keep depth drawn at custom sample locations at those locations, even after a transition that names them: the composite of uniform renders holds the guest's depth in 0 of 3005 sloped samples, so no path can be compared with it bit for bit
agc_host_import_sparse: skipped, host imports are unavailable: /dev/udmabuf is not accessible
agc_import_settle_initial_dcc: skipped, the device has no deferred settle of host imports: imports under 1 MiB wait for one only on Windows
agc_unit_shadow_completion_store: skipped, the device reports watched host memory written when it imports or reads it
guest_http_uri_locale: 
guest_net_ctl: skipped, the host has no IPv4 address
guest_video_out_color_space: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_hdr: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_high_frame_rate: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_output_mode: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_resolution_status: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_wqhd_detection: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
video_out_flip_aftergpu: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_backtoback: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_controls: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_cpuflip: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_cpuflip_in_order: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_dcc: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_deferred: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_eop: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_eop_aftergpu: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_eop_main: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_fps_limit: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_o
Reproduce
git fetch origin f7641873d2e49a62bc78cf5bb357914689673013 0dbf9fb7d2d3a16859a0b3bb7b8b86bc93bea9ef
git checkout 0dbf9fb7d2d3a16859a0b3bb7b8b86bc93bea9ef && git merge --no-ff f7641873d2e49a62bc78cf5bb357914689673013
git submodule update --init --recursive
cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=ON -DANYPS5_ENABLE_SPIRV_TOOLS=ON
cmake --build build && cmake --build build --target libs
VK_DRIVER_FILES=/nonexistent/icd.json ctest --test-dir build -E '^(agc|shader|recompiler|videoout|video_out|guest_video_out|vulkan)|gpu' --timeout 120 --parallel 32 --output-on-failure
ANYPS5_REQUIRE_VULKAN=1 ANYPS5_NO_SHADER_CACHE=1 VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation \
  ctest --test-dir build -R '^(agc|shader|recompiler|videoout|video_out|guest_video_out|vulkan)|gpu' --timeout 120 --parallel 4 --output-on-failure
# failures are then re-run alone: --timeout 360 --parallel 2; new validation IDs: the test alone, twice, --parallel 1

Automated build and test on a contributor's NVIDIA machine; it does not replace upstream CI. A new comment is posted for each new push. Full logs.

DefineInputs now declares the compute invocation builtins the IR reads
instead of every compute module declaring all three. Modules that do not
read them keep their previous SPIR-V and cache identity.
@edubox
edubox force-pushed the fix/shader-compute-builtins branch from f764187 to 594674b Compare November 2, 2026 02:13
@edubox

edubox commented Nov 2, 2026

Copy link
Copy Markdown
Contributor Author

Follow-up on 594674bf: DefineInputs now declares only the compute invocation builtins the IR actually reads, plus the existing lanes and LDS conditions, instead of every compute module declaring all three. A compute program that reads none of them keeps its previous SPIR-V and shader cache identity.

Added agc_compute_builtins, a compute emission test that fails on main with stage input 12 was not declared and passes with the change. Rebased onto 93dbd0f82a.

Focused tests pass; python3 tools/check_conventions.py --base origin/main and git diff --check are clean. agc_driver_image_packed_horizontal_gather and agc_driver_graphics pass with ANYPS5_REQUIRE_VULKAN=1, ANYPS5_NO_SHADER_CACHE=1 and VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation on this RTX 4080 SUPER, on main and on the change, with no new VUID- lines.

@github-actions github-actions Bot added build agc libSceAgc* labels Nov 2, 2026
@TechLevelZero

TechLevelZero commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

✅ NVIDIA GPU test: passed

PR head 594674b merged onto main 9ab6a28 · tested 2026-11-04 22:51 UTC · 10 min

GPU NVIDIA RTX 2000 Ada Generation, driver 595.99.02, Vulkan 1.4.329
Host Ubuntu 26.04.1 LTS, kernel 7.0.0-38-generic, container ubuntu:24.04
Logs build and test logs for this run
Build ✅ passed (Release, -DANYPS5_ENABLE_SPIRV_TOOLS=ON)
Tests ✅ 2,847 passed · 3 failed (0 new, 3 also fail on main) · 49 skipped (ANYPS5_REQUIRE_VULKAN=1)
Test setup CPU-only tests without a GPU; GPU tests (and any CPU-only test that failed or skipped without one) once on the GPU with VK_LAYER_KHRONOS_validation and ANYPS5_NO_SHADER_CACHE=1; main ran the same way
Test selection 2,899 of 3,909 tests run: the ones this change can reach, chosen by tools/select_tests.py from main's build graph
Vulkan validation ✅ no new validation IDs vs main (980 tests run on the GPU)
Slow tests (>30 s, GPU tests timed with validation layers) ⚠️ agc_driver_pipeline_journal 106.0 s
Failing on main too (3, not counted against this PR)
agc_driver_image_table
guest_mmap_executable
guest_socket_options_multicast
Validation IDs already present on main (13 IDs in 548 tests)
VUID-VkDeviceCreateInfo-pNext-pNext (548 tests): agc_driver, agc_driver_address_draw_space_reuse, agc_driver_address_passes, agc_driver_alias_copy_label, agc_driver_alpha_to_coverage, agc_driver_array_layer_target, agc_driver_bc_block_copy, agc_driver_bda_anchor_reads, and 540 more
VUID-VkShaderModuleCreateInfo-pCode-08737 (8 tests): agc_driver_dpp16_operands, agc_driver_dpp8_pk_fmac, agc_driver_dx10_clamp, agc_driver_float16_fma_rounding, agc_driver_float16_unused_nan_payloads, agc_driver_packed_alu, agc_driver_packed_f16_denormal_modes, agc_driver_packed_float16_native
VUID-VkShaderModuleCreateInfo-pCode-08739 (8 tests): agc_driver_dpp16_operands, agc_driver_dpp8_pk_fmac, agc_driver_dx10_clamp, agc_driver_float16_fma_rounding, agc_driver_float16_unused_nan_payloads, agc_driver_packed_alu, agc_driver_packed_f16_denormal_modes, agc_driver_packed_float16_native
VUID-VkShaderModuleCreateInfo-pCode-08741 (8 tests): agc_driver_dpp16_operands, agc_driver_dpp8_pk_fmac, agc_driver_dx10_clamp, agc_driver_float16_fma_rounding, agc_driver_float16_unused_nan_payloads, agc_driver_packed_alu, agc_driver_packed_f16_denormal_modes, agc_driver_packed_float16_native
VUID-vkCmdCopyImage-srcImage-01548 (7 tests): agc_driver_depth_clear_hle, agc_driver_depth_plane_buffer_memory, agc_driver_depth_plane_partial_store, agc_driver_depth_plane_sampling, agc_driver_depth_storage_dispatch, agc_driver_depth_surface_from_storage, agc_driver_dispatch_move_depth
VUID-vkCmdCopyImage-srcImage-01551 (7 tests): agc_driver_depth_clear_hle, agc_driver_depth_plane_buffer_memory, agc_driver_depth_plane_partial_store, agc_driver_depth_plane_sampling, agc_driver_depth_storage_dispatch, agc_driver_depth_surface_from_storage, agc_driver_dispatch_move_depth
VUID-VkPipelineColorBlendStateCreateInfo-pAttachments-00605 (2 tests): agc_driver_prepared_exports, agc_driver_srgb8_color_target
VUID-VkPipelineShaderStageCreateInfo-module-parameter (1 test): agc_driver_float16_unused_nan_payloads
VUID-VkPipelineShaderStageCreateInfo-pSpecializationInfo-06849 (1 test): agc_driver_float16_unused_nan_payloads
VUID-VkSamplerCreateInfo-addressModeU-01079 (1 test): agc_driver_image_gather_volume
VUID-VkSamplerCreateInfo-addressModeU-parameter (1 test): agc_driver_image_gather_volume
VUID-VkSamplerCreateInfo-addressModeV-parameter (1 test): agc_driver_image_gather_volume
VUID-VkSamplerCreateInfo-addressModeW-parameter (1 test): agc_driver_image_gather_volume
Skipped (49)
agc_driver_bda_reads_shadowed_results: skipped, the device reports watched host memory written when it imports or reads it
agc_driver_bda_reads_shadowed_results_per_build: skipped, the device reports watched host memory written when it imports or reads it
agc_driver_gcn_display_present: skipped, no display: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
agc_driver_sample_order_grid_reference: skipped, the device does not keep depth drawn at custom sample locations at those locations, even after a transition that names them: the composite of uniform renders holds the guest's depth in 0 of 3005 sloped samples, so no path can be compared with it bit for bit
agc_driver_scattered_runs: skipped, host imports are unavailable: /dev/udmabuf is not accessible
agc_driver_sparse_dma_buf_import: skipped, host imports are unavailable: /dev/udmabuf is not accessible
agc_driver_stencil_ref_export: skipped, the device lacks VK_EXT_shader_stencil_export
agc_host_import_sparse: skipped, host imports are unavailable: /dev/udmabuf is not accessible
agc_import_settle_initial_dcc: skipped, the device has no deferred settle of host imports: imports under 1 MiB wait for one only on Windows
agc_unit_shadow_completion_store: skipped, the device reports watched host memory written when it imports or reads it
guest_http_uri_locale: 
guest_net_ctl: skipped, the host has no IPv4 address
guest_video_out_color_space: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_hdr: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_high_frame_rate: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_output_mode: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_resolution_status: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_wqhd_detection: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
video_out_flip_aftergpu: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_backtoback: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_controls: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_cpuflip: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_cpuflip_in_order: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_dcc: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_deferred: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_eop: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_eop_aftergpu: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not av
Reproduce
git fetch origin 594674bfe3acd0a0b57c30840b3a560631c250d8 9ab6a281ea61c9e62a7eb07eb4cef8f19ede1c5c
git checkout 9ab6a281ea61c9e62a7eb07eb4cef8f19ede1c5c && git merge --no-ff 594674bfe3acd0a0b57c30840b3a560631c250d8
git submodule update --init --recursive
cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=ON -DANYPS5_ENABLE_SPIRV_TOOLS=ON
cmake --build build && cmake --build build --target libs
VK_DRIVER_FILES=/nonexistent/icd.json ctest --test-dir build -E '^(agc|shader|recompiler|videoout|video_out|guest_video_out|vulkan)|gpu' --timeout 120 --parallel 32 --output-on-failure
ANYPS5_REQUIRE_VULKAN=1 ANYPS5_NO_SHADER_CACHE=1 VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation \
  ctest --test-dir build -R '^(agc|shader|recompiler|videoout|video_out|guest_video_out|vulkan)|gpu' --timeout 120 --parallel 4 --output-on-failure
# failures are then re-run alone: --timeout 360 --parallel 2; new validation IDs: the test alone, twice, --parallel 1

Automated build and test on a contributor's NVIDIA machine; it does not replace upstream CI. A new comment is posted for each new push. Full logs.

@BunnyFl BunnyFl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed at 594674b, which answers all three points from my earlier comment:

  • Declarations now follow the GetBuiltin instructions in the final IR at emission. A builtin a lowering adds after CollectBuiltinInputs gets declared, and a compute module that reads none of them keeps its SPIR-V and cache identity.
  • The regression test is there.

Windows 11, WinLibs GCC 15.2.0 r7, Release, AMD Radeon RX 9070 XT (driver 32.0.32015.7029), Vulkan SDK 1.4.363 validation layer, ANYPS5_REQUIRE_VULKAN=1 ANYPS5_NO_SHADER_CACHE=1 VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation. Merged into upstream/main 93dbd0f:

  • agc_compute_builtins passes. Negative control: with main's SpirvEmitterHelpers.cpp it fails with SPIR-V module emission failed: stage input 12 was not declared.
  • agc_driver_image_packed_horizontal_gather and agc_driver_graphics pass on this branch and on main, none skipped. Neither log has the missing-layer message, a VUID- or a SYNC-HAZARD ID.

That adds an AMD run to your NVIDIA one.

@crastinator

Copy link
Copy Markdown

AMD RDNA 2 on RADV, which the Tested section and the NVIDIA report don't cover. No regression, and the newly declared builtins compile and dispatch on RADV. One limit: none of the GPU tests below reach the changed path, so the RADV evidence for it comes from the separate compile check.

Setup: RX 6700 XT (1002:73df, NAVI22), Mesa 26.2.5 (RADV), Linux 7.3.1 (CachyOS), GCC 16.2.1, Release, -DANYPS5_ENABLE_SPIRV_TOOLS=ON, VK_LAYER_KHRONOS_validation 1.4.363. VK_LOADER_DEBUG=layer shows the layer inserted (checked on one test run on this machine) and the log shows this GPU selected.

Commits: head 594674bfe3 merged locally onto main 98346b748c (merge cd934990c7, clean), against main 98346b748c.

tools/vulkan_validation_gate.py, exit 0, 3 iterations per side, 36 tests: agc_spirv_module, agc_guarded_memory_loads, agc_tessellation_lowering, agc_validate_program, agc_shader_disk_cache, agc_driver_image_packed_horizontal_gather, agc_driver_graphics and the matching agc_driver_compute_*, agc_driver_dispatch_* and agc_driver_lds_* tests.

  • All pass on both sides, with no VUID- messages on either side. Synchronization validation was not enabled.
  • agc_driver_dispatch_overlap_off declares a 60 s timeout, so the gate refuses it. Run separately, 2 of 2 on each side.

New test:

  • agc_compute_builtins passes 3 of 3 at the merge.
  • main with only this PR's test added fails 3 of 3 with SPIR-V module emission failed: stage input 12 was not declared.

Coverage of the change: I added a temporary print where DefineInputs adds a builtin, and ran the 36 tests and agc_compute_builtins. All 5 prints came from agc_compute_builtins, which only emits SPIR-V. None of the 36 tests had a builtin added, so on this GPU they show only that existing modules still work.

The new modules on RADV: to reach that path, I temporarily dumped the modules emit() in ComputeBuiltins.cpp produces, for WorkgroupId, GlobalInvocationId, LocalInvocationIndex and all three together, 7 modules across wave32 and wave64.

  • All 7 pass spirv-val --target-env vulkan1.1.
  • All 7 pass vkCreateComputePipelines and a vkCmdDispatch on this GPU under the validation layer, with no messages.
  • These are synthetic modules with no side effects, so this shows RADV accepts them, not that the values read are correct.

Same on both sides, unrelated to this PR:

  • agc_driver_graphics prints guest arena unavailable or not write-watched: heap mirrors not tested.
  • agc_driver_lds_mask_waterfall prints spanned mode skipped: the device has no full 32-lane compute subgroups.

Not tested: God of War Ragnarök or any other title, Windows.

AI-assisted: yes

@edubox

edubox commented Nov 2, 2026

Copy link
Copy Markdown
Contributor Author

Checked the current head against the review: DefineInputs collects the compute builtins from the IR GetBuiltin instructions (not unconditionally), the new agc_compute_builtins test covers the declared builtins and checks that unused ones stay undeclared, and the body notes the cache-identity effect. No further change needed.

tools/check_conventions.py --base origin/main passes; agc_compute_builtins passes 3 of 3 with ctest --repeat until-fail:3.

AI-assisted: yes

@edubox

edubox commented Nov 2, 2026

Copy link
Copy Markdown
Contributor Author

No code change was needed. This head is already approved at 594674bfe3, and the reported conventions: CANCELLED is one of two conventions runs on the same head; the other completed SUCCESS at 2026-11-02T02:13:49Z. Rebasing would only invalidate the GPU reports tied to this head.

Verification on 594674bfe3:

  • python3 tools/check_conventions.py --base origin/main: all checks passed.
  • ctest --test-dir build -R '^agc_compute_builtins$': passed.

The earlier review items are present in this head: DefineInputs adds only the compute builtins the IR reads, the new agc_compute_builtins test covers them, and the PR description notes the SPIR-V/cache identity change.

@edubox

edubox commented Nov 2, 2026

Copy link
Copy Markdown
Contributor Author

Current head 594674bfe3 already answers the review:

  • DefineInputs no longer declares all three builtins for every compute module. It scans program.Blocks() for GetBuiltin and adds only the builtins the IR actually reads, plus the existing lane/spanned-wave, ldsLock and LdsInDeviceMemory cases.
  • agc_compute_builtins is registered and covers WorkgroupId, wave64 GlobalInvocationId with its WorkgroupId/LocalInvocationIndex dependencies, LocalInvocationIndex, and an unused-builtin negative case. It passes here, and the RADV/NVIDIA reports at this head confirm the same.
  • The commit message records the SPIR-V identity point: modules that do not read these builtins keep their previous SPIR-V and cache identity.

The cancelled conventions check has since passed; gh pr checks 14870 now shows every required check passing or deliberately skipping. python3 tools/check_conventions.py --base origin/main passes here as well.

@edubox

edubox commented Nov 4, 2026

Copy link
Copy Markdown
Contributor Author

Re-checked the current approved head 594674bfe3. The review questions were answered in this head: the builtins are added only when the IR reads them (DefineInputs), the changed SPIR-V/cache identity is documented, and agc_compute_builtins is the regression that fails on main with stage input 12 was not declared.

Checks on 594674bfe3, with ANYPS5_REQUIRE_VULKAN=1, ANYPS5_NO_SHADER_CACHE=1:

  • agc_compute_builtins, agc_shader_disk_cache, agc_spirv_module, agc_validate_program: passed.
  • python3 tools/check_conventions.py --base origin/main: all checks passed.

No new change was pushed.

@oneandonlydean oneandonlydean self-assigned this Nov 5, 2026
@TechLevelZero

Copy link
Copy Markdown
Contributor

✅ AMD GPU test: passed

PR head 594674b merged onto main 4639eff · tested 2026-11-05 10:09 UTC · 15 min

GPU AMD Radeon RX 6300 (RADV NAVI24), driver Mesa 25.2.8-0ubuntu0.24.04.4, Vulkan 1.4.318
Host Ubuntu 26.04.1 LTS, kernel 7.0.0-42-generic, container ubuntu:24.04
Logs build and test logs for this run
Build ✅ passed (Release, -DANYPS5_ENABLE_SPIRV_TOOLS=ON)
Tests ✅ 2,866 passed · 10 failed (0 new, 8 also fail on main, 2 unstable on main) · 67 skipped (ANYPS5_REQUIRE_VULKAN=1)
Test setup CPU-only tests without a GPU; GPU tests (and any CPU-only test that failed or skipped without one) once on the GPU with VK_LAYER_KHRONOS_validation and ANYPS5_NO_SHADER_CACHE=1; main ran the same way
Test selection 2,943 of 3,962 tests run: the ones this change can reach, chosen by tools/select_tests.py from main's build graph
Vulkan validation ✅ no new validation IDs vs main (997 tests run on the GPU)
Slow tests (>30 s, GPU tests timed with validation layers) ⚠️ agc_driver_pipeline_journal 120.2 s, ⚠️ public_headers 86.5 s, ⚠️ check_skips 40.6 s, ⚠️ conventions_pr_template 34.3 s, ⚠️ agc_driver_stream_copy 31.7 s
Unstable on main (2, these also failed in recent main runs on this machine; not counted against this PR)
agc_driver_pipeline_journal: failed in 9 of the last 10 main runs
trace_digest_TraceDigestTests_test_cli_rejects_invalid_event_fields_without_partial_output: failed in 2 of the last 10 main runs
Failing on main too (8, not counted against this PR)
agc_driver_image_table
agc_driver_native_depth_bda_clear
agc_driver_native_multisample_depth
agc_driver_submit_progress
agc_submit_progress
agc_submit_progress_no_timeline
guest_mmap_executable
guest_socket_options_multicast
Validation IDs already present on main (4 IDs in 558 tests)
VUID-VkDeviceCreateInfo-pNext-pNext (558 tests): agc_driver, agc_driver_address_draw_space_reuse, agc_driver_address_passes, agc_driver_alias_copy_label, agc_driver_alpha_to_coverage, agc_driver_array_layer_target, agc_driver_bc_block_copy, agc_driver_bda_anchor_reads, and 550 more
VUID-vkCmdCopyImage-srcImage-01548 (4 tests): agc_driver_depth_clear_hle, agc_driver_depth_storage_dispatch, agc_driver_depth_surface_from_storage, agc_driver_dispatch_move_depth
VUID-vkCmdCopyImage-srcImage-01551 (4 tests): agc_driver_depth_clear_hle, agc_driver_depth_storage_dispatch, agc_driver_depth_surface_from_storage, agc_driver_dispatch_move_depth
VUID-VkPipelineColorBlendStateCreateInfo-pAttachments-00605 (1 test): agc_driver_srgb8_color_target
Skipped (67)
agc_driver_bda_reads_shadowed_results: skipped, the device reports watched host memory written when it imports or reads it
agc_driver_bda_reads_shadowed_results_per_build: skipped, the device reports watched host memory written when it imports or reads it
agc_driver_copied_writer_label_order: skipped, the device reports a minStorageBufferOffsetAlignment of 4, so the dispatch binds its record at +0x1004 in place and nothing is written back; the test needs an alignment above 4
agc_driver_float16_unused_nan_payloads: skipped, the device has no native f16 conversions with RTE rounding and kept f16 denormals
agc_driver_gcn_display_present: skipped, no display: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
agc_driver_image_table_vouched_lookups: skipped, the device reports 4 of 4 host-imported pages written after a GPU read, so a watched imported table cannot be vouched for
agc_driver_linear_target_direct_write_completion: skipped, the device does not keep host imports write-watched, so no completion stamp over the target is tracked
agc_driver_ngg_vertex_wave_reduction: skipped, s3 describes a 32-lane wave only on 32-lane subgroups, and the device has 64 lanes
agc_driver_scattered_runs: skipped, host imports are unavailable: /dev/udmabuf is not accessible
agc_driver_sparse_dma_buf_import: skipped, host imports are unavailable: /dev/udmabuf is not accessible
agc_driver_unit_shadow: skipped, the device does not import watched host memory for unit shadows
agc_driver_unit_shadow_budget: skipped, the device does not import watched host memory for unit shadows
agc_host_import_sparse: skipped, host imports are unavailable: /dev/udmabuf is not accessible
agc_idle_publish_wait: skipped, the device reports 4 of 4 host-imported pages written after a GPU read, so a forced watch of the import reads its submissions as CPU stores
agc_import_settle_initial_dcc: skipped, the device has no deferred settle of host imports: imports under 1 MiB wait for one only on Windows
agc_shadowed_key_stores: skipped, the device imports no watched host memory under a unit shadow
agc_small_tiled_shadow_label: skipped, host imports are unavailable: the test block is not imported and watched
agc_storage_clear_hand_over: skipped, the device imports no watched host memory for the clear-key hand-over
agc_storage_pending_key_refresh: skipped, refreshes over a pending key store are not testable here
agc_unit_shadow: skipped, the device does not import watched host memory for unit shadows
agc_unit_shadow_completion_store: skipped, the device reports watched host memory written when it imports or reads it
guest_http_uri_locale: 
guest_net_ctl: skipped, the host has no IPv4 address
guest_net_ctl_disconnected: skipped, the host has no IPv4 address
guest_video_out_color_space: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_hdr: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_high_frame_rate: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_host_resolution_disabled: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_host_resolution_four_k: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_host_resolution_height_only: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_host_resolution_limit: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_host_resolution_malformed: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayla
Reproduce
git fetch origin 594674bfe3acd0a0b57c30840b3a560631c250d8 4639eff7f4c23325a916875a0e37f1e312fa18f4
git checkout 4639eff7f4c23325a916875a0e37f1e312fa18f4 && git merge --no-ff 594674bfe3acd0a0b57c30840b3a560631c250d8
git submodule update --init --recursive
cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=ON -DANYPS5_ENABLE_SPIRV_TOOLS=ON
cmake --build build && cmake --build build --target libs
VK_DRIVER_FILES=/nonexistent/icd.json ctest --test-dir build -E '^(agc|shader|recompiler|videoout|video_out|guest_video_out|vulkan)|gpu' --timeout 120 --parallel 12 --output-on-failure
ANYPS5_REQUIRE_VULKAN=1 ANYPS5_NO_SHADER_CACHE=1 VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation \
  ctest --test-dir build -R '^(agc|shader|recompiler|videoout|video_out|guest_video_out|vulkan)|gpu' --timeout 120 --parallel 16 --output-on-failure
# failures are then re-run alone: --timeout 360 --parallel 2; new validation IDs: the test alone, twice, --parallel 1

Automated build and test on a contributor's AMD machine; it does not replace upstream CI. A new comment is posted for each new push. Full logs.

@edubox

edubox commented Nov 5, 2026

Copy link
Copy Markdown
Contributor Author

Re-checked the approved head 594674bfe3. The conventions: CANCELLED run is the earlier duplicate; the other conventions run on the same head passed, and the later hosted windows job passed.

On this head:

  • agc_compute_builtins passes with ANYPS5_REQUIRE_VULKAN=1 ANYPS5_NO_SHADER_CACHE=1.
  • python3 tools/check_conventions.py --base origin/main is clean.

No source change was needed and nothing was pushed.

AI-assisted: yes.

@edubox

edubox commented Nov 5, 2026

Copy link
Copy Markdown
Contributor Author

Verified the current head 594674bfe3: the current conventions run passes (the CANCELLED entry is a superseded duplicate), agc_compute_builtins, agc_spirv_module, agc_validate_program and agc_shader_disk_cache pass locally under ANYPS5_REQUIRE_VULKAN=1, ANYPS5_NO_SHADER_CACHE=1 and VK_LAYER_KHRONOS_validation (no missing-layer warning, no VUID- lines), and python3 tools/check_conventions.py --base origin/main passes. No code change was needed.

@edubox

edubox commented Nov 5, 2026

Copy link
Copy Markdown
Contributor Author

No changes needed: the current head 594674bfe3 is the one BunnyFl approved, and the conventions check that the summary showed as cancelled now passes along with relinker, relinker-asan, library-portability and windows. I did not rebase an approved PR. AI-assisted: yes.

@edubox

edubox commented Nov 6, 2026

Copy link
Copy Markdown
Contributor Author

Re-verified at head 594674bfe3: no source change is needed. The rollup still lists the earlier conventions: CANCELLED run, but the conventions run on the same head is SUCCESS, and BunnyFl approved this head after the earlier review points were addressed.

Locally: python3 tools/check_conventions.py --base origin/main passes and agc_compute_builtins passes. The third-party NVIDIA and AMD runs on this head are green. Nothing was pushed. AI-assisted: yes.

@edubox

edubox commented Nov 6, 2026

Copy link
Copy Markdown
Contributor Author

Re-verified approved head 594674bfe3. The conventions: CANCELLED entry in the rollup is a superseded run: conventions on this head is SUCCESS, and the relinker, relinker-asan, library-portability, windows and report checks pass.

The review points are already in this head: DefineInputs adds only the compute builtins the IR reads, agc_compute_builtins is the regression for the failing shape, and the commit records the SPIR-V/cache-identity effect. Locally on 594674bfe3:

  • python3 tools/check_conventions.py --base origin/main: all checks passed.
  • agc_compute_builtins with ANYPS5_REQUIRE_VULKAN=1, ANYPS5_NO_SHADER_CACHE=1 and VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation: passed.
  • git diff --check origin/main...HEAD: clean.

No source change was needed and the approved head was not rebased, so nothing was pushed. AI-assisted: yes.

@edubox

edubox commented Nov 6, 2026

Copy link
Copy Markdown
Contributor Author

The conventions run reported as CANCELLED is superseded: the live conventions check on this head (594674bfe3) is green, as are the rest of the checks, and the review was approved at this head after the three points were addressed. No code change was needed and the head is unchanged.

Re-ran locally on this head:

  • ctest --test-dir build -R '^agc_compute_builtins$': 1/1 passed.
  • python3 tools/check_conventions.py --base origin/main: all checks passed.

I did not re-run the GPU validation gate here; the head is unchanged.

AI-assisted: yes.

@edubox

edubox commented Nov 6, 2026

Copy link
Copy Markdown
Contributor Author

The watchdog flagged the earlier conventions run as CANCELLED. That run (2026-11-02T02:13:25Z) was superseded by the successful conventions run at 2026-11-02T02:13:35Z on head 594674bfe3, so no code change is needed. The review points from f7641873d2 are already addressed at this head (conditional builtin scan plus agc_compute_builtins).

Re-checked locally on 594674bfe3: agc_compute_builtins passed and python3 tools/check_conventions.py --base origin/main passed. Left untouched otherwise because the PR is approved.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agc libSceAgc* build shader core/shader

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants