Skip to content

fix(shader): give a device-path shader cache root the long-path form on Windows - #13004

Merged
adriadam10 merged 1 commit into
boykopovar:mainfrom
JJenkx:windows/shader-cache-device-long-path
Oct 30, 2026
Merged

adriadam10 merged 1 commit into
boykopovar:mainfrom
JJenkx:windows/shader-cache-device-long-path

Conversation

@JJenkx

@JJenkx JJenkx commented Oct 29, 2026

Copy link
Copy Markdown
Contributor

What

ShaderRecompiler::NativeCachePath_nid_no_patch turns a cache path into the long-path form (\\?\… or \\?\UNC\…) before the shader cache, pipeline cache and pipeline journal touch it. It returned a path that already started with \\.\ unchanged. Unlike \\?\, a \\.\ path is still parsed by Win32 and limited to MAX_PATH. So ANYPS5_SHADER_CACHE_DIR=\\.\C:\…, which the prebuilt-discovery test's device mode exercises, could not write entries once a path crossed 260 characters. The versioned layout's entry path crosses it with a root only moderately deep: a 140-character %TEMP% is enough for agc_shader_prebuilt_discovery_device to fail with device versioned: cannot seed the versioned prebuilt entry.

A \\.\ path now goes through GetFullPathNameW like the other paths, so ., .. and / are resolved as Win32 resolves them for that form. A drive path in that form (\\.\X:\…) then becomes the equivalent \\?\X:\…. Other device names (\\.\pipe\… and so on) are returned as GetFullPathNameW gives them. A path that already starts with \\?\ is still returned unchanged.

I found this with a full suite run on a 140-character TEMP. Only this test failed there besides ones already failing on main (ngs2_pitchshift, guest_filesystem) and one that passed on rerun. Earlier long-path fixes in the same area: #7933, #8554.

Tested

Windows 11 Enterprise 10.0.22621, WinLibs GCC 15.2.0 UCRT r7, Ninja, Release, BUILD_TESTING=ON, no Vulkan driver visible. This head (35efedd747) and its base main 951f374c87, both full builds.

main 951f374c87 this PR
agc_shader_prebuilt_discovery_device, 140-character TEMP, --repeat until-fail:3 fails (cannot seed the versioned prebuilt entry) passes 3 of 3
agc_shader_disk_cache_large_file alone, normal TEMP passes, 55.8 s passes, 47.0 s
  • With this PR, ctest -R 'shader_disk_cache|shader_cache|prebuilt_discovery|pipeline_cache|pipeline_journal|pipeline_store' -j8 --repeat until-fail:3 (46 tests) under both the 140-character and the normal TEMP: everything that runs here passes, except agc_shader_disk_cache_large_file, which hit its 240 s limit in that parallel run under both TEMP values. Alone it passes in 47 s, against 55.8 s on main. The four agc_pipeline_cache* / agc_driver_pipeline_journal entries skip without Vulkan.
  • python tools/check_conventions.py --base origin/main: all checks passed.
  • Not run here: Linux and macOS. The function is Windows-only.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: none
  • Unblocks: none
  • GPUs tested: n/a (shader cache path handling only; no AGC rendering or GPU test change)
  • AI-assisted: yes (Claude Code)

🤖 Generated with Claude Code

…on Windows

NativeCachePath_nid_no_patch returned a path that already started with
\.\ unchanged. Unlike \?\, a \.\ path is still parsed by Win32 and
limited to MAX_PATH, so with ANYPS5_SHADER_CACHE_DIR=\.\C:\... and a
moderately deep root (a ~140-character TEMP is enough) the versioned
cache entries crossed 260 characters and could not be written.

A \.\ path now goes through GetFullPathNameW like the others, and a
drive path in that form (\.\X:\...) becomes the equivalent \?\X:\...;
other device names are kept as they are.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed — looks good

The change handles drive paths in the \\.\ namespace while preserving existing extended paths and other device names; I found no blocking issue in the exact head reviewed.

What I ran

  • PR head: 35efedd74799182c9e9091d7b7cd1ac649764a24.
  • Base: upstream/main at d820fef1827f27247cd28da7b4eec7891904bae5; the merge preview completed cleanly as tree b24d94e2162b33b9d3c62bd8e65a25a522c960e3.
  • Environment: Linux x86_64, GCC 12.2.0; no project build or tests run locally.
git diff --check upstream/main...refs/review/pr-13004-head
git merge-tree --write-tree upstream/main refs/review/pr-13004-head
python tools/check_conventions.py --base upstream/main

Results

Check Result
Patch whitespace Pass; command exited 0 with no output.
Merge preview Pass; clean merge tree b24d94e2162b33b9d3c62bd8e65a25a522c960e3.
Repository conventions Pass.
Source review Pass; the conversion is limited to normalized \\.\X:\... drive paths; existing \\?\ paths and non-drive device names retain their intended handling.
Hosted CI Eligibility, conventions, intake, labels, Ubuntu and Windows relinker, and ASan passed. macOS relinker jobs were queued at final refresh; product linux and windows jobs were skipped.
Windows behavior test The PR description reports full Windows 11 builds for the base and this head, with agc_shader_prebuilt_discovery_device failing on the base and passing 3/3 on this head under a 140-character TEMP. I did not independently run these tests.
Unedited local command output

python tools/check_conventions.py --base upstream/main

All conventions checks passed.

git merge-tree --write-tree upstream/main refs/review/pr-13004-head

b24d94e2162b33b9d3c62bd8e65a25a522c960e3

git diff --check upstream/main...refs/review/pr-13004-head exited 0 and produced no output.

Not verified

  • I did not build or run tests locally. The Windows test results above are author-reported, not independently reproduced.
  • macOS CI was still queued; product Linux and Windows CI jobs were skipped. The changed implementation is Windows-only.

@adriadam10 adriadam10 self-assigned this Oct 30, 2026
@JJenkx

JJenkx commented Oct 30, 2026

Copy link
Copy Markdown
Contributor Author

Linux run at this PR's exact head 35efedd747 (35efedd74799182c9e9091d7b7cd1ac649764a24), not merged onto anything else; its base on main is 951f374c87.

Arch Linux, kernel 7.2.9, Ryzen 9 7950X, g++ (GCC) 16.2.1 20260810, glibc 2.44, x86-64. Release, BUILD_TESTING=ON, Ninja -j4, private worktree.

  • ctest -R '^(shader_cache_file_read|agc_shader_prebuilt_discovery_(plain|relative|plain-long)|agc_shader_disk_cache_(pack_ownership|large_file))$': passes in each of 3 runs (18 passes).

The change is in the Windows-only device-path branch; this run shows the Linux build and the shader cache tests are unaffected at this head. On Linux CMake registers only the plain, relative and plain-long prebuilt-discovery modes; the Windows device case is not exercised here.

@TechLevelZero

Copy link
Copy Markdown
Contributor

✅ NVIDIA GPU test: passed

PR head 35efedd merged onto main f6fddd6 · tested 2026-10-30 04:32 UTC · 23 min

GPU NVIDIA RTX 2000 Ada Generation, driver 595.99.02, Vulkan 1.4.329
Host Ubuntu 26.04.1 LTS, kernel 7.0.0-38-generic, container ubuntu:24.04
Logs build and test logs for this run
Build ✅ passed (Release, -DANYPS5_ENABLE_SPIRV_TOOLS=ON)
Tests ✅ 2,263 passed · 4 failed (0 new, 2 also fail on main, 2 unstable on main) · 41 skipped (ANYPS5_REQUIRE_VULKAN=1)
Test setup CPU-only tests without a GPU; GPU tests (and any CPU-only test that failed or skipped without one) once on the GPU with VK_LAYER_KHRONOS_validation and ANYPS5_NO_SHADER_CACHE=1; main ran the same way
Test selection 2,308 of 2,950 tests run: the ones this change can reach, chosen by tools/select_tests.py from main's build graph
Vulkan validation ✅ no new validation IDs vs main (863 tests run on the GPU)
Slow tests (>30 s, GPU tests timed with validation layers) ⚠️ agc_driver_bda_device 59.7 s, ⚠️ agc_driver_lazy_resolved_abi_switch_set 30.1 s
Unstable on main (2, these also failed in recent main runs on this machine; not counted against this PR)
agc_driver_submit_progress: failed in 5 of the last 7 main runs
agc_submit_progress_no_timeline: failed in 6 of the last 7 main runs
Failing on main too (2, not counted against this PR)
guest_mmap_executable
guest_socket_options_multicast
Passed on retry (1, failed in the parallel run, passed when re-run with a 360 s timeout; not counted against this PR)
agc_driver_job_manager_compute_capacity: Failed in the parallel run; also passed only on retry in 76 other runs in the last 7 days
Validation IDs already present on main (4 IDs in 391 tests)
VUID-VkDeviceCreateInfo-pNext-pNext (391 tests): agc_compute_pipeline_replay, agc_driver, agc_driver_address_passes, agc_driver_address_passes_draw_transitions, agc_driver_address_passes_targets_split, agc_driver_alias_copy_label, agc_driver_array_layer_target_cpu, agc_driver_bda_span_reads_draw_pipeline, and 383 more
VUID-vkCmdCopyImage-srcImage-01548 (11 tests): agc_driver_depth_clear_hle, agc_driver_depth_plane_buffer_memory, agc_driver_depth_plane_cpu_write_after_memo, agc_driver_depth_plane_cpu_write_after_page_unwatch, agc_driver_depth_plane_cpu_write_after_partial_unwatch, agc_driver_depth_plane_partial_store, agc_driver_depth_plane_partial_store_unimported, agc_driver_depth_storage_dispatch, and 3 more
VUID-vkCmdCopyImage-srcImage-01551 (11 tests): agc_driver_depth_clear_hle, agc_driver_depth_plane_buffer_memory, agc_driver_depth_plane_cpu_write_after_memo, agc_driver_depth_plane_cpu_write_after_page_unwatch, agc_driver_depth_plane_cpu_write_after_partial_unwatch, agc_driver_depth_plane_partial_store, agc_driver_depth_plane_partial_store_unimported, agc_driver_depth_storage_dispatch, and 3 more
VUID-VkPipelineColorBlendStateCreateInfo-pAttachments-00605 (2 tests): agc_driver_prepared_exports, agc_driver_srgb8_color_target
Skipped (41)
agc_driver_bda_reads_shadowed_results: skipped, the device reports watched host memory written when it imports or reads it
agc_driver_bda_reads_shadowed_results_per_build: skipped, the device reports watched host memory written when it imports or reads it
agc_driver_msaa_dcc_compared_import_2: skipped, the device reports host imports that keep write protection, so the write-watch probe compares none of them
agc_driver_msaa_dcc_compared_import_4: skipped, the device reports host imports that keep write protection, so the write-watch probe compares none of them
agc_driver_msaa_dcc_compared_import_8: skipped, the device reports host imports that keep write protection, so the write-watch probe compares none of them
agc_driver_sample_order_grid_reference: skipped, the device does not keep depth drawn at custom sample locations at those locations, even after a transition that names them: the composite of uniform renders holds the guest's depth in 0 of 3005 sloped samples, so no path can be compared with it bit for bit
agc_driver_sparse_dma_buf_import: skipped, host imports are unavailable: /dev/udmabuf is not accessible
agc_driver_stencil_ref_export: skipped, the device lacks VK_EXT_shader_stencil_export
agc_driver_vertex_elected_atomic_max: skipped, the device runs vertex programs in subgroups, so they keep their cross-lane reductions
agc_host_import_sparse: skipped, host imports are unavailable: /dev/udmabuf is not accessible
agc_import_settle_initial_dcc: skipped, the device has no deferred settle of host imports: imports under 1 MiB wait for one only on Windows
agc_unit_shadow_completion_store: skipped, the device reports watched host memory written when it imports or reads it
guest_http_uri_locale: 
guest_net_ctl: skipped, the host has no IPv4 address
guest_video_out_color_space: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_hdr: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_high_frame_rate: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_output_mode: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_resolution_status: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_wqhd_detection: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
video_out_flip_controls: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_cpuflip: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_cpuflip_in_order: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_eop: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_eop_aftergpu: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_eop_main: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_fresh: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or plat
Reproduce
git fetch origin 35efedd74799182c9e9091d7b7cd1ac649764a24 f6fddd60d826831f70f7c62e25423610ee742bcc
git checkout f6fddd60d826831f70f7c62e25423610ee742bcc && git merge --no-ff 35efedd74799182c9e9091d7b7cd1ac649764a24
git submodule update --init --recursive
cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=ON -DANYPS5_ENABLE_SPIRV_TOOLS=ON
cmake --build build && cmake --build build --target libs
VK_DRIVER_FILES=/nonexistent/icd.json ctest --test-dir build -E '^(agc|shader|recompiler|videoout|video_out|guest_video_out|vulkan)|gpu' --timeout 120 --parallel 32 --output-on-failure
ANYPS5_REQUIRE_VULKAN=1 ANYPS5_NO_SHADER_CACHE=1 VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation \
  ctest --test-dir build -R '^(agc|shader|recompiler|videoout|video_out|guest_video_out|vulkan)|gpu' --timeout 120 --parallel 4 --output-on-failure
# failures are then re-run alone: --timeout 360 --parallel 2; new validation IDs: the test alone, twice, --parallel 1

Automated build and test on a contributor's NVIDIA machine; it does not replace upstream CI. A new comment is posted for each new push. Full logs.

@github-actions github-actions Bot added the shader core/shader label Oct 30, 2026
@adriadam10
adriadam10 merged commit 521b461 into boykopovar:main Oct 30, 2026
13 checks passed
@adriadam10

Copy link
Copy Markdown
Collaborator

Checked the \\.\ handling in NativeCachePath_nid_no_patch: drive paths become \\?\X:\…, other device names and \\?\ paths are returned as before, and the [5] == ':' check matches the \\.\X: layout. Conventions and relinker checks are green; merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

shader core/shader

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants