Skip to content

fix(shader): discover prebuilt caches through native Windows paths - #8580

Merged
oneandonlydean merged 2 commits into
boykopovar:mainfrom
veosett:codex/windows-prebuilt-cache-discovery
Oct 20, 2026
Merged

oneandonlydean merged 2 commits into
boykopovar:mainfrom
veosett:codex/windows-prebuilt-cache-discovery

Conversation

@veosett

@veosett veosett commented Oct 20, 2026

Copy link
Copy Markdown
Contributor

What

On the supported WinLibs toolchain, prebuilt shader caches can be missed under long Windows paths. An extended-length directory can also be enumerated as the process working directory: with a.pack and z.pack in the cache but only z.pack in the working directory, discovery misses the pack containing the requested key.

Normalize candidates through the existing native cache-path helper before file/directory checks, and enumerate Windows directories with FindFirstFileW / FindNextFileW, closing the search handle through RAII. Preserve pack ordering, loose entries, direct pack-file inputs, and the versioned-directory fallback even when enumeration fails. Linux keeps its existing directory iterator.

Add isolated subprocess regressions for packed, loose, versioned, direct-file, empty and missing prebuilt inputs. Each run checks that both the cache input and a canary directory outside it remain byte-for-byte unchanged. Stop the cache writer before constructing the store so these tests exercise discovery and loading independently of housekeeping.

Partial follow-up for #7452. The housekeeping guard in #7945 and PipelineCache header reader in #8554 are separate changes; this PR does not depend on either.

Tested

Release builds with bundled dependencies: Windows 11 Pro 26100 / WinLibs GCC 15.2, and Ubuntu 24.04 WSL / GCC 13.3. Built agc_shader_disk_cache_tests, shader_cache_file_read_tests and agc_driver_pipeline_journal_tests on both systems.

Baseline production code: main 2fee39f1b, with only the test harness applied at b97b122f6. Fixed revision: 2190ccd20. Every registered test below ran three times, serially, with verbose CTest output and no skips:

Platform Baseline executions Fixed executions
Windows 27 pass; 9 expected regression failures 36 pass; 0 failures
Ubuntu WSL 21 pass; 0 failures 21 pass; 0 failures

Windows discovery modes: plain, relative, \\?\, \\?\ with forward slashes, \\.\, ordinary long paths, and extended long paths; every root contains non-ASCII characters. The last two long-path modes exceed 270 characters. Baseline failures are the ordinary long, extended long and extended-forward modes, each failing three times with prebuilt discovery returned the wrong result for pack; all pass after the fix. Linux covers plain, relative and long Unicode paths. Each mode tests all six layouts in separate seed/load processes with a five-second child timeout.

Existing tests on both revisions: agc_shader_disk_cache, shader_cache_file_read, shader_cache_native_paths, and Windows agc_shader_disk_cache_long_paths. Vulkan coverage on both revisions and both devices:

ctest --test-dir <build> -R '^agc_driver_pipeline_journal$' -V --repeat until-fail:3

GPU runs used ANYPS5_REQUIRE_VULKAN=1, ANYPS5_NO_SHADER_CACHE=1, VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation, and VK_LOADER_DEBUG=layer. The loader confirms the Khronos validation layer is inserted. Comparing baseline/fixed logs: zero VUID- entries and zero validation warnings/errors on either system; neither log reports a requested validation layer missing. The CPU discovery fixtures explicitly enable their isolated cache while keeping its writer stopped.

check_conventions.py --base origin/main passed. Ran pr_overlap.py --branch codex/windows-prebuilt-cache-discovery; its unresolved comparisons concern conflicted legacy PRs sharing these files. Comparing each listed head against its merge-base found no changes to discoverPrebuilt or additions/removals mentioning the relevant prebuilt/native-path discovery code or these test modes. A merge-tree check with #7945 is clean, as is the merge with main ae12a951f.

No retail game, full CTest suite, SMB share, or long device-namespace alias was tested. No performance claim.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: none
  • Unblocks: none
  • GPUs tested: AMD Radeon RX 6900 XT, Windows driver 32.0.21030.2001, Windows 11 Pro 26100; Mesa lavapipe 25.2.8 (llvmpipe LLVM 20.1.2), Ubuntu 24.04 WSL
  • AI-assisted: yes

@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 1368 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 1395 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 2434 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 5021 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 5371 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 5431 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 5499 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 5609 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.cpp
PR 5925 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/shader/recompiler/ShaderDiskCache.cpp
PR 6044 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6146 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6186 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6187 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6188 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6192 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6209 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6235 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
PR 6280 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.cpp
PR 6292 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6300 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6412 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.cpp
PR 6481 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6488 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.cpp
PR 6501 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6539 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.cpp
PR 6615 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6682 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6683 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6747 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6788 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6808 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6868 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6879 core/shader/recompiler/ShaderDiskCache.cpp
PR 6905 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6924 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.cpp
PR 6946 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/shader/recompiler/ShaderDiskCache.cpp
PR 6947 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.cpp
PR 6970 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.cpp
PR 6977 core/shader/recompiler/ShaderDiskCache.cpp
PR 7002 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 7048 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 7085 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.cpp
PR 7201 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 7395 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.cpp
PR 7446 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 7464 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 7484 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
PR 7505 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 7514 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 7550 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.cpp
PR 7604 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 7642 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.cpp
PR 7644 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 7677 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 7684 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.cpp
PR 7689 core/shader/recompiler/ShaderDiskCache.cpp
PR 7800 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 7888 core/shader/recompiler/ShaderDiskCache.cpp
PR 8077 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.cpp
PR 8206 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 8295 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
core/shader/recompiler/ShaderDiskCache.cpp
PR 8697 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp

@github-actions github-actions Bot added build shader core/shader agc libSceAgc* labels Oct 20, 2026
@oneandonlydean
oneandonlydean merged commit 01fbf10 into boykopovar:main Oct 20, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agc libSceAgc* build shader core/shader

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants