fix(agc): support long Windows pipeline journal paths by veosett · Pull Request #7933 · boykopovar/AnyPS5

veosett · 2026-10-19T14:47:23Z

What

An ordinary long Windows cache path can be written by #7462 but still fail at mapped entry/pack reads and journal locking/appends.

Resolve the journal directory once through NativeCachePath_nid_no_patch, then derive both journal and lock paths from it. Directory creation, existence/size checks, indexing, appends and complete-record/torn-tail recovery share that native path. Add the host-file-I/O helper to the shader-cache reviewed names.

As requested in review, also normalize the two mapped opens in MapFile and CachePack. Take Amoenus's regression and correction, preserving original authorship. The regression verifies exact published bytes, actual mapped SPIR-V, persistence/load round trips, and a long-path cache-pack load.

Rebased on main 1edbe4b5e03b4f655021da3a74b87d2dc511476f and retained both main's barycentric reviewed names and NativeCachePath_nid_no_patch. The branch includes #7462's exact shared-helper change and the previously offered cross-drive fixture correction, also preserving authorship. The mapped-test command dispatch was moved below the codec modes so it merges independently of #7945. Prerequisite commits can be dropped after #7462 absorbs them and merges.

The journal fixture covers short/ordinary-long/already-extended paths, Unicode, indexing, replay, deduplication, truncation and an externally appended complete record followed by torn bytes. It stops its writer and removes known journal files before cleaning the short temporary root. The external-record case exposes concurrent compute callbacks, so their shared result capture is protected by a mutex while retaining two replay workers. Remaining PipelineCache::readHeader/failed-read existence-check limits are recorded in the library debt file; the entry is placed to avoid a documentation-only insertion conflict with #7307. Prefixed directory-traversal safety is handled separately by #7945. This PR remains a partial follow-up for #7452.

Tested

Updated validation on 2026-10-20: final tested source 80024c48fd0d8b19bd7be8d66d16056c61cee5fc, baseline main 1edbe4b5e03b4f655021da3a74b87d2dc511476f, and mapped-regression test-only control 56778c97d25262e7f7973baeba3f44fa0e65d5db. Published PR head 80024c48fd0d8b19bd7be8d66d16056c61cee5fc has identical code and test sources to the tested head.

Release builds of the affected AGC library, shader recompiler and four selected test executables pass on Windows 11 Pro 26100 / supported WinLibs GCC 15.2.0 r7 and Ubuntu 24.04 WSL / GCC 13.3. Bundled dependencies, SPIRV-Tools ON.
Identical mapped-cache regression before the two-open correction fails 3/3 on Windows with a long-path cache entry was not persisted and mapped, after confirming that the written bytes match EncodeEntry. With the correction, mapped entry and pack round trips pass 3/3.
The same four existing groups pass three fresh CTest processes each on unmodified main and final head on both OSes: agc_shader_disk_cache, shader_cache_file_read, agc_pipeline_journal, and real Vulkan agc_driver_pipeline_journal (12 passes per OS per revision).
Final head also passes long journal and native-file regressions three times each on both OSes, plus the Windows extended-journal and mapped-cache variants. Windows repeats shader_cache_file_read 3/3 with C: TEMP/TMP and the build/working directory on D:. Final totals: Windows 27 passes, Linux 18 passes; no skips/timeouts, each test below 30 s.
An initial Windows run exposed heap corruption in the journal fixture's unsynchronized shared compute-hook capture when duplicate external records replayed concurrently. After protecting that capture, all three Windows mock-journal path variants pass 20 fresh processes each (60 additional passes, two replay workers retained), followed by the complete selected final matrix above. Production replay scheduling is unchanged.
Same real GPU test before/after: AMD Radeon RX 6900 XT, Windows driver 32.0.21030.2001; Linux lavapipe/Mesa 25.2.8, llvmpipe LLVM 20.1.2. All runs use verbose CTest, VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation, ANYPS5_NO_SHADER_CACHE=1, ANYPS5_REQUIRE_VULKAN=1. Loader logs confirm the layer is inserted on both devices and both revisions; no missing-layer warning, VUIDs or validation errors/warnings. Portable validation binaries: Khronos main build, source aea761f4d54b86682be4099c428dba27b6609940, manifest API 1.4.365.
The original journal-only negative control on 2026-10-19 reproduced the long-path lock failure 3/3; the fresh matrix above verifies the journal regression after rebasing.
Conventions, whitespace and test-registration checks pass on both OSes. A fresh merge check against main 3d54c5b2b2a90485e48d5890021c9380f65ab325 is clean, as are direct merges with fix(shader): avoid unsafe Windows cache directory traversal #7945 and feat(agc): add opt-in bounded submission progress #7307. fix(shader): support long Windows cache file paths #7462 is the declared prerequisite; offered mapped patches were folded in at the maintainer's request.
The required overlap checker completed. Its feat(agc): add opt-in bounded submission progress #7307 debt-note conflict was removed by moving our entry; fix(shader): support long Windows cache file paths #7462's fixture overlap is the declared dependency. Targeted inspection of 94 reported shared-file PR diffs found no duplicate mapped-open/journal path correction. Already-conflicting PRs are not claimed merge-clean.

Selected tests only. No full CTest suite, title execution, SMB end-to-end test or performance measurement. New Windows roots stay below MAX_PATH; known filenames cross it. Directory-enumeration safety is outside this PR.

Checklist

github-actions · 2026-10-19T14:59:53Z

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request	Conflicting files (lines on `main`)	Same exports	Same new files
PR 6959	`docs/dev/debt/libSceAgcDriver.md` L215
PR 7927	`docs/dev/debt/libSceAgcDriver.md` L215
PR 8014	`docs/dev/debt/libSceAgcDriver.md` L215

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request	Shared files not compared
PR 1368	`core/libs/prx/libSceAgcDriver/CMakeLists.txt`
PR 1395	`core/libs/prx/libSceAgcDriver/CMakeLists.txt` `core/libs/prx/libSceAgcDriver/tests/Graphics.cpp`
PR 2434	`core/libs/prx/libSceAgcDriver/CMakeLists.txt` `docs/dev/debt/libSceAgcDriver.md`
PR 4040	`core/libs/prx/libSceAgcDriver/tests/Graphics.cpp`
PR 4448	`docs/dev/debt/libSceAgcDriver.md`
PR 4449	`docs/dev/debt/libSceAgcDriver.md`
PR 5021	`core/libs/prx/libSceAgcDriver/CMakeLists.txt`
PR 5058	`docs/dev/debt/libSceAgcDriver.md`
PR 5371	`core/libs/prx/libSceAgcDriver/CMakeLists.txt`
PR 5426	`core/libs/prx/libSceAgcDriver/CMakeLists.txt`
PR 5431	`core/libs/prx/libSceAgcDriver/CMakeLists.txt`
PR 5441	`core/libs/prx/libSceAgcDriver/tests/Graphics.cpp`
PR 5499	`core/libs/prx/libSceAgcDriver/CMakeLists.txt` `core/libs/prx/libSceAgcDriver/tests/Graphics.cpp` `docs/dev/debt/libSceAgcDriver.md`
PR 5500	`core/libs/prx/libSceAgcDriver/tests/Graphics.cpp` `core/shader/recompiler/ShaderDiskCache.cpp`
PR 5609	`core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp` `core/shader/recompiler/ShaderDiskCache.cpp`
PR 5925	`core/libs/prx/libSceAgcDriver/CMakeLists.txt` `core/libs/prx/libSceAgcDriver/Graphics/src/PipelineJournal.cpp` `core/libs/prx/libSceAgcDriver/tests/Graphics.cpp` `core/shader/recompiler/CMakeLists.txt` `core/shader/recompiler/ShaderDiskCache.cpp` `docs/dev/debt/libSceAgcDriver.md`
PR 6044	`core/libs/prx/libSceAgcDriver/CMakeLists.txt`
PR 6146	`core/libs/prx/libSceAgcDriver/CMakeLists.txt` `core/libs/prx/libSceAgcDriver/tests/Graphics.cpp` `docs/dev/debt/libSceAgcDriver.md`
PR 6177	`core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp`
PR 6185	`core/libs/prx/libSceAgcDriver/CMakeLists.txt`
PR 6186	`core/libs/prx/libSceAgcDriver/CMakeLists.txt`
PR 6187	`core/libs/prx/libSceAgcDriver/CMakeLists.txt`
PR 6188	`core/libs/prx/libSceAgcDriver/CMakeLists.txt` `core/libs/prx/libSceAgcDriver/tests/Graphics.cpp`
PR 6192	`core/libs/prx/libSceAgcDriver/CMakeLists.txt`
PR 6194	`core/libs/prx/libSceAgcDriver/tests/Graphics.cpp` `docs/dev/debt/libSceAgcDriver.md`
PR 6209	`core/libs/prx/libSceAgcDriver/CMakeLists.txt`
PR 6223	`docs/dev/debt/libSceAgcDriver.md`
PR 6235	`core/libs/prx/libSceAgcDriver/tests/Graphics.cpp` `core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp`
PR 6273	`docs/dev/debt/libSceAgcDriver.md`
PR 6280	`core/libs/prx/libSceAgcDriver/CMakeLists.txt` `core/libs/prx/libSceAgcDriver/tests/Graphics.cpp` `core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp` `core/shader/recompiler/ShaderDiskCache.cpp` `docs/dev/debt/libSceAgcDriver.md`
PR 6300	`core/libs/prx/libSceAgcDriver/CMakeLists.txt` `docs/dev/debt/libSceAgcDriver.md`
PR 6412	`core/libs/prx/libSceAgcDriver/CMakeLists.txt` `core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp` `core/shader/recompiler/ShaderDiskCache.cpp` `docs/dev/debt/libSceAgcDriver.md`
PR 6481	`core/libs/prx/libSceAgcDriver/CMakeLists.txt` `core/libs/prx/libSceAgcDriver/tests/Graphics.cpp` `docs/dev/debt/libSceAgcDriver.md`
PR 6488	`core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp` `core/shader/recompiler/ShaderDiskCache.cpp`
PR 6498	`docs/dev/debt/libSceAgcDriver.md`
PR 6501	`core/libs/prx/libSceAgcDriver/CMakeLists.txt`
PR 6539	`core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp` `core/shader/recompiler/ShaderDiskCache.cpp`
PR 6548	`core/libs/prx/libSceAgcDriver/tests/Graphics.cpp` `docs/dev/debt/libSceAgcDriver.md`
PR 6549	`docs/dev/debt/libSceAgcDriver.md`
PR 6611	`core/libs/prx/libSceAgcDriver/CMakeLists.txt`
PR 6636	`core/libs/prx/libSceAgcDriver/tests/Graphics.cpp` `docs/dev/debt/libSceAgcDriver.md`
PR 6682	`core/libs/prx/libSceAgcDriver/CMakeLists.txt`
PR 6683	`core/libs/prx/libSceAgcDriver/CMakeLists.txt` `core/libs/prx/libSceAgcDriver/tests/Graphics.cpp` `docs/dev/debt/libSceAgcDriver.md`
PR 6727	`docs/dev/debt/libSceAgcDriver.md`
PR 6747	`core/libs/prx/libSceAgcDriver/CMakeLists.txt`
PR 6788	`core/libs/prx/libSceAgcDriver/CMakeLists.txt`
PR 6808	`core/libs/prx/libSceAgcDriver/CMakeLists.txt` `core/libs/prx/libSceAgcDriver/tests/Graphics.cpp` `docs/dev/debt/libSceAgcDriver.md`
PR 6820	`core/libs/prx/libSceAgcDriver/CMakeLists.txt`
PR 6867	`docs/dev/debt/libSceAgcDriver.md`
PR 6868	`core/libs/prx/libSceAgcDriver/CMakeLists.txt` `docs/dev/debt/libSceAgcDriver.md`
PR 6879	`core/shader/recompiler/CMakeLists.txt` `core/shader/recompiler/ShaderDiskCache.cpp`
PR 6926	`docs/dev/debt/libSceAgcDriver.md`
PR 6937	`core/libs/prx/libSceAgcDriver/CMakeLists.txt`
PR 6946	`core/libs/prx/libSceAgcDriver/CMakeLists.txt` `core/libs/prx/libSceAgcDriver/Graphics/src/PipelineJournal.cpp` `core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp` `core/shader/recompiler/CMakeLists.txt` `core/shader/recompiler/ShaderDiskCache.cpp` `docs/dev/debt/libSceAgcDriver.md`
PR 6947	`core/libs/prx/libSceAgcDriver/CMakeLists.txt` `core/libs/prx/libSceAgcDriver/Graphics/src/PipelineJournal.cpp` `core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp` `core/shader/recompiler/CMakeLists.txt` `core/shader/recompiler/ShaderDiskCache.cpp` `docs/dev/debt/libSceAgcDriver.md`
PR 6970	`core/libs/prx/libSceAgcDriver/tests/Graphics.cpp` `core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp` `core/shader/recompiler/ShaderDiskCache.cpp`
PR 6972	`core/libs/prx/libSceAgcDriver/tests/Graphics.cpp`
PR 6977	`core/shader/recompiler/CMakeLists.txt` `core/shader/recompiler/ShaderDiskCache.cpp` `docs/dev/debt/libSceAgcDriver.md`
PR 7002	`core/libs/prx/libSceAgcDriver/CMakeLists.txt` `docs/dev/debt/libSceAgcDriver.md`
PR 7048	`core/libs/prx/libSceAgcDriver/CMakeLists.txt`
PR 7085	`core/libs/prx/libSceAgcDriver/CMakeLists.txt` `core/libs/prx/libSceAgcDriver/tests/Graphics.cpp` `core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp` `core/shader/recompiler/ShaderDiskCache.cpp` `docs/dev/debt/libSceAgcDriver.md`
PR 7201	`core/libs/prx/libSceAgcDriver/CMakeLists.txt` `core/shader/recompiler/CMakeLists.txt`
PR 7223	`core/libs/prx/libSceAgcDriver/CMakeLists.txt` `core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp` `core/shader/recompiler/ShaderDiskCache.cpp`
PR 7256	`core/libs/prx/libSceAgcDriver/tests/Graphics.cpp` `docs/dev/debt/libSceAgcDriver.md`
PR 7332	`core/libs/prx/libSceAgcDriver/CMakeLists.txt` `core/libs/prx/libSceAgcDriver/tests/Graphics.cpp` `core/shader/recompiler/CMakeLists.txt` `docs/dev/debt/libSceAgcDriver.md`
PR 7395	`core/libs/prx/libSceAgcDriver/CMakeLists.txt` `core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp` `core/shader/recompiler/ShaderDiskCache.cpp`
PR 7446	`core/libs/prx/libSceAgcDriver/CMakeLists.txt` `docs/dev/debt/libSceAgcDriver.md`
PR 7449	`docs/dev/debt/libSceAgcDriver.md`
PR 7457	`core/libs/prx/libSceAgcDriver/CMakeLists.txt`
PR 7464	`core/libs/prx/libSceAgcDriver/CMakeLists.txt`
PR 7484	`core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp`
PR 7505	`core/libs/prx/libSceAgcDriver/CMakeLists.txt` `docs/dev/debt/libSceAgcDriver.md`
PR 7514	`core/libs/prx/libSceAgcDriver/CMakeLists.txt`
PR 7550	`core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp` `core/shader/recompiler/ShaderDiskCache.cpp`
PR 7604	`core/libs/prx/libSceAgcDriver/CMakeLists.txt` `docs/dev/debt/libSceAgcDriver.md`
PR 7615	`core/libs/prx/libSceAgcDriver/CMakeLists.txt`
PR 7618	`docs/dev/debt/libSceAgcDriver.md`
PR 7622	`core/libs/prx/libSceAgcDriver/Graphics/src/PipelineJournal.cpp` `core/shader/recompiler/CMakeLists.txt` `core/shader/recompiler/ShaderDiskCache.cpp`
PR 7642	`core/libs/prx/libSceAgcDriver/CMakeLists.txt` `core/libs/prx/libSceAgcDriver/Graphics/src/PipelineJournal.cpp` `core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp` `core/shader/recompiler/CMakeLists.txt` `core/shader/recompiler/ShaderDiskCache.cpp` `docs/dev/debt/libSceAgcDriver.md`
PR 7644	`core/libs/prx/libSceAgcDriver/CMakeLists.txt`
PR 7647	`core/libs/prx/libSceAgcDriver/CMakeLists.txt`
PR 7668	`core/libs/prx/libSceAgcDriver/tests/Graphics.cpp` `docs/dev/debt/libSceAgcDriver.md`
PR 7684	`core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp` `core/shader/recompiler/ShaderDiskCache.cpp`
PR 7689	`core/shader/recompiler/ShaderDiskCache.cpp`
PR 7862	`core/libs/prx/libSceAgcDriver/tests/Graphics.cpp` `docs/dev/debt/libSceAgcDriver.md`
PR 7888	`core/shader/recompiler/ShaderDiskCache.cpp`
PR 7922	`core/shader/recompiler/CMakeLists.txt`
PR 7948	`core/libs/prx/libSceAgcDriver/CMakeLists.txt` `docs/dev/debt/libSceAgcDriver.md`
PR 8004	`core/shader/recompiler/CMakeLists.txt`
PR 8099	`core/libs/prx/libSceAgcDriver/tests/Graphics.cpp` `docs/dev/debt/libSceAgcDriver.md`
PR 8105	`core/libs/prx/libSceAgcDriver/tests/Graphics.cpp`
PR 8145	`core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp` `core/shader/recompiler/ShaderDiskCache.cpp` `docs/dev/debt/libSceAgcDriver.md`
PR 8160	`core/libs/prx/libSceAgcDriver/CMakeLists.txt`
PR 8161	`core/libs/prx/libSceAgcDriver/CMakeLists.txt`
PR 8223	`docs/dev/debt/libSceAgcDriver.md`

oneandonlydean · 2026-10-20T01:53:29Z

Thanks, the journal part looks right: resolving the directory once covers the lock, the size checks, the appends and the tail recovery. Two things before it can go in.

It still has fix(shader): support long Windows cache file paths #7462's open issue: MapFile (ShaderDiskCache.cpp:198) and CachePack (ShaderCachePack.cpp:43) open the raw path, so a long-path entry is written but never mapped. Amoenus posted a small fix and a regression test for it on fix(shader): support long Windows cache file paths #7462 (0a3621bbbf, 83ad998790). Could you fold those in?
It needs a rebase. core/shader/recompiler/CMakeLists.txt conflicts with main's reviewed-names list; keep both sides.

blawesom · 2026-10-20T08:22:43Z

Rebase handoff — rebased onto current main (3fd1cc393, authorship unchanged). One symbol-list conflict in the recompiler's CMakeLists (main's newer LayoutBarycentricEmulation/MeshBarycentricLayoutOf vs your NativeCachePath_nid_no_patch), unioned.

Verified on Linux (CachyOS, kernel 7.2.9, GCC 16.2.1, Ninja, Release, BUILD_TESTING=ON, AMD RX 9070 XT / RADV): builds; 1673/1676 — green apart from the three known main failures (#7584 BDA, #7784 guest-memory, tessellation), none from this PR; conventions pass. Windows path normalization itself not exercised here (Linux run).

To pick it up:

git fetch https://github.com/blawesom/AnyPS5 rebase/7933
git checkout <your branch>
git merge --ff-only FETCH_HEAD

Head is 2e58f8246.

(cherry picked from commit 83ad998)

(cherry picked from commit 0a3621b)

github-actions Bot added the fix error Closes an issue (Fixes #N) label Oct 19, 2026

veosett mentioned this pull request Oct 19, 2026

fix(shader): avoid unsafe Windows cache directory traversal #7945

Merged

12 tasks done

github-actions Bot added build shader core/shader agc libSceAgc* needs rebase Conflicts with main, rebase needed labels Oct 19, 2026

oneandonlydean self-assigned this Oct 20, 2026

alexcallisaya-coder and others added 4 commits October 20, 2026 14:19

fix(shader): support long Windows cache file paths

a5ab12d

test(shader): keep cache relative paths on one drive

1481f03

test(agc): cover long pipeline journal paths and external tails

61fa2d2

fix(agc): normalize all pipeline journal file paths on Windows

2e4cae5

Amoenus and others added 4 commits October 20, 2026 14:25

test(shader): exercise long-path mapped cache persistence

56778c9

(cherry picked from commit 83ad998)

fix(shader): normalize mapped cache entry and pack paths

340f578

(cherry picked from commit 0a3621b)

docs(agc): record remaining long pipeline cache path limits

08fe08f

test(agc): serialize journal replay result capture

80024c4

veosett force-pushed the codex/windows-pipeline-journal-paths branch 2 times, most recently from 689c5cf to 80024c4 Compare October 20, 2026 08:43

github-actions Bot removed needs rebase Conflicts with main, rebase needed fix error Closes an issue (Fixes #N) labels Oct 20, 2026

oneandonlydean merged commit d14a186 into boykopovar:main Oct 20, 2026
15 of 24 checks passed

oneandonlydean mentioned this pull request Oct 20, 2026

fix(shader): support long Windows cache file paths #7462

Closed

12 tasks done

aarushkha mentioned this pull request Oct 20, 2026

fix(agc): read the pipeline cache header through the long-path form on Windows #8554

Merged

12 tasks done

Amoenus mentioned this pull request Oct 20, 2026

Windows shader and pipeline caches cannot persist under long host paths (PPSA99902) #7452

Closed

JJenkx mentioned this pull request Oct 29, 2026

fix(shader): give a device-path shader cache root the long-path form on Windows #13004

Merged

12 tasks done

Navigation Menu

Repository navigation

fix(agc): support long Windows pipeline journal paths - #7933

fix(agc): support long Windows pipeline journal paths#7933
oneandonlydean merged 8 commits into
boykopovar:mainfrom
veosett:codex/windows-pipeline-journal-paths

veosett commented Oct 19, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 19, 2026 •

edited

Loading

Uh oh!

oneandonlydean commented Oct 20, 2026

Uh oh!

blawesom commented Oct 20, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

veosett commented Oct 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Tested

Checklist

Uh oh!

github-actions Bot commented Oct 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

oneandonlydean commented Oct 20, 2026

Uh oh!

blawesom commented Oct 20, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

veosett commented Oct 19, 2026 •

edited

Loading

github-actions Bot commented Oct 19, 2026 •

edited

Loading