Skip to content

test(shader): skip open-entry cache replacements where SMB has no POSIX rename - #11434

Merged
oneandonlydean merged 1 commit into
boykopovar:mainfrom
LeandroLP23:test/shader-cache-smb-posix-rename
Oct 27, 2026
Merged

oneandonlydean merged 1 commit into
boykopovar:mainfrom
LeandroLP23:test/shader-cache-smb-posix-rename

Conversation

@LeandroLP23

@LeandroLP23 LeandroLP23 commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

What

Fixes #10833 the way agreed there (option 1): replacement stays atomic, and the tests skip only what an SMB share cannot do.

On Windows WriteFileAtomically replaces a cache entry with a POSIX-semantics rename (SetFileInformationByHandle class 22, FileRenameInfoEx), so readers and mappings of the old entry keep working. Through an SMB share that rename fails with ERROR_INVALID_PARAMETER, and the std::filesystem::rename fallback is refused while the destination is open or mapped. The issue's NAS and \\localhost\C$ (Windows' own SMB server, over NTFS) behave the same; the error codes are in #10833 (comment).

With the first skip in place, the tests reached three more places that need the same operations:

  • verifyStore and verifyPersist replace an entry that ShaderDiskCache itself has mapped with MapViewOfFile after loading it. A mapping blocks the plain rename just like an open reader.
  • The cleanup removeCacheDirectory deletes with FileDispositionInfoEx and FILE_DISPOSITION_FLAG_POSIX_SEMANTICS, which SMB also rejects with ERROR_INVALID_PARAMETER, and the entries the cache still maps can't be deleted there.
  • shader_cache_file_read built the \\?\ alias of a UNC path as \\?\\\server\share\…; the extended form of a UNC path is \\?\UNC\server\share\….

Changes:

  • ShaderCacheDirectory.hpp: the POSIX rename moves unchanged into PosixReplaceByHandle, which WriteFileAtomically calls as before, so production behaviour does not change. PosixReplaceUnsupported(directory) probes that rename on two scratch files in a directory and reports only ERROR_INVALID_PARAMETER.
  • agc_shader_disk_cache: where the probe reports it, verifyAtomicReplacement skips the retained-reader replacement, verifyStore skips the truncated- and damaged-entry checks, and verifyPersist skips replacing the mapped persisted entry, each printing why. Any other replacement failure still fails. The cleanup falls back to a plain delete only when the POSIX delete returns ERROR_INVALID_PARAMETER, and then leaves the directory behind with a note instead of failing.
  • shader_cache_file_read: the UNC alias fix, and the retained-reader replacement is skipped under the same probe. A plain replacement without a reader is still checked there.
  • docs/dev/debt/shader.md: a "Shader cache directories on SMB shares" entry records that such a cache directory cannot update an open or mapped entry, including one this process's cache has mapped, and why replacement stays atomic.

Tested

Rebased onto main 9656d366be (head 8259ecb26d); on the RTX 5060 Ti the whole table below and the validation run were repeated on the rebased head and on 9656d366be, with the same results. The first runs, on both machines, were on base 7b415520b6.

Same commands on two machines, TEMP/TMP set to a local NTFS directory or to \\localhost\C$\aps5-smbtmp, each test 3 times, on this head and on its base main 7b415520b6:

  • Windows 11 10.0.26300, AMD Ryzen AI 7 350 / Radeon 860M, AMD 24.30.58.08 (32.0.13058.8001), WinLibs GCC 15.2.0 r7, Release.
  • Windows 11 10.0.26300, Intel i5-10600K / NVIDIA GeForce RTX 5060 Ti, NVIDIA 621.01, same toolchain, Release.
Test TEMP this head (both machines) main (both machines)
agc_shader_disk_cache NTFS 3/3 3/3
shader_cache_file_read NTFS 3/3 3/3
agc_shader_disk_cache SMB 3/3, prints the skips 0/3, cannot replace cache entry with a retained reader
shader_cache_file_read SMB 3/3, prints the skips 0/3

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: Fixes Windows SMB: shader disk-cache retained-reader replacement fails; local NTFS passes #10833
  • Unblocks: none
  • GPUs tested: AMD Radeon 860M (integrated), AMD 24.30.58.08, Windows 11 10.0.26300; NVIDIA GeForce RTX 5060 Ti, NVIDIA 621.01, Windows 11 10.0.26300
  • AI-assisted: yes (Claude Code)

@MarcZX10

Copy link
Copy Markdown
Contributor

The tested base in the description is 7b415520b6, while current main is 035ac96edd (9 commits ahead). Although the intervening commits do not change the shader-cache implementation paths, CONTRIBUTING requires PRs to stay up to date. Please sync to current main and rerun conventions before final review.

@github-actions

github-actions Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 5021 docs/dev/debt/shader.md
PR 5371 docs/dev/debt/shader.md
PR 5609 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
PR 6235 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
docs/dev/debt/shader.md
PR 6280 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
docs/dev/debt/shader.md
PR 6412 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
docs/dev/debt/shader.md
PR 6488 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
docs/dev/debt/shader.md
PR 6858 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
PR 6924 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
docs/dev/debt/shader.md
PR 6959 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
docs/dev/debt/shader.md
PR 6970 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
PR 7395 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
docs/dev/debt/shader.md
PR 7410 docs/dev/debt/shader.md
PR 7684 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
docs/dev/debt/shader.md
PR 7694 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
PR 7922 docs/dev/debt/shader.md
PR 8004 docs/dev/debt/shader.md
PR 8939 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
docs/dev/debt/shader.md
PR 9020 docs/dev/debt/shader.md
PR 9193 docs/dev/debt/shader.md
PR 9335 docs/dev/debt/shader.md
PR 9387 docs/dev/debt/shader.md
PR 9457 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
docs/dev/debt/shader.md
PR 9633 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
docs/dev/debt/shader.md
PR 9748 docs/dev/debt/shader.md
PR 9749 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
docs/dev/debt/shader.md
PR 9790 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
docs/dev/debt/shader.md
PR 9852 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
PR 9889 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
docs/dev/debt/shader.md
PR 9897 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
PR 9969 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
docs/dev/debt/shader.md
PR 10000 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
docs/dev/debt/shader.md
PR 10049 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
PR 10233 docs/dev/debt/shader.md
PR 10489 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
docs/dev/debt/shader.md
PR 10556 docs/dev/debt/shader.md
PR 10803 docs/dev/debt/shader.md
PR 10846 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
docs/dev/debt/shader.md
PR 11483 docs/dev/debt/shader.md
PR 11551 core/libs/prx/libSceAgcDriver/tests/ShaderDiskCache.cpp
PR 11557 docs/dev/debt/shader.md

@LeandroLP23
LeandroLP23 force-pushed the test/shader-cache-smb-posix-rename branch from b26e0e8 to 8259ecb Compare October 26, 2026 13:44
@LeandroLP23

LeandroLP23 commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor Author

@MarcZX10 rebased onto current main 9656d366be (head 8259ecb26d, same single commit; between the old and new base only StencilReplay.cpp changed under core/shader/recompiler). check_conventions.py --base origin/main passes. I also reran the full check on the rebased head and on 9656d366be on the RTX 5060 Ti: agc_shader_disk_cache and shader_cache_file_read pass 3/3 on NTFS and on \\localhost\C$ with this head, while main fails 0/3 on the share; the 27 shader/pipeline cache tests pass with the validation layer on both, with no VUID- lines. The description is updated.

AI-assisted: yes (Claude Code)

@github-actions github-actions Bot added the fix error Closes an issue (Fixes #N) label Oct 26, 2026

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed the Windows SMB probing/fallback and its test changes on exact head 8259ecb26dac1890717b823dd42f9558eb4fa9a4. I found no implementation defect in the bounded FILE_RENAME_INFO_EX buffer construction or the fallback behavior. The author reports repeated Windows runs on two GPUs and local NTFS plus \\localhost\C$, including the cache test matrix and validation-layer comparison. I did not independently run those Windows tests. Current conventions, git diff --check, and the merge-tree preview pass; hosted Linux/Windows product jobs are path-skipped.

Before merge, please sync this branch to current main (b0f1183ca12b). Its tested base is 9656d366be; the intervening commits do not touch this PR's implementation, tests, or debt entry, and the preview merges cleanly. This is the remaining branch-freshness requirement from CONTRIBUTING.md.

…IX rename

On Windows, WriteFileAtomically replaces a cache entry with a POSIX
rename so that readers and mappings of the old entry keep working.
Through an SMB share that rename fails with ERROR_INVALID_PARAMETER, and
the plain rename it falls back to is refused while the destination is
open or mapped, so agc_shader_disk_cache and shader_cache_file_read
failed with their temporary directory on such a share (boykopovar#10833).

The POSIX rename moves into PosixReplaceByHandle, unchanged, and
PosixReplaceUnsupported probes it in a directory. Where it reports that
error, the tests skip only the replacements of an entry that a reader
holds open or the cache has mapped, and the disk cache test's cleanup
leaves its directory behind when the POSIX delete fails the same way;
any other failure still fails them. shader_cache_file_read also builds
its \?\ alias of a UNC path as \?\UNC\server\share. The limitation is
recorded in the shader debt file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@LeandroLP23
LeandroLP23 force-pushed the test/shader-cache-smb-posix-rename branch from 8259ecb to b95b22c Compare October 26, 2026 15:45
@LeandroLP23

Copy link
Copy Markdown
Contributor Author

@MarcZX10 synced: rebased onto current main da89f18ea7 (head b95b22cf16, the same single commit). None of the PR's files (ShaderCacheDirectory.hpp, CacheFileReadTests.cpp, ShaderDiskCache.cpp, docs/dev/debt/shader.md) changed between the tested base 9656d366be and da89f18ea7, and check_conventions.py --base origin/main passes.

AI-assisted: yes (Claude Code)

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed exact head b95b22cf1622, now based on live main da89f18ea7c1. The rebase preserves the reviewed one-commit implementation and reported Windows SMB/NTFS regression coverage; no source or test files changed between the tested base and this main. The current-main merge preview, conventions and whitespace checks pass. Linux and Windows relinker plus ASan checks pass; macOS relinker jobs are still running/queued, and runtime product tests are skipped by path policy. I did not independently run the Windows/GPU tests. The earlier branch-freshness request is resolved. Approved.

@github-actions github-actions Bot added shader core/shader docs agc libSceAgc* labels Oct 26, 2026
@oneandonlydean oneandonlydean self-assigned this Oct 26, 2026
@oneandonlydean
oneandonlydean merged commit a453b29 into boykopovar:main Oct 27, 2026
13 checks passed
@LeandroLP23
LeandroLP23 deleted the test/shader-cache-smb-posix-rename branch October 27, 2026 10:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agc libSceAgc* docs fix error Closes an issue (Fixes #N) shader core/shader

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Windows SMB: shader disk-cache retained-reader replacement fails; local NTFS passes

3 participants