Skip to content

fix(shader): read shader cache file times natively on Windows - #12666

Merged
oneandonlydean merged 1 commit into
boykopovar:mainfrom
JJenkx:windows/shader-cache-native-file-times
Oct 29, 2026
Merged

oneandonlydean merged 1 commit into
boykopovar:mainfrom
JJenkx:windows/shader-cache-native-file-times

Conversation

@JJenkx

@JJenkx JJenkx commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

What

On Windows, shader cache housekeeping deletes temporaries that were just written, or misdates its own cache stamp, whenever TZ differs from the Windows time zone.

Cause: with MinGW, std::filesystem::last_write_time reads and sets file times through UCRT's _wstat64 and _wutime64. These convert through local time using TZ, so with any TZ other than the Windows zone, a file's time comes back shifted by the difference. A standalone probe on a host set to US Eastern (summer time) measured the age of a file written in the same second:

TZ age reported
unset, EST5EDT 0 s
UTC0 14,400 s
IST-5:30 34,200 s
JST-9 46,800 s

When TZ differs: the user's environment can set it. So can a game: libc's applyGuestZone copies the guest's TZ into the host process with _putenv_s, and an empty guest TZ becomes UTC0.

Housekeeping compares those times with file_time_type::clock::now(), which isn't shifted:

  • TZ east of the host: a .tmp. file written moments ago looks hours old, and the 1-hour rule deletes it, possibly while another process is still writing it. Directories near the 14-day limit are evicted early.
  • TZ west of the host: the refreshed last-used stamp is written hours in the past.

agc_shader_cache_housekeeping_plain shows both: on main it fails under TZ=IST-5:30 (FileNotFoundError for fresh.tmp.1) and under TZ=AAA+12 ("active cache stamp was not refreshed"). It passed only because CI and most hosts leave TZ unset.

Fix: housekeeping now handles file times itself on Windows:

  • reads them with GetFileAttributesExW, converted to system_clock;
  • refreshes the stamp with SetFileTime from GetSystemTimePreciseAsFileTime;
  • compares against system_clock::now().

Other platforms keep std::filesystem, converted with file_clock::to_sys, as libSceSaveData.native already does. libkernel's stat decodes FILETIME itself, so guest-visible file times weren't affected.

Test: the housekeeping test now runs its child with a TZ 12 hours from the host's offset (AAA+12, or AAA-12 when the host is west of UTC). So the shift is exercised on UTC CI machines too.

Tested

Windows 11 (US Eastern), Ryzen 9 5950X, WinLibs GCC 15.2.0 UCRT r7, Release, main 3a2b919816.

TZ for ctest main this PR
unset (host Eastern), main's source with this PR's test, which picks AAA-12 plain fails (FileNotFoundError for fresh.tmp.1); extended, extended-long and device pass 9/9 pass, 3 of 3
IST-5:30, main as is plain fails the same way, 3 of 3 9/9 pass, 3 of 3
AAA+12, main as is plain fails: "active cache stamp was not refreshed" 9/9 pass, 3 of 3
UTC0 not run 9/9 pass, 3 of 3

The extended and device modes skip eviction for \\?\ and \\.\ roots, so only plain reaches the shifted comparisons.

This PR runs all 9 agc_shader_(cache_housekeeping|disk_cache) tests, 3 of 3 each (--repeat until-fail:3): plain, extended, extended-long and device housekeeping, plus the disk-cache tests. A full ctest -j32 run on main under TZ=IST-5:30 found this; its only other failures were guest_freebsd_clocks (#12660) and guest_pad, which passes 3/3 alone.

Conventions pass. tools/pr_overlap.py: 45 open PRs share files and merge cleanly. Sixteen that touch ShaderDiskCache.cpp already conflict with main and couldn't be compared, so I searched their diffs for housekeeping, last_write_time, last-used and .tmp.. Only #6924 matches: it adds a last_write_time call that orders cache entries by age, which a uniform shift doesn't change, and it doesn't touch housekeeping().

Not tested: Linux and macOS build or run (their branch only adds file_clock::to_sys, which libSceSaveData.native already uses); a game run.

Follow-up: #12669 fixes the same shift in libSceSaveData.native, whose fallback mtime for a save without stored parameters came from std::filesystem::last_write_time.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: none
  • Unblocks: none
  • GPUs tested: n/a
  • AI-assisted: yes (Claude Code)

🤖 Generated with Claude Code

The MinGW std::filesystem::last_write_time reads and sets file times
through the UCRT stat and utime calls, which convert through local time
using TZ. With a TZ other than the Windows zone, including the one libc
applies from the guest environment, times shifted by the difference:
under TZ=IST-5:30 on an Eastern host a temporary written that second
was 9.5 hours old, so housekeeping deleted it, and a TZ west of the host
set the active cache stamp hours in the past. Housekeeping now reads
times with GetFileAttributesExW, refreshes the stamp with SetFileTime
and compares in system_clock. The housekeeping test runs its child
with a TZ 12 hours from the host zone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 5609 core/shader/recompiler/ShaderDiskCache.cpp
PR 5925 core/shader/recompiler/ShaderDiskCache.cpp
PR 6280 core/shader/recompiler/ShaderDiskCache.cpp
PR 6412 core/shader/recompiler/ShaderDiskCache.cpp
PR 6488 core/shader/recompiler/ShaderDiskCache.cpp
PR 6924 core/shader/recompiler/ShaderDiskCache.cpp
PR 6970 core/shader/recompiler/ShaderDiskCache.cpp
PR 7085 core/shader/recompiler/ShaderDiskCache.cpp
PR 7642 core/shader/recompiler/ShaderDiskCache.cpp
PR 7684 core/shader/recompiler/ShaderDiskCache.cpp
PR 7689 core/shader/recompiler/ShaderDiskCache.cpp
PR 9356 core/shader/recompiler/ShaderDiskCache.cpp
PR 9387 core/shader/recompiler/ShaderDiskCache.cpp
PR 9508 core/shader/recompiler/ShaderDiskCache.cpp
PR 9889 core/shader/recompiler/ShaderDiskCache.cpp
PR 9897 core/shader/recompiler/ShaderDiskCache.cpp
PR 10049 core/shader/recompiler/ShaderDiskCache.cpp
PR 10193 core/shader/recompiler/ShaderDiskCache.cpp
PR 10351 core/shader/recompiler/ShaderDiskCache.cpp
PR 11551 core/shader/recompiler/ShaderDiskCache.cpp

@JJenkx

JJenkx commented Oct 29, 2026

Copy link
Copy Markdown
Contributor Author

Linux run at this PR's exact head e2f86a2047 (e2f86a2047e5990e87b3050260721862af72b0db), not merged onto anything else; its base on main is 3a2b919816.

Arch Linux, kernel 7.2.9, Ryzen 9 7950X, g++ (GCC) 16.2.1 20260810, glibc 2.44, x86-64. Release, BUILD_TESTING=ON, Ninja -j4, private worktree.

  • ctest -R '^agc_shader_(disk_cache.*|cache_housekeeping_.*)$': passes in each of 3 runs (15 passes).

On Linux CMake registers only the plain and plain-long housekeeping modes; the Windows-only modes (extended, extended-long, device) are not run here.

@github-actions github-actions Bot added shader core/shader agc libSceAgc* labels Oct 29, 2026
@oneandonlydean

Copy link
Copy Markdown
Collaborator

@JJenkx reviewed at e2f86a2. The Windows lastWriteTime reads GetFileAttributesExW and converts FILETIME with the right epoch and units, and refreshWriteTime sets it with SetFileTime, so housekeeping no longer sees times shifted by the TZ offset; a failed read yields no value and never deletes; the non-Windows branch is the old call plus file_clock::to_sys; no cache format change. The test runs its child 12 hours from the host zone. One caveat for merging: PR CI runs no Windows ctest, so the bar is our dispatched Windows run of the housekeeping tests. Not for this PR: libSceSaveData.native had the same Windows shift (now #12669, merged).

@adriadam10 we'll merge this once our dispatched Windows run of the housekeeping tests is green, unless you'd rather merge it yourself.

@JJenkx

JJenkx commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor Author

@oneandonlydean exact-head Windows run of the housekeeping tests, for the merge bar you set: head e2f86a2047 merged onto current main 52c09d567d (clean merge, local only). Windows 11 Enterprise 10.0.22621, Ryzen 9 5950X, WinLibs GCC 15.2.0 UCRT r7, Ninja, Release, BUILD_TESTING=ON, agc_shader_disk_cache_tests built explicitly (it is EXCLUDE_FROM_ALL).

  • This PR: agc_shader_cache_housekeeping_plain, _extended, _extended-long and _device all pass, 3 of 3 (ctest --repeat until-fail:3), none skipped.
  • Negative control: with core/shader/recompiler/ShaderDiskCache.cpp put back to main's version and the PR's test kept, agc_shader_cache_housekeeping_plain fails, and the other three pass. Every mode runs the child 12 hours away from the host zone, but only plain evicts; extended, extended-long and device expect housekeeping to skip, so the time shift cannot change their result.

This is my own run, not a dispatched one, so take it as supporting evidence if you still want yours.

Edited: an earlier version said plain was the only mode run 12 hours from the host zone; all modes are, and plain is the only one that evicts.

AI-assisted: yes (Claude Code).

@JJenkx

JJenkx commented Oct 29, 2026

Copy link
Copy Markdown
Contributor Author

Independent confirmation from a whole-suite zone sweep on current main edf6804cf5 (same machine; the host zone is Eastern Standard Time). I ran the full Windows ctest suite twice with an outer TZ set:

Outer zone main edf6804cf5 with this head e2f86a2047 merged
TZ=AAA+12 all 4 agc_shader_cache_housekeeping_* fail all 4 pass 3 of 3
TZ=AAA-12 agc_shader_cache_housekeeping_plain fails all 4 pass 3 of 3

Apart from those, the sweep fails only the two tests that also fail without TZ (guest_job_manager_worker_count, fixed by #12085, and guest_freebsd_clocks, fixed by #12660). No other test in the 2980 depends on the host zone.

AI-assisted: yes (Claude Code).

@oneandonlydean oneandonlydean self-assigned this Oct 29, 2026
@oneandonlydean
oneandonlydean merged commit 70003eb into boykopovar:main Oct 29, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agc libSceAgc* shader core/shader

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants