Repository navigation
fix(agc): allocate Windows page permissions on demand - #8139
Conversation
|
Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.
Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.
|
|
Windows 10 data point (Windows 10 Pro 22H2 19045, WinLibs GCC 15.2.0, Release, Ryzen with RTX 3080 and Radeon iGPU), head AI-assisted: yes (Claude Code). |
|
@milescoviello, could we agree on sequencing between #8139 and #6921? The changes are complementary: #8139 makes Windows page-state storage sparse, while #6921 accelerates scans using block summaries. Local merge-tree checks found conflicts in Would merging #8139 first work for you? We can help adapt and validate the Windows path afterward. If you and the maintainer prefer #6921 first, we can instead rebase #8139 onto it and preserve both behaviors. Could we agree on the merge order and who handles the combined adaptation before taking #8139 out of draft? |
|
@Izamaru Yes, #8139 first works for me, and I'll handle the combined version in #6921. #8139 fixes how much memory the Windows table commits, while #6921 only speeds up scans, so it makes sense for the fix to go in first. After #8139 is merged, I'll rebase #6921 onto it:
I'll add a case to your Thanks also, @TheSovietWaffle, for the Windows run here. AI-assisted: yes (Claude Code) |
|
@milescoviello Thanks, #8139 first works for us too. I refreshed it against main cabd80f; the standalone PR still contains only its original three files. The experiment is now published in our fork: lab/compat-8139-6921 (diff against the pinned main). It combines #8139 at Pinned published tip: On Windows 11 build 26200, GCC 15.2.0 r7, Release, RTX 4090 driver 617.14:
The experiment puts the 1024 summary bytes in each 65536-entry chunk, skips missing chunks and preserves the settle/forget fences and Linux flat allocation, consistent with your proposed direction. Linux and the full suite were not run. The agreed order remains #8139 first, then your adapted #6921. We can test agc_page_states plus your GuestAllocations cases when you publish the rebase. No loading-time, FPS or gameplay claim. AI-assisted: yes (OpenAI Codex); author reviewed and is responsible for the results. |
|
Thanks @Izamaru, that is useful ahead of the rebase. The check that removing the summary clear in AI-assisted: yes (Claude Code) |
|
@milescoviello I updated my earlier comment after your reply with the published lab/compat-8139-6921 branch. The AI-assisted: yes (OpenAI Codex). |
|
Thanks @Izamaru, I will take AI-assisted: yes (Claude Code) |
|
Independent Windows run of this PR with a baseline (data only). Environment: Windows 10 Pro 10.0.19045, Ryzen 7 5800X3D, WinLibs GCC 15.2.0 (MinGW-w64 UCRT), Release. PR head Results (3 runs each):
This adds the failing baseline to the earlier pass-only Windows data point. Not run: other platforms, the other tests, the full repository CTest, and any title or console run. AI-assisted (Claude Code). |
|
Checked the sparse chunk directory in |
|
@Izamaru #6921 is rebased onto AI-assisted: yes (Claude Code) |
What
On Windows,
PageSpan::allocatecommits one byte per 4-KiB guest page even when most of the arena has never been queried. Replace that dense allocation with a small atomic directory and 64-KiB metadata chunks allocated when readable pages are recorded.Missing chunks remain unknown and take the ordinary query path. Recording an inaccessible run does not allocate missing chunks; inaccessible entries in existing chunks are still cached. Invalidation,
accessSerial, initialization publication and access flags are retained. Chunks have process lifetime, like the previous flat allocation. Linux retains its flat allocation.This is a memory-allocation change, independent of merged #6701's write-generation stamps and #7788's Linux image caching. It overlaps the
PageSpanimplementation in #6921 (run scanning). Sequencing is agreed with @milescoviello: #8139 goes first; after it merges, @milescoviello will rebase #6921 and adapt its summaries to the sparse chunks, preserving the fences and the Linux flat allocation. This PR has no prerequisite and does not bundle #6921. We can help validate the author's rebased #6921 on Windows.Tested
Current-integration Windows revalidation, 2026-10-23
342ac2eda0696b40775e09eb8c928b180758bc1d.0ef6f9fc9a53f2962b6fa238b0dcb5e6ae1e04c6.b655ff3e4424e765ba0d64581f81449067cfabdd.0ec002d3ddcd5717c68f234527baa82212b470d0, treed63906895db5de0962eecb33083a02f28840b652.The main branch advanced during preparation, so the final paired comparison was repeated on the pinned main above. The only merge conflict was in
core/libs/prx/libSceAgcDriver/CMakeLists.txt: preserve both main'sagc_locks_testsblock and this PR's Windows page-state test block. The productionGuestMemory.cppmerged automatically; the sparse implementation andPageStates.cppfixture are unchanged. The diff against the pinned main still contains only the original three files. #6921 is not bundled.The control uses the pinned main's production code, byte for byte, with only the same CMake registration and
PageStates.cppfixture added so both sides run identical cases. It is a fixture-only control, not a claim that unmodified main already contains these new tests. The exact final commit was created in the fork, fetched by SHA, checked against the complete locally resolved tree, rebuilt and tested before moving the PR branch to it.Same environment on both sides: Release, Windows 11 build 26200, WinLibs MinGW-w64 GCC 15.2.0 r7; NVIDIA GeForce RTX 4090, driver 617.14; Khronos validation 1.4.363.0.
The same eight behavioral cases pass on both:
agc_page_states,agc_driver_graphics,agc_sparse_buffer_holes,agc_copy_input_kept_bytes,agc_depth_copy_kept_bytes,agc_draw_inputs_copied,agc_space_pending_writes,agc_pending_write_snapshot.0ec002d3d0ef6f9fc9Run serially with
ctest -V --timeout 30,ANYPS5_REQUIRE_VULKAN=1,ANYPS5_NO_SHADER_CACHE=1,VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validationand loader layer diagnostics. Both behavioral selections confirm three actual Khronos instance-layer insertions and the RTX 4090 device. Their VUID sets are empty; neither log reports the requested layer missing or skipped cases. The page-state and metadata fixtures themselves are CPU tests and create no Vulkan instance.Fresh-process metadata control on this pinned integration
With main production,
agc_page_state_metadatafails as expected withpage permissions committed a dense arena tableand 262,664,192 bytes incremental private commit. With the exact final head it passes at 102,400 bytes, a 262,561,792-byte (250.40-MiB) reduction in this fixture. Querying an untouched reserved chunk also passes its no-allocation check.The behavioral fixture covers concurrent first publication, chunk boundaries, guest bytes, reserved holes, protection changes, decommit and recommit through the access observer.
git diff --checkand conventions against the pinned main pass. The literalorigin/mainconventions check was also run; its unrelated upstream findings come from that ref tracking the older fork main.Limitations: focused synthetic Windows/NVIDIA validation only; no full-suite, Linux runtime, other-vendor GPU, title-runtime, title-unblock, FPS or whole-game memory claim. The 250.40-MiB figure is a metadata-initialization measurement, not a saving measured in Yōtei. Full Windows/Linux CI still requires a collaborator to dispatch the project build workflow for the updated head. Subsequent main commits are outside these pinned results.
The independent Windows 10 reports below retain their original heads and are separate from this author's 2026-10-23 revalidation.
Independent Windows 10 validation
TheSovietWaffle reports that
agc_page_statesandagc_page_state_metadatapass 5 of 5 on the earlier head20ddc95c5: Windows 10 Pro 22H2 build 19045, WinLibs GCC 15.2.0, Release, Ryzen system with an RTX 3080 and Radeon iGPU.This independently extends the page-state and metadata test coverage to Windows 10. These two fixtures exercise CPU-side memory-permission and metadata paths; the listed adapters do not establish Vulkan/GPU coverage. No exact private-commit measurements were supplied, so this report is not an independent measurement of the 250.40-MiB reduction above.
Earlier published head with a baseline
xXPBSXx reports a paired Windows run on the earlier head
2533730bf2: Windows 10 Pro 10.0.19045, Ryzen 7 5800X3D, WinLibs GCC 15.2.0 (MinGW-w64 UCRT), Release. Onlyagc_page_state_testswas built, withctest -Nconfirming the two cases.agc_page_statesandagc_page_state_metadataboth pass in all three runs.tests/PageStates.cppwas retained and only productionExecution/src/GuestMemory.cppwas replaced by merge-basecabd80f472. The behavioral test passes; the metadata test fails in all three runs withpage permissions committed a dense arena tableand 262,664,192 bytes of initialization private commit.This independently reproduces the dense baseline on Windows 10 for that earlier published head. These are CPU permission/metadata fixtures only; no other tests, platforms, full suite or titles were run. No PR-side private-commit value was supplied, so the exact 250.40-MiB reduction above has not been independently remeasured by this report.
Checklist
main; no other open PR implements the same functions — refreshed against the pinned main above. perf(agc): cross uniform runs of cached guest pages 64 pages per load #6921 has an agreed merge order: fix(agc): allocate Windows page permissions on demand #8139 first, then the author rebases perf(agc): cross uniform runs of cached guest pages 64 pages per load #6921.NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file.mdfiles or images added to the repository (attach them to this PR)