refactor(libc): map a run of shared guest pages as one view on Windows by EagleBearerYS · Pull Request #9505 · boykopovar/AnyPS5

EagleBearerYS · 2026-10-21T23:43:56Z

What

On Windows, WindowsMappings::Map maps shared guest memory one 16 KiB page at a time. That covers direct memory while shared imports are attached, shm_open memory and JIT shared memory. Each page costs a placeholder split, a view, a protect and three VirtualQuery calls: two to find the placeholder the page splits from, one to see whether its section page still needs a commit. All of it runs under the mappings mutex. A 1 GiB map places 65,536 views.

This ports two of fable-void's commits from his fork (wip/new-gpu-path, 027bff5 and 3d695fb), with his authorship. On top of them are our two adjustments for main's file mappings, a bench, and a change after review:

6c1f636 (bench): windows_shared_map_cost maps 4 MiB, 256 MiB and 1 GiB of shared memory through WindowsMappings::Map, then releases it, five times each. It prints the median map and release times and the number of views. It only measures and always passes.
312fb24 (fable-void): a page-by-page mapping commits its section range once, through one window. Each page then splits off the placeholder the previous page left, whose extent is already known. A page now costs a placeholder split, its view and its protection.
a16262b (adapts 027bff5): only mappings that pass no section size, i.e. direct and shared memory, commit the range up front. A file mapping passes the bytes its file still has. A read-write window over it can run past the end of a short file, and a read-only file section refuses one, so file mappings keep committing each view as before.
bdf9bfe (fable-void): each run of shared pages that need a view becomes one view: one placeholder split, one map at read-write-execute (the cap later guest protections need) and one protect per run. Per-page protections and write tracking still change 4 KiB subpages inside the view. A view splits only where a release or a remap cuts it, and the split restores every 4 KiB protection. An alias of a page elsewhere, or a per-page protection, leaves the view whole.
1f19ce0 (adapts 3d695fb): a run view needs a read-write-execute section and the whole run committed, which a file mapping can't give. File mappings take the per-page path they took before: one view per page at the file's view protection, with the last page cut at the end of the file.
a2e0bf1 (after MarcZX10's review): the VirtualQuery counter that 312fb24 adds for windows_write_watch is compiled only into that test, through APS5_COUNT_MAPPING_QUERIES, so the runtime's query() does no atomic update per query. main has had the same counter since perf(libc): carve fresh private guest pages out of their placeholder #10070, which took it from this PR so that whichever landed second would drop its copy, so after the merge with main the PR's diff no longer carries it.

fable-void's message on 312fb24 lists agc_private_collect among the tests he ran. That test comes with his private-collect commit (2760ad8 on his branch), which isn't part of this PR.

Measured on GitHub's Windows runner, three runs each: main with the bench (6c1f636, run) against this PR (run).

shared map	`main`: map median (views)	this PR	release median, `main` → this PR
4 MiB	1.60–1.69 ms (256)	0.16–0.19 ms (1)	0.85–0.88 → 0.25–0.27 ms
256 MiB	151–181 ms (16,384)	10.0–12.0 ms (1)	63–64 → 17.6–18.4 ms
1 GiB	549–614 ms (65,536)	49–50 ms (1)	273–277 → 71.7–72.3 ms

On his machine, fable-void measured the 1 GiB shared map and import in agc_driver_committed_imports at 190–196 ms on the mapping thread, against 418–427 ms before.

These measure the mapping itself. A Resident Evil Village (PPSA01556) pair (under Tested) shows no measurable change in a 40 s window of its load, so this PR makes no frame-rate claim.

Overlap: checked with git merge-tree against every open PR touching these files, on main a6dcea9. Two of ours overlap, on adjacent added lines only:

perf(libc): read private write watches per allocation without a region query #9098 changes the private-memory collect, not shared mappings. On their current heads (git merge-tree) it conflicts on a test registration in core/libs/CMakeLists.txt, on the query counter in WindowsMappings.hpp (its accessor and the count in query(), which this PR now guards with APS5_COUNT_MAPPING_QUERIES), and on a test function before main() in WindowsWriteWatch.cpp.
fix(agc): keep a rewatched import's memory compared once it is shared #9159 has merged since, and this PR's merge of main 0ea0f25 takes it in: its aliasDemoted reset and unreported marks are in both of this PR's loops, and both sides' cases run in WindowsWriteWatch.cpp (see Tested).

Both are mine. Merge order: #9098 goes after this PR, lists it under Depends on, and resolves these conflicts when it rebases.

The eight PRs that conflict with main themselves were checked by their diffs: none changes WindowsMappings.hpp. #6044, #7201, #7275, #8285, #8650 and #9224 add root library test registrations in core/libs/CMakeLists.txt next to this PR's windows_shared_map_cost and the windows_write_watch_tests definition. Whichever PR rebases keeps every registration, and checks after resolving that ctest -N still lists all of them.

Tested

GitHub's Windows runner at 1f19ce0 (run):
- windows_shared_map_cost, windows_write_watch, guest_memory, guest_jit_shared_memory, guest_shm_open and guest_raise_exception pass ×3;
- the libc (142) and libkernel (207) suites pass.
- main's code doesn't build windows_write_watch there, because fable-void's two new cases use the query counter 312fb24 adds. His messages give their reds: "775 queries for 256 pages" and "a shared gibibyte was mapped as 65536 views instead of one".
a2e0bf1 on GitHub's Windows runner (run): windows_shared_map_cost and windows_write_watch pass 5 runs in a row, and the libc suite passes (142). In both attempts of that run, libkernel's guest_raise_in_system_call timed out after 30 s near the end of the suite. Run alone 20 times in a row, it and guest_raise_exception pass at a2e0bf1 (run) and at 1f19ce0 (run), while main's a6dcea9 times out guest_raise_exception within its 20 (run).
5fd76af, merged with main 1e20650 (run). The one conflict was the list of cases in WindowsWriteWatch.cpp's main(), where main added three private-commit cases next to this PR's two; all five run. main now has the same APS5_COUNT_MAPPING_QUERIES query counter (from perf(libc): carve fresh private guest pages out of their placeholder #10070), so after the merge this PR no longer changes those lines. Its other added and removed lines are unchanged.
- on Windows, windows_write_watch, windows_shared_map_cost, guest_memory, guest_shm_open, guest_jit_shared_memory and guest_raise_exception pass ×10: 6 run, 6 pass, 0 skip, 0 fail;
- on Linux, guest_memory, guest_shm_open, guest_jit_shared_memory and guest_raise_exception pass ×10: 4 run, 4 pass, 0 skip, 0 fail;
- the libc and libkernel suites pass on both (147 and 213 tests on Windows, 120 and 203 on Linux), run with APS5_SKIP_TIMING_TESTS=1.
fbf50df, merged with main 0ea0f25 (run). Both conflicts come from fix(agc): keep a rewatched import's memory compared once it is shared #9159, which main has taken since:
- in WindowsMappings.hpp, fix(agc): keep a rewatched import's memory compared once it is shared #9159 added two lines to the per-page loop this PR replaces: the aliasDemoted = false reset after a resident alias is demoted, and the unreported = allSubpages mark on a new view of compared shared memory. The merge carries both into this PR's per-page loop and into its run loop, where each run view takes the mark from its shared page. The PR's other added and removed lines are unchanged;
- in WindowsWriteWatch.cpp's main(), the list of cases keeps this PR's two and fix(agc): keep a rewatched import's memory compared once it is shared #9159's four;
- on Windows, windows_write_watch, windows_shared_map_cost, the write-tracking tests (fix(agc): keep a rewatched import's memory compared once it is shared #9159's agc_rewatched_direct_shared, agc_import_alias_shared and agc_compared_shared_generations among them), guest_memory, guest_shm_open, guest_jit_shared_memory and guest_raise_exception pass ×10: 14 run, 13 pass, 1 skip (agc_driver_compared_shared_snapshot, no Vulkan device), 0 fail. With main's implementation, windows_write_watch fails: "a shared mapping queried the address space for every page: 775 queries for 256 pages";
- on Linux, agc_write_tracking, agc_write_watch_coverage, guest_memory, guest_shm_open, guest_jit_shared_memory and guest_raise_exception pass ×10: 6 run, 6 pass, 0 skip, 0 fail;
- an ^agc sweep, once (run): Linux 987 run, 923 pass, 64 skip, 0 fail. Windows 1,017 run, 287 pass, 726 skip, 4 fail: agc_submit_progress, agc_submit_progress_no_timeline, agc_driver_pipelined_metadata_pass and agc_driver_pipelined_metadata_pass_draw_pipeline stop at vkCreateInstance (-9) on GitHub's runner, which has no Vulkan device, and fail the same way on main c9b2660 alone (run);
- the libc and libkernel suites pass on both (170 and 221 tests on Windows, 143 and 205 on Linux), run with APS5_SKIP_TIMING_TESTS=1, which relaxes precise_wait's median bounds on GitHub's hosted runners.
Our two adjustments each have a red from existing tests on the Windows runner, in guest_memory's regular file mapping case. Those runs were on an earlier branch of this port that also carried his private-collect commit, which doesn't touch file mappings.
- Without a16262b (run), a read-only shared map of a read-only file fails: "Guest memory check failed at core/libs/tests/GuestMemory.cpp:2922".
- Without 1f19ce0 (run), a shared map of a two-page file fails: "Guest memory check failed at core/libs/tests/GuestMemory.cpp:2899". guest_jit_shared_memory and guest_shm_open fail there too.
guest_raise_exception timed out once (30 s) in an earlier run of the port. It passed in a recheck (run) and in all three runs above. This PR doesn't touch libkernel.
Linux (same run): WindowsMappings.hpp is Windows-only, so Linux builds unchanged code. guest_memory, guest_jit_shared_memory, guest_shm_open and guest_raise_exception pass ×3, and libkernel's suite passes (197). libc's suite has one failure, precise_wait, a timing test that flakes on that runner.
JJenkx's run (comment): Ryzen 9 5950X, Windows 11, WinLibs GCC 15.2.0 UCRT, Release, this PR's a2e0bf1 merged into main d9c99c9. windows_shared_map_cost (not on main), windows_write_watch, guest_memory, guest_jit_shared_memory and guest_shm_open pass 3 of 3, and libc's suite passes (142). guest_raise_exception times out there with and without this PR, as guest_raise_in_host_code does once on main; JJenkx's breakdown of that time is libkernel: guest_raise_exception exceeds its 30 s budget on Windows; 14.8 s of it is raises on a thread looping in sceKernelSyncOnAddressWake #9610.
check_conventions.py passes, with no comment lines added.
Resident Evil Village (PPSA01556), a traced pair on an RTX 3070 Laptop GPU (NVIDIA 617.42, Windows 11). GPU clocks weren't held: P0 for 99.6% of samples on both, 780-1785 MHz graphics, median 1290.
- Trees: A is a local test tree, d56bdc5: main 1ff5c8a plus four local AGC driver commits, none in libc. D is A with this PR's six commits on top (ca13192259, the same diff as this PR).
- Runs:
  - the same environment on both (APS5_LAZY_SHADER_PREPARE=1, APS5_NO_GPL=1, APS5_HOST_IMPORT_MIB=6400, APS5_TRACE_SECONDS=900:40), the same route and a warm shader cache;
  - each 40 s window falls inside the dark load after New Game's Yes: about 119 s after it on A (presents 4935-5055), and about 68 s after it on D (presents 5249-5371).
- A: 120 presents, frame p50 323.5 / p90 383.1 / p99 577.4 / max 1447.3 ms, GPU busy 75.8 ms per present.
- D: 122 presents, frame p50 336.1 / p90 384.2 / p99 433.6 / max 495.3 ms, GPU busy 74.1 ms per present.
- No measurable effect in this window:
  - p50 +4% and GPU busy −2%, with dispatch and memory wait within 1%;
  - A's p99 and max come from a single 1.45 s shader-resolve stall (present 5047), so the difference there isn't a gain.
- Files: A's and D's trace, aps5-summary.jsonl line and tools/trace_digest.py output.

Checklist

windows_shared_map_cost maps 256, 16384 and 65536 pages of shared memory (4 MiB, 256 MiB and 1 GiB) through WindowsMappings::Map, then releases them, five times each, and prints the median map and release times, the time per page and the number of mapped views. It only measures and always passes, so a build of this commit and of a later one can be compared on the same machine.

Mapping shared guest memory page by page queried the address space three times for every 16 KiB page: twice to find the placeholder the page splits from, and once more to see whether the section page still had to be committed, which it then committed on its own. For a range of fresh direct memory that is seven system calls a page, and with shared imports attached every direct mapping takes this path on the guest thread that maps it, under the mapping lock. A mapping that goes page by page now commits its section range once through one window, and splits each page off the placeholder the previous page left, whose extent it already knows; only a page after a resident run or a demotion looks the placeholder up again. A page now costs a placeholder split, the view and its protection. Tested: built; windows_write_watch gains a case that maps 256 shared pages and counts the address-space queries the mapping makes, then stores to and reads back every page. With the per-page lookups it fails (775 queries for 256 pages) and with this change it passes. agc_driver_committed_imports, agc_write_watch_coverage, agc_private_collect, agc_write_guards, agc_driver_dcc_keys_first_write, agc_driver_residency_dirty_mirrors, agc_driver_residency_driver, guest_memory and the host import, window import and alias store tests pass.

…ections Committing the section range through one read-write window before the pages are mapped works for direct and shared memory, whose sections are committable and at least as long as the mapping. A file mapping passes the bytes its file still has: the window can run past the end of a short file, and a read-only file section refuses a read-write window, so every file mapping would fail. Only mappings that pass no section size commit the range up front; file mappings keep committing each view as before. guest_memory's regular file mapping case maps a read-only file and a file tail shorter than a page on Windows. Adapts fable-void's 027bff5 to main's file mappings.

Mapping shared direct memory while shared maps are imported placed one view per 16 KiB page: a placeholder split, a map and a protect for every page. A 1 GiB map made about 200,000 system calls, held the mapping thread for about 0.4 s and took and dropped the mappings mutex on every page, so threads taking write faults queued behind it. Map now maps each run of shared pages that need a view as one view: one placeholder split, one map at read-write-execute (the cap later guest protections need) and one protect per run, and builds the per-page bookkeeping under the mutex while yielding to waiting faults. The views move the run's page references instead of copying them, so releasing the run does not hold the mutex. Protections and write tracking change subpages inside the view as before. A view is split only where a release or a remap cuts it: reset unmaps the view, maps the two pieces again over the same section offsets and restores every 4 KiB protection, so armed write tracking and guest protections survive the split. An alias of a page elsewhere and a per-page protection leave the view whole. Tested: windows_write_watch gains a case that maps 1 GiB of shared memory (65536 views before, one after), then protects one page, aliases one page, arms write tracking on one page, releases one page and remaps another: the run splits only at the released and remapped pages (2 and 4 views), every other page keeps its bytes and protection, and the armed page still faults and is collected. It fails before ("a shared gibibyte was mapped as 65536 views instead of one") and passes after. agc_driver_committed_imports: the 1 GiB shared map and import takes 190-196 ms on the mapping thread instead of 418-427 ms, and the longest write fault on another thread stays at 46-97 us. Built; the write watch, committed import, guest memory, write guard, host and window import, residency and mapping tests pass (58 tests).

A run view maps the section at read-write-execute over the whole run and commits the run through a read-write window. A file mapping passes the protection its file allows and the bytes its file still has, so a read-only file or a file that ends inside the run cannot be mapped that way. File mappings now take the per-page path they took before the run views: one view per page at the file's view protection, with the last page cut at the end of the file. Mappings that pass no section size, direct and shared memory, still map each run as one view. guest_memory's regular file mapping case maps a read-only file and a file tail shorter than a page on Windows. Adapts fable-void's 3d695fb to main's file mappings.

github-actions · 2026-10-21T23:44:08Z

This pull request is labeled perf, so it needs the summary line of a capture on main and one with the change, from the same title, scene and machine, as the Performance report form asks. Please paste both aps5-summary.jsonl lines, attach the two traces and add the output of tools/trace_digest.py.

MarcZX10

The implementation has substantial Windows coverage: the PR reports successful shared-view, write-watch, guest-memory, JIT, shm, libc/libkernel and relinker runs, plus repeated same-runner map/release measurements. Eligibility, conventions, intake, labels, report and the reported matrix checks pass. I did not run Windows tests locally.

I’m requesting changes for these items:

The required performance report is still missing. The perf label bot correctly asks for the aps5-summary.jsonl lines for main and this change, traces for both, and tools/trace_digest.py output. CONTRIBUTING.md requires that evidence for a speed claim. The custom windows_shared_map_cost timing table is useful, but it does not provide those report artifacts. Please attach them for the same workload and machine, or revise the performance claim and checklist to match the evidence available.
Please complete overlap coordination for #9098 and #9159. The live overlap scan reports shared edits in WindowsMappings.hpp and WindowsWriteWatch.cpp (and the CMake test registration for #9098). The PR description says the changes compose, but does not record agreement with those authors about merge order. Please confirm the order with them; add Depends on if this PR must follow either one, then rebase/recheck the resulting combined diff.
The test query counter adds an unconditional atomic operation to production. WindowsMappings::query() now performs queries.fetch_add() for every VirtualQuery, even in the runtime library where the counter is not read. Please compile the counter only into the test target (or otherwise keep it out of production), so a performance change does not add a global atomic RMW to this hot path.

The measured view-count and map-time improvements are promising; these requests concern the required report, merge-order process, and the production cost of the new instrumentation.

The VirtualQuery counter that windows_write_watch_tests reads is now compiled only into that test, through APS5_COUNT_MAPPING_QUERIES, so the runtime's WindowsMappings::query does no atomic update per query.

github-actions · 2026-10-22T01:12:30Z

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request	Conflicting files (lines on `main`)	Same exports	Same new files
PR 11152	`core/libs/prx/libc/include/WindowsMappings.hpp` L135-138, L167-184, L984-988 `core/libs/tests/WindowsWriteWatch.cpp` L549

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request	Shared files not compared
PR 5021	`core/libs/CMakeLists.txt`
PR 7201	`core/libs/CMakeLists.txt`
PR 7275	`core/libs/CMakeLists.txt`
PR 8285	`core/libs/CMakeLists.txt`
PR 8650	`core/libs/CMakeLists.txt`
PR 9156	`core/libs/CMakeLists.txt`
PR 9284	`core/libs/CMakeLists.txt`
PR 9293	`core/libs/CMakeLists.txt`
PR 9297	`core/libs/CMakeLists.txt`
PR 9302	`core/libs/CMakeLists.txt`
PR 9383	`core/libs/CMakeLists.txt`
PR 9535	`core/libs/CMakeLists.txt`
PR 9536	`core/libs/CMakeLists.txt`
PR 9571	`core/libs/CMakeLists.txt`
PR 10255	`core/libs/CMakeLists.txt`

MarcZX10

The performance evidence and hosted product checks are now present and green, so I withdraw the earlier performance-report concern. I verified the current heads directly with git merge-tree: #9505 and #9098 conflict in core/libs/CMakeLists.txt, core/libs/prx/libc/include/WindowsMappings.hpp, and core/libs/tests/WindowsWriteWatch.cpp. In particular the shared APS5_COUNT_MAPPING_QUERIES accessor/counter hunks in WindowsMappings.hpp conflict, so the PR description’s statement that these parts merge cleanly is stale. #9505 and #9159 also conflict in WindowsWriteWatch.cpp. The contribution rule requires author-agreed order for actual shared-line overlaps and Depends on for prerequisites. Please coordinate with #9098 and #9159 authors, update this PR’s overlap account/checklist, and rerun the merge-tree/overlap check after any rebase. The content-level work appears distinct; I’m requesting changes for the inaccurate overlap/order record only. Hosted eligibility, conventions, relinker matrix, ASan and write-watch checks pass; full Linux/Windows product jobs are skipped. I did not run local Windows tests.

EagleBearerYS · 2026-10-22T01:38:00Z

Thanks, @MarcZX10, on both reviews.

a2e0bf1 compiles the VirtualQuery counter only into windows_write_watch_tests, through APS5_COUNT_MAPPING_QUERIES, so the runtime's WindowsMappings::query no longer updates it (run).
perf(libc): read private write watches per allocation without a region query #9098 and fix(agc): keep a rewatched import's memory compared once it is shared #9159 are both mine, so the order is settled: perf(libc): read private write watches per allocation without a region query #9098 goes after this PR, lists it under Depends on and resolves the CMakeLists.txt, WindowsMappings.hpp and WindowsWriteWatch.cpp hunks when it rebases. This PR and fix(agc): keep a rewatched import's memory compared once it is shared #9159 don't wait on each other, and whichever lands second rebases. The Overlap section now says so, and no longer says the counter parts merge cleanly with perf(libc): read private write watches per allocation without a region query #9098.
Thanks for withdrawing the performance-report point. A Resident Evil Village (PPSA01556) capture pair still runs, as extra evidence: its load, on the same base with and without this PR, with the summary lines, a trace for each and the trace_digest.py output. I'll post it here when it's in.

The description also gains the GPUs tested line it was missing.

MarcZX10 · 2026-10-22T02:07:40Z

Thanks for documenting #9098 as a dependent rebase and #9159 as independent. I checked the remaining overlap warnings: #5021 does not change core/libs/CMakeLists.txt; #6044, #7201, #7275, #8285, #8650 and #9224 do add root library test registrations, so the shared CMake insertions need to be preserved on rebase even if the changes are otherwise independent. Please note that plan and verify all registrations remain after resolving the CMake conflicts. One separate repository rule remains: this PR still has title perf(...) and label perf, while its benchmark is a test-only mapping microbenchmark rather than the prescribed same-title/same-scene warm-cache capture with summary lines, attached traces and digests. The Resident Evil Village pair is still described as running. Please add the required report artifacts if retaining the perf title/label, or change the title/scope so it no longer claims a performance change. I did not run tests locally.

MarcZX10

The Windows mapping change is supported by broad author-reported regression coverage and a dedicated mapping benchmark. I am requesting changes for repository policy/evidence: the perf(...) title/label requires valid same-title, same-scene, same-machine warm-cache summary lines, attached base/change traces and tools/trace_digest.py output. The current mapping benchmark does not meet that title-scene report requirement, and the described Resident Evil Village pair has not been posted yet. Please attach the compliant capture report or remove the performance claim/perf labeling. Also preserve the documented #9098/#9159 order and root test registrations during rebases. I did not run tests locally; hosted product build jobs are skipped.

EagleBearerYS · 2026-10-22T02:19:37Z

Thanks, @MarcZX10. The perf title stays for now. A Resident Evil Village (PPSA01556) capture pair is queued on this PC in a matched rotation: the same base with and without this PR, a warm cache, and the same machine and scene. I'll attach both summary lines, a trace for each arm and the trace_digest.py output here. If it shows no measurable effect in that title, the description will cut the claim to the mapping bench, and the title will drop perf. The CMake registrations are in the Overlap section now.

MarcZX10

The #9098/#9159 ordering is now documented, the runtime query counter is test-only, and the CMake registration conflicts are accounted for. I am withdrawing the earlier overlap coordination request. One repository requirement remains: this PR still has a perf(...) title and perf label, but its body still says the PPSA01556 capture pair is queued and includes no summary lines, attached traces or trace_digest.py output. The mapping benchmark does not replace the prescribed same-title, same-scene, same-machine warm-cache report. Please attach that complete capture pair or remove the perf title/claim and revise the description accordingly. No local Windows/title tests were run by me.

JJenkx · 2026-10-22T04:23:14Z

Independent Windows run of this PR, as data only. The machine is a Ryzen 9 5950X on Windows 11 (10.0.22621), built with WinLibs GCC 15.2.0 UCRT r7, Release, BUILD_TESTING=ON. I compared main d9c99c9 with a local merge of your head a2e0bf1 into it (c66a5dbc72, no conflicts).

Focused tests, 3 runs each:

	merge	`main`
`windows_shared_map_cost`	pass 3/3	not on `main`
`windows_write_watch`, `guest_memory`, `guest_jit_shared_memory`, `guest_shm_open`	pass 3/3	pass 3/3
`guest_raise_exception`	timeout 3/3 (30 s)	timeout 2/3

Suites:

	merge	`main`
libc	142/142 pass	not run
libkernel	205/207; `guest_raise_exception` and `guest_raise_in_host_code` time out	run 1: 207/207; run 2: `guest_raise_in_host_code` times out

The libkernel timeouts aren't from this PR: main shows them too, and this PR doesn't touch libkernel. Run outside ctest, guest_raise_exception passes but takes 35.6 s and 37.4 s on this machine. I filed the breakdown as #9610: 14.8 s of it is raises on a thread looping in sceKernelSyncOnAddressWake.

Not tested: a title, or a performance pair.

EagleBearerYS · 2026-10-22T05:39:32Z

@JJenkx thanks for the Windows run on the 5950X, and for tracking guest_raise_exception's time down in #9610. It's in Tested now.

EagleBearerYS · 2026-10-23T01:28:20Z

Thanks, @MarcZX10. The Resident Evil Village (PPSA01556) pair is attached now: the summary lines, a trace per arm and tools/trace_digest.py output, from the same machine, route and warm cache. It shows no measurable effect in its 40 s load window, so the title is now refactor(libc) and the description makes no frame-rate claim. The windows_shared_map_cost table stays as a measurement of the mapping itself.

EagleBearerYS · 2026-10-24T03:11:10Z

@MarcZX10 the Resident Evil Village (PPSA01556) pair is attached with summary lines, traces and trace_digest output, and the PR is now refactor(libc) with no performance claim. Could you re-review?

A private guest commit on Windows (every anonymous kernel map commits with a 16 KiB granule) ran reset() on each page before replacing it: 10 VirtualQuery calls, a placeholder split and the VirtualAlloc2, so 12 calls per page. At the arena's first-fit frontier the placeholder is the arena's free tail, hundreds of GiB. Commit now queries a placeholder once, cuts the range out of it (one split where the range starts inside the placeholder, one where the placeholder extends past the range), and carves the pages from that bounded placeholder: one split and one VirtualAlloc2 per page, and no split for the last page. Each page is still its own allocation with the same protection, write watch and created ranges, so munmap, MAP_FIXED remaps, views and collects see the same layout as before. A piece that would cross the end of its placeholder (a whole-range granule over several placeholders) still takes the coalescing path through reset(), and committed memory in the range still only changes its protection. A zero granule now throws std::invalid_argument before touching the address space. windows_write_watch counts the queries with a compile-time counter that only that test defines (APS5_COUNT_MAPPING_QUERIES, copied from #9505) and checks: 256 fresh pages one page into a 1 GiB placeholder take one query (main's loop makes 2,561), each page is its own allocation and the placeholders around them are intact; the release and recommit of one page; a protection-only commit; the write watch of carved pages; a whole-range granule inside one placeholder and across two; committed pieces and adjacent placeholders inside the range; a partial release and a fixed remap that keep their neighbours; a granule that would cross the end of the placeholder being carved, which takes the coalescing path; a short last piece; a commit that runs to the end of its placeholder; an unwatched commit; and the zero-granule throw. PPSA14363 boot on an RX 6700 XT, idle PC, 4 runs on main and 3 with this change: the first 10 s after the first present go from 546-552 presents with a 578-704 ms frame to 588-594 presents with a 97-117 ms frame. Co-Authored-By: Claude <noreply@anthropic.com>

Dismissing a stale review: the requested changes were addressed after it (new head) and the merge queue blocks on it. Please request changes again if anything is still open.

adriadam10 · 2026-10-25T21:34:27Z

Re-checked on the current main (7a25f87): main only added test registrations to core/libs/CMakeLists.txt next to this PR's if(WIN32) block; no duplicate targets in the merged tree and no code file of this PR changed on main. The head is unchanged since the last review, so the verdict stands. Checked the mapping run split/merge in WindowsMappings.hpp, the new windows_write_watch cases and the registered windows_shared_map_cost bench; the Windows runs (ours and @JJenkx's) cover what we can't run here, and the Resident Evil Village pair shows no measurable effect, so the claim is the mapping bench only. Merged.

thuanquang · 2026-10-26T04:49:34Z

@EagleBearerYS here is a post-merge review of the run views, done by reading main be9effaeb0; nothing was built or run. All names are in WindowsMappings.hpp on main.

Commit over a run view updates only one View per region. In the committed branch, Commit resets views.find(cursor) (protection, armed = 0, invalidate), then calls VirtualProtect(cursor, stop - cursor, ...), where stop is the end of the VirtualQuery region. Before refactor(libc): map a run of shared guest pages as one view on Windows #9505 a shared view was one page, so its region never covered more than one page. A run view's region covers every page once Collect has armed them alike. A single-threaded sequence:
1. Map a 4-page run.
2. Collect(All, clear): all four pages are armed read-only, as one region.
3. Commit(run, 4 pages, PAGE_READWRITE, ...).
4. Store to page 1, then Collect(Armed) over page 1.
Page 1 is now writable, but its View keeps armed set and seen == generation, so the store isn't reported. Guest mprotect reaches this path through SetProtection followed by Commit.
breakView maps the rest of the run PAGE_EXECUTE_READWRITE before it restores the recorded protections. Between mapRun and the VirtualProtect loop, a store from another thread to an armed page of the surviving part doesn't fault, so HandleWrite never runs. The page then goes back to read-only with armed, seen and its clean range unchanged, and Collect never reports it. Mapping both halves PAGE_EXECUTE_READ would close this: writes then fault and wait for the mutex, and the loop raises each part to its recorded protection.
The whole run is a placeholder between breakView's unmap and its mapRun. User-mode loads and stores there fault and retry after the mutex, but kernel-mode copies don't. A ReadFile into another page of the run (sceKernelPread) fails with ERROR_NOACCESS and returns EIO. An instruction fetch from the run in that window doesn't reach the read and write handlers either.
breakView has no rollback. A failure after the unmap leaves the rest of the run unmapped, while views still lists it as mapped.
windows_shared_map_cost can't fail. It is registered with add_test, but its main() returns 0 unconditionally.

I can send fixes with tests for 1 and 2; 1 has a single-threaded red test. Or would you rather take them?

AI-assisted: yes (Claude Code).

EagleBearerYS · 2026-10-26T04:54:41Z

@thuanquang thank you for the careful read. Yes please, send the fixes with tests for 1 and 2; I'll review them, and the single-threaded red for 1 is exactly what it needs. Points 3 to 5 are noted: if you'd like to include any of them as well, that's welcome, otherwise we'll follow up on them.

thuanquang · 2026-10-26T18:35:55Z

@EagleBearerYS both fixes are up: #11344 for point 1, which MarcZX10 has approved, and #11552 for point 2.

A correction to my suggestion for point 2: mapping the halves read-only and raising them afterwards isn't possible. VirtualProtect can't raise a view mapped PAGE_EXECUTE_READ or PAGE_READONLY to read-write (error 87). #11552 therefore compares the armed subpages with a copy taken before the unmap, which is exact. A split of a fully armed run costs more, and the measured table is in its description. Points 3 to 5 aren't addressed.

EagleBearerYS · 2026-10-26T19:02:08Z

@thuanquang thank you for both fixes, and for the correction on point 2; good to know VirtualProtect won't raise a read-only view (error 87). For merge order: our #9098 touches the same two files on other lines, and #9098, #11344 and #11552 merge cleanly with each other in any order.

EagleBearerYS and others added 5 commits October 21, 2026 22:07

github-actions Bot added perf Performance change perf numbers missing perf pull request without a before and after summary line labels Oct 21, 2026

MarcZX10 suggested changes Oct 22, 2026

View reviewed changes

perf(libc): count address-space queries only in the write-watch test

a2e0bf1

The VirtualQuery counter that windows_write_watch_tests reads is now compiled only into that test, through APS5_COUNT_MAPPING_QUERIES, so the runtime's WindowsMappings::query does no atomic update per query.

MarcZX10 suggested changes Oct 22, 2026

View reviewed changes

MarcZX10 previously requested changes Oct 22, 2026

View reviewed changes

github-actions Bot added libc build labels Oct 22, 2026

EagleBearerYS mentioned this pull request Oct 22, 2026

test: organize library tests and label CTest suites #6044

Closed

11 tasks done

EagleBearerYS added a commit to EagleBearerYS/AnyPS5 that referenced this pull request Oct 23, 2026

assets: boykopovar#9505 Resident Evil Village trace pair

0916aca

EagleBearerYS changed the title ~~perf(libc): map a run of shared guest pages as one view on Windows~~ refactor(libc): map a run of shared guest pages as one view on Windows Oct 23, 2026

github-actions Bot removed perf Performance change perf numbers missing perf pull request without a before and after summary line labels Oct 23, 2026

This was referenced Oct 23, 2026

perf(libc): read private write watches per allocation without a region query #9098

Merged

fix(agc): keep a rewatched import's memory compared once it is shared #9159

Merged

thuanquang mentioned this pull request Oct 23, 2026

perf(libc): carve fresh private guest pages out of their placeholder #10070

Merged

12 tasks done

github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 24, 2026

Merge main into port/fable-shared-map-views

5fd76af

github-actions Bot added needs rebase Conflicts with main, rebase needed and removed needs rebase Conflicts with main, rebase needed labels Oct 24, 2026

Merge main into port/fable-shared-map-views

fbf50df

github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 24, 2026

adriadam10 self-assigned this Oct 25, 2026

thuanquang mentioned this pull request Oct 25, 2026

perf(libc): keep no per-page physical state for resident guest pages on Windows #11152

Merged

12 tasks done

adriadam10 merged commit a940685 into boykopovar:main Oct 25, 2026
21 checks passed

This was referenced Oct 26, 2026

fix(libc): reset every page of a run view that Commit reprotects #11344

Merged

fix(libc): keep write tracking when a run of shared guest pages is split #11552

Open

oneandonlydean pushed a commit that referenced this pull request Oct 27, 2026

Merge #9505's main merge into perf/libc-collect-without-region-query

98fd280

oneandonlydean pushed a commit that referenced this pull request Oct 27, 2026

Merge #9505's main merge into perf/libc-collect-without-region-query

bc0971a

Conversation

EagleBearerYS commented Oct 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Tested

Checklist

Uh oh!

github-actions Bot commented Oct 21, 2026

Uh oh!

MarcZX10 left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Oct 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MarcZX10 left a comment

Choose a reason for hiding this comment

Uh oh!

EagleBearerYS commented Oct 22, 2026

Uh oh!

MarcZX10 commented Oct 22, 2026

Uh oh!

MarcZX10 left a comment

Choose a reason for hiding this comment

Uh oh!

EagleBearerYS commented Oct 22, 2026

Uh oh!

MarcZX10 left a comment

Choose a reason for hiding this comment

Uh oh!

JJenkx commented Oct 22, 2026

Uh oh!

EagleBearerYS commented Oct 22, 2026

Uh oh!

EagleBearerYS commented Oct 23, 2026

Uh oh!

EagleBearerYS commented Oct 24, 2026

Uh oh!

Uh oh!

adriadam10 commented Oct 25, 2026

Uh oh!

thuanquang commented Oct 26, 2026

Uh oh!

EagleBearerYS commented Oct 26, 2026

Uh oh!

thuanquang commented Oct 26, 2026

Uh oh!

EagleBearerYS commented Oct 26, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

EagleBearerYS commented Oct 21, 2026 •

edited

Loading

github-actions Bot commented Oct 22, 2026 •

edited

Loading