Repository navigation
perf(agc): settle Windows imports under 1 MiB without waiting - #6678
Merged
oneandonlydean merged 11 commits intoOct 18, 2026
Merged
Conversation
GuestMemory::Rewatch takes ranges that left the write watch back into it. Only whole tracker blocks whose pages can be collected return: their write-watch marks are collected and dropped, and every block is stamped as a CPU store with a new generation, so a snapshot or a proof made before the block left the watch reads as changed, and the blocks' untracked hints stop applying. It does nothing outside Windows.
Since boykopovar#6060 a Windows host import of private write-watched memory stays in the write watch only after its settle, a 4-byte GPU read that the importing thread waits for, and only imports of 1 MiB or more are settled: SuperTuxKart imports a 64 KiB buffer around every submission, and the wait cost it about 19% of its frame rate. A smaller import is compared instead, and its range leaves the write watch for good. Resident Evil Village maps its GPU heap in 64 KiB pieces, so every texture over that heap is compared with guest memory. An import under 1 MiB is still compared when it is made, but its read is now queued. The next import lookup that holds the device lock records the reads queued since into one batch on the settle queue and submits it, so the thread that maps and imports neither submits nor waits. Once the batch's fence has signalled, a later lookup returns the imports' blocks to the write watch, except where storage results are pending over them, which waits for a later lookup. An import retired while its read is in flight waits for that batch first. A settled import whose range had already left the write watch, such as a whole range imported over a piece that was compared, returns to it the same way.
ARCHITECTURE.md and the debt entry describe how imports under 1 MiB are settled without waiting and return to the write watch, which the debt entry listed as not implemented.
|
Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.
Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.
|
11 tasks done
Retiring a Windows import whose settle read was still running waited for the read's batch under the imports mutex, so lookups on every thread waited with it. The retired import is now kept with that batch and freed once the batch has completed. Only a guest unmap of its memory still waits for the batch, in the pin waiter, which waits for the main queue there anyway. The [buffers] profile line counts these retirements and the unmaps that waited, apart from the imports of 1 MiB or more settled inside their window, which the old wait counter mixed with the retirement waits. agc_import_settle_retire replaces a 64 KiB window import with the import of its whole range, 64 times, right after its settle read was submitted. No replacement may wait, at least one must find the read running, and the last import must return to the write watch.
…mport A Windows import under 1 MiB returns to the write watch once its settle read has completed, though main-queue work submitted before the read may have used the import first, and an earlier import of the same pages may not have run its first use yet. On the NVIDIA driver (617.42) neither marks the pages once the read has completed, so the rewatch needs no wait for the main queue. The docs said the marking lasts until the import's first GPU use completes; they now say that a completed GPU use ends it, the settle read included. agc_import_settle_first_use submits 4096 copies of a 64 KiB import on the main queue before its settle read is queued. The import must return to the write watch while the copies still run, and the driver must mark none of its pages afterwards. agc_import_settle_earlier_import records 4096 copies of a 64 KiB window import, replaces it with the import of its whole 128 KiB range, and submits the copies only once the range is back in the write watch. The driver must mark none of the range's pages while they run. A driver that keeps marking fails either test.
12 tasks done
This was referenced Oct 16, 2026
|
Data point from Metal Gear Solid 4 (PPSA18194, Master Collection) on Windows 11, RTX 3080 (NVIDIA 610.74), with this PR picked onto main 5d9dcc4 plus #6188, #6192/#6187, #6273 and #6292.
AI-assisted (Claude Code); numbers from runs on my machine with |
…first The first-use check needs the main-queue copies of an import to be still in flight when the import returns to the write watch. On a fast device they can finish first, and the test then failed although the driver did nothing wrong. Retry with 4 and 16 times as many copies, and report the case as not tested when the copies finish first every time.
Contributor
Author
|
@ps54all thanks for trying it on Metal Gear Solid 4 (PPSA18194), and for the numbers.
|
11 tasks done
Contributor
|
A Windows NVIDIA data point, and one that goes the other way: RTX 3060 12 GB (NVIDIA 617.42), i5-12400F, Windows 10 19045, MinGW GCC Release. Minecraft Dungeons II (PPSA16064 01.100.100), title screen, no input, 200 s rows,
The idle gaps shrink as intended (-10 ms per frame), but GPU busy grows by 13 ms per frame, so the frame rate drops a little. The command classes are unchanged (draw 32.3 / 31.5 ms, storage write-back 17.4 / 17.2 ms per present); the growth is all in the title's compute dispatches. By rank (the program addresses move between runs), both rounds:
So the heaviest dispatches run 1.6-2.4x longer on the GPU when the sub-1 MiB settles no longer wait. My guess is that those dispatches now read buffers that are still on the slower path because their import wasn't settled yet when they were recorded; a AI-assisted: yes (Claude Code) |
Contributor
Author
|
@parl0v thank you, this is the cost side we hadn't measured: the idle gaps shrink by 10 ms per frame as intended, but the heaviest compute dispatches take 1.6–2.4× longer, so the frame rate drops a little on the 3060. We'll look at where those dispatches' buffers come from with and without this PR, a settled import read from host memory or a device copy, with an import breakdown, and report back here. |
Contributor
Author
|
Thanks for the numbers, @parl0v. From the code, #6678 doesn't change which buffer a read binds. An import under 1 MiB is read in place from host memory on both trees; main creates it too, unwatched. What #6678 changes is the write tracking. Once its settle read lands, the import is back in the write watch, and watched memory takes the unit-shadow path ( Could you run #6678 once more with Two lines from each tree, over the same window, would also help:
If your per-program lines have dispatch counts too, the per-dispatch time would rule out a change in how often they run. |
Main's boykopovar#5905 settles Linux dma-buf imports with settleImport, which is now built on both platforms. The batched settle of Windows imports under 1 MiB, its counters and adoptSettleDevice's batch reset stay Windows-only, and ClearHostImports drains the batches only there.
…ERRED_SETTLES The deferred settle of small Windows imports could only be turned off through the test hook. The variable turns it off for a run, so a title can be compared with and without it on one build; unset, nothing changes.
Contributor
Author
EagleBearerYS
added a commit
to EagleBearerYS/AnyPS5
that referenced
this pull request
Oct 17, 2026
…a pin wait The Windows release of imports retired during a settle read belongs after the whole overlapsImport block in the not-holding-the-lock branch of WaitForLeasesAndImports, as in the original change, not inside it. It was placed inside while main was merged in.
This was referenced Oct 18, 2026
Merged
11 tasks done
This was referenced Oct 18, 2026
12 tasks done
Scramblrrrr
pushed a commit
to Scramblrrrr/AnyPS5
that referenced
this pull request
Oct 18, 2026
…ps it Since boykopovar#6678, a Windows host import under 1 MiB returns to the write watch once its deferred settle read has completed, and GuestMemory::Rewatch stamps every block of it as a CPU store. The return waited only while storage results were pending over the import. A storage image can take its generation before the stamp and have results pending only after it: an image's first upload takes its generation and then imports its block, a recorded draw marks its targets pending once it is recorded, and the settle is polled at the import lookups between the two. The image's next refresh then takes the stamp for a CPU store over its results, drops them and uploads the guest bytes again, so the GPU's output is lost. On an RTX 3070 this also splits the first two draws of agc_driver_address_passes into two passes. A settled import that a live storage image overlaps now stays compared, as every import under 1 MiB was before boykopovar#6678. Imports with no image over them still return to the write watch. Keeping the import compared, rather than waiting for the image to go, checks each settled import against the live images once instead of at every poll. The [buffers] line counts the imports kept compared. agc_import_settle_storage_image makes an image whose first upload imports its block, waits for the settle read, clears the image on the GPU, then refreshes and stores it. On the old code the refresh drops the clear.
13 tasks done
12 tasks done
12 tasks done
adriadam10
pushed a commit
that referenced
this pull request
Oct 24, 2026
Since #6678 a Windows host import under 1 MiB returns to the write watch when its deferred settle completes (rewatchSettledImports). Grand Theft Auto V (PPSA04264) imports buffers that are resident direct memory at that point and later become shared 16 KiB views, when a second mapping of the same direct memory demotes them. From then on the write watch arms them, every guest store into them faults and is resumed, and the resumed faults damage the red zone the game's zlib inflate_fast reloads its pointers from, so the open world crashes in every run (#9136). Before #6678 such a range stayed out of the write watch for good and was compared. Rewatch now marks each range it returns to the watch with GuestArenaCompareWhenShared first. While the memory stays resident it is write-watched, as #6678 intends. Once it is shared its collects fail and it is compared again, as before #6678, instead of being armed. agc_rewatched_direct_shared maps direct memory resident, takes a block out of the watch and rewatches it as a settle does, then maps the same memory a second time. The rewatched block must collect while resident, then collect as 0 through either mapping, stay unarmed and show the store through both, while a block that never left the watch is still collected.
10 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
This implements the non-blocking settle that the debt entry for pages under an import window lists as missing. The entry (23fccd7, merged with #6060) ends: "Settling without blocking the importing thread (submit the read, keep the import compared until its fence signals, then switch it to the write watch) is not implemented."
Why. Since #6060, a Windows host import of private write-watched memory stays in the write watch only after its settle, a 4-byte GPU read that the importing thread waits for. Only imports of 1 MiB or more are settled (23fccd7): SuperTuxKart (PPSA69421) maps, imports and unmaps a 64 KiB buffer around every submission, and waiting for each settle cost it about 19% of its frame rate (Dripl1x, on #6060). A smaller import is compared instead, and its range leaves the write watch for good.
Resident Evil Village (PPSA01556) maps its GPU heap in 64 KiB pieces of resident direct memory. In a profiled boot to the menu, 12,628 of the 14,881 imports were made at map time, in such pieces. So on
main's settle code that whole heap is compared, and so is every texture sampled from it.APS5_PROFILE_DRAW=1, per 10 s:[textures]kept 16.5 M untracked snapshots by their hint and compared 283,785, and[guestmem]compared 33 GB in 3.4 s.cachedTexture's snapshot compare.The game's menu, unprofiled, as the median of the 60-s intervals 5–15 min after the menu stands, on our Resident Evil Village test tree (
mainplus the open fixes the title needs, ours and others'), Windows 11, RTX 3070 Laptop:main'smain's settle code. The PR's row ran b62a057, the revision before this head. The head only adds theSetHostImportDeferredSettlestest hook, which is on by default, and tests, so it runs the same code.main's settle code.#5692 tracks the same comparison cost on other titles. This PR covers the part of it that comes from imports under 1 MiB. I don't know whether the titles there make such imports, so it's listed as related.
How:
GpuMutexwhile the imports match the registry records the reads queued since into one command buffer on the settle queue (up to 256 reads per batch, four batches in flight) and submits it. The thread that maps and imports neither submits nor waits.GuestMemory::Rewatch, the first commit, collects and drops the blocks' write-watch marks and stamps every block as a CPU store with a new generation. So anything derived from the range while it was compared (a snapshot without a generation, a proof by its untracked hint) reads as changed once.Rewatchthat refuses a page, which happens only for pages an import wouldn't have pinned.SetHostImportDeferredSettles(false), a test hook likeSetHostImportBudget, keeps imports under 1 MiB compared as before.agc_driver_dcc_keys_first_write's hinted-import case uses it, because that case tests the untracked hint of an import that stays compared. That still happens on a device without a settle queue, and until the read completes.APS5_NO_DEFERRED_SETTLES(any value) does the same for a run (5f822a5), so a title can be compared with and without the deferral on one build.SuperTuxKart (PPSA69421) stays whole. Nothing that maps, imports or unmaps its per-submission buffer waits for a settle:
GpuMutex, only queues its read. Reads go out only from a lookup underGpuMutex.WaitForLeasesAndImports), as onmain. A read still queued is dropped. One already submitted has almost always finished by then, and the retire checks its fence before it would wait.Why not #4448's
GuestMemory::Rewatch. On Windows its body is empty: it re-registers the range with Linux's write watch, for imports retired after sixty idle frames. This PR'sRewatchis the Windows re-include, by whole tracker blocks. The two take different arguments, so they would sit side by side as overloads.The commits:
feat(agc):GuestMemory::Rewatch, with a write-tracking case.perf(agc): the settle, with its tests. It also keepsagc_driver_dcc_keys_first_write's hinted-import case on a compared import.docs(agc): ARCHITECTURE.md and the debt entry.perf(agc): an import retired during its settle read is freed after the read instead of waiting for it. The[buffers]line counts settle reads submitted, back in the write watch, retired during their read, the unmaps that waited for one, and pending.test(agc): the two canaries for the driver's marking, and the docs to match.Overlap.
fix(agc): settle dma-buf imports on Linux on the settle queue #5905 (hugodearaujo, merged) settles dma-buf imports on Linux on the settle queue. This PR's merge of
main(c7e07ca) keeps both:settleImportbuilds on both platforms, and the batched settle of Windows imports under 1 MiB and its counters stay Windows-only. The paragraph and the debt entry keep both texts.fix(agc): retain import settle resources per device #7269 (JJenkx, merged) holds the synchronous settle's buffer, memory and fence per device in
ImportSettle. This PR's merge ofmain(8f03535) keys its settle batches the same way: each device'sImportSettlealso holds its batch target, memory, batches and their fences, so a device switch keeps every device's batches, andClearHostImportswaits for and destroys only that device's. A submitted settle records its device, and the per-import map ispendingSettles, clear of fix(agc): retain import settle resources per device #7269'ssettles.perf(agc): import adjacent host ranges as aligned blocks and stop re-trying budget refusals #6457 and perf(agc): take the host import budget from the device's memory budget #6479 (mine) conflict with this PR in
GuestBufferMemory.cpp, at four places:importRunAt's andimportAllocation's signatures;Each resolution keeps both sides, as already done in my Resident Evil Village test tree. Whichever PR lands second takes it.
feat(agc): import shared ranges whose backing offset is not 64 KiB aligned #5390 shares the debt file.
fix(agc): keep half the host heap out of the host import budget #6472 (ahir061) adds
HostImportBudgetnext to this PR'sSetHostImportDeferredSettles, inGuestBufferMemory.hppand.cpp. Whichever lands second keeps both.perf(agc): keep a host import across a split of its guest range #6499 (parl0v) also retires its trimmed imports when
refreshImportsfinds the device changed. This PR drains the pending settles there first, so whichever lands second keepsdrainSettles(state)ahead of perf(agc): keep a host import across a split of its guest range #6499's loop over both maps.fix(libSceAgcDriver): refresh heap mirrors when the range is not write-watched #6546 (mariuszkreft) refreshes a heap mirror conservatively where its range has left the write watch. On Windows an import under 1 MiB is one way a range leaves it. With this PR such ranges come back after their settle, so that fallback is taken less often. The two change different functions in
GuestBufferMemory.cppand merge cleanly.perf(agc): fast revalidation over untracked memory proves a surface by its stamp until the next ordering point #6192 (TheAggie) makes revalidation over untracked memory cheaper, and fix(agc): run the unwatch hook when a watched import leaves the write watch #6308 (LeandroLP23) runs the unwatch hook in
ImportWatched's fallback. Both merge cleanly with this one, and both keep mattering for what stays compared.fix(agc): import committed spans of partially accessible windows #6918 (Scramblrrrr, draft) clips a partly committed import window to the aligned committed span that holds the request. It merges cleanly with this PR. A clipped window is aligned to the device's import alignment, not to 64 KiB, so one that isn't whole 64 KiB blocks stays compared, like any import this PR doesn't settle.
Related: #6060, #5851, #4448, #6192, #6308, #6546, #5692.
Tested
agc_write_tracking(CheckRewatch): a rewatched block is watched and collected again, reads as changed since before it left and as CPU-stored, its untracked hint stops applying, and the blocks around it keep their stamps. Part of a block, and a block with uncommitted pages, stay out.agc_import_settle_deferredandagc_import_settle_deferred_other_family(Windows, a Vulkan device with a settle queue):main'sGuestBufferMemory.cpp, without their counter checks, fail with "(d) a 64 KiB import did not return to the write watch after its settle".agc_write_tracking,agc_import_watchandagc_import_watch_other_family.agc_driver_host_import_windows,_at_map,agc_host_import_retire,agc_driver_dcc_keys_first_write,agc_driver_layered_volume_submit,agc_driver_multisample_fill_host_only,agc_driver_store_over_untrackedandagc_storage_cpu_snapshot.agc_import_settle_retire("64 while the read ran, 0 waits, 0 unmaps waited for a read"; with retirement waiting for the read again it fails 3 of 3, "0 while the read ran, 64 waits" and "replacing an import during its settle read waited for the read"), and the canariesagc_import_settle_first_use(back in the write watch 21.7 to 30.4 ms after 4,096 main-queue copies of the import were submitted, while they still ran, with no driver marks afterwards) andagc_import_settle_earlier_import(an earlier import's first use runs only after the range is back in the watch, with no marks). The canaries have no red by design: they fail only on a driver that keeps marking. The other settle and watch tests pass 2 of 2.main66e290f with this PR, perf(agc): share one image table per heap and write epoch through capture, compare, resolution and descriptors #6454 and fix(agc): keep a compute data hit's live words with the regions that span them #6911 merged (the other two don't touch this path):agc_import_settle_first_usepasses 10 of 10, racing in its first round (back in the write watch 26.7 ms after 4,096 main-queue copies, which were reaped by 47.9 ms; no driver marks after its return).agc_driver_store_over_untrackedsometimes fails onmainon this PC too ("the result was never held pending").^agctests (run; test(agc): align the indirect arguments block at run time #6452's test fix applied there only, becausemaindoesn't build in full with MinGW since fix(agc): keep the gate run going past indirect dispatches whose arguments stay on the GPU #5747):agc_driver_pm4_replay_tool(0xc0000139) fails onmainon that runner too.agc_driver_flip_computeprinted "AGC flip and suspend tests passed" and then timed out at exit. That's the pattern of the exit hang fix(agc): stop and join the driver's thread pools at shutdown #6641 fixes. In 20 more runs it passed every time, on this branch alone (run) and with fix(agc): stop and join the driver's thread pools at shutdown #6641's commits added (run).guest_kqueueandprecise_waitin the libc and libkernel suites, which fail onmainon those runners too.guest_kqueueon Windows, a kqueue test this PR doesn't touch.HostImportSettles, which returns zeros there:guest_datagram_*tests andguest_tcp: that machine has no IPv6, and they fail the same way onmain.agc_write_trackingandagc_import_watchpass 20 of 20.mainonly byGuestMemory::Rewatch, which is empty outside Windows, and bysettleImport's device setup, which ran the same steps fromadoptSettleDevice; since the merge ofmain98c251f that setup issettleFor, with the same steps. Everything else this PR adds is inside#ifdef _WIN32.^agcpasses 606 of 606 (43 skip), and the preprocessed Linux sources change only by two counter fields.main's settle code ran 0.029 fps there (one earlier run at control). All four runs fill the 6400 MiB import budget in the scene. With this PR the budget fills during the load, about 100 s before control, because imports under 1 MiB no longer wait; with synchronous settling it fills about 100 s after control, once the scene has imported its ranges. While the budget is full, every import an address-based build maps is stamped as recently used, so none of them can be evicted, and a range first needed after that, here one build import at 0x2372400000 (512 KiB to 1 MiB), is copied on the CPU at each use. That comes from the import budget's eviction, not from this PR's settle.main; it carries the open fixes the title needs) with fix(agc): stamp the watched blocks of a store into partly watched memory #6455 and this PR, Windows 11, RTX 3070 Laptop, warm shader cache, noAPS5_PROFILE_*(the trace summary on). The meadow save is loaded from the title menu, then the player stands still; each value is the median of the five 60-s intervals from 180 to 480 s after the first present. One build, run OFF (APS5_NO_DEFERRED_SETTLES=1, the only difference between the launches), ON, then OFF again:gpu_busy_ms_per_present): 50.3, 54.4 and 44.0 ms. ON is at the high end, but inside the 6.3 ms between the OFF runs, so it can't be told apart from run-to-run spread. parl0v's RTX 3060 rows below go the same direction.APS5_PROFILE_DRAW=1andAPS5_ADDRESS_COPY_MAX_MIB=8192for 90 s (comment):main3b10c7e it aborts after 25 s witha heap mirror's range is no longer write-watched.main47a266c with fix(agc): stamp the watched blocks of a store into partly watched memory #6455 and this PR merged on top (comment). In game at the start of Act 1, 60 s of frames standing still from 120 s after Continue, no profiler, one run per build (noise about ±5 %):main: 3.63 fps. With fix(agc): stamp the watched blocks of a store into partly watched memory #6455 and this PR: 8.12 fps.mainwith the settle floor lowered from 1 MiB to 64 KiB (synchronous settles, a local control): 8.45 fps.mainthe render targets' DCC key ranges sit in compared imports: none of about 400,000 clear-key proofs per 10 s is reused (CollectWritesreturns 0 for each),[dcc]reports about 570,000 key scans (78 GiB) per 10 s, and 15 % of the draw committer is inAllKeysEqual. With this PR, 33,207 proofs per 10 s are reused.APS5_PROFILE_DRAW=1 APS5_PROFILE_GPU=1,main93e54ce againstmainwith this PR at 141d605, two rounds (comment):main, 6.50 and 6.53 with this PR (−1.4 %).main.APS5_NO_UNIT_SHADOW=1, withAPS5_NO_DEFERRED_SETTLES=1, and with the settle-batching commits.APS5_TRACE_SECONDS=60:30). The base ismain84b0d2f with fix(agc): import native multisample depth whose HTILE is not ITERATE_256 #6557, fix(agc): retire an unwritten native multisample depth surface on guest access #6727 and fix(agc): replace with a stencil operation value the pixel shader exports #6762, 6 runs (3 before and 3 after this PR's); this PR is merged on top of it, 3 runs (comment):Driver.Packet.memory_waitper frame, in run 1 of each: 107.0 ms on the base, 78.5 ms with this PR. GPU busy per present is unchanged at 27.4 to 27.6 ms (GPU clocks weren't reported).main1b1f848 (c7e07ca, with 5f822a5), on GitHub's runners (run): the settle, import and write-tracking tests and fix(agc): settle dma-buf imports on Linux on the settle queue #5905'sagc_driver_sparse_dma_buf_importpass 3 of 3 on Linux (45 tests) and Windows (53, of which 50 skip without Vulkan). The run shows as failed because the Windows libkernel suite'sguest_kqueuefails on that runner with and without this PR (nomaincomparison was run).main98c251f (8f03535, the per-device settle batches after fix(agc): retain import settle resources per device #7269):agc_import_settle_deferred,agc_import_settle_deferred_other_family,agc_import_settle_retire,agc_import_settle_first_use,agc_write_tracking,agc_import_watch,agc_import_watch_other_family,agc_driver_host_import_windows,agc_host_import_retireandagc_storage_cpu_snapshotpass 5 runs in a row each, with none skipped.^agctests pass 3 runs in a row on Linux. On Windows, where the settle tests skip without a Vulkan device, the only failures areagc_driver_pm4_replay_toolandagc_shader_replay_tool, which fail the same way on that runner for an unrelated branch (run).python3 tools/check_conventions.py --base origin/main: passed.Checklist
main5d9dcc4, merged withmain1b1f848; no other open PR implements the same functionsNotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file.mdfiles or images added to the repository (attach them to this PR)