[SleglJan]

What

A descriptor heap that a shader binds whole while only the pieces from its base are mapped (a sparse heap) is copied on the CPU for every build that binds it, unless a mirror serves it. The sparse heap mirror (#3167) takes a heap whose needed extent lies in one releasable registered range. A heap the title maps piece by piece, each mapping its own allocation and the pieces back to back from the heap's base, is several registered ranges, so every build copied its mapped run again.

Such a region is now bound in place from an import of that run: an import that already covers it, the run's own import, or one joined over exactly the run (#4517).

• Scope: any sparse heap no mirror took, including under APS5_NO_LEASE_MIRROR or APS5_NO_SPARSE_HEAP_MIRROR.
• Growth: a join is made only for a region that starts at a registered range, once the previous bind of that address saw the same last mapped byte. A heap that grows between every two of its binds keeps the copy; one bound more often than it grows is joined again after each growth, as a buffer bound exactly over the run is on main since perf(agc-driver): join the imports of adjacent ranges of any size that a buffer is bound exactly over #4517. Joins stop after 64 per joined run base, with one message; from then on the heap is bound in place only while an existing import covers its run, and copied otherwise. The records go when their range does. The host-imports debt entry says so.
• Range: the descriptor and BDA ranges end at the import's end, which covers the mapped run and the last view, as the copy does (fix(agc): copy a sparse descriptor heap only up to its last backed byte #2992).
• Left to the copy: a region whose mapped part does not start at its base, with a view past the run, written by its shader, or whose run is not imported, or imported at an offset not bindable in place.
• Unchanged: a direct region whose import ends before it outside this path still fails in Descriptor, as on main.

A set over such a region stays reusable in the resource cache:

• DirectRegions and InPlaceReads name the region up to its import's end, so the import's serial proves it. Without that, a set over a region bound in place would stay unreusable, as one over a copied region is on main, because no import covers the whole region.
• A reuse also requires the heap to be mapped exactly up to the cut (CutRegions). That check runs whenever the imports or the registry changed, which every mapping does, so a mapping past the cut makes the set miss instead of reading zeros through it.
• DirectRegions refuses a set any of whose regions bound in place no longer holds the import found at its base (a later join in the same build retired it). This applies to every region bound in place.

There is no switch: the fallback is the existing copy path for every case it does not take.

Tested

Linux (CachyOS, kernel 7.2.9), GCC 16.2.1, Release. Windows was not built or run.

• New test sparseRunImportTests in the recorder tests, as agc_sparse_run_import and agc_sparse_run_import_copied. Three 64 KiB allocations mapped back to back at the start of a 512 KiB range whose tail is unmapped:

  • the first bind copies (the run has not been seen yet), the second is bound in place from the joined import, with the view and BDA range cut at the third allocation's end, DirectRegions, InPlaceReads and CutRegions naming the cut, and a GPU copy through the view returning the guest bytes;
  • a written binding and a binding with a view past the run are copied;
  • a compute set over the heap is reusable and proves its first reuse; after a fourth allocation is mapped behind the run, its next proof fails, the next bind copies and the one after it is joined over all four;
  • agc_sparse_run_import_copied: once the run has been seen, a bind with a view past the run is copied, nothing is read in place, and a GPU copy through the view returns the guest's mapped bytes.
  • When the device refuses to import the test's memory, the case exits 77 (skipped).

• Red/green on lavapipe (Mesa llvmpipe): the in-place test fails on this branch with the binding removed ("a sparse run heap is not bound in place from its run's import"), with the cut reverted (Descriptor refuses the view), with DirectRegions/InPlaceReads naming the whole region ("… is not named up to its import's end for reuse"), without the reuse check of the mapped extent ("a set over a sparse run heap mapped past its cut was reused") and without the wait for an unchanged run ("a sparse run heap was joined the first time its run was seen"). It passes with the change. On a4d8c29, with the last view ignored (neededEnd set to the committed end), both tests fail ("a sparse run heap with a view past its run is bound in place", "… past its stable run is bound in place"); the five earlier reds were run on the in-place test before the switch was dropped.

• Suite on lavapipe, ctest -j 8 --timeout 300, on an earlier base, main 0f2683c: 1035 of 1046 pass with the change and 1032 of 1044 without. The same 11 fail on both in that container (hardware_oracle, guest_syslog, agc_driver_depth_compare_copy, autorun_exit_status, guest_none_relocations, guest_module_args and five linux_* executable tests); guest_audio_in_capture aborted once on main under the parallel run and passes alone on both. On main d1b3aa1, which has perf(agc-driver): join the imports of adjacent ranges of any size that a buffer is bound exactly over #4517 and feat(agc): host import windows, least-recent eviction under the budget #290, the two new tests and 494 of 496 agc_ tests pass on 9c8fd76, and 492 of 494 on main alone; the same two fail on both in that container: agc_driver_depth_compare_copy and agc_driver_bc_block_copy (the guest block is not write-watched there). a4d8c29 (on main a5f493f, the switch dropped) gives the same: both new tests pass and 494 of 496 agc_ with the same two failing. tools/check_conventions.py --base upstream/main and tools/check_test_registration.py pass.

• Title: 007 First Light (PPSA11386), NVIDIA GeForce RTX 3080 (615.71.09). The title does not reach the scene on main alone, so the numbers come from one build of our 007 integration stack (upstream/main 3e0e01f of 2026-10-12 plus 164 commits of our 007 integration work, including cross-tests of feat(agc): implement image and sampler tables indexed at runtime offsets #757, feat(agc): implement image tables loaded through 64-bit pointers #758, perf(agc): pipeline queue-0 draw commits behind the packet worker #1201, perf(agc-driver): join the imports of adjacent large ranges when a buffer is bound across them #4515, perf(agc-driver): join the imports of adjacent ranges of any size that a buffer is bound exactly over #4517, fix(agc): wait for recorded writes over a sampled texture's DCC keys before reading them #4554, fix(agc): preserve storage image dependencies during ahead dispatch capture #4724 and fix(agc): store small pending storage results at each flip #4841 (perf(agc-driver): join the imports of adjacent large ranges when a buffer is bound across them #4515, perf(agc-driver): join the imports of adjacent ranges of any size that a buffer is bound exactly over #4517 and fix(agc): wait for recorded writes over a sampled texture's DCC keys before reading them #4554 have merged since and fix(agc): preserve storage image dependencies during ahead dispatch capture #4724 was closed)), whose version of this change has this PR's behaviour behind the stack's own opt-in switch APS5_SPARSE_RUN_IMPORT and prints counters of what it did. In the run with the change on, binds waited for an unchanged run in 10 of the 142 ten-second reports (1 to 5 binds each: four before the cinematic, five in its first 35 frames and one later), and no heap reached the join limit. With that switch off and on, from the same save, warm caches, 1500 s each; neither run crashed:

  	change on	change off
  menu, frames 450–700, median FPS	2.44	2.36
  in-engine cinematic, first 150 frames, median FPS	0.57	0.24
  cinematic, frames 150–237 into it (the run without the change ended there), median FPS	0.43	0.22
  frame reached at 1500 s	1771	1609
  sparse heap regions copied on the CPU per 10 s in the cinematic (median)	4	about 1,140
  sparse heap regions bound in place per 10 s in the cinematic (median)	about 1,970	0

  • The copied regions are the title's two descriptor heaps, each a 512 MiB view over 6–42 MiB of mapped pieces: about 30 GiB copied per 10 s without the change.

  • In each run's 60 s trace window (900–960 s: frames 248–263 of the cinematic with the change, 126–139 without), a resource build (Resources.Complete, all threads) took 1.8 ms on average with the change against 4.9 ms without; builds per frame were about equal (619 against 570). The trace with the change dropped events: queue worker 0x0 reached its event limit at 50.9 s, so its window ends at the last present before that. The windows are at different frames, so the runs' summary lines are not a before/after pair (p50 frame time 4,598 ms with the change, 4,199 ms without).

  • Later stretches of the cinematic vary a lot between runs; on earlier builds of our integration line without this change, the first 150 frames sat at 0.20–0.24.

  • Capture: 3840x2160. Both runs set further APS5_TRACE_* switches, APS5_PROFILE_VRAM=1 and APS5_DUMP_SHADERS=1, and redirected the log, unlike the Performance report form; their environments differ only in the integration stack's switch for this change.

  • The integration stack also carries a local change that keeps draws over copied read-only buffers reusable (sparse heaps included), so the column without this change is better than main's copy path for these heaps.

  • Summary lines, with env reduced to the integration stack's switch for this change (the traces are attached as aps5-trace-on.json.gz and aps5-trace-off.json.gz):

    With the change:

    {"commit":"d5422af08d7e","dirty":false,"env":{"APS5_SPARSE_RUN_IMPORT":"1"},"gpu":"NVIDIA GeForce RTX 3080","driver":"NVIDIA 615.71.09","os":"Linux 7.2.9-1-cachyos","resolution":"3840x2160","window":{"spec":"seconds 900:60","first_present":1629,"ms":62328.397},"presents":16,"frame_ms":{"p50":4598.014,"p90":5235.786,"p99":5268.656,"max":5268.656},"gpu_busy_ms_per_present":571.314,"worker_busy_pct":{"queue worker 0x0":99.936,"queue worker 0x20":99.941,"queue worker 0x38":99.986,"queue worker 0x28":98.878,"queue worker 0x30":99.98},"top_phases_ms_per_frame":{"Driver.Packet":19384.613,"Driver.Packet.memory_wait":15189.661,"Driver.Packet.draw":3108.319,"Driver.Draw":3107.283,"Graphics.Draw":2830.808,"Graphics.Draw.draw_and_resource_release":2770.853,"Committer.Commit":2644.087,"Driver.Draw.draw_enqueued":1842.702},"trace_dropped":true,"trace":"aps5-trace-on.json.gz"}

    Without the change:

    {"commit":"d5422af08d7e","dirty":false,"env":{},"gpu":"NVIDIA GeForce RTX 3080","driver":"NVIDIA 615.71.09","os":"Linux 7.2.9-1-cachyos","resolution":"3840x2160","window":{"spec":"seconds 900:60","first_present":1498,"ms":60806.67},"presents":14,"frame_ms":{"p50":4199.032,"p90":4999.966,"p99":5253.996,"max":5253.996},"gpu_busy_ms_per_present":634.232,"worker_busy_pct":{"queue worker 0x0":99.929,"queue worker 0x20":99.797,"queue worker 0x38":99.981,"queue worker 0x28":99.982,"queue worker 0x30":99.989},"top_phases_ms_per_frame":{"Driver.Packet":21644.861,"Driver.Packet.memory_wait":16467.943,"Resources.Complete":2785.334,"Driver.Packet.dispatch":2565.201,"Driver.Dispatch":2563.671,"Driver.Packet.draw":2380.272,"Driver.Draw":2379.792,"Graphics.Draw":2348.481},"trace_dropped":false,"trace":"aps5-trace-off.json.gz"}

InPlaceReads sits next to #4978's SplitInPlaceReads. At #4978's current head a6970f1 the two conflict only on the if(WIN32) PATH list line in libSceAgcDriver/CMakeLists.txt, where #4978 adds agc_recorder_read_list and this PR its two tests. SplitInPlaceReads names its own regions up to region.end; whichever merges second should end them at inPlaceEnd(region), as InPlaceReads does here.

Checklist

• Based on current main; no other open PR implements the same functions
• One topic per PR; follow-ups go in a new PR
• No comments in code except technical debt
• Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
• No notes, investigation .md files or images added to the repository (attach them to this PR)
• New third-party code is a submodule built from source
• Generic behaviour, not specific to one title
• Depends on: none
• Unblocks: none
• GPUs tested: lavapipe (Mesa 26.2.4, LLVM 23.1.1), Arch Linux container, kernel 6.12.91: the suite, the red/green and both new tests, and on a4d8c29 both new tests, the new red and agc_, on d34564b (its rebase onto main 3e577ed; only the if(WIN32) test list merged) both new tests; NVIDIA GeForce RTX 3080, driver 615.71.09, Linux (CachyOS, kernel 7.2.9): at 510f926, before the rebase onto feat(agc): host import windows, least-recent eviction under the budget #290 and the switch removal, both new tests passed and no agc_ test failed (487 run, agc_driver_stencil_ref_export skipped itself), and the title runs on our integration stack with this change behind that stack's opt-in switch
• AI-assisted: yes (Claude Code)

🤖 Generated with Claude Code

https://claude.ai/code/session_01Fnrk5Dazjb2tLooVSpdRQq

[SleglJan]

aps5-trace-off.json.gz
aps5-trace-on.json.gz

[SleglJan]

@MichonGoddijn231849 The overlap check lists #290. Every line except GuestBufferMemory.cpp L3015-3017 is #4515's and #4517's, which this PR carries. This PR's own change adds one condition to the import lookup in UploadFinish there, which #290 makes windowed. Neither needs the other. This PR merges after #4517 either way; the order of #290 and #4515/#4517 is for you and @AlxcBTW. If #290 lands first, I rebase onto it and add it to Depends on. After that rebase, a joined run import that #290's budget evicts is joined again at the next bind and counts toward this PR's limit of 64 joins per base; I'll check that case then.

AI-assisted: yes (Claude Code).

[Scramblrrrr]

Windows cross-test of the sparse-run change: author-preserved integration281506e81 plus older-layout CTest correction693182a5b65dde056084f6e6dad0b5199903ad96, https://github.com/Scramblrrrr/AnyPS5/tree/codex/first-light-sparse-run-import . Reviewed rebased headc75fac9a: sparse-run production differs fromc6e only by whitespace. Our stack already carries4515/4517 plus window-budget integrationc990fc176; it preserves non-sparse importAccess and the separate copied-read reuse checks.

RTX5070Ti /Windows /GCC15.2 /Vulkan required: final8/8 controls PASS7.76s, including imported and opt-out copied sparse runs, mapped-run growth/reusable cut, import unmap/retirement, alias handovers, graphics/BDA. Targeted production-negative disables sparseRunImportsEnabled with fixtures unchanged: imported test FAILS (not bound in place), copied test PASSES. Restored/touched/rebuilt positives pass. Initial two CTest registrations failed DLL lookup in this older Windows layout, corrected before these results. No skips at the CTest level; this is not full CI or a fresh-current-main build.

Our bounded title diagnostic matches the scope:12of16 sampled descriptors512MiB over10MiB covered by2 imported ranges. First title attempt on this port was guard-ended during startup with current host RAM pressure, before the scene or sparse-run samples, so no FPS/copy reduction claim yet. Guard/settings retained; investigating memory headroom before retry. Native evidence is independent of that incomplete title run. AI-assisted: yes (Codex); no title data attached.

[blawesom]

Rebase handoff — rebased onto current main (staleness only, applied cleanly), authorship unchanged.

Verified on Linux (CachyOS, kernel 7.2.9, GCC 16.2.1, Ninja, Release, BUILD_TESTING=ON, AMD RX 9070 XT / RADV):

• builds; tools/check_conventions.py --base origin/main passes
• full suite passes except the known main regression (agc_driver_tessellation segfault, feat(shader): read hull inputs at a control-point index computed from data #4727) — not caused by this PR

To pick it up:

git fetch https://github.com/blawesom/AnyPS5 rebase/5249
git checkout <your branch>
git merge --ff-only FETCH_HEAD

Head is 49d2a5cf7.

[xXPBSXx]

On Windows the two new tests agc_sparse_run_import and agc_sparse_run_import_copied exit with 0xc0000135 (DLL not found) and never start. They are not in the if(WIN32) set_tests_properties(... ENVIRONMENT_MODIFICATION "PATH=path_list_prepend:$<TARGET_FILE_DIR:libc>") list that the other agc_driver_recorder_tests entries are in. With that directory prepended to PATH by hand, both pass in 3 of 3 runs; without it, both fail in 3 of 3 runs.

Built on main 18c438b with this PR's commit. Windows 10 (19045), MinGW-w64 GCC 15.2, Release, NVIDIA GeForce RTX 4080 SUPER (driver 610.74). Not tested: Linux, AMD.

AI-assisted (Claude Code).

[SleglJan]

@Scramblrrrr @xXPBSXx Thanks for the Windows runs. This was a gap in the PR, not in the older layout: both new tests were missing the if(WIN32) PATH entry that every other agc_driver_recorder_tests test has. 510f926 adds agc_sparse_run_import and agc_sparse_run_import_copied to the list with agc_host_import_retirement, which prepends the libc directory to PATH as xXPBSXx did by hand; it is the same one-line change as Scramblrrrr's 693182a. Not run on Windows here.

@blawesom Thanks for the RADV run. I'm keeping the branch on #4517's head f6c289e, so the #4517 commit here stays identical to AlxcBTW's. Your 49d2a5c matches c75fac9 patch for patch; 510f926 merges with current main (30f9612) without conflicts.

AI-assisted: yes (Claude Code)

[oneandonlydean]

#290 is in now. It changed the import lookup in GuestBufferMemory.cpp (windowed imports through importAccess, with eviction under the budget), so this conflicts there, in tests/BdaResources.cpp and in the debt file. Can you rebase it? The evicted joined-run case you mentioned is the one to check after.

[oneandonlydean]

One more for the rebase: please drop APS5_NO_SPARSE_RUN_IMPORT; new paths don't get env switches.

[SleglJan]

@oneandonlydean Rebased onto main a5f493f (with #4517 and #290) and dropped the switch; this PR is now the one commit a4d8c29.

• The only code conflict was the import lookup in UploadFinish. A region that isn't a sparse run takes feat(agc): host import windows, least-recent eviction under the budget #290's importAccess; a sparse run takes the path this PR adds, unchanged. The debt file conflicted only on the two entries above this PR's line.
• tests/BdaResources.cpp no longer conflicts, since the perf(agc-driver): join the imports of adjacent ranges of any size that a buffer is bound exactly over #4517 commit this PR carried is gone.
• Joins still happen with feat(agc): host import windows, least-recent eviction under the budget #290 in: on main, importRunAt skips the held check for a join. agc_sparse_run_import checks the join over all four pieces after the growth.
• APS5_NO_SPARSE_RUN_IMPORT is gone. agc_sparse_run_import_copied now reaches the copy through a view past the run and checks the copied bytes with a GPU copy through the view.
• lavapipe (Mesa 26.2.4): on a4d8c29 both new tests pass and 494 of 496 agc_, the same as 9c8fd76 on main d1b3aa1, where main alone passes 492 of 494; the same two fail throughout (agc_driver_depth_compare_copy, and agc_driver_bc_block_copy: "the guest block is not write-watched" in that container). With the last view ignored, both new tests fail.

Evicted joined runs, from the code (no test evicts a joined import yet):

• When a new import would go over the budget, feat(agc): host import windows, least-recent eviction under the budget #290 retires the least recently used import that no build holds and that wasn't looked up in the last 256 lookups.
• Each build that binds a sparse heap in place looks its import up, so it isn't evicted while builds keep binding it. A draw or dispatch that reuses a cached set proves the import by its serial without a lookup, so a heap bound only through reused sets for a while can be evicted under budget pressure, like any other import.
• After an eviction the reused set fails its proof. The next build joins the run again (the one after it, if the heap grew meanwhile), and that join counts toward the limit of 64 joins per base, which joins after growth share. Once 64 are used, the next eviction leaves the heap copied until no registered range starts at its base or the device changes.
• If you'd rather a re-join after an eviction not count toward the limit, I can add that with a test that retires the joined import.

AI-assisted: yes (Claude Code)

[SleglJan]

@adriadam10 Thanks. The list of failing lines came through empty in your comment (the same empty list went to six other PRs between 10:32 and 10:37 UTC). The description passes tools/check_conventions.py --title … --body … from current main (3e577ed), and the conventions check is green on the new head. I've cut Depends on to the template's none (#4517 and #290 are merged). The branch did conflict with main since #4935, on the if(WIN32) test PATH list in libSceAgcDriver/CMakeLists.txt; it's rebased onto 3e577ed with both lists kept, head d34564b. If something still fails on your side, which lines are they?

AI-assisted: yes (Claude Code)

[github-actions]

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request	Conflicting files (lines on main)	Same exports	Same new files
#5372	docs/dev/debt/libSceAgcDriver.md L84

[adriadam10]

Checked the head merged onto current main: scoped build and test run of libSceAgcDriver, both new tests pass; the only failures are agc_driver_tessellation (#4727) and agc_recorder_kept_bytes, which fails the same with this PR's changes reverted. The new tests use CutRegions, so they do not build without the change. Windows PATH entry, dropped env switch and debt entry are in place; merging once the Windows Build is green.