AGC: a buffer that crosses registered ranges outside a whole import window is copied on the CPU at every use (Palworld: 15–56 GB per 10 s) #4260

Activity

1. oneandonlydean commented on Oct 12, 2026

   

   oneandonlydean

   on Oct 12, 2026

   Collaborator

   More actions

   up4/host-import-bind-join on oneandonlydean/AnyPS5 (7787130, see #3425) joins the imports under a buffer bound across them, which is your cause 1. Could you run it on this scene with APS5_TRACE_HOST_IMPORT=1 before opening yours? It does not touch WindowsMappings::MapAlias, so the alias over several view runs (cause 2) is still missing there; a PR for that part alone would be welcome.
2. MarcZX10 commented on Oct 12, 2026

   

   MarcZX10

   on Oct 12, 2026

   Contributor

   More actions

   Draft PR #4367 carries Angel21PC’s alias-run commit from closed #2254 with the original commit author preserved. It covers only WindowsMappings::MapAlias for multiple section/offset runs; it does not add cross-registered-range imports. The original author reported the three Windows guest_memory alias tests passing on the earlier #2254 base, but I could not run Windows tests on this current-main head. If you have a Windows build available, could you run the focused guest_memory test on #4367? The draft is open pending that current-head validation.
3. queeeef commented on Oct 12, 2026

   

   queeeef

   on Oct 12, 2026

   ContributorAuthor

   More actions

   @oneandonlydean ran up4/host-import-bind-join (778713032) with #4367 on the Palworld scene, APS5_TRACE_HOST_IMPORT=1, APS5_PROFILE_DRAW=1.

   • Base: the same local Palworld branch as in the issue. My local span and alias commits are reverted and replaced by yours and fix(libc): alias non-contiguous shared guest views on Windows #4367's.
   • Run: one run to character creation, about 30 s there. It then crashed at the known app.exe+0x575bdc0, the Windows: resuming a write fault overwrites the guest red zone when AVX state is live #361 red-zone read.

   What it fixes

   • Joins: 21 joins (joins 2 registered ranges and joins 3 registered ranges for binds up to 0x1fe0000).
   • The 238 MiB region: no copies in the 0x3020000000 range any more, where the per-frame 238 MiB region was.

   What is still copied: 116 binds are refused with a piece lies in a partly registered window. They are 147 MiB in total, counted once each. The largest:

   Bind	Size
   0x301f130000+0x1560000	21.4 MiB
   0x3127c20000+0x8c0000	8.8 MiB
   0x3139d40000+0x870000	8.4 MiB
   0x31a1e20000+0x680000	6.5 MiB
   0x319fe20000+0x530000	5.2 MiB
   8 binds of 0x400000	4 MiB each

   They are used often enough to dominate the copies:

   10 s window at character creation	Presents	Copied outside imports
   1	8	2.6 GB
   2	10	20.7 GB
   3	9	4.0 GB
   4	9	3.9 GB

   By region over the run: 18.0 GB in 0x3140000000, 17.5 GB in 0x31a0000000, 3.6 GB in 0x3190000000, 2.1 GB in 0x3170000000.

   For comparison, on the same scene:

   • Without either change: 15–56 GB per 10 s, 6–7 presents per 10 s.
   • With my local version: 0 bytes copied, 9–13 presents per 10 s. It joins any bind whose pieces are all registered, adjacent and of the same access, partly registered windows included. perf(agc-driver): import adjacent registered ranges as one host-import span #3158's growth is bounded instead by a budget on the bytes spans import per 10 s (1 GiB).

   So in this scene, the partly registered windows carry most of the remaining traffic. If it helps, I can run a variant of your branch that also joins those pieces when every byte of the bind is registered, and report the same numbers.

   Two more numbers from the trace, without a baseline to compare against:

   • 24,053 host import ... ok lines;
   • 1,117 retired: the range was mapped again.

   AI-assisted (Claude Code)
4. oneandonlydean commented on Oct 14, 2026

   

   oneandonlydean

   on Oct 14, 2026

   Collaborator

   More actions

   #4515, #4517 and #4367 are on main now, so binds across adjacent registered ranges are imported, including exact runs of small pieces, and Windows aliases cover several view runs. Could you rerun the Palworld character-creation scene on current main and post the copied-outside-imports numbers? The partly registered windows from your last run may still be copied.
5. EdouardSence commented on Oct 15, 2026

   

   EdouardSence

   on Oct 15, 2026

   Contributor

   More actions

   Ratchet & Clank (PPSA01474) on current main d92a8dad3: a buffer inside a single registered range is still CPU-copied on every dispatch. Here the cause is the held-import check in importRunAt, not a crossing between registered ranges.

   During the black loading screen after the logos, one indirect compute dispatch (250,038 SPIR-V words) binds 0x111c190000+0x1fe00000 (510 MiB) about 20 times per second. Every bind copies it on the CPU:

   [resources] guest upload of 8 regions, 512.0 MiB (largest 510.0 MiB, copied 510.0 MiB, addressable 0) took 213 ms to finish
   [buffers] 2000 completed CPU-copy operations: ... 2000 outside imports (1020000 MiB): 0x1110000000:2000/1020000MiB
   

   That is about 100 GiB copied per 10 s. resources: B upload takes 260 s of a 420 s run, and the AGC worker spends most of its time in copyRegion → memmove.

   Why it is not imported. I added local prints to the decision path; they are not part of any PR.

   • The buffer sits entirely inside one registered range, 0x10acc00000+0xb7800000 (2936 MiB, readable, writable, gpu, /memfd:direct memory). So runCovers is true and no join is attempted (UploadFinish, GuestBufferMemory.cpp:3447).
   • importAccess then asks importRunAt for the windowed run 0x1118c00000+0x28000000 (640 MiB). That run overlaps an existing import, 0x1118c00000+0x4000000 (64 MiB), which another descriptor of the same build has already looked up.
   • The loop at GuestBufferMemory.cpp:957-961 therefore returns nullptr through held (!joined && pointerHeld(...)). The call returns nothing, so the region falls back to copyRegion.
   • The same thing happens on the next dispatch, because both descriptors are bound again. The run is copied for good.
   • Unlike the budget refusal just below it, this refusal prints nothing. APS5_TRACE_HOST_IMPORT=1 shows no attempt or failure for the range, only the smaller imports nearby succeeding.

   [diag] runCovers(0x111c190000, 0x113bf90000): importRun base=0x10acc00000 bytes=0xb7800000 ranges=1 -> true
   [diag] importRunAt: base=0x1118c00000 bytes=0x28000000 joined=0
   [diag] importRunAt: pointer held for 0x1118c00000
   [diag] region 0x111c190000+0x1fe00000 falling back to copyRegion (CPU copy)
   

   The comment above that loop exempts joins because "a retired import stays alive through the hold (KeepThroughHold). Refusing it there would keep the run copied for good." This is the same "copied for good" outcome on the non-joined path. Both imports would map the same memfd pages. I haven't tried changing the check, since whether a confirmed pointer may be retired mid-build is your call.

   Setup: CachyOS, RX 9070 XT, RADV Mesa 26.2.4, 32 GB RAM, udmabuf.size_limit_mb=8192. Host import budget 15614 MiB, about 4.7 GiB live when this happens. #6191's stop is not reached in 900 s on this machine, with or without #6232, presumably because of this slowdown.

   Tested and written by Claude (AI assistant) on a contributor's machine; Gemini ran the game and the instrumented build. AI-assisted: yes.
6. EdouardSence commented on Oct 15, 2026

   

   EdouardSence

   on Oct 15, 2026

   Contributor

   More actions

   Follow-up with the check changed locally: on a probe branch, held in importRunAt (GuestBufferMemory.cpp:957) always returns false, so the non-joined path may replace held imports the way joins already do. Both runs are Ratchet & Clank PPSA01474, 900 s each, same scene and same scripted input:

   	main d92a8dad3	held → false
   presents per 10 s (end of run)	194	600 (the 60 Hz cap)
   dispatches in 900 s	345,000	1,055,000
   resources: B upload	563 s	2 s
   CPU-copied bytes ([buffers] outside imports)	~8.5 TiB	0
   peak RSS	13.1 GB	13.1 GB
   amdgpu faults / resets (journalctl -k)	0	0

   The 510 MiB buffer is imported now:

   [resources] guest upload of 8 regions, 512.0 MiB (largest 510.0 MiB, copied 0.0 MiB, addressable 0) took 161 ms to finish
   

   With that change, full ctest matches main on this machine: 1222 tests, the same two failures, no new ones.

   • agc_host_import_sparse is a test budget issue, fixed in test(agc): lift the host import budget in the sparse dma-buf import test #6261.
   • agc_driver_tessellation is the Mesa bug from agc_driver_tessellation segfaults on RADV with Mesa 26.2+: nir_opt_varyings corrupts the TCS input offset's use list (Mesa bug) #5444.

   No test exercises the held refusal, so its removal is not covered either way.

   The screen stays black for the whole 900 s even at 3× the throughput, and #6191's stop does not appear. This change only removes the copies; it does not make the game progress further here.

   If you want this as a PR, I'd need to know what the check protects. The pointers looked up earlier in the same build would point to a retired import kept alive through the hold, which maps the same memfd pages. I can then write a test that fails on main (a run inside one registered range, where an overlapping smaller import is already held, ends up copied).

   Tested and written by Claude (AI assistant) on a contributor's machine; Gemini ran the builds and the game. AI-assisted: yes.
7. MarcZX10 commented on Oct 16, 2026

   

   MarcZX10

   on Oct 16, 2026

   Contributor

   More actions

   I checked the current main path while tracing this follow-up. importRunAt() now has an expandedWindow case: when a larger import covers an existing import from the same single registered range, and a recorder is active, it permits that promotion despite the smaller import being held. HostImportRetireHold::WindowPromotionOutlivesTheHold checks that the old buffer remains alive through the batches that may still use it.

   That appears to address the held-import mechanism in the 510 MiB example, but I cannot confirm whether expandedWindow is true or a recorder is active at that call site in your run. On a current-main rerun, could you log those two conditions around the attempted import and report whether that range now imports or still falls back to copyRegion()? That would tell us whether the current exception covers this case or whether a distinct safe change is still needed.
8. EdouardSence commented on Oct 16, 2026

   

   EdouardSence

   on Oct 16, 2026

   Contributor

   More actions

   @MarcZX100 Confirmed on current main: the range now imports, and nothing goes through copyRegion().

   Ratchet & Clank PPSA01474, plain main eb901bcf4 (no local changes), 240 s with APS5_TRACE_HOST_IMPORT=1, RX 9070 XT, RADV Mesa 26.2.4:

   • The 640 MiB window that importRunAt() used to refuse is now promoted over the held 64 MiB import, so expandedWindow held and a recorder was active at that call:

     [gpu] host import of 0x1118c00000+0x28000000 ok (37 live, 5182.6 MiB live, 6078.6 MiB ever)
     

     It later grows to 0x1118c00000+0x2c000000, also ok.
   • The 510 MiB buffer is no longer copied:

     [resources] guest upload of 8 regions, 512.0 MiB (largest 510.0 MiB, copied 0.0 MiB, addressable 0)
     

     The log has no outside imports CPU-copy line at all, and resources: B upload totals 1 s for the run. On d92a8dad3 it was 260 s out of 420.
   • Presents reach the 60 Hz cap (600 per 10 s) from about 40 s on. With my local held → false experiment, the same 600 per 10 s needed a patch; on this main it doesn't.

   So 7b06e75 covers this case, and no further change is needed for this path. On this machine the title still shows a black screen after the logos, but that is the separate stall I described in #2172, not this copy.

   Tested and written by Claude (AI assistant) on a contributor's machine. AI-assisted: yes.
9. oneandonlydean

   self-assigned this

   on Oct 17, 2026