Skip to content

perf(agc): settle Windows private imports larger than 64 KiB at the import - #10443

Draft
Dripl1x wants to merge 8 commits into
boykopovar:mainfrom
Dripl1x:perf/agc-settle-private-imports-over-64k
Draft

Dripl1x wants to merge 8 commits into
boykopovar:mainfrom
Dripl1x:perf/agc-settle-private-imports-over-64k

Conversation

@Dripl1x

@Dripl1x Dripl1x commented Oct 24, 2026

Copy link
Copy Markdown
Contributor

What

On Windows a host import of private memory under 1 MiB is compared until its deferred settle, and the deferred return to the write watch is skipped while a live storage image overlaps the range (the rewatch stamps it as a CPU store, which the image would take as a write over its results). SuperTuxKart (PPSA69421) views its small textures (96 KiB to 352 KiB) from storage images, so they stayed compared for good and every draw sampling one took the full revalidation walk with a byte compare.

  • Imports larger than one 64 KiB block now settle at the import, as the 1 MiB ones did, and stay in the write watch. 64 KiB imports, the size a title maps around every submission, keep the deferred settle.
  • The cost moves to the import: each new private import from 128 KiB up to 1 MiB waits for its settle read (the [buffers] line's "settled inside their window" counts them). In SuperTuxKart they are made once; a title that makes such imports every frame would pay a wait per import.
  • APS5_SETTLE_FLOOR_KIB sets the floor (1024 restores the old one). The retire test needs a 128 KiB import on the deferred path and sets it to 1024; with the default floor, an import replaced while its deferred read runs is now only reached by 64 KiB imports and has no test of its own.

SuperTuxKart main menu with the tutorial prompt (about 59 draws a frame), on a stack based on main a0900b9d99 (#7464, #9402, two local direct-memory changes and three other local changes), APS5_PROFILE_DRAW over 10 s, the old floor against this one on the same build:

revalidations time full walks draw recorded in
APS5_SETTLE_FLOOR_KIB=1024 33583 1339 ms 19435 (19355 for a failed collect) 104 us
this PR 18287 315 ms 204 (audits only) 82 us

Dumped menu frames 0 to 6 of both matched within a mean difference of 0.001. The menu runs at the title's 60 FPS cap either way, so this shows as spare time per frame, not FPS: on current heads (main 9d4fb4fb70 + #7464 + #9402 + this and three other local changes, the menu without the tutorial prompt, about 33 draws a frame) 59.94 FPS with and without it, the longest idle gap of queue worker 0 and the committer per frame 6.7–7.0 ms (median) with it against 5.9 and 6.2 ms without (a third run without it lost the window focus and is left out).

Overlap: #9941 also changes importRunAt (it marks partial-block imports settleOnDemand and settles them on demand before an initial multisampled DCC import, for #9724). With this PR a private import of more than 64 KiB, partial or not, settles at the import, which may already cover #9724's 784 KiB import; that was not tested on those titles. Whichever lands second needs a small rebase there.

Tested

This head: main 9d4fb4fb70 + this commit; merges cleanly into 5e6ae6e3d0. Windows 11 Pro 26300, Ryzen 7 5800H, RTX 3070 Laptop GPU (NVIDIA 621.01), WinLibs GCC 15.2.0 r7, Release: every ctest entry of agc_driver_recorder_tests (86, the agc_import_settle_* entries included), agc_driver_host_import_windows_tests, agc_driver_host_import_at_map_tests, agc_driver_host_import_teardown_tests, agc_host_import_retire_tests and agc_driver_guest_memory_lifetime_tests (6) passes.

Linux (Ubuntu 26.04 in WSL, llvmpipe), on the measurement stack: agc_driver_recorder passes (the change is Windows-only code).

Titles: SuperTuxKart menu as above. vkQuake (PPSA99010), on the earlier stack: 27 imports settled at the import over a run, 82–85 FPS with this series and without it. Not run: the full CTest suite, other titles, other GPUs.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed in TechnicalDebt
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Unblocks: none
  • AI-assisted: yes (Claude Code)

…mport

A Windows import of private memory under 1 MiB was compared until its
deferred settle, and the deferred return to the write watch is skipped
while a live storage image overlaps the range (the rewatch stamps it as
a CPU store, which the image would take as a write over its results).
SuperTuxKart views its small textures (96 KiB to 352 KiB) from storage
images, so they stayed compared for good and every draw sampling one
took the full revalidation walk with a byte compare.

Imports larger than one 64 KiB block now settle at the import, as the
1 MiB ones did, and stay in the write watch; 64 KiB imports, the size a
title maps around every submission, keep the deferred settle. The cost
moves to the import: each new private import from 128 KiB up to 1 MiB
waits for its settle read (the [buffers] line's "settled inside their
window" counts them). APS5_SETTLE_FLOOR_KIB sets the floor (1024
restores the old one), and the retire test, which needs a 128 KiB
import on the deferred path, sets it so.

SuperTuxKart (PPSA69421) main menu without the tutorial prompt (about 33
draws a frame) on Windows 11, RTX 3070 Laptop, Ryzen 7 5800H; main 9d4fb4f with boykopovar#7464, boykopovar#9402 and three local changes (early
unimported labels, an idle spin of the queue worker and committer, import
refresh spans diffed in place) plus this one; protocol runs interleaved.
With APS5_SETTLE_FLOOR_KIB=1024: 59.94 / 59.94 FPS (a third run lost the
window focus and is left out), the same as with this change at the
title's vblank cap, and the longest idle gap of queue worker 0 and the
committer per frame 5.9 and 6.2 ms (median) against 6.7-7.0 ms. On an
earlier stack (main a0900b9 with boykopovar#7464, boykopovar#9402, two local direct-memory
changes and the other three changes named above; the menu with the tutorial
prompt, about 59 draws a frame), APS5_PROFILE_DRAW over
10 s with the old floor against this one:
revalidations 33583 in 1339 ms with 19435 full walks (19355 for a failed
collect) against 18287 in 315 ms with 204 (audits only), a draw recorded
in 104 us against 82, and dumped menu frames 0 to 6 of both matched within
a mean difference of 0.001.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

This pull request is labeled perf, so it needs the summary line of a capture on main and one with the change, from the same title, scene and machine, as the Performance report form asks. Please paste both aps5-summary.jsonl lines, attach the two traces and add the output of tools/trace_digest.py.

@github-actions github-actions Bot added perf Performance change perf numbers missing perf pull request without a before and after summary line labels Oct 24, 2026
@Dripl1x
Dripl1x marked this pull request as draft October 24, 2026 00:36
@github-actions

github-actions Bot commented Oct 24, 2026 •

Copy link
Copy Markdown

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 1368 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
PR 1395 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
PR 1896 docs/dev/ENVIRONMENT.md
PR 4472 docs/dev/ENVIRONMENT.md
PR 5021 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
PR 5925 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
PR 6186 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
core/libs/prx/libSceAgcDriver/tests/cmake/agc_driver_recorder_tests.cmake
docs/dev/ENVIRONMENT.md
PR 6187 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
PR 6188 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
PR 6192 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
PR 6197 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
PR 6209 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
PR 6481 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
PR 6918 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
PR 6924 docs/dev/ENVIRONMENT.md
PR 7085 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
core/libs/prx/libSceAgcDriver/tests/cmake/agc_driver_recorder_tests.cmake
PR 7449 docs/dev/ENVIRONMENT.md
PR 7457 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
docs/dev/ENVIRONMENT.md
PR 7514 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
docs/dev/ENVIRONMENT.md
PR 7604 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
docs/dev/ENVIRONMENT.md
PR 7642 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
core/libs/prx/libSceAgcDriver/tests/cmake/agc_driver_recorder_tests.cmake
docs/dev/ENVIRONMENT.md
PR 7643 docs/dev/ENVIRONMENT.md
PR 8004 docs/dev/ENVIRONMENT.md
PR 8484 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
docs/dev/ENVIRONMENT.md
PR 9012 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
docs/dev/ENVIRONMENT.md
PR 9289 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
core/libs/prx/libSceAgcDriver/tests/cmake/agc_driver_recorder_tests.cmake
docs/dev/ENVIRONMENT.md
PR 9345 docs/dev/ENVIRONMENT.md
PR 9356 docs/dev/ENVIRONMENT.md
PR 9387 docs/dev/ENVIRONMENT.md
PR 9852 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
docs/dev/ENVIRONMENT.md
PR 9884 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
core/libs/prx/libSceAgcDriver/tests/cmake/agc_driver_recorder_tests.cmake
PR 10193 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
core/libs/prx/libSceAgcDriver/tests/cmake/agc_driver_recorder_tests.cmake
docs/dev/ENVIRONMENT.md
PR 10678 docs/dev/ENVIRONMENT.md
PR 10757 docs/dev/ENVIRONMENT.md
PR 10894 docs/dev/ENVIRONMENT.md
PR 10920 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
PR 11435 docs/dev/ENVIRONMENT.md
PR 11436 docs/dev/ENVIRONMENT.md
PR 12157 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
PR 12324 docs/dev/ENVIRONMENT.md
PR 12544 docs/dev/ENVIRONMENT.md
PR 12746 docs/dev/ENVIRONMENT.md
PR 12848 docs/dev/ENVIRONMENT.md
PR 12849 docs/dev/ENVIRONMENT.md
PR 12970 docs/dev/ENVIRONMENT.md
PR 13148 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
PR 13251 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
core/libs/prx/libSceAgcDriver/tests/cmake/agc_driver_recorder_tests.cmake
PR 13486 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
docs/dev/ENVIRONMENT.md
PR 13707 docs/dev/ENVIRONMENT.md
PR 13793 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
core/libs/prx/libSceAgcDriver/tests/cmake/agc_driver_recorder_tests.cmake
docs/dev/ENVIRONMENT.md
PR 13809 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
core/libs/prx/libSceAgcDriver/tests/cmake/agc_driver_recorder_tests.cmake
docs/dev/ENVIRONMENT.md
PR 13863 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
core/libs/prx/libSceAgcDriver/tests/cmake/agc_driver_recorder_tests.cmake
docs/dev/ENVIRONMENT.md
PR 13920 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
core/libs/prx/libSceAgcDriver/tests/cmake/agc_driver_recorder_tests.cmake
docs/dev/ENVIRONMENT.md
PR 13928 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
core/libs/prx/libSceAgcDriver/tests/cmake/agc_driver_recorder_tests.cmake
docs/dev/ENVIRONMENT.md
PR 13929 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
core/libs/prx/libSceAgcDriver/tests/cmake/agc_driver_recorder_tests.cmake
docs/dev/ENVIRONMENT.md
PR 14051 docs/dev/ENVIRONMENT.md
PR 14344 docs/dev/ENVIRONMENT.md
PR 15059 docs/dev/ENVIRONMENT.md
PR 15136 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
core/libs/prx/libSceAgcDriver/tests/cmake/agc_driver_recorder_tests.cmake
docs/dev/ENVIRONMENT.md
PR 15169 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
core/libs/prx/libSceAgcDriver/tests/cmake/agc_driver_recorder_tests.cmake
docs/dev/ENVIRONMENT.md
PR 15296 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
PR 15302 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
PR 15529 core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
core/libs/prx/libSceAgcDriver/tests/cmake/agc_driver_recorder_tests.cmake
docs/dev/ENVIRONMENT.md
PR 15548 docs/dev/ENVIRONMENT.md
PR 15570 core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
core/libs/prx/libSceAgcDriver/tests/Recorder.cpp
docs/dev/ENVIRONMENT.md

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed exact head 90b9574ad52da6ecf26c1cec51bd1fea2587d592 against refreshed upstream main c8d34aa8f4d8058d57169cc1e423ba6db00faaf0.

The threshold change is narrowly scoped to Windows private imports and preserves the 64 KiB deferred-settle path; the new 128 KiB retirement case restores the old 1 MiB floor for that test. The recorded Windows suite and SuperTuxKart measurements are useful, but they were run on a stack based on 9d4fb4fb70 with three additional local changes. The current branch is based on that same 9d4fb4fb70 commit, while current main has 20 commits not present here, including subsequent AGC import/write-watch changes. A merge-tree preview is clean, but that does not verify the changed settle behavior with the current integrated import paths.

Please update this branch to current main and rerun the focused Windows tests on the final composition: the recorder suite, host-import Windows/map/teardown suites, retire and guest-memory lifetime tests. Report which tests actually ran and whether the existing GPU/title comparison still holds on that base. Keep the environment floor override exercised so the retire test still covers deferred settling.

Local review: current-main merge-tree is clean; current-main conventions and diff-check pass. Hosted eligibility, intake, labels, report, and conventions pass; hosted product/relinker jobs are skipped. I cannot reproduce Windows/GPU tests locally. This request is for fresh integration evidence, not a claim that the implementation is incorrect.

@tibzejoker

Copy link
Copy Markdown
Contributor

One thing from Grand Theft Auto V (PPSA04264) that may matter for the larger floor, since it involves the same path on Windows.

If it helps, I can run GTA V's open world on your head and on its main, without --frame-red-zone, with the per-rip resumed-fault log on, and post the counts by memory type and whether the streamer crash comes back.

AI-assisted: yes.

@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 25, 2026
Dripl1x and others added 2 commits October 25, 2026 23:23
# Conflicts:
#	core/libs/prx/libSceAgcDriver/CMakeLists.txt
#	core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
agc_import_settle_initial_dcc (boykopovar#9941) needs a 320 KiB private import that
is compared until its deferred settle, so that the initial multisampled DCC
import finds a pending settle. With this PR such an import settles at the
import instead, so the test runs with APS5_SETTLE_FLOOR_KIB=1024, as the
retire test does, and still checks the path it was written for.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 25, 2026
@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 26, 2026
# Conflicts:
#	core/libs/prx/libSceAgcDriver/Graphics/src/GuestBufferMemory.cpp
@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 26, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot added needs rebase Conflicts with main, rebase needed and removed needs rebase Conflicts with main, rebase needed labels Oct 29, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 31, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Nov 2, 2026
…test

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Nov 3, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agc libSceAgc* build docs perf numbers missing perf pull request without a before and after summary line perf Performance change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants