Repository navigation
perf(agc): keep an untracked fast proof across collect epochs on Windows while its write-watch hint has not moved - #12921
Conversation
…is forgotten Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…son memo Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…y its stamp until the next ordering point APS5_VERIFY_UNTRACKED_PROOFS=1 runs the full walk beside every such proof and counts disagreements. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n a collect epoch Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s its detiler The test's last refresh records a detile dispatch that is still pending when the function returns. The detiler, declared after the Unregister guard, was destroyed first, freeing the pipeline and descriptor pools that dispatch uses; Unregister then removed the allocations, which syncs the recorder, and lavapipe executed the dispatch with the freed pipeline and crashed on its queue thread. Sync the recorder before the detiler goes away, as the neighbouring untracked tests already do.
…ts write-watch hint has not moved
|
Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.
Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.
|
✅ NVIDIA GPU test: passedPR head
Unstable on main (1, these also failed in recent main runs on this machine; not counted against this PR)Failing on main too (5, not counted against this PR)Validated but not compared (3, not in the main baseline's validation pass)Validation IDs already present on main (4 IDs in 436 tests)Skipped (39)ReproduceAutomated build and test on a contributor's NVIDIA machine; it does not replace upstream CI. A new comment is posted for each new push. Full logs. |
|
GTA V (PPSA04263, 01.011.001) on Windows: no FPS change at story start, and 0 disagreements in the proof self-check. GTA's collect fast-fails are almost all on ranges with no write-watch hint, so the hint key doesn't apply to them. Setup
Results (FPS)
The difference is inside the run-to-run spread. Why (local counters, not part of either PR)
Correctness
AI-assisted (Claude Code). All numbers come from runs on the machine above. |
There was a problem hiding this comment.
The measurement does not isolate this change, and the fast path it extends gets more expensive per proof. Neither point is addressed in the description.
1. The +29.9 % is #6192 plus this PR. The description says so: "#6192 alone wasn't run". The branch carries #6192's four commits, #6187 under them, and a merge of main, so an unrelated diff of about 450 lines is reviewed as one PR. The arms are 4 runs of 0.51–0.83 fps with a 19–25 % spread inside each arm. B ahead of A by 30 % on two runs per arm is not distinguishable from the spread. The only independent run so far is chitters2004's GTA V one above (12.60 vs 12.31 fps, inside the noise), and it says the fast-fails there are on ranges with no hint, where this change does not apply. Please run main + #6192 against main + #6192 + this PR on the Village menu, same machine and scene, with tools/trace_digest.py summaries of both. That is the only comparison that measures this PR.
2. The proof now does more work per surface. For an untracked surface, currentUntrackedStamp calls hintedProof: UntrackedHint, NewestStamp, ForgetStamp and NewestWriteNote. ForgetStamp, which this branch carries from #6192, loops over every 64 KiB block of the range with an acquire load per block. A 3840x2160 RGBA16F target is about 66 MB, which is about 1,000 loads per call, per surface, per proof. GTA shows about 135k fast proofs per 10 s, and each proof can hold several untracked surfaces. The fast path exists because it is cheaper than the full walk; with #6192's loop it costs O(range size) per surface instead of O(1). Keep a single global counter that the forget path bumps, and use the per-block slots only when that counter has moved. Or show the proof cost in the trace.
3. Correctness depends on the hint being up to date. The description says "a store into it moves the hint at the next walk of that page's marks". Between two ordering points no walk happens, so a CPU store after the compare is only visible if something walks those marks before the next check. Please say what guarantees a walk between the compare and the next check on a surface nobody reads through CollectWrites. APS5_VERIFY_UNTRACKED_PROOFS covers the runs listed, but a title that rewrites an untracked surface once per frame from the CPU would show a stale texture instead of a slow frame. The new test would need a case that stores into the range from another thread right after the compare.
4. Smaller points.
- The "where it comes from" numbers use
APS5_PROFILE_DRAW=1. The Performance report form does not accept those for a speed claim. They are fine as explanation, but the claim needs theaps5-summary.jsonlfigures from item 1. - On Linux and for unhinted ranges nothing changes, so the PR is Windows-only in effect. Say that in the title or in
What. - "A new storage image has no hinted entry until its first equal compare" means a freshly created image pays the full walk once per recreation. If the title recreates targets per scene change, that cost should be in the numbers too.
|
@CnRJay, on point 2, with the proof cost measured on GTA V instead of estimated. Where the loop comes from. Measured. I timed each
On point 1, the GTA run above is that isolated comparison: #6192 against #6192 + this PR, same machine, scene and save, arms interleaved.
The PR arm averages 2.2 % lower, and the ranges overlap. The measured proof cost above is about 0.2 % of an 80 ms frame, so it doesn't explain a 2 % gap if there is one. Three runs per arm can't separate that from noise. This is GTA, not the Village menu, so it doesn't replace the run you asked EagleBearerYS for. Your global-counter idea would cut the epoch path's loop. For the hinted path, the cost to look at is AI-assisted: yes (Claude Code) |
|
@CnRJay, thanks for the close read. The answers are below, in the order 3, 2, 4, 1, with links at this head (fdc7004). @chitters2004, thank you for the Grand Theft Auto V runs and the timing by stamp kind. 3. What makes a store after the compare visible. The check walks the write-watch marks itself, so it doesn't depend on anyone else calling
2. Cost per proof. chitters2004's measurement stands, and so does their point about where the loop comes from.
4. The smaller points.
1. The isolation pair is queued, on the same machine and scene as before.
|
# Conflicts: # core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
… the proof's compare and check
The conflict in core/libs/prx/libSceAgcDriver/CMakeLists.txt from boykopovar#12862's split is resolved with tools/agc_test_split.py resolve, which moves this PR's test registrations into tests/cmake/.
✅ NVIDIA GPU test: passedPR head
Intermittent validation messages (1, seen once in the parallel run, not reproduced when the test was re-run alone 2 times; not counted against this PR)Unstable on main (3, these also failed in recent main runs on this machine; not counted against this PR)Failing on main too (4, not counted against this PR)Validated but not compared (3, not in the main baseline's validation pass)Validation IDs already present on main (5 IDs in 476 tests)Skipped (44)ReproduceAutomated build and test on a contributor's NVIDIA machine; it does not replace upstream CI. A new comment is posted for each new push. Full logs. |
|
@EagleBearerYS the change itself looks right to us: the per-block forget stamps only err on the side of comparing again, and the cross-thread hint test covers the race. One fix in the tests: Also, this PR and #12745 both change |
|
@oneandonlydean thanks. Done in 793fc7f:
|
✅ AMD GPU test: passedPR head
Failing on main too (9, not counted against this PR)Validated but not compared (3, not in the main baseline's validation pass)Validation IDs already present on main (5 IDs in 481 tests)Skipped (54)ReproduceAutomated build and test on a contributor's AMD machine; it does not replace upstream CI. A new comment is posted for each new push. Full logs. |
✅ NVIDIA GPU test: passedPR head
Unstable on main (3, these also failed in recent main runs on this machine; not counted against this PR)Failing on main too (3, not counted against this PR)Validated but not compared (3, not in the main baseline's validation pass)Validation IDs already present on main (5 IDs in 484 tests)Skipped (45)ReproduceAutomated build and test on a contributor's NVIDIA machine; it does not replace upstream CI. A new comment is posted for each new push. Full logs. |
The counter commit below reads APS5_COUNT_WALKS without a row in docs/dev/ENVIRONMENT.md, which the conventions check refuses. This adds the row. It goes with the counter and is not for a pull request either. The report of this branch follows, as in the counter commit fb0be86. Report (v2) Branch draft/revalidate-walks-v2, based on upstream main 4ac683f. Commits: - 3e89f8a perf(agc): look up only the changed textures when a template's memory moved (unchanged from v1) - 795405a perf(agc): fail a template on a released texture without the full walk (unchanged from v1) - 3ac1c0d perf(agc): find a template's changed elements in one tracker query - 983e273 perf(agc): relook changed elements from lookup recipes captured once - this counter v1's report (draft/revalidate-walks, aefbec8) still holds for every reason a walk runs and for the relook and replaced arguments. What each addition costs, per call and per frame: - ChangedSince, proof passes: the same single tracker lock and one range check per query as UnchangedSinceAll, plus a clear of a thread-local vector and one branch per query. Nothing measurable per Revalidate. - ChangedSince, 'memory changed': v1 took UnchangedSinceAll's lock, then one more lock, covers check and range check for each of Q queries to name the changed ones. Now one lock and Q range checks. That is Q lock round trips saved per failure. Calls per frame: the [walks] relook counts (same + differing + fell back). - Recipes, capture: once per template, on its first relook. One pass over its bindings and table elements, which is what v1's relook did on every call, plus about 64 bytes per element held by the template. - Recipes, each later relook: v1 re-enumerated all bindings and table elements (narrowing and null-checking every sampled descriptor) to reach k changed elements. Now k recipe reads, a sort of k entries by build order, and the k lookups. Calls per frame: as above. - Neither adds work to a proof that passes. Why the recipes are the walk's lookups: - The walk derives each lookup from the bindings it is passed. ShaderResources::ContentKey puts every non-data binding's descriptor words, image shapes and compare bits into the cache key, so a template found under that key has its build's descriptors. - The recipe holds those raw words. Sampled elements are narrowed by firstLayerWords at each lookup, as the walk does. Components and multisample view type are derived as the walk derives them. Storage elements keep the build's mip, written flag and dedupe (index - 1). - Lookups run in the walk's order: bindings stage by stage, then tables. - A layout the walk would refuse (no program, more than 8 words per element, a table index out of step, counts not matching textures[] or storageTextures[]) marks the recipes refused. Those templates take the full walk. Conservative fallbacks, unchanged and tested: - The relook starts only when 'memory changed' is the fast proof's sole failure. That check runs after the pending-registry, key and metadata, depth and cache checks. - The rerun repeats all of those. - Import identity, the serial loop and staging checks of revalidateFull's steps 2 and 3 run after either path, as before. - A DCC key change fails on keys, and the test shows it takes the full walk with no relook. Lookup-induced eviction: a sampled cache hit never evicts. admitTexture evicts only on inserting a new texture, and a lookup that inserts has returned a new object, so the relook returns Differs at that element. It fails before any rerun, and no evicted object is proved. The test shows it: a one-byte budget, the changed texture's new copy evicts the other bound texture, and the proof fails after one relooked lookup. Tests: ChangedRelook, now two ctest entries. - agc_driver_changed_relook (imported memory): the 96-texture table cases from v1, plus the layered, DCC and eviction cases. - agc_driver_changed_relook_copied (APS5_NO_HOST_IMPORT=1, APS5_VERIFY_PROOFS=1): the same synthetic cases plus a storage image and its sampled view over one surface. The table set is not reusable without imports, so it runs only in the first entry. - Results: - Storage and sampled alias, same bytes: relooked, 2 lookups, current. Changed bytes: relooked, not current. - First layer of a 2D array: same bytes relooked with 1 lookup and current; changed not current. - DCC: same bytes under uncompressed keys relooked, 1 lookup, current. Keys cleared to 0x00: full walk, 0 relooked, not current. - Eviction: relooked, 1 lookup, not current, the other bound texture released. - Every outcome's current flag matched a run with APS5_NO_CHANGED_RELOOK=1 (the full walk alone), checked by hand. The copied entry also runs the walk beside every relooked proof that passes, through APS5_VERIFY_PROOFS. Mutations, all caught: - Recipe without the image shape (no first-layer narrowing): "a same-byte store in the sampled first layer of a 2D array took the relook ... not current", in both modes. - ChangedSince stopping at the first changed query: "stores over 2 textures looked 1 textures up again" (imported); the storage alias case "took the full walk" (copied). - v1's mutations still apply. Also run on lavapipe (Mesa 25.2.8 llvmpipe, LLVM 20.1.2, Ubuntu 24.04), ANYPS5_NO_SHADER_CACHE=1, 33 of 33 passing with 3 skipped by their own checks: agc_driver_graphics (with maps fallback), agc_driver_recorder, agc_copy_guest_bytes, alpha_to_coverage, array_layer_target, cached_image_multisample_guard, changed_relook (both), image_table_reuse, lookup_log_fits_build, multisample_fill_host_only, native_multisample_depth, own_refresh_repack_alias, packed_mip_tail_target, partially_resident_*, sample_multisample_target, sampled_cache_budget_*, storage_cache_budget, storage_one_slice_alias, list_restart, quad_list_index_overflow. check_conventions passes. Not reached: - Storage images and views over imported memory: lavapipe compares imports instead of write-watching them, so their collect refuses and they always take the full walk there. Open boykopovar#14262, boykopovar#6192 and boykopovar#12921 address that reason. - Views whose source is a storage image in an image table. - Depth surfaces, which never produce a changed query. - A hardware GPU, the validation layer, and the title's real cost. Run APS5_COUNT_WALKS=1 on GTA V for calls per frame. Overlaps as in v1: boykopovar#10894, boykopovar#9012, boykopovar#11444, boykopovar#14262, boykopovar#12575. No speed claims. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011mLh2cdhBwJrK5hBZGnNyy Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011mLh2cdhBwJrK5hBZGnNyy
# Conflicts: # core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp # core/libs/prx/libSceAgcDriver/tests/cmake/agc_driver_recorder_tests.cmake
✅ NVIDIA GPU test: passedPR head
Unstable on main (3, these also failed in recent main runs on this machine; not counted against this PR)Failing on main too (3, not counted against this PR)Validated but not compared (3, not in the main baseline's validation pass)Validation IDs already present on main (4 IDs in 525 tests)Skipped (49)ReproduceAutomated build and test on a contributor's NVIDIA machine; it does not replace upstream CI. A new comment is posted for each new push. Full logs. |
|
@EagleBearerYS, a Windows/AMD data point for this PR: GTA V (PPSA04264, 01.011.001) in the Performance graphics mode, gameplay of the opening level, warm caches.
Details
|
XairRrR
left a comment
There was a problem hiding this comment.
Reviewed — comments
On Linux/lavapipe the merge is clean, it builds, the checks pass, and the three new agc_untracked_* tests pass here and fail on main's production code. Two limits: the PR's own hinted stamp only runs on Windows, so this machine can't exercise it, and the APS5_VERIFY_UNTRACKED_PROOFS self-check cannot detect a stale view or storage image.
What I ran
Head 8468301cddcc8d5ba3992077b94084c449293b1f, main 4f41f4c375f5abb487dbd184fbc150f31994d762, merged tree 5ae2065315ccc0445f9f5092dfbd6ec1d619d5ce. Debian 13 VM (Xeon Gold 6136), GCC 14.2.0, Release, lavapipe (Mesa 26.1.6, LLVM 19).
git diff --check 4f41f4c375f5abb487dbd184fbc150f31994d762...8468301cddcc8d5ba3992077b94084c449293b1f
python3 tools/check_conventions.py --base 4f41f4c375f5abb487dbd184fbc150f31994d762
env VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation ANYPS5_NO_SHADER_CACHE=1 ANYPS5_REQUIRE_VULKAN=1 ctest --test-dir build -V -R '^agc_driver_native_depth_uniform_planes$'
# negative control: the 6 production files (GuestMemory.hpp/.cpp, ShaderResources.hpp/.cpp, Texture.hpp/.cpp) restored from main
ninja -C build agc_driver_recorder_tests agc_driver_native_multisample_depth_tests
ctest --test-dir build --output-on-failure -R '^agc_untracked_'
env VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation ANYPS5_NO_SHADER_CACHE=1 ANYPS5_REQUIRE_VULKAN=1 ctest --test-dir build -V -R '^agc_driver_native_depth_uniform_planes$'
# merged files restored, rebuilt
ctest --test-dir build --output-on-failure --repeat until-fail:5 -R '^agc_untracked_|^agc_write_tracking$'
git status --porcelain # empty
Results
| Check | Result |
|---|---|
git diff --check |
clean |
check_conventions.py |
all checks passed |
| Configure / build / libs (script) | ok |
| Selected tests (script) | 2,232 run, 2,111 passed, 114 skipped (all expected), 0 new failures, 0 timeouts; the 7 known failures also fail on main |
agc_untracked_forget, agc_untracked_revalidate, agc_untracked_hint_proof, agc_write_tracking |
pass, 5 of 5 in a row; each takes under 0.2 s |
Validation (script): new VUID in agc_driver_native_depth_uniform_planes |
not from this PR. The test is newer than the baseline (d1ae628bc2), so the script had no main result to compare with. With main's production files I get the same 4 × VUID-vkCmdDispatch-groupCountX-00386. The test passes in both. |
| Validation layer loaded | 769 of 770 tests; layer_missing_in is empty |
uniform_planes with main's production files
1/1 Test #1852: agc_driver_native_depth_uniform_planes ... Passed 0.40 sec
4 × Validation Error: [ VUID-vkCmdDispatch-groupCountX-00386 ] | MessageID = 0x8e927036
Negative control
I restored only the six production files from main and rebuilt. All three new tests fail:
output
agc_untracked_revalidate: AGC graphics: a sampled template over untracked memory took the full walk in the same collect epoch
agc_untracked_hint_proof: the block has no write-watch hint: its proofs are expected to follow the collect epoch
AGC graphics: a sampled template over untracked memory took the full walk in the same collect epoch
0% tests passed, 3 tests failed out of 3
These failures come from #6192's in-epoch part, which this branch carries. On Linux, UntrackedHint returns 0, so agc_untracked_hint_proof takes its "no hint" branch. The hinted stamp this PR adds (untrackedRangeStamp's hinted[0] != 0 branch, HintedComparison, the hinted proofKey) never runs on this machine, and this control does not cover it. The red/green for that path is the author's RTX 3070 run on Windows. I restored the merged files afterwards; git status --porcelain is empty.
Findings
Blocking
None.
Non-blocking
APS5_VERIFY_UNTRACKED_PROOFScan't detect a stale view or storage image. The check is atcore/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp:3880. It counts a disagreement only whenfullWalk()returns false.fullWalkcompares only pointer identity:cachedTexture(...) != textures[textureIndex]atShaderResources.cpp:3806andexpected != storageTextures[storageIndex]atShaderResources.cpp:3817.- A storage image (and a view of one) is refreshed in place, so the lookup returns the same pointer. A missed store into a view's or storage image's untracked range is therefore never counted.
- The full walk also refreshes the image as a side effect, so verify mode hides the stale content it is meant to catch.
- In this PR the hinted stamp applies exactly to views and storage images (through
HintedComparison). The "0 disagreed" figures (38,000 in Village, 300,000 in GTA V) are therefore evidence only for snapshots. - Suggested fix: record each
validatedTextures[i].sourceandstorageTextures[i]Version()before the full walk, and also count a disagreement when any of them moves.
- The speed claim doesn't isolate the hinted stamp. CnRJay raised this, and it is still open in the description:
- The Village pair and Nesyth's GTA V pair measure #6192 and this PR together.
- The only isolated runs, chitters2004's North Yankton pair and Story split, show no measurable gain from the hinted stamp.
- The isolated Village pair the author said was queued is not in the description yet.
- The rules are met (summary lines from two commits and traces are attached), but a run of
main+ #6192 againstmain+ #6192 + this PR on the Village menu would show whether this PR's own lines pay off.
- Not verified here: thread safety of
HintedComparison. It reads the mutable, non-atomiccomparedRangesof aStorageTexturefromcaptureValidation;compareUntrackedwrites that array. Please confirm that both run under the same lock (the GPU mutex) or on the same thread.
Not verified
- The hinted path itself (Windows only). I took the hint-key behaviour, the cross-thread cases in
agc_untracked_hint_proof, andCheckUntrackedHintAcrossThreadsinagc_write_tracking(compiled only under_WIN32) from the author's RTX 3070 report. - Titles. I launched none; per the review rules, no title or game is run.
- Other GPUs. I tested only lavapipe.
To get this to a good state
- Make the self-check compare image versions (finding 1). Then rerun it on a title where hinted stamps are taken, so that "0 disagreed" also covers views and storage images.
- Add the isolated #6192-vs-#6192+this-PR Village measurement, or state in the description that the gain shown is #6192's.
- Confirm the lock under which
HintedComparisonreadscomparedRanges.
✅ NVIDIA GPU test: passedPR head
Unstable on main (4, these also failed in recent main runs on this machine; not counted against this PR)Failing on main too (3, not counted against this PR)Validated but not compared (3, not in the main baseline's validation pass)Validation IDs already present on main (4 IDs in 517 tests)Skipped (48)ReproduceAutomated build and test on a contributor's NVIDIA machine; it does not replace upstream CI. A new comment is posted for each new push. Full logs. |
✅ AMD GPU test: passedPR head
Unstable on main (2, these also failed in recent main runs on this machine; not counted against this PR)Failing on main too (8, not counted against this PR)Validated but not compared (3, not in the main baseline's validation pass)Validation IDs already present on main (4 IDs in 517 tests)Skipped (58)ReproduceAutomated build and test on a contributor's AMD machine; it does not replace upstream CI. A new comment is posted for each new push. Full logs. |
|
Closing this PR. Thanks to XairRrR for the reviews, to chitters2004 and Nesyth for the Grand Theft Auto V runs, and to QRMOE for agreeing an order with #12575. The isolated runs, chitters2004's North Yankton pair and Story split, show no measurable gain from the hinted stamp on its own, and a hinted check costs about 1.1 µs. The measured gain in these runs is #6192's, which #14262 carries forward. @QRMOE, #12575 no longer waits on this PR, so the The branch stays on the fork. |
What
Windows only in effect. Write-watch hints exist only on Windows (
GuestMemory::UntrackedHintreturns 0 on other platforms), so on Linux this changes nothing: every untracked proof keeps #6192's epoch stamp.#6192 lets a fast revalidation over untracked memory accept a surface by its stamp until the next ordering point: the stamp is
{collect epoch, NewestStamp, ForgetStamp, write note}. In Resident Evil Village (PPSA01556)'s static menu, collect epochs advance about 80 times a second, so most of those proofs expire before they're reused, and the next one walks the full table again.On Windows, an unwatched range has a write-watch hint (
GuestMemory::UntrackedHint), and a store into it moves the hint at the next walk of that page's marks. The check runs that walk itself:UntrackedHintreads and resets the range's marks (walkHints) unless this thread already walked the range in the current collect epoch. This change keeps a proof across epochs while the hint hasn't moved.{hint, NewestStamp, ForgetStamp, write note, 1}. Every field is taken just before the bytes the proof stands on are read or compared, not when the proof is captured:untrackedKey, carried in the lookup record (LookupRecord::untracked);StorageTexture::compareUntracked, throughStorageTexture::HintedComparison.NewestStamp, a forget movesForgetStamp, and a new recorded GPU write moves the note. So at the next ordering point the check walks again.APS5_VERIFY_UNTRACKED_PROOFS=1still cross-checks every fast proof that used an untracked stamp, hinted or epoch, with the full walk.The branch. It holds #6192's four commits unchanged (with #6187 under them), then four of ours:
main. perf(agc): fast revalidation over untracked memory proves a surface by its stamp until the next ordering point #6192 conflicts with current main inTexture.cpp(main's per-rangecomparedRangesmemo kept, with perf(agc): an untracked comparison memo holds until memory it covers is forgotten #6187'sForgetStampin place ofForgetSerial),ShaderResources.cpp(main'sLastRecordIndices,GenerationForand depth guard kept, with the untracked stamps added) andCMakeLists.txt(the Windows test list).ENVIRONMENT.mdrow forAPS5_VERIFY_UNTRACKED_PROOFS, which main's conventions check requires.agc_untracked_forget's teardown fix, offered on perf(agc): an untracked comparison memo holds until memory it covers is forgotten #6187: the test freed its detiler while a recorded dispatch still used its pipeline, which crashed lavapipe's queue thread. It's test-only.Overlap: checked with
git merge-treeagainst each pull request the overlap comment lists, at head fdc7004.main: the same four commits of TheAggie's, rebased and kept under TheAggie's name, with oneandonlydean's test fixes. This PR carries the original commits. Once perf(agc): fast revalidation over untracked memory proves a surface by its stamp until the next ordering point #14262 lands, this PR's next merge ofmaintakes their changes frommain, and this PR's diff is then only its hinted stamp.GuestSnapshots()where this PR addsHintedComparison(), inTexture.hppandTexture.cpp. The clash is textual only: whichever lands second keeps both.cachedTexture's sampled-cache hit branch as a per-entry check. This PR adds one argument in that branch: the untracked proof's key, passed tologLookup. QRMOE agreed in perf(agc): reuse a texture the sampled cache refuses while frames keep using it #12575's thread that this PR goes first, and carries itslogLookupargument into the per-entry form when perf(agc): reuse a texture the sampled cache refuses while frames keep using it #12575 is rebased (comment).CMakeLists.txt), and perf(agc): reuse a cached resource build that samples a texture remade per use #10894 (ShaderResources.cpp), conflict with perf(agc): fast revalidation over untracked memory proves a surface by its stamp until the next ordering point #6192's commits, which this PR carries. The conflicting hunks are the same with perf(agc): fast revalidation over untracked memory proves a surface by its stamp until the next ordering point #6192's head merged withmain, and this PR's own lines don't touch them, so their order follows perf(agc): fast revalidation over untracked memory proves a surface by its stamp until the next ordering point #6192's. feat(agc): count why a template's collect was refused #11550 has merged, and 31dff93 merges it (Tested).DepthSurface's HTILE clear check of untracked memory on the sameUntrackedHint. It's a different consumer of the same hint.Resident Evil Village (PPSA01556)
Static menu, one block of four unprofiled runs, B A A B.
main319621b plus fix(shader): load and store 16-bit data of converted packed images #11580 and fix(agc): keep buffer owners when a staged address space dissolves #10304, with a localscePadOpenanswer for port types 3 and 4 (103015a). B is the same tree plus perf(agc): fast revalidation over untracked memory proves a surface by its stamp until the next ordering point #6192's four commits and this PR's commit (24b56bc: the same added and removed lines as this head, apart from theagc_untracked_forgetteardown fix, which only changes a test). So the pair measures perf(agc): fast revalidation over untracked memory proves a surface by its stamp until the next ordering point #6192 and this PR together; perf(agc): fast revalidation over untracked memory proves a surface by its stamp until the next ordering point #6192 alone wasn't run.APS5_LAZY_SHADER_PREPARE=1 APS5_NO_GPL=1 APS5_HOST_IMPORT_MIB=6400 APS5_TRACE_SUMMARY_EVERY=10, with scripted pad input. The change has no switch, so nothing else differs.Driver.Packet3,926 / 3,368 against 4,944 / 4,253, about 21 % lower.Driver.Packet.memory_wait2,277 / 1,943 against 2,898 / 2,482.Graphics.Draw782 / 689 against 1,056 / 907.APS5_PROFILE_DRAW=1; menu +10..15, 31 ten-second[rescache]blocks):APS5_VERIFY_UNTRACKED_PROOFS=1, so its revalidation time includes the self-check's extra full walks.diag/village-combined-on-103015a9cac9cff8c, the A tree above plus perf(agc): bypass the draw cache while its hit rate is under 5% #12478, perf(agc): prove cached image tables from a write index behind APS5_TABLE_INDEX #12745 and perf(agc): keep an untracked fast proof across collect epochs on Windows while its write-watch hint has not moved #12921, run withAPS5_TABLE_INDEX=1.Driver.Packetper frame: 3,822 and 3,568 ms against 5,187 and 5,301 ms.memory_wait: 2,199 and 2,032 ms against 3,090 and 3,167 ms.APS5_TABLE_INDEX=1:agc_untracked_revalidateandagc_untracked_hint_proof, expect a full walk after a collect epoch. The index proves those draws by looking the elements up again, which re-reads the memory, so the content still follows every store.test/untracked-proof-index-on-12745, for perf(agc): prove cached image tables from a write index behind APS5_TABLE_INDEX #12745 if perf(agc): keep an untracked fast proof across collect epochs on Windows while its write-watch hint has not moved #12921 lands first;test/untracked-proof-index-on-12921, for perf(agc): keep an untracked fast proof across collect epochs on Windows while its write-watch hint has not moved #12921 if perf(agc): prove cached image tables from a write index behind APS5_TABLE_INDEX #12745 lands first.ctest -R '^agc'gives 1,048 passed, 73 skipped and 0 failed in both modes.Summary lines at menu +10 of the first B and A runs (B 07f624d, A 103015a). Each row is one 10 s interval, so its frame times aren't the window's figures above.
{"commit":"07f624d854fb","dirty":false,"env":{"APS5_HOST_IMPORT_MIB":"6400","APS5_LAZY_SHADER_PREPARE":"1","APS5_NO_GPL":"1","APS5_PAD_INPUT_FILE":"<path>","APS5_TRACE_DIR":"<path>","APS5_TRACE_SUMMARY_EVERY":"10"},"gpu":"NVIDIA GeForce RTX 3070 Laptop GPU","driver":"NVIDIA 617.42","os":"Windows 10.0.26300","cpu":"AMD Ryzen 9 5900HS with Radeon Graphics","cpu_threads":16,"memory_mib":15776,"title_id":"PPSA01556","title":"Resident Evil Village","content_version":"01.210.000","resolution":"3840x2160","window":{"spec":"summary_every 10","first_present":1605,"ms":10237.489},"interval_start_s":836.978,"interval_end_s":847.215,"presents":6,"frame_ms":{"p50":1738.099,"p90":2336.798,"p99":2336.798,"max":2336.798},"gpu_busy_ms_per_present":243.520,"worker_busy_pct":{"queue worker 0x0":99.570,"queue worker 0x38":0.112,"queue worker 0x20":89.391,"queue worker 0x28":94.215},"top_phases_ms_per_frame":{"Driver.Packet":4828.637,"Driver.Packet.memory_wait":2822.733,"Driver.Packet.draw":1269.158,"Driver.Draw":1267.783,"Graphics.Draw":996.479,"Graphics.Draw.draw_and_resource_release":970.327,"Driver.Packet.dispatch":693.763,"Driver.Dispatch":639.245},"trace_dropped":false,"trace":""} {"commit":"103015a9ca66","dirty":false,"env":{"APS5_HOST_IMPORT_MIB":"6400","APS5_LAZY_SHADER_PREPARE":"1","APS5_NO_GPL":"1","APS5_PAD_INPUT_FILE":"<path>","APS5_TRACE_DIR":"<path>","APS5_TRACE_SUMMARY_EVERY":"10"},"gpu":"NVIDIA GeForce RTX 3070 Laptop GPU","driver":"NVIDIA 617.42","os":"Windows 10.0.26300","cpu":"AMD Ryzen 9 5900HS with Radeon Graphics","cpu_threads":16,"memory_mib":15776,"title_id":"PPSA01556","title":"Resident Evil Village","content_version":"01.210.000","resolution":"3840x2160","window":{"spec":"summary_every 10","first_present":3369,"ms":11746.745},"interval_start_s":979.116,"interval_end_s":990.863,"presents":6,"frame_ms":{"p50":2124.387,"p90":2317.005,"p99":2317.005,"max":2317.005},"gpu_busy_ms_per_present":285.700,"worker_busy_pct":{"queue worker 0x0":99.617,"queue worker 0x38":0.114,"queue worker 0x20":90.888,"queue worker 0x28":95.018},"top_phases_ms_per_frame":{"Driver.Packet":5590.476,"Driver.Packet.memory_wait":3348.407,"Driver.Packet.draw":1541.600,"Driver.Draw":1540.185,"Graphics.Draw":1190.041,"Graphics.Draw.draw_and_resource_release":1161.247,"Driver.Packet.dispatch":661.397,"Driver.Dispatch":602.462},"trace_dropped":false,"trace":""}APS5_TRACE_SECONDS=960:30and no other trace or profile switch:16eab88b27f0843a5aa46433a83dbb69cc1fd960ff2a521acade7d20b52fb809;bb08a9ac103f778937429c3ead8343ed6b3e87b6abf7bc7ff8c471b125913f0b.trace_digest.py of A (103015a)
seconds 960:30 from present 2901: 21 presents, frame p50 1497.6 / p90 1783.2 / p99 1903.7 / max 1903.7 ms, GPU busy 230.6 ms per present
NVIDIA GeForce RTX 3070 Laptop GPU, NVIDIA 617.42, Windows 10.0.26300, 3840x2160, commit 103015a, dropped: no
{"commit":"103015a9ca66","dirty":false,"env":{"APS5_HOST_IMPORT_MIB":"6400","APS5_LAZY_SHADER_PREPARE":"1","APS5_NO_GPL":"1","APS5_PAD_INPUT_FILE":"<path>","APS5_TRACE_SECONDS":"960:30"},"gpu":"NVIDIA GeForce RTX 3070 Laptop GPU","driver":"NVIDIA 617.42","os":"Windows 10.0.26300","cpu":"AMD Ryzen 9 5900HS with Radeon Graphics","cpu_threads":16,"memory_mib":15776,"title_id":"PPSA01556","title":"Resident Evil Village","content_version":"01.210.000","resolution":"3840x2160","window":{"spec":"seconds 960:30","first_present":2901,"ms":31778.408},"presents":21,"frame_ms":{"p50":1497.603,"p90":1783.23,"p99":1903.699,"max":1903.699},"gpu_busy_ms_per_present":230.559,"worker_busy_pct":{"queue worker 0x0":99.554,"queue worker 0x38":0.057,"queue worker 0x28":93.25,"queue worker 0x20":89.065},"top_phases_ms_per_frame":{"Driver.Packet":4261.674,"Driver.Packet.memory_wait":2505.215,"Driver.Packet.draw":1156.599,"Driver.Draw":1155.503,"Graphics.Draw":928.353,"Graphics.Draw.draw_and_resource_release":907.06,"Driver.Packet.dispatch":560.368,"Driver.Dispatch":510.148},"trace_dropped":false,"trace":"aps5-trace-103015a9ca66-33724.json.gz"}Slowest frames (largest slices inside each, inclusive)
Threads (ms per frame)
Largest slices (ms per frame, inclusive, all threads): Driver.Packet 4250.6, Driver.Packet.memory_wait 2505.1, Driver.WorkerWait 1594.9, Driver.WorkerWait.dequeue_order 1338.7, Driver.Packet.draw 1155.6, Driver.Draw 1154.5, Graphics.Draw 927.6, Graphics.Draw.draw_and_resource_release 906.3, Driver.Packet.dispatch 559.1, Driver.Dispatch 508.7
Longest waits
Largest GPU batches (GPU ranges inside, and the CPU slices that recorded each, inclusive)
trace_digest.py of B (07f624d)
seconds 960:30 from present 1815: 19 presents, frame p50 1438.8 / p90 2203.1 / p99 2511.8 / max 2511.8 ms, GPU busy 255.7 ms per present
NVIDIA GeForce RTX 3070 Laptop GPU, NVIDIA 617.42, Windows 10.0.26300, 3840x2160, commit 07f624d, dropped: no
{"commit":"07f624d854fb","dirty":false,"env":{"APS5_HOST_IMPORT_MIB":"6400","APS5_LAZY_SHADER_PREPARE":"1","APS5_NO_GPL":"1","APS5_PAD_INPUT_FILE":"<path>","APS5_TRACE_SECONDS":"960:30"},"gpu":"NVIDIA GeForce RTX 3070 Laptop GPU","driver":"NVIDIA 617.42","os":"Windows 10.0.26300","cpu":"AMD Ryzen 9 5900HS with Radeon Graphics","cpu_threads":16,"memory_mib":15776,"title_id":"PPSA01556","title":"Resident Evil Village","content_version":"01.210.000","resolution":"3840x2160","window":{"spec":"seconds 960:30","first_present":1815,"ms":30085.902},"presents":19,"frame_ms":{"p50":1438.787,"p90":2203.059,"p99":2511.784,"max":2511.784},"gpu_busy_ms_per_present":255.744,"worker_busy_pct":{"queue worker 0x0":94.143,"queue worker 0x38":0.263,"queue worker 0x28":87.619,"queue worker 0x20":83.028},"top_phases_ms_per_frame":{"Driver.Packet":4195.016,"Driver.Packet.memory_wait":2410.236,"Driver.Packet.draw":1091.817,"Driver.Draw":1090.727,"Graphics.Draw":809.924,"Graphics.Draw.draw_and_resource_release":788.767,"Driver.Packet.dispatch":649.726,"Driver.Dispatch":595.737},"trace_dropped":false,"trace":"aps5-trace-07f624d854fb-36204.json.gz"}Slowest frames (largest slices inside each, inclusive)
Threads (ms per frame)
Largest slices (ms per frame, inclusive, all threads): Driver.Packet 4184.1, Driver.Packet.memory_wait 2410.1, Driver.WorkerWait 1639.1, Driver.WorkerWait.dequeue_order 1381.6, Driver.Packet.draw 1090.9, Driver.Draw 1089.8, Graphics.Draw 809.3, Graphics.Draw.draw_and_resource_release 787.9, Driver.Packet.dispatch 648.6, Driver.Dispatch 594.5
Longest waits
Largest GPU batches (GPU ranges inside, and the CPU slices that recorded each, inclusive)
Grand Theft Auto V (PPSA04263), by @chitters2004
On Windows 11, a Ryzen 7 5800X and an RTX 3080 (driver 621.01), with the same machine, scene and save in both arms.
e8efeed58a+ fix(shader): load and store 16-bit data of converted packed images #11580, sincemainstops at Story load. The change merges 23ef222 into it, so it adds perf(agc): fast revalidation over untracked memory proves a surface by its stamp until the next ordering point #6192's four commits and this PR's together.APS5_PROFILE_DRAW=1,Revalidatefull walks refused bycollectfell from 3,380 to 1,530 ms per 10 s.untrackedRangeStamp's hinted branch is removed, so every proof takes perf(agc): fast revalidation over untracked memory proves a surface by its stamp until the next ordering point #6192's epoch stamp. Two runs each, alternating, with no crashes.collectrefusals per 10 s are on plain sampled snapshots recorded with the epoch stamp.GuestArenaCollectWritesrefuses their ranges: in one 10 s sample, all 19,091 refusals came fromcollectable, on direct-memory pages markedcomparedWhenShared. SowalkHintsfails andUntrackedHintreturns 0. This is the shared-pages case under What's "Unhinted ranges".main52c09d5 + fix(shader): load and store 16-bit data of converted packed images #11580 + perf(agc): fast revalidation over untracked memory proves a surface by its stamp until the next ordering point #6192 against the same plus this PR, three runs each.APS5_VERIFY_UNTRACKED_PROOFS=1 APS5_VERIFY_PROOFS=1) compared 300,000 fast proofs with the full walk, with 0 disagreements.Grand Theft Auto V (PPSA04264), by @Nesyth
Windows 11 Pro 26300, a Radeon RX 7900 XTX (AMD Software 26.10.1), a Ryzen 7 7800X3D, 3840x2160 output. Version 01.011.001 in the Performance graphics mode, gameplay of the opening level with warm caches (comment). The arms are
main9090972 and the same commit with this PR's head 793fc7f merged, so the second carries #6192's four commits as well as this PR's. Each run is a separate boot, and FPS is the window title's counter over 57 s of scripted gameplay.main, against 17.32 and 18.70 with the change (+20 to +29 %). WithAPS5_PROFILE_DRAW=1: 12.78 and 13.56, against 15.60 and 16.36.main). Draw commit: 39.9 and 37.6 µs, against 31.3 and 30.3 µs. Collect full walks: 3.83 and 3.58 s per 10 s, against 2.42 and 2.22 s. Collect refusals per 10 s: 50,181 over 163 surfaces and 41,172 over 125, against 30,269 over 120 and 30,118 over 119. Every remaining refusal is counted as unwatched.mainwith perf(agc): fast revalidation over untracked memory proves a surface by its stamp until the next ordering point #6192 and this PR together, so it doesn't separate their shares. On the Story save above, the split put the gain on perf(agc): fast revalidation over untracked memory proves a surface by its stamp until the next ordering point #6192.Tested
agc_untracked_hint_proof(new), with two sampled 64 KiB surfaces over write-watched, host-imported, then unwatched memory:MarkWrittendriver store; a recorded GPU write note, before and after the write is reaped; and a CPU store made after the hint is sampled but before the stamp is recorded, through a flush hook.settlere-proves the stamps after each one. No case asserts a throw.agc_untracked_hint_proof,agc_untracked_revalidateandagc_untracked_forgetpassed 10 of 10 each.agc_untracked_hint_proof,agc_untracked_revalidateandagc_untracked_forgetpassed 20 of 20 each. Linux has no hints, so the new test runs there under the epoch rule and checks that nothing changes.ctest -R '^agc' --parallel 4: 1,119 tests: 1,046 passed, 73 skipped, 0 failed.agc_untracked_forgetcrashes on lavapipe 20 of 20 times.check_conventions.pyandcheck_test_registration.pypass. MinGW-w64-fsyntax-onlyadds no errors or warnings tomain's. Nothing undercore/shader/**changes, so the shader cache key is unchanged.mainplus the teardown fix (e440b74), so it differs from this head only by the hint key.agc_untracked_hint_proof,agc_untracked_revalidateandagc_untracked_forgetpassed 10 of 10 under validation.^agcunder validation: 1,119 run, 73 skipped, 1 failed. The failure isagc_driver_image_table, whose only validation message isVUID-VkDeviceCreateInfo-pNext-pNextfromvkCreateDevice("unknown VkStructureType (1000574000) ... version 275 of the Vulkan header"). That's the runner's validation layer being older than the headers. The comparison arm fails that test the same way, and the change adds no VUID.APS5_SKIP_TIMING_TESTS=1, pass: 191 and 291 on Linux, 220 and 314 on Windows.agc_driver_image_tablewithout the validation layer (run 40890382693) passes on both sides. On Linux, it ran with the branch's 78 tests: 71 passed, 7 skipped, 0 failed. On Windows, 94 ran, 4 passed and 90 skipped, since the runner has no device. The suites match the run above.main673fa86, which brings johandavidcm's feat(agc): count why a template's collect was refused #11550 (run).fastRevalidate, in lines perf(agc): fast revalidation over untracked memory proves a surface by its stamp until the next ordering point #6192 changes. feat(agc): count why a template's collect was refused #11550 counts each collect refusal withcountCollectRefusal, and perf(agc): fast revalidation over untracked memory proves a surface by its stamp until the next ordering point #6192 lets an untracked surface whose stamp still matches pass instead of refusing.countCollectRefusalnow sits inside perf(agc): fast revalidation over untracked memory proves a surface by its stamp until the next ordering point #6192's stamp check, just beforereturn fail(FastFail::Collect). So only collects that are still refused are counted, and an untracked surface its stamp proves isn't.agc_driver_resource_cache_collect_refusals; the untracked-proof testsagc_untracked_revalidate,agc_untracked_forgetandagc_untracked_hint_proof;agc_write_tracking; and the^agc_driver_image_tabletests, with the tests for the files the merge changes.APS5_SKIP_TIMING_TESTS=1, which relaxesprecise_wait's median bounds on GitHub's hosted runners.agc_untracked_hint_proofgains three cases:agc_write_trackinggains a device-free case on Windows:agc_untracked_hint_proof,agc_untracked_revalidate,agc_untracked_forgetandagc_write_trackingpass 10 of 10 each.agc_write_trackingalone passes 20 of 20.agc_write_tracking's device-free cases run there.APS5_SKIP_TIMING_TESTS=1, which relaxesprecise_wait's median bounds on GitHub's hosted runners.UntrackedHintmemo made to ignore the collect epoch (a measurement branch, not part of this PR):agc_write_trackingfails with "the last store of a racing thread did not move the hint past every value read before it", andagc_untracked_hint_prooffails at its existing step, "a CPU store after a collect epoch kept the fast proof".agc_write_trackingwith the same message, andagc_untracked_hint_proofskips there. Linux passes: it has no write-watch hints, so its ranges keep the epoch stamp.main8304cca (run).CMakeLists.txt, the WindowsENVIRONMENT_MODIFICATIONtest list, where fix(agc): free a completed batch's command buffer instead of keeping it #9565 addsagc_recorder_command_buffer_release. The resolution is the union of both sides' names, each once.^agc_untracked_tests,agc_write_tracking,agc_driver_recorderand main'sagc_recorder_command_buffer_release.APS5_SKIP_TIMING_TESTS=1, which relaxesprecise_wait's median bounds on GitHub's hosted runners.main0836dfd (run). build(agc): register each AGC test executable in its own CMake fragment #12862 movedlibSceAgcDriver's test registrations intotests/cmake/; the conflict inCMakeLists.txtis resolved withtools/agc_test_split.py resolve, which moves this PR's registrations intotests/cmake/agc_driver_recorder_tests.cmake(+7 −1). Nothing else conflicted. Apart from the move, this PR's diff is unchanged. With-DBUILD_TESTING=ON,ctest -Ngives 3078 tests:main's 3075 plus its 3 tests, each listed once.agc_driver_recorder,agc_recorder_command_buffer_release,agc_untracked_forget,agc_untracked_hint_proof,agc_untracked_revalidate,agc_write_trackingpass 10 runs in a row. Linux: 6 run, 6 pass, 0 skip, 0 fail. Windows: 6 run, 1 pass, 5 skip (no Vulkan device), 0 fail. Suites: libc and libkernel pass on Linux (199 and 308) and on Windows (226 and 330). The suites ran withAPS5_SKIP_TIMING_TESTS=1, which relaxesprecise_wait's median bounds on GitHub's hosted runners.tests/Recorder.cpp).untrackedForgetTests,untrackedRevalidateTestsanduntrackedHintProofTestsreturnfalseat each of their 10 early outs andtrueat the end, and their--untracked-*-onlybranches return? 0 : 77, asSKIP_RETURN_CODE 77expects.APS5_SKIP_TIMING_TESTS=1, which relaxesprecise_wait's median bounds on GitHub's hosted runners.main291e02d, Ubuntu 26.04.1, 2,540 of 3,155 tests selected.main.VK_LAYER_KHRONOS_validation, no new validation IDs over 934 GPU tests.agc_untracked_forget,agc_untracked_revalidateandagc_untracked_hint_proofpassed on the GPU, so they ran in full there, since this head reports 77 when they can't. They aren't inmain's baseline, so their validation wasn't compared.maine6f5730, driver 595.99.02, Ubuntu 26.04.1, 2,535 of 3,148 tests selected.main, and 3 are unstable onmain.VK_LAYER_KHRONOS_validation, no new validation IDs over 934 GPU tests. The threeagc_untracked_*tests passed on the GPU, so they ran in full there too. They aren't inmain's baseline, so their validation wasn't compared.mainf6232fc. It resolves the conflicts inShaderResources.hppandtests/cmake/agc_driver_recorder_tests.cmake. This PR's own lines are unchanged. The only difference in its diff is the WindowsENVIRONMENT_MODIFICATIONlist line, which now carries main'sagc_window_import_unmap.mainf6232fc, driver 595.99.02, Ubuntu 26.04.1, 2,795 of 3,743 tests selected. 2,740 passed, 6 failed and 49 skipped. None of the failures is new: 3 also fail onmain, and 3 are unstable onmain. WithVK_LAYER_KHRONOS_validation, no new validation IDs over 937 GPU tests.main4f41f4c (run).Texture.hpp,main's newOccasionalLabelStoresdeclaration and this PR'sHintedComparisondeclaration are adjacent, and both stay. Intests/cmake/agc_driver_recorder_tests.cmake, the WindowsENVIRONMENT_MODIFICATIONlist, wheremainaddsagc_changed_copy, the resolution is the union of both sides' names, each once. This PR's own lines are unchanged, and that list line is the only difference in its diff.agc_changed_copy,agc_recorder_read_listand every test matching^agc_untrackedor^guest_memory. Linux: 9 run, 9 pass, 0 skip, 0 fail. Windows: 8 run, 3 pass, 5 skip (no Vulkan device), 0 fail. The libc and libkernel suites run 300 and 322 tests on Linux and 327 and 342 on Windows, with 0 failed. The suites ran withAPS5_SKIP_TIMING_TESTS=1, which relaxesprecise_wait's median bounds on GitHub's hosted runners.Checklist
main; no other open PR implements the same functionsNotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file.mdfiles or images added to the repository (attach them to this PR)Graphics/src/ShaderResources.cppand conflict there, and this PR takes the second merge with the test-only update ontest/untracked-proof-index-on-12921