Repository navigation
perf(libc, libkernel): keep aliased direct memory on the Linux write watch and collect stores through every alias - #12452
Conversation
|
Data for the GTA V A/B in the description, one zip per run, in run order:
Each zip holds:
All four runs used AI-assisted: yes (Claude Code) |
|
This pull request is labeled |
MarcZX10
left a comment
There was a problem hiding this comment.
The new GTA V A/B archives are useful: I downloaded all four, verified their README and summary metadata, and ran tools/trace_digest.py on the attached aps5-summary.jsonl files. They provide same-machine/same-scene data for main + #11580 (48919964c63c) versus the earlier PR build (690ca444159f). However, the PR is currently at b6535bf6cb6231cd39fd336234b15fbe85652692, and the report explicitly says the change capture predates the rebase onto current main; I cannot confirm that those numbers represent the current head. Also, the repository requires a perf PR description to contain two valid summary lines with different commits, and the current body has neither line; GitHub still shows perf numbers missing. Please update the description with paired current-main/current-head aps5-summary.jsonl lines, attach their traces, and include the tools/trace_digest.py output. The performance claim remains unverified for the current head until then. The overlap scan reports only the independent debt-file hunk in #10760; the hosted build and platform checks are still running. I am requesting changes for the missing current-head performance report, not alleging a code defect.
adriadam10
left a comment
There was a problem hiding this comment.
The change reads correctly: links are kept in both directions, cut on Register/Unregister, and the three collects report every page at each linked view. The new CheckDirectMemoryWriteWatchSeesWritesThroughAliases pins the reported pages for two and three views, and the updated expectations in the existing tests fail on main. What is left:
- The GTA V A/B was taken before the rebase (
690ca44415, base48919964c6), and the head is nowb6535bf6cb. @MarcZX10 asked for the pair on the current head and that is still open. Please rerun the base/PR pair onb6535bf6cb(or on the head after any further push), update the twoaps5-summary.jsonllines and thetrace_digest.pyoutput in the description, and attach the traces. Thelabelscheck is still pending on that. WindowCopy::Holdsbounds check (core/libs/prx/libSceAgcDriver/Execution/src/GuestMemory.cpp:930) has no test: only GTA V reached it. A test where a collect over an import window reports a store at a linked page outside the window would show the crash it prevents.- The link cut on
Registerover an already linked range (GuestWriteWatch.cpp:64) is not reached by any test either; remapping a view over an aliased one and checking that no store is reported at the old partner would cover it.
|
Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.
Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.
|
b6535bf to
dc64a22
Compare
MarcZX10
left a comment
There was a problem hiding this comment.
Follow-up on the new head dc64a22147: the added CheckDirectMemoryWriteWatchDropsAliasOnRemap and CheckImportWindowOverAlias tests now exercise the two previously uncovered paths (cutting a link on Register and rejecting a linked page outside an import window). I confirmed those cases are present in the exact-head diff, and python3 tools/check_conventions.py --base upstream/main plus git diff --check upstream/main...HEAD pass locally. I have not run the C++ tests on this host. The remaining blocker from my earlier review is still the perf report: the attached A/B captures use the pre-rebase build 690ca44415, not this head, and GitHub still reports perf numbers missing. Please attach a same-machine/title/scene main vs current-head pair, the two summary lines, traces, and tools/trace_digest.py output; the performance claim cannot yet be checked for this commit.
adriadam10
left a comment
There was a problem hiding this comment.
Both test gaps are covered: CheckDirectMemoryWriteWatchDropsAliasOnRemap pins that neither view reports the other's stores after a remap to other physical pages, and CheckImportWindowOverAlias pins the window collect reporting the linked page below the window. Each fails without the corresponding code (the link cut on Register, the Holds bound).
One point is left, the one @MarcZX10 still has open: the perf report. The GTA V pair and both aps5-summary.jsonl lines are from 690ca44415 on base 48919964c6; the head is now dc64a22147. The patch under test did not change in the rebase (range-diff shows =), so a pair on the current head is enough, no new analysis: rerun base/PR on the current head, update the two summary lines and the trace_digest.py output in the description, and attach the traces. Once that is in, this is ready to merge.
dc64a22 to
99fdd71
Compare
|
@MarcZX10 @adriadam10 The GTA V A/B is now on the current head. The description has the new tables, the two
One zip per run, in run order. The traces are recompressed with xz to fit the attachment limit;
Each zip holds:
The second PR run was left in gameplay after cutscene 2 and stopped by the 30-minute run limit. Its tables use only the cutscene windows. AI-assisted: yes (Claude Code) |
MarcZX10
left a comment
There was a problem hiding this comment.
Re-checked — changes requested
The new head adds the two regression cases requested in the earlier review: remapping one alias now checks that tracking is detached correctly, and the import-window case checks that a linked store outside the window is bounded. Those close the previously identified test gaps. The remaining blocker is still the performance comparison baseline.
What I ran
- Reviewed current head
99fdd715b0857c44abf120d4c1974c21d142f84aagainst currentmain319621ba2e578bd053b0baba2877fe70520e0737. - Compared the new head with the previously reviewed
dc64a2214708ca7bf3e69d0ff18aae4ebb3397a0; the new commit changes only the test file. - Inspected the two new alias/remap and import-window cases.
- Ran
git diff --check refs/review/base...refs/review/pr-12452. - Generated a merge preview against current
main. - Checked #11580’s current state and refreshed this PR’s hosted checks.
Results
| Check | Result |
|---|---|
git diff --check |
pass |
Merge preview against current main |
clean; tree d46006acea9dd9f061eedae09c99d781c4309703 |
| New remap and import-window tests | present at the current head; not run locally |
| Hosted conventions, policy, and platform checks | pass |
| Hosted Linux/product job | skipped |
| Author-reported test suite and GPU validation runs | useful evidence; not independently reproduced |
| Performance report | still uses a baseline including open #11580, not current main |
Findings
Blocking
The performance data remains a comparison of current PR head 99fdd715b0 against main 446bd8776c plus #11580. #11580 is still open, so that baseline is not current main 319621ba2e. The paired traces may measure the optimization accurately on top of #11580, but they do not meet the repository’s required main-versus-change comparison. Please recapture on current main or obtain maintainer agreement to an explicitly declared stacked baseline.
Non-blocking
The new tests address the two test-coverage gaps from the previous review. The remaining open limitation in the report is the baseline, not those test cases.
Sources
- Current alias/remap tests
- Current import-window regression
- Current PR description and performance report
- Still-open baseline PR #11580
Not verified
I did not rerun the C++ build, CTest, Vulkan validation, GTA V captures, or trace digests. Hosted Linux/product checks were skipped.
To get this to a good state
Provide a matched same-machine/title/scene capture pair using current main and this head, with both summary lines, traces, and digest output; alternatively, document maintainer-approved use of the stacked baseline and declare the dependency.
|
Correction: the cutscene 1 fps row double-counted each run's 60 s trace-window summary line. Recomputed from the 10 s lines alone, it is 12.53 / 13.51 / 13.80 / 13.22 (was 12.71 / 13.77 / 13.80 / 13.31), so the cutscene 1 gain is 2–10 %, not the 3–9 % in my comment above. Both PR runs are still above both base runs, and the description is updated; the cutscene 2 and comparison figures are unchanged. AI-assisted: yes (Claude Code) |
|
@MarcZX10 Thanks. On the baseline: GTA V can't reach these cutscenes on plain This PR doesn't depend on #11580 in code: the two touch different files, and the merge preview against
In the meantime, here is the same comparison on current
The comparisons and refused collects go away as in the description. Cutscene 2 gains 3–8 %; in cutscene 1 one run gains about 5 % and the other matches the base. AI-assisted: yes (Claude Code) |
99fdd71 to
60df389
Compare
MarcZX10
left a comment
There was a problem hiding this comment.
Re-checked — changes requested
The added alias and import-window regression cases address the earlier test gaps, but the performance report still does not provide a complete, accepted comparison against the current base.
What I ran
- Reviewed exact head
60df38935c6c26fbb05d47e38d8b9a791df05c61; the head's merge base with the fetched currentmain3a2b91981651569163768854a1fcdc7193b6667bisc40a9fbd1c373134505c12ce71013d903430e128. - Ran
git merge-tree --write-tree upstream/main HEAD; the current-main merge preview is clean (beb1054126a9341f130e7868627e1c3a177dc9c7). - Ran
python3 tools/check_conventions.py --base upstream/mainandgit diff --check upstream/main...HEAD; both passed. - Refreshed hosted checks: Windows, Ubuntu relinker, Windows relinker, ASan, Ubuntu register-context, conventions, eligibility and labels pass. macOS relinker/register-context/write-watch jobs remain pending; Linux product tests are skipped.
- Read the updated performance comment and the PR body. The body still documents the earlier pair based on
main446bd8776c+ #11580. The comment adds a table formain319621ba2e+ #11580, but it does not include the summary lines, trace files ortrace_digest.pyoutput for that pair. I did not rerun the C++ tests or captures locally.
Results
| Check | Result |
|---|---|
| Current-main merge preview | Clean, tree beb1054126a9341f130e7868627e1c3a177dc9c7 |
| Conventions and diff check | Pass |
| Hosted Windows and Ubuntu relinker checks | Pass |
| Hosted ASan and Ubuntu register-context checks | Pass |
| Hosted macOS relinker/register-context/write-watch checks | Pending |
| Hosted Linux product job | Skipped |
| Current-base performance report | Incomplete; table only in comment, without the paired summaries, traces and digest |
Findings
Blocking
- The required performance comparison is not yet reviewable against the current base. The complete report in the PR body uses the earlier
main446bd8776c+ #11580 baseline. The newer comment reportsmain319621ba2e+ #11580 results, but omits the exactaps5-summary.jsonllines, attached traces andtools/trace_digest.pyoutput for those runs. I also do not see maintainer agreement in the thread to use a stacked baseline; #11580 remains open. - The reported baseline predates a relevant current-main change. Current main includes #8930 (
33acc03447034b61424cc1c36143cc40ea02bad3), which changescore/libs/prx/libSceAgcDriver/Execution/src/GuestMemory.cpp; this PR also changes that file. The merge preview is clean, but the posted tests and captures do not verify the integrated tree or the current baseline.
Sources
- Updated PR test cases
- PR's
GuestMemory.cppchange - Current main's #8930
GuestMemory.cppchange - Newer baseline discussion
- Performance report requirements
Not verified
I did not independently run the C++ tests, the GPU captures, or trace digests. The posted exact summary lines and trace attachments remain the earlier pair; the current main + #11580 table is author-reported.
To get this to a good state
Rebase onto current main and rerun the focused alias/write-watch tests with #8930 present. Then provide the full same-machine/title/scene report for the current main and change (both summary lines, attached traces and digest output). If retaining main + #11580 as the baseline because plain main cannot reach the scene, record maintainer agreement to that stacked baseline and provide those same artifacts for the exact runs.
|
@MarcZX10 @adriadam10 This PR is now rebased onto current
One zip per run, in run order. The traces are recompressed with xz to fit the attachment limit;
Each zip holds the summary lines ( AI-assisted: yes (Claude Code) |
60df389 to
d10b767
Compare
…port window over an alias CheckDirectMemoryWriteWatchDropsAliasOnRemap remaps an aliased direct memory view with MAP_FIXED to other physical pages and checks that neither view reports the other's stores afterwards. CheckImportWindowOverAlias opens an import window over one of two linked views and stores through the other, so the window collect reports a page below the window. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
d10b767 to
a160f78
Compare
|
@MarcZX10 Rebased onto current
AI-assisted: yes (Claude Code) |
MarcZX10
left a comment
There was a problem hiding this comment.
Re-checked — changes requested
Your update resolves the prior stale-baseline request against aaccd0ba39: it rebases to 1db077e876, reports all 18 focused tests and the negative controls on the new head, and explains why the captures still cover an unchanged production path. Current main is now ba5b5330b3. Please merge that current head and rerun the 18 focused write-watch/import/alias tests and negative controls. The current-main changes in the PR's files are limited to its debt entry, so the existing performance captures can remain if the updated diff confirms the measured production path is unchanged.
|
@MarcZX10 Merged
AI-assisted: yes (Claude Code) |
MarcZX10
left a comment
There was a problem hiding this comment.
Re-checked — new head, looks good
The requested current-main refresh and exact-head focused evidence are present, the measured production paths are unchanged since the capture baseline, and the required hosted checks pass.
What I ran
- Reviewed head
4621b11d0034ac851ca5691829050dc621504c8fagainst currentmain7aabd03d0a32bb928311ef1323b95dffb4618f7e; the merge preview is clean. - Compared the PR file list with changes from the reported measurement base
1db077e876to currentmain. The only overlapping path is the debt document; the measured write-watch, direct-memory, and import-window implementation paths are unchanged. - Refreshed hosted checks for the exact head. This review ran from a Linux shell; I did not invoke a compiler or run C++ tests locally.
- Read the author's exact-head test and negative-control report; those runs are author-reported, not independently reproduced here.
git fetch upstream main
git fetch upstream pull/12452/head:refs/remotes/review/pr-12452
git merge-tree --write-tree upstream/main refs/remotes/review/pr-12452
gh pr view 12452 -R boykopovar/AnyPS5 --json files --jq '.files[].path'
git diff --name-only 1db077e876 upstream/main
gh pr checks 12452 -R boykopovar/AnyPS5Results
| Check | Result |
|---|---|
Merge preview against current main |
Pass; clean tree 7e84b4ba1296e790168b317d2b7c652988478574 |
| Relevant path comparison since measured base | Only shared path is docs/dev/debt/libSceAgcDriver.md; measured production paths are unchanged |
| Hosted conventions, eligibility, labels, report, Windows, register-context, relinker, ASan, and macOS write-watch checks | Pass |
| Hosted Linux product job | Skipped |
| 18 focused write-watch/import/alias tests on exact head | Author reports pass with Vulkan validation; not locally reproduced |
991/992 guest_* tests on exact head |
Author reports 991 pass; guest_freebsd_clocks has the same failure on main |
| Three negative controls | Author reports each regression test fails when its corresponding fix is removed |
Unedited command output
7e84b4ba1296e790168b317d2b7c652988478574
docs/dev/debt/libSceAgcDriver.md
conventions pass 16s https://github.com/boykopovar/AnyPS5/actions/runs/40867583050/job/123426148406
eligibility pass 8s https://github.com/boykopovar/AnyPS5/actions/runs/40867580933/job/123426140489
labels pass 7s https://github.com/boykopovar/AnyPS5/actions/runs/40867583016/job/123426201836
register-context (macos-15-intel, Debug) pass 1m11s https://github.com/boykopovar/AnyPS5/actions/runs/40867580933/job/123426211921
register-context (macos-15-intel, Release) pass 1m13s https://github.com/boykopovar/AnyPS5/actions/runs/40867580933/job/123426211890
register-context (ubuntu-latest, Debug) pass 35s https://github.com/boykopovar/AnyPS5/actions/runs/40867580933/job/123426211919
register-context (ubuntu-latest, Release) pass 35s https://github.com/boykopovar/AnyPS5/actions/runs/40867580933/job/123426211986
relinker (macos-latest, clang++) pass 5m25s https://github.com/boykopovar/AnyPS5/actions/runs/40867580933/job/123426211893
relinker (macos-latest, clang++, x86_64) pass 6m22s https://github.com/boykopovar/AnyPS5/actions/runs/40867580933/job/123426211952
relinker (ubuntu-latest, g++) pass 2m42s https://github.com/boykopovar/AnyPS5/actions/runs/40867580933/job/123426212008
relinker (windows-latest, g++) pass 9m23s https://github.com/boykopovar/AnyPS5/actions/runs/40867580933/job/123426212076
relinker-asan pass 1m49s https://github.com/boykopovar/AnyPS5/actions/runs/40867580933/job/123426211857
windows pass 10m8s https://github.com/boykopovar/AnyPS5/actions/runs/40867580933/job/123426211959
linux skipping 0 https://github.com/boykopovar/AnyPS5/actions/runs/40867580933/job/123426213868
report pass 40s https://github.com/boykopovar/AnyPS5/actions/runs/40867580927/job/123426135322
write-watch (macos-15) pass 41s https://github.com/boykopovar/AnyPS5/actions/runs/40867580933/job/123426211914
write-watch (macos-15-intel) pass 1m51s https://github.com/boykopovar/AnyPS5/actions/runs/40867580933/job/123426211799
Negative control
| Assertion | Change made to code under test | New test failed? |
|---|---|---|
| Alias stores are observed through each linked view | Restored main's GuestWriteWatch.cpp and DirectMemory.cpp |
Yes, author-reported |
| Remapped aliases stop reporting stale linked stores | Removed the link cut in Register |
Yes, author-reported |
| Import-window alias collection stays within the window buffer | Removed the WindowCopy::Holds bound |
Yes, author-reported |
Findings
Blocking
None found in the reviewed exact head.
Non-blocking
The hosted Linux product job is skipped, and I did not independently reproduce the author's Linux/GPU test and capture runs.
Sources
- PR implementation and test files at reviewed head
- Performance report template at reviewed head
- Author's exact-head test and negative-control report
Not verified
I did not run a local C++ build, CTest, Vulkan validation, GTA V capture, trace digest, or console test. The focused Linux results and GPU captures are author-reported.
|
@Tehshortbus, a quick merge-order check for MarcZX10's review of #12565. #12565 changes one word ( AI-assisted: yes (Claude Code) |
✅ NVIDIA GPU test: passedPR head
Unstable on main (3, these also failed in recent main runs on this machine; not counted against this PR)Failing on main too (3, not counted against this PR)Validation IDs already present on main (5 IDs in 686 tests)Skipped (42)ReproduceAutomated build and test on a contributor's NVIDIA machine; it does not replace upstream CI. A new comment is posted for each new push. Full logs. |
|
@chitters2004 Thanks for checking. Since this PR was opened first, I'd keep it ahead of #12565. MarcZX10 has approved the current head The rebase is one line. I merged both heads ( AI-assisted: yes (Claude Code) |
|
@Tehshortbus Fair enough, and thanks for checking the merge. I get the same single conflict between AI-assisted: yes (Claude Code) |
|
Independent check in gameplay, since the description measures the opening cutscenes. GTA V EU (PPSA04263 01.010.002), Linux, RTX 3080 Ti (615.71.09), 4K.
So the effect holds in gameplay too, on another GPU and the EU release. AI-assisted; the runs are from my machine(s). |
… capture note Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@MarcZX10 @adriadam10
AI-assisted: yes (Claude Code) |
✅ NVIDIA GPU test: passedPR head
Unstable on main (3, these also failed in recent main runs on this machine; not counted against this PR)Failing on main too (3, not counted against this PR)Validation IDs already present on main (5 IDs in 711 tests)Skipped (51)ReproduceAutomated build and test on a contributor's NVIDIA machine; it does not replace upstream CI. A new comment is posted for each new push. Full logs. |
…alias capture note Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@MarcZX10 @adriadam10 #10760 merged, and its "Capture trace output" entry conflicted with this PR's edit of the next line in
AI-assisted: yes (Claude Code) |
✅ NVIDIA GPU test: passedPR head
Unstable on main (2, these also failed in recent main runs on this machine; not counted against this PR)Failing on main too (3, not counted against this PR)Validation IDs already present on main (4 IDs in 746 tests)Skipped (52)ReproduceAutomated build and test on a contributor's NVIDIA machine; it does not replace upstream CI. A new comment is posted for each new push. Full logs. |
✅ AMD GPU test: passedPR head
Unstable on main (2, these also failed in recent main runs on this machine; not counted against this PR)Failing on main too (13, not counted against this PR)Validation IDs already present on main (4 IDs in 750 tests)Skipped (59)ReproduceAutomated build and test on a contributor's AMD machine; it does not replace upstream CI. A new comment is posted for each new push. Full logs. |
|
Hi adriadam10, Your change request from 10-28 (at
|
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@MarcZX10 @adriadam10
AI-assisted: yes (Claude Code) |
✅ AMD GPU test: passedPR head
Unstable on main (4, these also failed in recent main runs on this machine; not counted against this PR)Failing on main too (13, not counted against this PR)Validation IDs already present on main (4 IDs in 764 tests)Skipped (60)ReproduceAutomated build and test on a contributor's AMD machine; it does not replace upstream CI. A new comment is posted for each new push. Full logs. |
✅ NVIDIA GPU test: passedPR head
Unstable on main (3, these also failed in recent main runs on this machine; not counted against this PR)Failing on main too (3, not counted against this PR)Validation IDs already present on main (4 IDs in 764 tests)Skipped (52)ReproduceAutomated build and test on a contributor's NVIDIA machine; it does not replace upstream CI. A new comment is posted for each new push. Full logs. |
What
On Linux, when a title maps one direct memory block at a second guest address,
WatchMappinginlibkernel/DirectMemory/DirectMemory.cppunregisters both views from the write watch. From then on the driver's tracker refuses every collect over them (GuestWriteWatchCoversis false). Textures over that memory fall back to whole-surface comparisons, and the resource caches over it to full revalidation walks.GTA V maps direct memory this way. In the first two Story cutscenes it compares 8,000–22,000 textures per 10 s (3–9.5 GiB), and the committer pays for it per draw.
The two views share physical pages, but the userfaultfd write-protect bit and
PAGEMAP_SCANare per virtual page: a store through one view marks only that view's page written. So both views can stay registered, as long as a collect over one also scans the other and reports its stores at both addresses. This PR does that:GuestWriteWatchAlias_nid_postfix(first, second, bytes)(libc): new export. It records a link in each direction between two page-aligned, watched ranges of the same size, and returns false (changing nothing) when the watch is unavailable or either range is not fully watched. It returns false on macOS and Windows.Collect,CollectFreshandCollectArmed(GuestWriteWatch.cpp): over a range with links, each also walks the linked ranges. Every written page found through any view is reported at its own address and at the address of each view linked to it.WatchMappinglinks every pair, so one level of indirection covers all of them.RegisterandUnregistercut the links of the range they change: the linked parts outside it are kept, on both sides.WatchMapping(libkernel): links an overlapping pair withGuestWriteWatchAlias_nid_postfix, and unregisters both views as before only when that returns false.WindowCopy::Holds(libSceAgcDriver/Execution/src/GuestMemory.cpp): returns false for a page outside the window. A collect over an import window can now report a store at a linked page outside it. Without the check,stampWindowRunsread pasthashesand GTA V crashed (SIGSEGV) in the first cutscene.docs/dev/debt/libSceAgcDriver.md): Linux now records a store through one alias at every alias, instead of recording none.Tested
CachyOS Linux (kernel 7.3.0-4-cachyos), GCC 16.2.1, Release, Ryzen 7 5800X, NVIDIA GeForce RTX 4070 SUPER (615.78.08).
Tests (head
4621b11d00,mainba5b5330b3merged intoa160f789ec;range-diffofa160f789ecagainst the measured headd10b767163shows both commits unchanged, and the merge preview ontomainis clean):guest_memory:CheckDirectMemoryWriteWatchSeesWritesThroughAliasesmaps one physical page at three addresses: the upper page of a two-page mapping, and two one-page mappings. It checks that:CollectFresh,CollectArmedandCollecteach report a store through one view at every view;CheckDirectMemoryWriteWatchDropsAliasOnRemaplinks two views of one page, then maps other physical memory over the second withsceKernelMapDirectMemory(fixed). It checks that the second view stays watched, that a collect over the first no longer reports the second's pages, and that a store through either view is reported only at that view.CheckDirectMemoryWriteWatchandCheckDirectMemoryWriteWatchFollowsRemovedMappingsnow expect aliased views to stay covered.agc_write_tracking:CheckImportWindowOverAlias(Linux) links two views of one memfd block and imports the upper view with a window. A store through the window is reported at both views, including the lower one, outside the window.4621b11d00):main'sGuestWriteWatch.cppandDirectMemory.cpp,guest_memoryfails on the first coverage check of an aliased view (core/libs/tests/GuestMemory.cpp:3036).Register,guest_memoryfails inCheckDirectMemoryWriteWatchDropsAliasOnRemap: the first view still reports the remapped view's pages.WindowCopy::Holds,agc_write_trackingcrashes (SIGSEGV) inCheckImportWindowOverAlias.Alias, the alias walks of all three collects, the link cut on bothRegisterandUnregister, and theWindowCopy::Holdsbound.guest_*tests pass (ctest -R '^guest_').guest_freebsd_clocksaborts, the same way on plainmain(Tests: guest_freebsd_clocks aborts where the host zone is not UTC #12736, fixed by the open fix(tests): drop the guest_freebsd_clocks check that the host zone is UTC #12660 and test(libkernel): pin the zone before the kernel timezone check #12747); this PR does not touch clocks.ANYPS5_REQUIRE_VULKAN=1 ANYPS5_NO_SHADER_CACHE=1 VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation, with noVUID-line or missing-layer message. They includeagc_write_watch_coverage,agc_import_watch,agc_shared_write,agc_write_tracking,agc_driver_alias_copy_labeland the heap mirror unwatch tests.python3 tools/check_conventions.py --base origin/mainandgit diff --checkpass.GTA V (PPSA04264 01.011.001, 3840x2160, warm shader cache):
main: it stops while the save loads (GTA V (PPSA04264) stops when the save loads: compute shader reads or writes a converted float image with 16-bit data #11425), which fix(shader): load and store 16-bit data of converted packed images #11580 fixes. Both arms carry fix(shader): load and store 16-bit data of converted packed images #11580, and Depends on lists it, so this merges after fix(shader): load and store 16-bit data of converted packed images #11580. The code shares no files with fix(shader): load and store 16-bit data of converted packed images #11580.d10b767163onmain2960e9a6e7. The current head is the same two commits on1db077e876, withmainba5b5330b3merged in. Of the merges since2960e9a6e7, only test(libkernel): cover the block sceKernelAllocateMainDirectMemory returns and its type #11681 (a new direct-memory allocation test incore/libs/tests/GuestMemory.cpp), feat(agc): blend into an unclamped normalized color target in the pixel shader #11684 and fix(agc): CLEAR_STATE leaves SPI_PS_IN_CONTROL at NUM_INTERP 2 #9595 (debt entries, and the unclamped-blend path, which GTA V never uses) feat(libSceAgcDriver): draw PS4 (GCN) command buffers #9431 (PS4-only fill-kernel matching inBufferFill.cpp) and fix(agc): draw compacted restart lists and restart-free strips through the barycentric patches #12706 (a different debt entry) touch this PR's files or the driver's memory code. None changes the write watch, direct memory orWindowCopy, so the measured path is unchanged.WriteChangedinExecution/src/GuestMemory.cpp; this PR changes one line ofWindowCopy::Holdsin the same file, about 1,900 lines away.main2960e9a6e7+ fix(shader): load and store 16-bit data of converted packed images #11580 (b8f4e6343a).d10b767163(98efd2e7b2).APS5_PROFILE_DRAW=1 APS5_PIPELINE_STATS=1 APS5_TRACE_SUMMARY_EVERY=10 APS5_TRACE_SECONDS=100:60in every run.Cutscene 1, per 10 s:
[guestmem] compare)collectfast-failsCutscene 2, per 10 s:
collectfast-failsgta-alias10-*.zip) in this comment, with the summary lines, the trace, the[guestmem]/[rescache]/[draw]lines and a README with the build, switches, cutscene marks and where the trace window falls. The traces are recompressed with xz to fit GitHub's attachment limit;xz -dgives the.jsonthattools/trace_digest.pyreads.Summary lines and trace digests (the
APS5_TRACE_SECONDS=100:60window of each trace):Base, round 1 (
b8f4e6343a:main2960e9a6e7+ #11580):{"commit":"b8f4e6343a96","dirty":false,"env":{"APS5_PIPELINE_STATS":"1","APS5_PROFILE_DRAW":"1","APS5_TRACE_DIR":"<path>","APS5_TRACE_EVENTS":"4194304","APS5_TRACE_SECONDS":"100:60","APS5_TRACE_SUMMARY_EVERY":"10"},"gpu":"NVIDIA GeForce RTX 4070 SUPER","driver":"NVIDIA 615.78.08","os":"Linux 7.3.0-4-cachyos","cpu":"AMD Ryzen 7 5800X 8-Core Processor","cpu_threads":16,"memory_mib":64203,"title_id":"PPSA04264","title":"Grand Theft Auto V","content_version":"01.011.001","resolution":"3840x2160","window":{"spec":"seconds 100:60","first_present":2761,"ms":60100.850},"presents":829,"frame_ms":{"p50":71.412,"p90":77.050,"p99":114.076,"max":203.223},"gpu_busy_ms_per_present":36.892,"worker_busy_pct":{"queue worker 0x0":97.246},"top_phases_ms_per_frame":{"Driver.Packet":67.123,"Graphics.Draw":38.415,"Driver.Packet.draw":35.829,"Driver.Draw":35.268,"Graphics.Draw.draw_and_resource_release":32.713,"Driver.Draw.shader_compile_and_link":24.068,"Driver.Packet.dispatch":23.982,"Shader.DrawStage":18.429},"trace_dropped":false,"trace":"aps5-trace-b8f4e6343a96-2334160.json.gz"}tools/trace_digest.pyof the base, round 1seconds 100:60 from present 2761: 829 presents, frame p50 71.4 / p90 77.0 / p99 114.1 / max 203.2 ms, GPU busy 36.9 ms per present
NVIDIA GeForce RTX 4070 SUPER, NVIDIA 615.78.08, Linux 7.3.0-4-cachyos, 3840x2160, commit b8f4e6343a96, dropped: no
{"commit":"b8f4e6343a96","dirty":false,"env":{"APS5_PIPELINE_STATS":"1","APS5_PROFILE_DRAW":"1","APS5_TRACE_DIR":"<path>","APS5_TRACE_EVENTS":"4194304","APS5_TRACE_SECONDS":"100:60","APS5_TRACE_SUMMARY_EVERY":"10"},"gpu":"NVIDIA GeForce RTX 4070 SUPER","driver":"NVIDIA 615.78.08","os":"Linux 7.3.0-4-cachyos","cpu":"AMD Ryzen 7 5800X 8-Core Processor","cpu_threads":16,"memory_mib":64203,"title_id":"PPSA04264","title":"Grand Theft Auto V","content_version":"01.011.001","resolution":"3840x2160","window":{"spec":"seconds 100:60","first_present":2761,"ms":60100.85},"presents":829,"frame_ms":{"p50":71.412,"p90":77.05,"p99":114.076,"max":203.223},"gpu_busy_ms_per_present":36.892,"worker_busy_pct":{"queue worker 0x0":97.246},"top_phases_ms_per_frame":{"Driver.Packet":67.123,"Graphics.Draw":38.415,"Driver.Packet.draw":35.829,"Driver.Draw":35.268,"Graphics.Draw.draw_and_resource_release":32.713,"Driver.Draw.shader_compile_and_link":24.068,"Driver.Packet.dispatch":23.982,"Shader.DrawStage":18.429},"trace_dropped":false,"trace":"aps5-trace-b8f4e6343a96-2334160.json.gz"}Slowest frames (largest slices inside each, inclusive)
Threads (ms per frame)
Largest slices (ms per frame, inclusive, all threads): Driver.Packet 52.8, batch 36.9, Graphics.Draw 28.6, Driver.Packet.draw 26.0, Driver.Draw 25.5, Driver.Packet.dispatch 23.8, Graphics.Draw.draw_and_resource_release 23.6, Recorder.Sync 15.4, Driver.Draw.shader_compile_and_link 15.4, VideoOut.Flip 14.2
Longest waits
Largest GPU batches (GPU ranges inside, and the CPU slices that recorded each, inclusive)
PR, run 3 (
98efd2e7b2: the same + this PR atd10b767163):{"commit":"98efd2e7b261","dirty":false,"env":{"APS5_PIPELINE_STATS":"1","APS5_PROFILE_DRAW":"1","APS5_TRACE_DIR":"<path>","APS5_TRACE_EVENTS":"4194304","APS5_TRACE_SECONDS":"100:60","APS5_TRACE_SUMMARY_EVERY":"10"},"gpu":"NVIDIA GeForce RTX 4070 SUPER","driver":"NVIDIA 615.78.08","os":"Linux 7.3.0-4-cachyos","cpu":"AMD Ryzen 7 5800X 8-Core Processor","cpu_threads":16,"memory_mib":64203,"title_id":"PPSA04264","title":"Grand Theft Auto V","content_version":"01.011.001","resolution":"3840x2160","window":{"spec":"seconds 100:60","first_present":2706,"ms":60006.587},"presents":832,"frame_ms":{"p50":70.569,"p90":77.423,"p99":114.056,"max":197.225},"gpu_busy_ms_per_present":36.331,"worker_busy_pct":{"queue worker 0x0":97.171},"top_phases_ms_per_frame":{"Driver.Packet":66.659,"Graphics.Draw":37.803,"Driver.Packet.draw":35.694,"Driver.Draw":35.107,"Graphics.Draw.draw_and_resource_release":31.991,"Driver.Draw.shader_compile_and_link":24.045,"Driver.Packet.dispatch":23.608,"Shader.DrawStage":18.446},"trace_dropped":false,"trace":"aps5-trace-98efd2e7b261-2341520.json.gz"}tools/trace_digest.pyof PR, run 3seconds 100:60 from present 2706: 832 presents, frame p50 70.6 / p90 77.4 / p99 114.1 / max 197.2 ms, GPU busy 36.3 ms per present
NVIDIA GeForce RTX 4070 SUPER, NVIDIA 615.78.08, Linux 7.3.0-4-cachyos, 3840x2160, commit 98efd2e7b261, dropped: no
{"commit":"98efd2e7b261","dirty":false,"env":{"APS5_PIPELINE_STATS":"1","APS5_PROFILE_DRAW":"1","APS5_TRACE_DIR":"<path>","APS5_TRACE_EVENTS":"4194304","APS5_TRACE_SECONDS":"100:60","APS5_TRACE_SUMMARY_EVERY":"10"},"gpu":"NVIDIA GeForce RTX 4070 SUPER","driver":"NVIDIA 615.78.08","os":"Linux 7.3.0-4-cachyos","cpu":"AMD Ryzen 7 5800X 8-Core Processor","cpu_threads":16,"memory_mib":64203,"title_id":"PPSA04264","title":"Grand Theft Auto V","content_version":"01.011.001","resolution":"3840x2160","window":{"spec":"seconds 100:60","first_present":2706,"ms":60006.587},"presents":832,"frame_ms":{"p50":70.569,"p90":77.423,"p99":114.056,"max":197.225},"gpu_busy_ms_per_present":36.331,"worker_busy_pct":{"queue worker 0x0":97.171},"top_phases_ms_per_frame":{"Driver.Packet":66.659,"Graphics.Draw":37.803,"Driver.Packet.draw":35.694,"Driver.Draw":35.107,"Graphics.Draw.draw_and_resource_release":31.991,"Driver.Draw.shader_compile_and_link":24.045,"Driver.Packet.dispatch":23.608,"Shader.DrawStage":18.446},"trace_dropped":false,"trace":"aps5-trace-98efd2e7b261-2341520.json.gz"}Slowest frames (largest slices inside each, inclusive)
Threads (ms per frame)
Largest slices (ms per frame, inclusive, all threads): Driver.Packet 52.1, batch 36.3, Graphics.Draw 28.0, Driver.Packet.draw 25.8, Driver.Draw 25.3, Driver.Packet.dispatch 23.5, Graphics.Draw.draw_and_resource_release 22.9, Driver.Draw.shader_compile_and_link 15.2, Recorder.Sync 14.8, VideoOut.Flip 13.8
Longest waits
Largest GPU batches (GPU ranges inside, and the CPU slices that recorded each, inclusive)
Overlap:
tools/pr_overlap.pylists five conflicts, all indocs/dev/debt/libSceAgcDriver.md:DirectMemory.cpporWindowCopy.WatchMapping, the write-watch exports,WindowCopyand the write-watch tests:SharedBackingandCreateDirectMemoryBackinginDirectMemory.cpp, the driver's import probes and a backing test. None of those is touched here. Its call in themainofcore/libs/tests/GuestMemory.cppis 3 lines from ours, so whichever merges later may need a trivial rebase.#if defined(__linux__)aroundWatchMappingto macOS. That is compatible: on macOSGuestWriteWatchAlias_nid_postfixreturns false, so aliased views are unregistered there as now.CheckImportWindowinWriteTracking.cpp.CheckImportWindowOverAliasis placed afterCheckImportWindowAcrossThreads, away from that hunk.Checklist
main; no other open PR implements the same functionsNotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file.mdfiles or images added to the repository (attach them to this PR)🤖 Generated with Claude Code