[ivica00]

What

Asynchronous GPU sample counter dumps used to store the sample count and ready bit 63 from one compute dispatch. A guest CPU that polls the dump can observe that bit set while the count words are still the previous or zero values.

The dump now publishes in three dispatches. Each one waits until the previous stores are visible to the host:

1. Write the high half with bit 63 clear.
2. Write the low half.
3. Store the whole high word with bit 63 set. That store does not read the dump, so a masked extra iteration cannot load the next list entry. The counter is published in this dispatch, after the ready bit.

APS5_SYNC_SAMPLE_DUMPS or APS5_NO_GPU_SAMPLE_DUMPS still forces the synchronous CPU dump. GPU dumps stay the default.

agc_sample_dumps runs sampleDumpTests through agc_driver_recorder_tests --sample-dumps-only.

Tested

SampleDumps.comp compiles with glslang vulkan1.1. agc_sample_dumps was not run on lavapipe or another GPU in this pass.

Checklist

• Based on current main; no other open PR implements the same functions
• One topic per PR; follow-ups go in a new PR
• No comments in code except technical debt
• Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
• No notes, investigation .md files or images added to the repository (attach them to this PR)
• New third-party code is a submodule built from source
• Generic behaviour, not specific to one title
• Depends on: none
• Unblocks: none
• GPUs tested: n/a for this push; earlier env-var path was AMD Radeon RX 9070 XT, driver 26.20.15024.1001, Windows 11
• AI-assisted: yes

[xXPBSXx]

Thanks for tracking this down. Reading the diff against PacketMemory.cpp: with DumpSamplesOnGpu returning false by default, wroteOnGpu stays false and the sample-dump packet falls into the drains branch (sampleDump is in the list there), so every sample counter dump now drains the queue (SubmitAndEpoch / WaitIdle). That is the intended synchronous behaviour, but it applies to every title that issues sample dumps, and the description only reports one title (PPSA03522) and no timing. A frame-time number from a title with many occlusion queries (before/after) would show what the default costs.

Also, the description says the asynchronous dump lacks a host-visible barrier; if that is the cause, adding the barrier on the GPU path would keep it asynchronous. I only read the code and did not run anything.

AI-assisted (Claude Code).

[ivica00]

Good points, thanks @xXPBSXx!

Regarding the host-visible barrier:
The phrasing in the initial description was misleading. oldSamples and closeStoreRun already emit barriers to VK_PIPELINE_STAGE_HOST_BIT / VK_ACCESS_HOST_READ_BIT in the command stream. However, in Vulkan, pipeline barriers only establish visibility once execution completes and the host synchronizes (e.g. via fence or idle); they do not block host CPU threads while the command buffer is unsubmitted (such as when queued in open->run.queued under LabelRunsPerBatch) or executing in flight on the GPU. Titles like Unreal Engine 4 directly poll the query counter on the CPU without waiting on a GPU fence; because GuestMemory uses write-watching rather than read-trapping, CPU loads do not invoke flush hooks and observe unready memory (ready bit = 0) or zero samples, causing false-negative culling and flickering geometry.

Regarding the default:
You are completely right that draining by default would risk regressions on titles with heavy occlusion queries like PPSA21564. I have updated the PR so that asynchronous GPU sample dumps remain enabled by default. Titles whose CPU polls the counters directly without GPU synchronization can now opt in to synchronous dumps via APS5_SYNC_SAMPLE_DUMPS=1 or APS5_NO_GPU_SAMPLE_DUMPS=1.

[CnRJay]

As it stands the PR is an opt-in switch, so the default behaviour is unchanged. That makes it a reasonable diagnostic, but it isn't a fix users will find: Hello Neighbor 2 still flickers unless someone sets APS5_SYNC_SAMPLE_DUMPS=1.

A CPU seeing "not ready" while the GPU hasn't finished is also what happens on the console. UE4's visibility thread handles a not-ready query by keeping the previous result, so polling before completion shouldn't cause flicker by itself. Flicker suggests the CPU sees the result as ready with wrong counts, which points to the order in which the dump's words become visible:

• If the sample counts and the ready/end marker are written by the same command, or by commands with no dependency between them, nothing orders those writes as the host observes them while the batch is in flight. A barrier to HOST_READ only makes writes visible once the host waits on a fence. A polling CPU can then see the marker before the counts.
• Writing the counts first, then the marker in a separate command after an execution and memory dependency, with the marker store being the last write of the dump, matches how the console's CP writes them and closes that window in practice on all desktop drivers. That would keep dumps asynchronous for every title and only change the order of two stores.

To confirm the cause: a trace of one flickering query slot, logging the dump's words at the moment the guest reads them, would show whether the ready word is set while the counts are still the previous or zero values.

[Tamalero]

High on Life (PPSA15388, UE4) is another title this fixes, together with #5232. Tested at the upstairs window from #5488 and the #5232 report.

Build	Visible objects dropping out for a frame	FPS at the window
#5232 alone	about once in 50 frames, a run of queries reads 0, so the window trim, a rug, frames or a cup vanish for one frame	6–14
#5232 + this PR (APS5_SYNC_SAMPLE_DUMPS behaviour)	none, over 55 traced frames and some minutes of play	about 12

A second symptom also went away. With #5232 alone, objects that the first-person weapon passed over came back a few frames late. I had put that down to UE4 reading occlusion results from earlier frames, but with this PR added they no longer disappear behind the weapon at all.

The occlusion trace from #5488 agrees: the three large steady queries (0.7–1.6 M samples) read correctly in 55 of 55 frames, with no zero frame.

The combination matters. Without #5232, this same CPU-side path only counted finished work. Many queries in a row then shared one stale total, and at low frame rates doors vanished (on RDNA2 at about 1 FPS, see #5488). With #5232's ordering in place, the per-query totals on this path are real, and they move smoothly as the camera sways.

Details

• Linux, RTX 4090 (driver 615.78.08), i9-14900K.
• Build: main 18c438b10 plus feat(agc): sample non-sRGB textures through FORCE_DEGAMMA samplers as sRGB views #4487, feat(libSceAgcDriver): support secondary LOD bias and relax unnormalized coordinate filters #5156, feat(agc): storage image access to depth planes through 2D array views #5003, fix(libScePlayGo): report missing chunk metadata #5494, local diagnostics and fix(agc): keep stores after occlusion dumps after them in a queued run #5232. This PR's two-line change matches a local switch I already had (DumpSamplesOnGpu returns false), so the run used that switch rather than the PR commit.
• Trace: the local ZPASS_DONE trace from High on Life (PPSA15388): occlusion-query results hide walls and props in view (all fine when every query reports visible) #5488, armed about 6 s, 55 frames, 45–63 queries per frame. Every dump took the CPU path.

Tested and written by Claude (AI assistant) on a contributor's machine.

[ivica00]

The default path no longer stores the count and bit 63 in one dispatch. Each dump now clears bit 63 and writes the high half, then writes the low half, then sets bit 63. Each of those dispatches waits until the previous stores are visible to the host, so a poll that sees the ready bit also sees the counts. APS5_SYNC_SAMPLE_DUMPS / APS5_NO_GPU_SAMPLE_DUMPS remain a diagnostic for the synchronous CPU dump.

Pushed in 6b164f2.

[KauaSantos702]

NVIDIA run at this head (6b164f21a), adding a vendor to the AMD results in the body: no agc_* test fails.

• NVIDIA GeForce RTX 5070 (Blackwell), driver 32.0.16.1714, Windows 11
• WinLibs GCC 15.2.0 r7, Release, BUILD_TESTING=ON
• ctest -R '^agc_' -j2: 484 tests, 483 passed, 0 failed, 1 skipped. The skipped test is agc_driver_stencil_ref_export, because the device lacks VK_EXT_shader_stencil_export; it is also skipped on main.

[nikowolf123]

The three phases are in the right order, but at 6b164f2 phase 2 crashes lavapipe, and the occlusion dump tests that would catch it are not in ctest.

Phase 2 reads the dump back, and lavapipe faults on that read. With this PR merged into main, sampleDumpTests segfaults at its first queued dump on lavapipe (Mesa 26.0.8, LLVM 21.1.8). Main passes the same test. The fault is the |= 0x80000000u in the phase-2 loop of SampleDumps.comp#L18-L24. llvmpipe's JIT runs that loop body once more after the loop has ended (its registers show dump = 1 with dumpCount = 1). The stores in that pass are masked, but the load through target is not. It reads the next list entry, which was never written and is 0, as the target and faults at address 0x4. Phases 0 and 1 only store to the target, so they don't hit this, and the RTX 5080 runs the same shader without a fault. CI runs the GPU tests only on lavapipe (CONTRIBUTING.md), so these tests can't be registered while phase 2 reads.

If phase 2 stores the whole high word from the same fold, the read goes away and the order stays the same: both count halves first, bit 63 last. The counter then has to move in phase 2 instead of phase 1:

SampleDumps.comp

 void main() {
-    if (parameters.phase == 2u) {
-        for (uint dump = parameters.firstDump; dump < parameters.firstDump + parameters.dumpCount; ++dump) {
-            Words target = Words(uvec2(parameters.dumps.words[dump * 4u], parameters.dumps.words[dump * 4u + 1u]));
-            for (uint db = 0u; db < 16u; ++db) target.words[db * 4u + 1u] |= 0x80000000u;
-        }
-        return;
-    }
     uint low = parameters.counter.words[0];
@@
         for (uint db = 0u; db < 16u; ++db) {
             if (parameters.phase == 0u) target.words[db * 4u + 1u] = db == 0u ? (high & 0x7fffffffu) : 0u;
-            else target.words[db * 4u] = db == 0u ? low : 0u;
+            else if (parameters.phase == 1u) target.words[db * 4u] = db == 0u ? low : 0u;
+            else target.words[db * 4u + 1u] = (db == 0u ? high : 0u) | 0x80000000u;
         }
     }
-    if (parameters.phase == 0u && parameters.dumpCount != 0u) return;
+    if (parameters.phase != 2u && parameters.dumpCount != 0u) return;

SampleCounter.comp can keep its |=, because its target is the pushed address, never a list entry. With APS5_LABEL_RUNS_INLINE=1, which sends the dumps through foldSamples, sampleDumpTests passes every value check on lavapipe and stops where it should, at "occlusion dumps are recorded at once".

The dump tests aren't in ctest. sampleDumpTests only runs when agc_driver_recorder_tests is started with no argument (tests/Recorder.cpp#L7272), and no add_test does that. On main that run also stops earlier, at "(9) a scan under a pending write was kept", on both lavapipe and the RTX 5080. So neither CI nor the ctest -R '^agc_' run above executes the dump shaders. An entry next to --pending-key-stores-only covers them, with skip lines that tools/check_skips.py accepts:

tests/Recorder.cpp and CMakeLists.txt (against main)

             pendingKeyStoreTests(device, recorder);
             std::cout << "Pending DCC key store tests passed\n";
             return 0;
         }
+        if (argc == 2 && std::string_view(argv[1]) == "--sample-dumps-only") {
+            if (device.GetContext().hostImportAlignment == 0) {
+                std::cout << "skipped, host imports are unavailable\n";
+                return 77;
+            }
+            if (!device.GetContext().bufferDeviceAddress) {
+                std::cout << "skipped, the device lacks buffer device addresses\n";
+                return 77;
+            }
+            sampleDumpTests(device, recorder);
+            std::cout << "Occlusion counter dump tests passed\n";
+            return 0;
+        }

     set_tests_properties(agc_shadowed_key_stores PROPERTIES SKIP_RETURN_CODE 77 TIMEOUT 30)
+    add_test(NAME agc_sample_dumps COMMAND agc_driver_recorder_tests --sample-dumps-only)
+    set_tests_properties(agc_sample_dumps PROPERTIES SKIP_RETURN_CODE 77 TIMEOUT 30)
     if(WIN32)
-        set_tests_properties(agc_pending_key_stores agc_shadowed_key_stores PROPERTIES ENVIRONMENT_MODIFICATION
+        set_tests_properties(agc_pending_key_stores agc_shadowed_key_stores agc_sample_dumps PROPERTIES ENVIRONMENT_MODIFICATION
             "PATH=path_list_prepend:$<TARGET_FILE_DIR:libc>")
     endif()

With both changes on this PR merged into main e34c479, agc_sample_dumps passes 3 of 3 times on lavapipe (--repeat until-fail:3, 0.07 s), the 32 registered recorder ctests pass (agc_shadowed_key_stores skips, as on main), and tools/check_test_registration.py and tools/check_skips.py are clean. With the shader of 6b164f2, the same test fails with SEGFAULT. Main's shader passes it. On the RTX 5080, the test passes 3 of 3 times with either shader, and the registered recorder ctests pass for main, this PR, and this PR with the change.

What the default path changes, and #5488. A dump ends with the same words as on main: DB 0 holds the total with bit 63 set, DBs 1 to 15 hold bit 63 alone, and the counter moves once. Only the order in which a polling CPU can see them changes. To check that order, I used a local probe (not for this PR). It records 256 query slots per batch (clear, draw, begin dump, draw, end dump), and the CPU polls DB 0's begin and end words and DB 15's end word while the batch runs:

Build	End dumps seen ready	Of those, ready with a wrong count or begin	Polls that saw a count without the ready bit
main, RTX 5080 (3 runs)	153,600	0	0
this PR, RTX 5080 (3 runs)	153,600	0	about 2.7 M per run
this PR with the change, RTX 5080 (3 runs)	153,600	0	about 1.1 M per run
main, lavapipe	12,800	0	103
this PR with the change, lavapipe	12,800	0	9.7 M

With this PR the poller does catch dumps in flight. On main, the RTX 5080 never showed it a half-written dump, so this probe can't reproduce the race there. The new order is still the safe one, but this is no evidence that the #5488 dropouts come from this race.

A readback taken after the GPU goes idle, like the per-flip trace in #5488, reads the same values at this head as on main. @Tamalero's 55 clean frames (comment) used the synchronous path (DumpSamplesOnGpu returning false), which this head makes opt-in. Nobody has run this head's default path on PPSA15388 yet. With #5232 alone, a run of queries still read 0 about once in 50 frames, and if those zeros are in the post-idle trace, this head won't change them. A run of this head without APS5_SYNC_SAMPLE_DUMPS at the same window would show that, on top of the current-main run @oneandonlydean asked for in #5488. Until then, #5488 should stay open when this merges.

Cost. Each fold now runs three dispatches and two more barriers. Since #5232, a store after a dump starts a new group, so with a clear before each query slot (the pattern in #5232's test) this happens once per slot. On the RTX 5080, one probe batch of 256 query slots, from recording to idle, took a median of 2.37 ms on main, 6.8 ms on this PR and 4.8 ms with the change above (3 runs of 200 batches each). That is about 17 µs per query slot for this PR and about 9.4 µs with the change. What that costs in a title needs a frame-time run, like the one @xXPBSXx asked for.

AI-assisted: written with Claude Code; I checked the diff at 6b164f2 against main's dump path, the words each phase writes, the llvmpipe JIT code at the crash, tools/check_skips.py, this thread and #5488, and ran agc_driver_recorder_tests (the dump tests through a --sample-dumps-only entry, the registered recorder ctests and a local CPU polling probe) for main, this PR merged into main, and that merge with the shader change, on lavapipe in WSL (main 763ad20 and e34c479, gdb on the crash) and on an RTX 5080 (driver 616.92, Windows 11, WinLibs GCC, main 763ad20).

[adriadam10]

The three-phase order (high half without bit 63, low half, bit 63 last) is right, but two things block merging:

• core/libs/prx/libSceAgcDriver/Graphics/shaders/SampleDumps.comp:18-24: phase 2 does target.words[...] |= 0x80000000u, which reads through target. As @nikowolf123 reported, lavapipe runs one extra masked iteration, loads the next (unwritten, 0) list entry and faults at 0x4 in sampleDumpTests. Phases 0 and 1 only store, so they are fine. Please apply the fix from their comment: phase 2 stores the whole high word from the same fold ((db == 0u ? high : 0u) | 0x80000000u) and phase 1 no longer folds the counter, so the read goes away and bit 63 is still the last write.
• The occlusion dump tests (sampleDumpTests, tests/Recorder.cpp:5278) only run when agc_driver_recorder_tests is started by hand and are not in ctest, so CI cannot catch a regression like the one above. Please register a ctest entry for them (CI runs these GPU tests on lavapipe) and check it passes there.

Note on testing: we could not run sampleDumpTests to the end on our lavapipe setup, so the lavapipe crash is taken from the report above; the ordering fix needs a run on lavapipe after the change.

[ivica00]

Phase 2 now stores the whole high word, (db == 0u ? high : 0u) | 0x80000000u, and does not read the dump. The counter is written in that dispatch, so phase 1 no longer publishes it. agc_sample_dumps runs sampleDumpTests via --sample-dumps-only and skips with code 77 when host imports or buffer device addresses are missing. SampleDumps.comp compiles with glslang vulkan1.1. I have not run agc_sample_dumps on lavapipe in this pass.

[adriadam10]

Checked the re-push at b526a11:

• Phase 2 of SampleDumps.comp now stores the whole high word and never reads the dump; --sample-dumps-only on lavapipe (Mesa 25.2.8, LLVM 20) passes, without the fault reported at 6b164f2.
• agc_sample_dumps is registered in ctest and passes on RADV; the rest of the libSceAgcDriver test scope is green.

Merging once the full Build is green.