Skip to content

fix(agc): stop resolveGroupAhead on unknown writes and reopen gate after HLE - #11439

Merged
oneandonlydean merged 8 commits into
boykopovar:mainfrom
hamb3r:fix/agc-dispatch-ahead-unknown-writes
Nov 2, 2026
Merged

oneandonlydean merged 8 commits into
boykopovar:mainfrom
hamb3r:fix/agc-dispatch-ahead-unknown-writes

Conversation

@hamb3r

@hamb3r hamb3r commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

What

When Driver::resolveGroupAhead scans a compute command buffer ahead of an in-flight WAIT_REG_MEM gate, it captures SRT/BDA memory for upcoming dispatches so their shaders and resources are ready before the gate opens. Four issues in the ahead-resolution path caused it to either capture stale pre-write descriptors across earlier GPU writes, pollute the global dispatchCache with rejected captures, or stop ahead resolution prematurely:

  • Keep FaultBuffer-only shaders bounded in DispatchWriteRanges (WrittenBuffers.cpp):
    • DispatchWriteRanges treated binding.kind == BufferBindingKind::FaultBuffer the same as Gds and BdaPagetable (bounded = false). However, FaultBuffer is an internal fault-reporting buffer added by the recompiler to specialized descriptor-indexed atomic shaders and does not write to arbitrary guest memory addresses. Treating FaultBuffer as bounded (unless Gds or BdaPagetable is present) prevents specialized compute shaders from prematurely terminating ahead group resolution.
  • Match fill kernels during resolvingAhead(), record unvalued GPU copies, and reopen gateOpened() after HLE (Dispatch.cpp, BufferCopy.cpp):
    • During resolvingAhead(), tryDwordPatternFillBuffer, tryConstantDwordFillBuffer, tryClampedGlobalFillBuffer, and tryMemoryValueFillBuffer were bypassed (!resolvingAhead()), causing buffer-fill dispatches to be captured as regular compute shaders instead of stopping ahead resolution at the HLE fill boundary.
    • When an HLE clear, fill, or copy dispatch executed while resolvedAhead() was empty, closeGate() closed the ahead gate without reopening it for the dispatches that follow the HLE operation in the same group; reopening openGate() when resolvedAhead_.empty() allows subsequent dispatches in the submission to be resolved ahead against the updated post-HLE state.
    • In Driver::noteCopyWriter, GPU-backed HLE copies (!cpuDone) that did not populate knownCopyValues (e.g., when the source range had a pending GPU write) only called recorder->notePendingWrite without recording a writer in writtenBuffers, so DispatchWriteRanges and pendingWrite treated the destination as having no writer. Recording writtenBuffers->noteWrite(destination, size, 0) marks the destination under foreign writer policy (program == 0).
  • Stop Driver::resolveGroupAhead on unknown writes and defer dispatchCache insertion (Driver.hpp, Dispatch.cpp, DispatchAhead.cpp):
    • Switched Driver::resolveGroupAhead from WrittenBufferRanges (which only inspects direct SRT V# buffers and misses BDA table buffers, StorageImage writes, and unbounded flags) to DispatchWriteRanges. When a dispatch is not resolved or DispatchWriteRanges reports !bounded, resolveGroupAhead increments stats.unknownWrites and breaks immediately (break;) instead of continuing to capture later dispatches that may depend on the unknown write.
    • Added bool cacheInserted = false to Driver::CaptureResult and deferred insertDispatch during resolvingAhead() until resolveGroupAhead verifies !OverlapsLabel and !OverlapsRange, preventing rejected ahead captures from polluting the global dispatchCache with stale pre-write snapshots.
  • Regression tests (WriteDataWrittenAhead.cpp):
    • Extended agc_driver_write_data_written_ahead to verify that rejected ahead captures do not insert into dispatchCache, that CPU-backed and GPU-backed HLE StreamCopy, FillKernel, and DwordPatternFill update descriptors before subsequent direct and IndirectGpu dispatches, and that StorageImage write overlap and unbounded Gds writes stop ahead resolution cleanly.

Tested

  • Windows 10 (10.0.19045), NVIDIA GeForce RTX 3080 (driver 617.14, Vulkan 1.4.351):
    • ctest --test-dir build-portable -R "^(agc_shader_table_snapshot_reuse|agc_shader_table_snapshot_reuse_unmasked|agc_shader_type_zero_image_materialization|agc_shader_memory|agc_shader_result_memo_budget|agc_shader_result_memo_budget_threads|agc_shader_result_memo_budget_masked|agc_driver_constant_dword_fill|agc_driver_dispatch_ahead_adopt_check|agc_driver_gated_dispatch_capture|agc_driver_gpu_indirect_arguments_written_ahead|agc_driver_indirect_arguments_written_ahead|agc_driver_pipelined_dispatch|agc_driver_stream_copy|agc_driver_write_data_written_ahead|agc_driver_write_data_written_ahead_draw_pipeline|agc_driver_gated_dispatch_capture_draw_pipeline|agc_driver_gated_dispatch_capture_short_timeout|agc_driver_pipelined_dispatch_item)$" -V with ANYPS5_REQUIRE_VULKAN=1, ANYPS5_NO_SHADER_CACHE=1, and VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation on both main (08a10d225306) and this branch (cb5d7fe3ef74) — all 19 tests pass, neither log contains was not found but was requested by env var VK_INSTANCE_LAYERS, and 0 VUID- validation errors are logged.
    • Verified that the new regression cases in agc_driver_write_data_written_ahead fail deterministically on main (rejected ahead capture polluted global dispatchCache with an extra insert and HLE StreamCopy (CPU-backed): dispatches after descriptor update used stale descriptor (stale 2, fresh 0)) and pass on this branch (Fresh[0] == 2..3, Stale[0] == 0).
  • Windows 10, PPSA01341 (Demon's Souls v1.07): verified that Driver::resolveGroupAhead no longer captures stale pre-copy/pre-fill descriptors (fixing VUID-vkCmdCopyBuffer-size-00115 and intermittent EXCEPTION_ACCESS_VIOLATION crashes in INTRO), stops on unknown writes, and reopens ahead resolution after HLE copies/fills.

Before (main):

[vulkan] validation: Validation Error: [ VUID-vkCmdCopyBuffer-size-00115 ] | MessageID = 0x99ef6537
vkCmdCopyBuffer(): pRegions[0].size is 0.
The Vulkan spec states: The size member of each element of pRegions must be greater than 0
Unhandled exception 0xC0000005 (EXCEPTION_ACCESS_VIOLATION) at 0x000000014330DC96
RIP = 0x000000014330DC96 (eboot.bin + 0x330DC96), RAX = 0x0000000000000000
[groupcap] queue 0x20 (10 s): 591 runs resolved 147 dispatches (71 cache hits) in 13.6 ms (max 2.26 ms); 148 adopted 0.01 ms after the resolve on average, max 0.83 ms; mismatches: address 0 key 0; 0 resolve failures, 0 stale, 0 label overlaps, 0 read an earlier dispatch's output, 0 rejected by the device, 0 dropped because the argument re-read differed
[groupcap] queue 0x38 (10 s): 207 runs resolved 146 dispatches (43 cache hits) in 37.1 ms (max 23.00 ms); 146 adopted 0.53 ms after the resolve on average, max 31.82 ms; mismatches: address 0 key 0; 0 resolve failures, 0 stale, 0 label overlaps, 0 read an earlier dispatch's output, 0 rejected by the device, 0 dropped because the argument re-read differed
[groupcap] queue 0x50 (10 s): 73 runs resolved 292 dispatches (264 cache hits) in 52.9 ms (max 22.04 ms); 292 adopted 7.86 ms after the resolve on average, max 132.53 ms; mismatches: address 0 key 0; 0 resolve failures, 0 stale, 0 label overlaps, 0 read an earlier dispatch's output, 0 rejected by the device, 0 dropped because the argument re-read differed

After (this PR):

[groupcap] queue 0x20 (10 s): 1908 runs resolved 869 dispatches (545 cache hits) in 26.4 ms (max 1.21 ms); 869 adopted 1.06 ms after the resolve on average, max 14.65 ms; mismatches: address 0 key 0; 0 resolve failures, 0 stale, 0 label overlaps, 0 read an earlier dispatch's output, 1671 stopped at unknown writes, 0 rejected by the device, 0 dropped because the argument re-read differed
[groupcap] queue 0x38 (10 s): 696 runs resolved 464 dispatches (229 cache hits) in 26.3 ms (max 2.14 ms); 464 adopted 0.01 ms after the resolve on average, max 0.84 ms; mismatches: address 0 key 0; 0 resolve failures, 0 stale, 0 label overlaps, 0 read an earlier dispatch's output, 232 stopped at unknown writes, 0 rejected by the device, 0 dropped because the argument re-read differed
[groupcap] queue 0x50 (10 s): 231 runs resolved 924 dispatches (827 cache hits) in 21.2 ms (max 6.81 ms); 924 adopted 0.35 ms after the resolve on average, max 12.82 ms; mismatches: address 0 key 0; 0 resolve failures, 0 stale, 0 label overlaps, 0 read an earlier dispatch's output, 0 stopped at unknown writes, 0 rejected by the device, 0 dropped because the argument re-read differed

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: none
  • Unblocks: none
  • GPUs tested: NVIDIA GeForce RTX 3080, driver 617.14 (Vulkan 1.4.351), Windows 10 (10.0.19045)
  • AI-assisted: yes

@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 26, 2026
@hamb3r
hamb3r force-pushed the fix/agc-dispatch-ahead-unknown-writes branch from dd3a169 to c53c696 Compare October 26, 2026 13:37

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the rebased head c53c696e97cb7b27c12081f9c28f877689b18160 against current main bf77a9e9cba4b02c358779b35295176dd9cae164. The code changes are consistent with the stated ahead-capture safeguards, and I found no additional functional defect in the inspected hunks. However, the PR’s reported 15-test Vulkan comparison uses main acb900c19432 and the prior PR head dd3a169b5682. Current main includes accae4b62d, which changes dispatch adoption and DispatchAhead.cpp in the same paths this PR modifies; the hosted product jobs are skipped. Please rerun the documented focused tests on current main and this rebased head with the stated Vulkan validation settings, compare the logs for new VUID- errors, and update the report before requesting approval.

@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 26, 2026
@hamb3r
hamb3r force-pushed the fix/agc-dispatch-ahead-unknown-writes branch from c53c696 to eaeafab Compare October 26, 2026 13:51
@hamb3r

hamb3r commented Oct 26, 2026

Copy link
Copy Markdown
Contributor Author

@MarcZX10 Rebased onto current main (9656d366be90, which includes accae4b62d), reran the 16 focused tests (including agc_driver_dispatch_ahead_adopt_check) on both main (9656d366be90) and the rebased head (eaeafab6fb0e) with ANYPS5_REQUIRE_VULKAN=1, ANYPS5_NO_SHADER_CACHE=1, and VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation (16/16 passed, 0 VUID- validation errors on either log), and updated the ### Tested section in the PR description.

@github-actions

github-actions Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 1896 core/libs/prx/libSceAgcDriver/Execution/src/Driver/Dispatch/Dispatch.cpp
PR 5609 core/libs/prx/libSceAgcDriver/Execution/include/Driver/Driver.hpp
PR 5925 core/libs/prx/libSceAgcDriver/Execution/include/Driver/Driver.hpp
core/libs/prx/libSceAgcDriver/Execution/src/Driver/Dispatch/Dispatch.cpp
core/libs/prx/libSceAgcDriver/Execution/src/Driver/Dispatch/DispatchAhead.cpp
PR 6188 core/libs/prx/libSceAgcDriver/Execution/include/Driver/Driver.hpp
PR 6501 core/libs/prx/libSceAgcDriver/Execution/include/Driver/Driver.hpp
PR 7275 core/libs/prx/libSceAgcDriver/Execution/include/Driver/Driver.hpp
PR 7369 core/libs/prx/libSceAgcDriver/Execution/include/Driver/Driver.hpp
PR 7410 core/libs/prx/libSceAgcDriver/Execution/src/Driver/Dispatch/Dispatch.cpp
PR 7642 core/libs/prx/libSceAgcDriver/Execution/include/Driver/Driver.hpp
core/libs/prx/libSceAgcDriver/Execution/src/Driver/Dispatch/Dispatch.cpp
core/libs/prx/libSceAgcDriver/Execution/src/Driver/Memory/WrittenBuffers.cpp
PR 9248 core/libs/prx/libSceAgcDriver/Execution/include/Driver/Driver.hpp
PR 9335 core/libs/prx/libSceAgcDriver/Execution/include/Driver/Driver.hpp
PR 9508 core/libs/prx/libSceAgcDriver/Execution/include/Driver/Driver.hpp
core/libs/prx/libSceAgcDriver/Execution/src/Driver/Dispatch/Dispatch.cpp
core/libs/prx/libSceAgcDriver/Execution/src/Driver/Memory/WrittenBuffers.cpp
PR 9898 core/libs/prx/libSceAgcDriver/Execution/include/Driver/Driver.hpp
PR 10193 core/libs/prx/libSceAgcDriver/Execution/include/Driver/Driver.hpp
core/libs/prx/libSceAgcDriver/Execution/src/Driver/Dispatch/Dispatch.cpp
core/libs/prx/libSceAgcDriver/Execution/src/Driver/Memory/WrittenBuffers.cpp
PR 10359 core/libs/prx/libSceAgcDriver/Execution/include/Driver/Driver.hpp
PR 11825 core/libs/prx/libSceAgcDriver/Execution/src/Driver/Memory/WrittenBuffers.cpp
PR 12568 core/libs/prx/libSceAgcDriver/Execution/include/Driver/Driver.hpp
PR 13863 core/libs/prx/libSceAgcDriver/Execution/src/Driver/Dispatch/Dispatch.cpp
PR 13920 core/libs/prx/libSceAgcDriver/Execution/src/Driver/Dispatch/Dispatch.cpp
PR 13928 core/libs/prx/libSceAgcDriver/Execution/include/Driver/Driver.hpp
core/libs/prx/libSceAgcDriver/Execution/src/Driver/Dispatch/Dispatch.cpp
core/libs/prx/libSceAgcDriver/Execution/src/Driver/Memory/WrittenBuffers.cpp
PR 13929 core/libs/prx/libSceAgcDriver/Execution/include/Driver/Driver.hpp
core/libs/prx/libSceAgcDriver/Execution/src/Driver/Dispatch/Dispatch.cpp
core/libs/prx/libSceAgcDriver/Execution/src/Driver/Memory/WrittenBuffers.cpp
PR 14944 core/libs/prx/libSceAgcDriver/Execution/src/Driver/Memory/WrittenBuffers.cpp

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new head and test report address my earlier request: the PR now tests against main at 9656d366be90 and reports all 16 focused Vulkan tests passing on both main and the change, with no missing-layer messages or VUID- output. I have not independently reproduced the Windows/NVIDIA run.

One merge-order item remains. I ran a three-way comparison with open #8105 from their common base; it has a content conflict in Dispatch.cpp at the declaration after htileClearCompletion: this PR adds codeSpan and #8105 adds Graphics::HtileMaskClear htileMask. CONTRIBUTING.md requires authors to agree on order and declare it with Depends on. Please coordinate with #8105's author and update this PR before merge. I manually checked #7201's Dispatch.cpp hunk (NoteRecentWork near the function start); it is separate from this conflict.

@hamb3r
hamb3r force-pushed the fix/agc-dispatch-ahead-unknown-writes branch from eaeafab to 0f8b37f Compare October 26, 2026 14:49
@hamb3r

hamb3r commented Oct 26, 2026

Copy link
Copy Markdown
Contributor Author

@MarcZX10 Dropped the local codeSpan refactoring after htileClearCompletion in Dispatch.cpp (commit 0f8b37f008e3), keeping std::span(snapshot.code).subspan(codeOffset) on the modified lines and leaving the htileClearCompletion / HtileDepthClearAddress lines untouched. Verified via git merge-tree --write-tree HEAD origin/pull/8105/head that this PR and #8105 now merge cleanly with zero conflicts in either order, and reran the 16 focused tests with VK_LAYER_KHRONOS_validation (16/16 passed, 0 VUID- errors).

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I rechecked head 0f8b37f008e3e58d3d0307f353caf2c59481f79d. The previous Dispatch.cpp overlap with #8105 is resolved: I confirmed the merge-tree is clean in both merge orders. The hosted conventions, eligibility and report checks pass; AGC product jobs are skipped by path policy.

Two follow-ups remain before approval:

  • git diff --check upstream/main...upstream/pr-11439 reports a new blank line at EOF in core/libs/prx/libSceAgcDriver/tests/execution/WriteDataWrittenAhead.cpp:408.
  • The branch's merge base and the Vulkan comparison base are 9656d366be90, while current main is 7903201bb426. I found no intervening changes to this PR's touched source/test paths, but CONTRIBUTING requires branches to stay current. Please sync to current main and rerun the documented focused Vulkan-validation comparison on the same GPU, including the no-missing-layer and no-new-VUID- checks.

I did not independently run the Windows/NVIDIA suite. The previous dependency-order finding is withdrawn because the current heads merge cleanly in either order.

@hamb3r
hamb3r force-pushed the fix/agc-dispatch-ahead-unknown-writes branch from 0f8b37f to 2c2843d Compare October 26, 2026 15:32
@hamb3r

hamb3r commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor Author

@MarcZX10 Removed the trailing blank line at EOF in WriteDataWrittenAhead.cpp (git diff --check is now clean), rebased onto live main (570089a806c1) at head fb6613fa5b5d, and reran the 19 focused Vulkan tests with ANYPS5_REQUIRE_VULKAN=1, ANYPS5_NO_SHADER_CACHE=1, and VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation (19/19 passed, no missing-layer messages, 0 VUID- errors). Updated the Tested section with the exact revisions.

@hamb3r
hamb3r force-pushed the fix/agc-dispatch-ahead-unknown-writes branch 2 times, most recently from 1c26236 to fb6613f Compare October 26, 2026 16:19
@github-actions github-actions Bot added the agc libSceAgc* label Oct 26, 2026
@adriadam10 adriadam10 self-assigned this Oct 27, 2026

@jotitadev jotitadev left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First NVIDIA run on Linux for this head fb6613fa5.

Linux run: Ubuntu 26.04, GCC 15.2, Release. RTX 3070 (NVIDIA 595.99.02) and lavapipe (Mesa 26.0.3), one at a time via VK_DRIVER_FILES. Khronos validation layer 1.4.341 with ANYPS5_REQUIRE_VULKAN=1 ANYPS5_NO_SHADER_CACHE=1; VK_LOADER_DEBUG=layer shows the layer inserted, no "was not found but was requested", 0 VUID- lines.

Base main 808a557ac; the merge is clean.

  • Red: main + only this PR's WriteDataWrittenAhead.cpp: agc_driver_write_data_written_ahead and _draw_pipeline fail on both devices with normal dispatch after rejected ahead capture received the rejected stale variant from global dispatchCache (stale 3, fresh 0).
  • Green: 16 of the description's 19 focal tests pass 3/3 on each device. I left out the three agc_shader_result_memo_budget* tests. 0 VUID-.
  • ^agc suite (no flip/display/lds64, -j1): 3070 1052/1052, lavapipe 1051/1052. The one failure is agc_driver_multisample_retirement, which also fails on main. On main, agc_submit_progress_no_timeline failed once on the 3070. The GPU was shared, and that test passed with this PR.
  • Builds on Linux/GCC; public_headers passes.

GPUs tested: NVIDIA RTX 3070 (Linux), lavapipe.

Written with AI assistance (Claude Code). I ran the builds and tests above on my machine.

@MarcZX10

Copy link
Copy Markdown
Contributor

Thanks for clearing the trailing blank line and reporting the 19 focused Vulkan-validation tests on both main and the rebased head. I rechecked the current head fb6613fa5b5d76128c6ac9cc8c75372df6ec00ce against live main 4c7d275bef89d309313d16bc2bc05d37a1b50be2:

  • Current-main merge preview is clean, and git diff --check passes.
  • python3 tools/check_conventions.py --base upstream/main --head refs/review/upstream-11439 passes.
  • Main has since changed three files also touched by this PR (Driver.hpp, Dispatch.cpp, BufferCopy.cpp). I inspected those hunks: they add dispatch dimension handling and cache/overlay APIs, separate from this PR's dispatch-ahead logic, but the reported test comparison is still on the earlier 570089a base.

Please sync with current main and rerun the focused comparison on the updated base/head, including validation-layer load and VUID results. I did not run the GPU tests locally; this environment has no Vulkan device. My existing changes-requested review remains limited to current-base validation.

Symptom / Log:
- agc_driver_write_data_written_ahead: specialized atomic compute shaders with FaultBuffer but no BdaPagetable were classified as unbounded in DispatchWriteRanges, stopping ahead group resolution.

Root Cause:
- core/libs/prx/libSceAgcDriver/Execution/src/Driver/Memory/WrittenBuffers.cpp:DispatchWriteRanges:L56-L61 — FaultBuffer bindings were treated as unbounded pointer writes even when no BdaPagetable binding was present.

Change:
- Allow FaultBuffer bindings in DispatchWriteRanges without clearing bounded unless Gds or BdaPagetable is present.

Metric / Verification:
- DispatchWriteRanges returns bounded == true for descriptor-indexed atomic shaders with FaultBuffer and no BdaPagetable.
Symptom / Log:
- 26102026_03_45_39.log:L2476-L2492, 26102026_03_53_05.log:L1968-L1985: VUID-vkCmdCopyBufferToImage-pRegions-00171 / VUID-vkCmdDispatch-None-08600 / EXCEPTION_ACCESS_VIOLATION (0xC0000005) on queue 0x20 after HLE copy and fill dispatches.

Root Cause:
- core/libs/prx/libSceAgcDriver/Execution/src/Driver/Dispatch/Dispatch.cpp:dispatch:L125-L154 — HLE buffer fill and copy matchers skipped ahead capture without reopening gateOpened() after execution, while BufferCopy.cpp:noteCopyWriter:L131 recorded GPU-backed copies without known values under the copy program address instead of foreign writer policy.

Change:
- Match dword-pattern, constant-dword, clamped-global, and memory-value buffer fills during resolvingAhead(), reopen gateOpened() when resolvedAhead() is empty after HLE clear/fill/copy execution, and record GPU-backed copies without known values under writer program 0.

Metric / Verification:
- Post-HLE dispatches synchronize against pending GPU-backed copies and reopen ahead group resolution once resolvedAhead() drains.
…sert

Symptom / Log:
- 26102026_03_45_39.log:L2476-L2492: resolveGroupAhead continued capturing dispatches after an unknown write (found == end or unbounded write), and dispatch() inserted ahead captures into global dispatchCache before OverlapsLabel / OverlapsRange rejection.

Root Cause:
- core/libs/prx/libSceAgcDriver/Execution/src/Driver/Dispatch/DispatchAhead.cpp:resolveGroupAhead:L290-L299 and Dispatch.cpp:dispatch:L399 — unknown writes only set unknownWrites = true without stopping the loop, WritesThroughPointers missed StorageImage and GDS write hazards, and insertDispatch ran before overlap checks.

Change:
- Stop resolveGroupAhead immediately (++stats.unknownWrites; break;) when a dispatch is not resolved or DispatchWriteRanges reports !bounded, and defer insertDispatch (plus compute pipeline prebuild) until OverlapsLabel and OverlapsRange pass.

Metric / Verification:
- Rejected ahead captures do not insert into global dispatchCache (inserts delta == 1), and resolveGroupAhead stops at unknown or unbounded writes.
… insert

Symptom / Log:
- Missing regression coverage in WriteDataWrittenAhead for post-HLE ahead gate reopening, rejected ahead capture cache pollution, StorageImage write overlap, and unbounded GDS writes.

Root Cause:
- core/libs/prx/libSceAgcDriver/tests/execution/WriteDataWrittenAhead.cpp only covered direct PM4 WRITE_DATA before an ahead dispatch.

Change:
- Extend WriteDataWrittenAhead to test rejected ahead capture cache pollution, CPU-backed and GPU-backed HLE StreamCopy / FillKernel / DwordPatternFill before direct and IndirectGpu dispatches, StorageImage write overlap, and unbounded GDS write stop.

Metric / Verification:
- ctest -R agc_driver_write_data_written_ahead passes with VK_LAYER_KHRONOS_validation and 0 VUID errors.
@hamb3r
hamb3r force-pushed the fix/agc-dispatch-ahead-unknown-writes branch from fb6613f to c4ddc2d Compare October 29, 2026 10:16
@hamb3r

hamb3r commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor Author

@MarcZX10 Rebased onto current main (1b580f13cf7e) at head c4ddc2d40465 (git diff --check and check_conventions.py clean) and reran the 19 focused Vulkan tests with ANYPS5_REQUIRE_VULKAN=1, ANYPS5_NO_SHADER_CACHE=1, and VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation on both main (1b580f13cf7e, 13.07s) and this branch (c4ddc2d40465, 12.02s): 19/19 passed, neither log contains was not found but was requested by env var VK_INSTANCE_LAYERS, and 0 VUID- validation errors are logged. Updated the Tested section with the new revisions.

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-check — changes requested for current-main integration\n\nThe author’s 19/19 focused Vulkan-validation result on both main and the PR head addresses the previous test-evidence request. The validation-layer load and zero-VUID reporting are also documented.\n\nThe comparison now uses main 1b580f13cf, while current upstream main is d2b15d8802. Since that base, core/libs/prx/libSceAgcDriver/CMakeLists.txt and docs/dev/debt/libSceAgcDriver.md changed, along with related AGC tests. The merge preview against current main is clean, and I independently confirmed git diff --check and check_conventions.py pass. CONTRIBUTING requires the branch to stay current; please refresh onto d2b15d8802 and rerun the focused comparison on the merge-ready head so the updated registration and debt context are covered.\n\nI did not run the Vulkan tests locally; product GPU checks are path-skipped by hosted CI.

@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 29, 2026
…d-unknown-writes

# Conflicts:
#	core/libs/prx/libSceAgcDriver/Execution/src/Driver/Memory/BufferCopy.cpp

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-checked — new head, changes requested

The branch now merges current main, but the focused Vulkan comparison still covers the prior integration, including three source files changed by the new base.

What I ran

Reviewed exact head 5cc2975a20f7beb519a072fa14a34e60999be417, whose second parent is current main 9624d8c6bb9e96b7aa92b815f55690f69b0a88d5. I compared the tested main 1b580f13cf7e with current main and inspected the overlap in Driver.hpp, BufferCopy.cpp, and WrittenBuffers.cpp. Environment: Linux x86-64, Git and GitHub CLI; no build or GPU test was run locally.

git diff --check 9624d8c6bb9e96b7aa92b815f55690f69b0a88d5 refs/review/pr-11439-head
git merge-base --is-ancestor 9624d8c6bb9e96b7aa92b815f55690f69b0a88d5 refs/review/pr-11439-head
git diff --name-only 1b580f13cf7e 9624d8c6bb9e96b7aa92b815f55690f69b0a88d5 -- core/libs/prx/libSceAgcDriver/Execution/include/Driver/Driver.hpp core/libs/prx/libSceAgcDriver/Execution/src/Driver/Memory/BufferCopy.cpp core/libs/prx/libSceAgcDriver/Execution/src/Driver/Memory/WrittenBuffers.cpp

Results

Check Result
Patch whitespace Pass
Current-main ancestry Pass; current main is the merge commit's second parent
Changes since the reported test base Driver.hpp, BufferCopy.cpp, and WrittenBuffers.cpp changed on main; those are also modified by this PR
Focused Vulkan comparison Author reports 19/19 tests and negative controls on main 1b580f13cf7e and head c4ddc2d40465; this is before the current-main merge and does not cover the three overlapping main updates
Hosted exact-head jobs Windows in progress; conventions and labels queued; report and eligibility pass; Linux/product and other specialized jobs skipped
Local build/GPU tests Not run
Unedited local command output
diffcheck-exit=0
current-main-is-ancestor-exit=0
c4ddc2d404657c35b6822c37dc3c4311edf76872 9624d8c6bb9e96b7aa92b815f55690f69b0a88d5

Findings

Blocking

  • The current-main merge changes the WrittenBufferIndex storage and writer-recording paths in files this PR modifies. The previous 19-test comparison used main 1b580f13cf7e and head c4ddc2d40465, so it does not validate the combined behavior now submitted at 5cc2975a20f7. Please rerun the focused Vulkan suite and the relevant negative controls on current main 9624d8c6bb9e and this exact merged head, with validation-layer load and VUID results recorded.

I found no additional source-level defect in the rechecked diff.

Sources

Not verified

I did not build or run the current merged head locally or on a GPU. The exact-head Windows job was still running; conventions and labels were queued. Linux product and specialized jobs were skipped.

To get this to a good state

Rerun the same 19 focused tests and issue-specific negative controls against current main and exact head 5cc2975a20f7, recording the required Vulkan layer and VUID checks. I will recheck the resulting head and hosted statuses.

@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 29, 2026
@hamb3r

hamb3r commented Oct 29, 2026

Copy link
Copy Markdown
Contributor Author

@MarcZX10 Merged current main (c40cef843681, including the WrittenBufferIndex changes in Driver.hpp, BufferCopy.cpp, and WrittenBuffers.cpp) into this branch at head 0711645eef72 (git diff --check and check_conventions.py clean) and reran all 19 focused Vulkan tests with ANYPS5_REQUIRE_VULKAN=1, ANYPS5_NO_SHADER_CACHE=1, and VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation on both main (c40cef843681, 12.88s) and this merged head (0711645eef72, 13.01s): 19/19 passed, neither log contains was not found but was requested by env var VK_INSTANCE_LAYERS, and 0 VUID- validation errors are logged. Updated the Tested section with the new revisions.

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-checked — new head, changes requested

The exact-head regression evidence and current-base merge are satisfactory, but the branch conflicts with two open PRs and its dependency checklist still says none.

What I ran

  • PR head: 0711645eef72b5e26934a6245c417bfe52bd5b2f.
  • Base: upstream/main at d820fef1827f27247cd28da7b4eec7891904bae5; the merge preview was clean (99eca890510413d8115e076ea406fec1aa377cba). No files touched by this PR changed on main after the tested base c40cef843681.
  • Environment: Linux x86_64; no compiler, build, or GPU tests run locally.
git diff --check upstream/main...refs/review/pr-11439-head
git merge-tree --write-tree upstream/main refs/review/pr-11439-head
python tools/check_conventions.py --base upstream/main --head refs/review/pr-11439-head
git merge-tree --write-tree refs/review/pr-11439-head refs/review/pr-12147-head
git merge-tree --write-tree refs/review/pr-11439-head refs/review/pr-12657-head

Results

Check Result
Patch whitespace Pass; command exited 0 with no output.
Current-main merge preview Pass; clean tree 99eca890510413d8115e076ea406fec1aa377cba.
Repository conventions Pass.
Pairwise integration with #12147 Conflict in core/libs/prx/libSceAgcDriver/Execution/src/Driver/Dispatch/Dispatch.cpp.
Pairwise integration with #12657 Conflict in the same Dispatch.cpp. Both PRs remain open; #12657's description says it depends on #12147.
Hosted CI Windows product job passed; eligibility, conventions, labels, and report checks passed. Linux and specialized jobs were skipped.
Focused Vulkan tests The PR description reports 19/19 focused Vulkan validation tests on both main c40cef843681 and this head, with the validation layer loaded and zero VUID errors. This is author-reported; I did not run the GPU tests.
Unedited local command output

python tools/check_conventions.py --base upstream/main --head refs/review/pr-11439-head

All conventions checks passed.

git merge-tree --write-tree upstream/main refs/review/pr-11439-head

99eca890510413d8115e076ea406fec1aa377cba

git diff --check upstream/main...refs/review/pr-11439-head exited 0 and produced no output.

Both pairwise merge previews exited 1 and reported:

CONFLICT (content): Merge conflict in core/libs/prx/libSceAgcDriver/Execution/src/Driver/Dispatch/Dispatch.cpp

Findings

Blocking

  • Unresolved PR overlap and merge order: Pairwise merge previews conflict in Dispatch.cpp with open #12147 and #12657. The PR currently lists Depends on: none. CONTRIBUTING.md requires authors to agree on which overlapping PR goes first and declare that order with Depends on (pinned contribution rule). Please coordinate the order with the affected authors and update the dependency line; then I can recheck the combined merge.

Non-blocking

  • No additional functional issue was identified in this source-level recheck. The reported GPU test comparison is author-provided, not independently reproduced here.

Not verified

  • I did not build or run tests locally; this environment has no verified Vulkan device. Linux, macOS, and specialized hosted jobs were skipped.

To get this to a good state

Please record the agreed order for the overlapping PRs in Depends on. I will rerun the merge checks and review the updated head.

@hamb3r

hamb3r commented Oct 30, 2026

Copy link
Copy Markdown
Contributor Author

@MarcZX10 Kept the SubmitLibraryPrebuild block in Dispatch.cpp untouched (leaving only insertDispatch deferred to DispatchAhead.cpp in cb5d7fe3ef74, merged with current main 08a10d225306). Verified that both git merge-tree --write-tree HEAD origin/pull/12147/head and git merge-tree --write-tree HEAD origin/pull/12657/head now exit 0 with zero conflicts in either merge order, and reran all 19 focused Vulkan tests with VK_LAYER_KHRONOS_validation on cb5d7fe3ef74 (19/19 passed, no missing-layer messages, 0 VUID- errors).

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-checked — new head, looks good

The update preserves the unrelated prebuild block, and I confirmed the branch combines cleanly with current main and both cited PRs in either merge order.

What I ran

Reviewed head cb5d7fe3ef741780004cfd8e4e677d1b033e7311 against current main 5ecef095b61e7a406b279c97896f3a8a6fbd6353; current-main merge tree is 907b51b66dc5a7a17e45994202854189394b9941.

git merge-tree --write-tree refs/remotes/upstream/main refs/remotes/upstream/pr-11439
git diff --check refs/remotes/upstream/main...refs/remotes/upstream/pr-11439
git merge-tree --write-tree refs/remotes/upstream/pr-11439 refs/remotes/upstream/pr-12147
git merge-tree --write-tree refs/remotes/upstream/pr-12147 refs/remotes/upstream/pr-11439
git merge-tree --write-tree refs/remotes/upstream/pr-11439 refs/remotes/upstream/pr-12657
git merge-tree --write-tree refs/remotes/upstream/pr-12657 refs/remotes/upstream/pr-11439

Results

Check Result
Current-main merge preview Clean
Diff check Passed; no output
Merge with #12147 Clean in both orders
Merge with #12657 Clean in both orders
Hosted conventions and Windows Passed
Focused Vulkan tests Author reports 19/19 on the updated head, with validation enabled and zero VUIDs; not independently run

Not verified

I did not run a local build or Vulkan tests. Product Linux and specialized hosted checks were skipped for this change.

@TechLevelZero

Copy link
Copy Markdown
Contributor

✅ NVIDIA GPU test: passed

PR head cb5d7fe merged onto main 98ddd9c · tested 2026-10-30 16:09 UTC · 34 min

GPU NVIDIA RTX 2000 Ada Generation, driver 595.99.02, Vulkan 1.4.329
Host Ubuntu 26.04.1 LTS, kernel 7.0.0-38-generic, container ubuntu:24.04
Logs build and test logs for this run
Build ✅ passed (Release, -DANYPS5_ENABLE_SPIRV_TOOLS=ON)
Tests ✅ 2,280 passed · 6 failed (0 new, 2 also fail on main, 4 unstable on main) · 40 skipped (ANYPS5_REQUIRE_VULKAN=1)
Test setup CPU-only tests without a GPU; GPU tests (and any CPU-only test that failed or skipped without one) once on the GPU with VK_LAYER_KHRONOS_validation and ANYPS5_NO_SHADER_CACHE=1; main ran the same way
Test selection 2,326 of 2,987 tests run: the ones this change can reach, chosen by tools/select_tests.py from main's build graph
Vulkan validation ✅ no new validation IDs vs main (861 tests run on the GPU)
Slow tests (>30 s, GPU tests timed with validation layers) ⚠️ agc_driver_bda_device 42.2 s, ⚠️ agc_driver_recorder 30.1 s
Unstable on main (4, these also failed in recent main runs on this machine; not counted against this PR)
agc_driver_recorder: failed in 2 of the last 9 main runs
agc_driver_submit_progress: failed in 5 of the last 9 main runs
agc_submit_progress: failed in 5 of the last 9 main runs
agc_submit_progress_no_timeline: failed in 7 of the last 9 main runs
Failing on main too (2, not counted against this PR)
guest_mmap_executable
guest_socket_options_multicast
Passed on retry (1, failed in the parallel run, passed when re-run with a 360 s timeout; not counted against this PR)
agc_driver_job_manager_compute_capacity: Failed in the parallel run; also passed only on retry in 116 other runs in the last 7 days
Validation IDs already present on main (4 IDs in 390 tests)
VUID-VkDeviceCreateInfo-pNext-pNext (390 tests): agc_compute_pipeline_replay, agc_driver, agc_driver_address_passes_draw_transitions, agc_driver_address_passes_targets_split, agc_driver_array_layer_target_cpu, agc_driver_bda_span_reads_draw_pipeline, agc_driver_bindless_images, agc_driver_branch_past_endpgm, and 382 more
VUID-vkCmdCopyImage-srcImage-01548 (13 tests): agc_driver_depth_clear_hle, agc_driver_depth_plane_buffer_memory, agc_driver_depth_plane_cpu_write_after_memo, agc_driver_depth_plane_cpu_write_after_page_unwatch, agc_driver_depth_plane_cpu_write_after_partial_unwatch, agc_driver_depth_plane_partial_store, agc_driver_depth_plane_partial_store_unimported, agc_driver_depth_plane_sampling, and 5 more
VUID-vkCmdCopyImage-srcImage-01551 (13 tests): agc_driver_depth_clear_hle, agc_driver_depth_plane_buffer_memory, agc_driver_depth_plane_cpu_write_after_memo, agc_driver_depth_plane_cpu_write_after_page_unwatch, agc_driver_depth_plane_cpu_write_after_partial_unwatch, agc_driver_depth_plane_partial_store, agc_driver_depth_plane_partial_store_unimported, agc_driver_depth_plane_sampling, and 5 more
VUID-VkPipelineColorBlendStateCreateInfo-pAttachments-00605 (2 tests): agc_driver_prepared_exports, agc_driver_srgb8_color_target
Skipped (40)
agc_driver_bda_reads_shadowed_results: skipped, the device reports watched host memory written when it imports or reads it
agc_driver_bda_reads_shadowed_results_per_build: skipped, the device reports watched host memory written when it imports or reads it
agc_driver_msaa_dcc_compared_import_2: skipped, the device reports host imports that keep write protection, so the write-watch probe compares none of them
agc_driver_msaa_dcc_compared_import_4: skipped, the device reports host imports that keep write protection, so the write-watch probe compares none of them
agc_driver_msaa_dcc_compared_import_8: skipped, the device reports host imports that keep write protection, so the write-watch probe compares none of them
agc_driver_sample_order_grid_reference: skipped, the device does not keep depth drawn at custom sample locations at those locations, even after a transition that names them: the composite of uniform renders holds the guest's depth in 0 of 3005 sloped samples, so no path can be compared with it bit for bit
agc_driver_stencil_ref_export: skipped, the device lacks VK_EXT_shader_stencil_export
agc_driver_vertex_elected_atomic_max: skipped, the device runs vertex programs in subgroups, so they keep their cross-lane reductions
agc_host_import_sparse: skipped, host imports are unavailable: /dev/udmabuf is not accessible
agc_import_settle_initial_dcc: skipped, the device has no deferred settle of host imports: imports under 1 MiB wait for one only on Windows
agc_unit_shadow_completion_store: skipped, the device reports watched host memory written when it imports or reads it
guest_http_uri_locale: 
guest_net_ctl: skipped, the host has no IPv4 address
guest_video_out_color_space: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_hdr: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_high_frame_rate: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_output_mode: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_resolution_status: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
guest_video_out_wqhd_detection: skipped, no display or Vulkan device: SDL_InitSubSystem(VIDEO | GAMECONTROLLER) failed: wayland,x11 not available
video_out_flip_controls: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_cpuflip: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_cpuflip_in_order: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_eop: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_eop_aftergpu: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_eop_main: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_fresh: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is either not configured in SDL or not available in current SDL video driver (offscreen) or platform
video_out_flip_labels: skipped, no display or Vulkan device: DisplayWindow: Vulkan support is eithe
Reproduce
git fetch origin cb5d7fe3ef741780004cfd8e4e677d1b033e7311 98ddd9c1792bbab016ceed5db267a93d1d61f00a
git checkout 98ddd9c1792bbab016ceed5db267a93d1d61f00a && git merge --no-ff cb5d7fe3ef741780004cfd8e4e677d1b033e7311
git submodule update --init --recursive
cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=ON -DANYPS5_ENABLE_SPIRV_TOOLS=ON
cmake --build build && cmake --build build --target libs
VK_DRIVER_FILES=/nonexistent/icd.json ctest --test-dir build -E '^(agc|shader|recompiler|videoout|video_out|guest_video_out|vulkan)|gpu' --timeout 120 --parallel 32 --output-on-failure
ANYPS5_REQUIRE_VULKAN=1 ANYPS5_NO_SHADER_CACHE=1 VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation \
  ctest --test-dir build -R '^(agc|shader|recompiler|videoout|video_out|guest_video_out|vulkan)|gpu' --timeout 120 --parallel 4 --output-on-failure
# failures are then re-run alone: --timeout 360 --parallel 2; new validation IDs: the test alone, twice, --parallel 1

Automated build and test on a contributor's NVIDIA machine; it does not replace upstream CI. A new comment is posted for each new push. Full logs.

@oneandonlydean
oneandonlydean merged commit 8ea8213 into boykopovar:main Nov 2, 2026
11 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agc libSceAgc*

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants