Skip to content

fix(agc): keep guest buffer offsets in shader data when stencil replay adds a push constant - #9427

Closed
TheSovietWaffle wants to merge 1 commit into
boykopovar:mainfrom
TheSovietWaffle:fix/agc-stencil-replay-buffer-shader-data
Closed

TheSovietWaffle wants to merge 1 commit into
boykopovar:mainfrom
TheSovietWaffle:fix/agc-stencil-replay-buffer-shader-data

Conversation

@TheSovietWaffle

Copy link
Copy Markdown
Contributor

What

On devices without VK_EXT_shader_stencil_export, stencil replay appends a four-byte dynamic reference to the pixel shader's push constants (Recompile resizes pushConstants, BindingAllocator puts stencilReferenceDword right after the shader data, or at dword 0 when the layout keeps its shader data in a buffer).

The second case was broken. When a pixel shader's shader data does not fit its push range (or it uses runtime images), IrBindingLayout::UsesPushData() is false and the guest buffer offsets live in the shader-data buffer; the stencil reference is then the only push constant. ShaderResources decided where the guest buffer offsets are with !push.empty(), so it looked for them in the four-byte reference and refused the draw with guest buffer offset lies outside the shader's push constants.

It now counts only the shader-data part of the push constants: all of them without stencil replay, and the bytes before stencilReferenceDword with it (the same rule Recompile and the prepared-artifact check already use). A layout without push data then takes the shader-data buffer path, as it does without stencil replay.

Found while running Onimusha: Way of the Sword (PPSA27836) on an RTX 3080 with a local stack of the open stencil replay changes: the first replayed draw failed this way.

Tested

Windows 10 Pro 22H2 (19045), WinLibs GCC 15.2.0 r7, Release, on main a6dcea927.

  • agc_driver_stencil_replay, new case: the finite-export shader with its color read by s_buffer_load_dword through a buffer descriptor in user data, with the fragment push range shrunk to the reference dword so the shader data stays in a buffer; the replayed draw must color every pixel from the buffer. On main it fails on both GPUs with AGC graphics: guest buffer offset lies outside the shader's push constants; with this change it passes.
  • With ANYPS5_REQUIRE_VULKAN=1 and ANYPS5_NO_SHADER_CACHE=1, on main and on this change: agc_driver_stencil_replay, agc_driver_stencil_replay_no_gpl, agc_driver_stencil_ref_export, agc_driver_stencil_ref_export_unused, agc_driver_stencil_known_replace, agc_driver_stencil_null_feedback, agc_driver_sample_mask_depth_stencil, agc_masked_select, agc_driver_graphics, agc_driver_draw_cache_relocation, the five agc_driver_draw_cache_reuse* variants, agc_driver_prepared_graphics with its three prebuild variants, agc_driver_prepared_unregistered_graphics and guest_buffer_reuse (20 tests) pass on both GPUs below, except agc_driver_stencil_ref_export skipped on the RTX 3080 (no VK_EXT_shader_stencil_export). Results are identical on main and on this change (main runs its own StencilReplay.cpp, without the new case).
  • In the title: with this change the replayed Onimusha draw no longer fails (the run then continues, though the draw's unbounded reference makes it very slow; that is unrelated to this change).
  • Not run: the Khronos validation layer (not installed on this host, so VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation has nothing to load; neither log contains a VUID- line).
  • tools/check_conventions.py --base origin/main and git diff --check pass. tools/pr_overlap.py was not run (no gh on this host).

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: none
  • Unblocks: none (PPSA27836 Onimusha: Way of the Sword (PPSA27836) #5769 also needs the open stencil replay mask changes and a bounded reference)
  • GPUs tested: NVIDIA RTX 3080, 617.42, Windows 10 22H2; AMD Radeon RDNA 2 iGPU (1002:164e), 32.0.21045.5002, Windows 10 22H2
  • AI-assisted: yes

🤖 Generated with Claude Code

…y adds a push constant

Stencil replay appends a four-byte reference to a pixel shader's push
constants. When the shader's binding layout keeps its shader data in a
buffer (it does not fit the push range, or the shader uses runtime
images), the reference is the only push constant. ShaderResources took
any non-empty push constants to hold the guest buffer offsets, read them
from the push constants and refused the draw. It now counts only the
shader data part of the push constants, which ends at the stencil
reference dword when the shader replays its stencil export.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request Conflicting files (lines on main) Same exports Same new files
PR 8615 core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp L1998, L2000

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 6192 core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 6194 core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 6202 core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 6223 core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 6292 core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 6412 core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 6481 core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 6615 core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 6946 core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 7642 core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 7689 core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 8026 core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 8672 core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 8998 core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 9508 core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 9592 core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 9674 core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp

@jotitadev jotitadev left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed red→green on Linux on two vendors (the PR's runs are both on Windows: RTX 3080 and an RDNA 2 iGPU).

Tested

Ubuntu 26.04, GCC 15.2.0, Release, BUILD_TESTING=ON, ANYPS5_NO_SHADER_CACHE=1 and ANYPS5_REQUIRE_VULKAN=1, device selected with VK_ICD_FILENAMES, run as ./agc_driver_stencil_replay_tests, on main a6dcea927 (this PR applies cleanly on it; head 62a5b6cea).

agc_driver_stencil_replay NVIDIA RTX 3070 (595.99.02) lavapipe (Mesa 26.0.3, LLVM 21.1.8)
main a6dcea927 + the PR's test only (ShaderResources.cpp reverted) fail fail
this PR pass pass

With ShaderResources.cpp reverted and the PR's new case kept, the run exits non-zero on both devices with:

AGC graphics: guest buffer offset lies outside the shader's push constants

That is the Require this PR relaxes: with the stencil-replay path the fragment push range is shrunk to the reference dword, so a s_buffer_load_dword of the color through a buffer descriptor in user data leaves the shader data in a buffer past the push constants. With the full PR, the new case ("dynamic reference keeps guest buffer offsets in shader data outside the push constants") colors every pixel from the buffer and passes on both devices.

GPUs tested: NVIDIA GeForce RTX 3070 (595.99.02, Linux) and lavapipe (Mesa 26.0.3, LLVM 21.1.8), Ubuntu 26.04.

Written with AI assistance (Claude Code). I ran the builds and tests above on my machine.

@github-actions github-actions Bot added the agc libSceAgc* label Oct 22, 2026

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This reproduces a real failure and adds a useful buffer-backed regression. There is, however, an earlier open PR (#8615) that implements the same fix in the same ShaderResources::buildPrepare branch: it also replaces !push.empty() with a check of the shader-data bytes before stencilReferenceDword, and routes the no-shader-data case through dataByte. This is the same behavior, not just an adjacent edit, so the checklist claim that no open PR implements it is currently inaccurate. Please coordinate with #8615's author and the maintainers so the fix has one authoritative PR; if your buffer-offset variant adds distinct coverage, agree on transferring/combining that test without duplicating the implementation.

The reported test runs cover the regression on Windows across an RTX 3080 and an AMD RDNA 2 iGPU, but the repository requires the paired AGC validation-layer run with VK_LAYER_KHRONOS_validation loaded and a VUID- comparison. The description says that layer was unavailable. Please provide that paired run on a host where the layer is installed, and rerun the focused tests after updating from current main (the comparison base is a6dcea927, 53 commits behind the current tip).

The hosted eligibility, conventions, intake, labels, and report checks pass; Linux/Windows product jobs are skipped. A separate reviewer reports Linux Vulkan runs on two vendors, but those are on the same older a6dcea927 baseline and do not replace the required validation-layer comparison. I did not run the tests locally because no matching checkout/build is available here.

@Lindamea1

Copy link
Copy Markdown
Contributor

I'm the author of #8615. Its buildPrepare change is the same as this one: it counts only the push bytes before stencilReferenceDword as shader data, and writes the offsets to the shader data buffer when there are none. #8615 already has the paired validation-layer run on main and on the change (RTX 5090, Linux), and @ycbuzi ran its two stencil replay tests on Windows (RTX 4070 SUPER).

Your case covers a route that #8615's doesn't. #8615's shader gets a shader data buffer only through its image, which needs registration modules (the pipeline library). With APS5_NO_GPL=1 its shader data fits in push constants, so with main's ShaderResources.cpp its case still passes in agc_driver_stencil_replay_no_gpl. Your shader's data is in a buffer because the push range is shrunk, so with main's ShaderResources.cpp your case fails in both tests.

I ran your case on top of #8615's change: main d6eafa5, RTX 5090 (595.99.02), Gentoo Linux, with VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation, ANYPS5_NO_SHADER_CACHE=1 and ANYPS5_REQUIRE_VULKAN=1:

  • agc_driver_stencil_replay and agc_driver_stencil_replay_no_gpl pass. The layer loaded, and neither log has a VUID- line.
  • With ShaderResources.cpp from main, both fail with AGC graphics: guest buffer offset lies outside the shader's push constants.

To keep one implementation, either of these works for me:

  1. I add your test case to fix(agc): put a stencil replay shader's guest buffer offsets where its shader data is #8615 as a separate commit with you as its author, and credit the Onimusha find (PPSA27836) in the description. You could then close this one.
  2. You keep this PR for the test case only, rebased onto fix(agc): put a stencil replay shader's guest buffer offsets where its shader data is #8615, with Depends on: #8615.

Which would you prefer? cc @MarcZX10

AI-assisted: yes (Claude Code).

@TheSovietWaffle

Copy link
Copy Markdown
Contributor Author

@Lindamea1 thanks, option 1 works for me: please add the test case to #8615 as a separate commit with me as its author. I'll close this PR once it's there.

One note for the description: the case fails in both agc_driver_stencil_replay and agc_driver_stencil_replay_no_gpl with main's ShaderResources.cpp because the fragment push range is shrunk to the reference dword (pixelPushBytes = 4), so its shader data is in a buffer with or without the pipeline library.

AI-assisted: yes (Claude Code).

@Lindamea1

Copy link
Copy Markdown
Contributor

@TheSovietWaffle Done: your test case is now the second commit of #8615 (15803c7), with you as its author and its lines unchanged. I rewrote the commit message to describe only the test, since the fix is #8615's first commit. The description credits you and the Onimusha find (PPSA27836, #5769), includes your note on why the case fails in both tests (pixelPushBytes = 4), and cites your Windows runs and jotitadev's Linux runs from here.

#8615 is rebased onto main 4777cac. On that base, with the RTX 5090 (595.99.02, Gentoo Linux):

  • With ShaderResources.cpp from main, agc_driver_stencil_replay and agc_driver_stencil_replay_no_gpl both fail with AGC graphics: guest buffer offset lies outside the shader's push constants. With the fix, both pass in 20 of 20 runs each.
  • The paired validation-layer run on main and on the branch (20 focused tests, listed in fix(agc): put a stencil replay shader's guest buffer offsets where its shader data is #8615's description) gives the same results on both: 19 pass and agc_driver_stencil_ref_export skips. The layer loaded, and neither log has a VUID- line.

It's in, so this one can be closed whenever you're ready. Thanks for the find and the test. cc @MarcZX10

AI-assisted: yes (Claude Code).

@TheSovietWaffle

Copy link
Copy Markdown
Contributor Author

Closing: the fix is #8615 and this test is now its second commit (15803c7). Thanks @Lindamea1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agc libSceAgc*

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants