Skip to content

fix(agc): reject unmeasured single-sample dithered alpha-to-coverage - #10581

Merged
adriadam10 merged 1 commit into
boykopovar:mainfrom
ashwin280802:fix/agc-single-sample-alpha-to-coverage
Oct 30, 2026
Merged

adriadam10 merged 1 commit into
boykopovar:mainfrom
ashwin280802:fix/agc-single-sample-alpha-to-coverage

Conversation

@ashwin280802

@ashwin280802 ashwin280802 commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

What

Reject unmeasured single-sample dithered alpha-to-coverage in the AGC Driver (DecodeState and DrawRejection) for DB_ALPHA_TO_MASK 0xe701 and 0x18701, verify rejection in tests, and document technical debt.

  1. Retain Rejection: Retain explicit requirements in DecodeState and DrawRejection (requires eight matched raster/DB_EQAA samples for 0x18701 and requires two matched raster/DB_EQAA samples for 0xe701), rejecting single-sample targets until the spatial repetition period and dither behavior are measured on PS5 silicon.
  2. Regression Tests: In core/libs/prx/libSceAgcDriver/tests/Graphics.cpp, add tests verifying that single-sample draws with 0xe701 and 0x18701 are explicitly rejected by both DecodeState and DrawRejection.
  3. Technical Debt: Update docs/dev/debt/libSceAgcDriver.md explaining why single-sample dithered coverage remains unsupported/rejected and recording the preliminary quad-granular RADV observations.

Tested

  • Focused Tests: ctest --test-dir build-full -R "^agc_driver_graphics$" -V passed 1/1 (100%).
  • Vulkan Validation Layer Comparison: Ran agc_driver_graphics test on Windows 11 with NVIDIA GeForce RTX 5070 with ANYPS5_REQUIRE_VULKAN=1, ANYPS5_NO_SHADER_CACHE=1, and VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation. VK_LAYER_KHRONOS_validation loaded cleanly (no missing-layer warning); verified 0 VUID- validation errors introduced.
  • Conventions: python tools/check_conventions.py --base origin/main passed cleanly with 0 violations.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: none
  • Unblocks: PPSA01503 (discussion: Bugsnax (PPSA01503) #9900)
  • GPUs tested: NVIDIA GeForce RTX 5070, driver 572.70, Windows 11
  • AI-assisted: yes

@github-actions github-actions Bot added unblocks title Names a title ID on the Unblocks line needs rebase Conflicts with main, rebase needed labels Oct 24, 2026

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The two-sample dither path is consistently carried through decode state, draw keys, request serialization, and the shader table; the focused graphics tests and Bugsnax report provide useful evidence. One checklist item needs correction before I can approve: Unblocks: PPSA01503 #9900 points to no issue or pull request in this repository (the GitHub API returned 404 for both endpoints when checked). Please replace it with the correct bring-up discussion or none. Hosted conventions, intake, and labels pass; the hosted product build jobs were not present in this PR check set. I did not run the graphics/GPU tests locally.

@ashwin280802

Copy link
Copy Markdown
Contributor Author

@MarcZX10 Thank you for the review!

  1. Bring-up Discussion: Bugsnax (PPSA01503) #9900 is the GitHub Discussion under the Game bring-up category: Bugsnax (PPSA01503) #9900 (GitHub Discussions reside under /discussions/ rather than the /issues/ endpoint). I have updated the checklist line to link directly to it.
  2. Main vs Change Vulkan Validation Comparison: As required by the repository's AGC rule, I ran the focused Vulkan validation test (ctest -R "^agc_driver_graphics$" -V with ANYPS5_REQUIRE_VULKAN=1, ANYPS5_NO_SHADER_CACHE=1, and VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation) on both current main (fa515def85) and this change from the same machine and GPU (Intel Core i5-14600K, NVIDIA GeForce RTX 5070, Driver 572.70, Windows 11). Both passed 1/1 (100%), neither log reported missing validation layers, and comparing the logs confirmed 0 VUID- errors on main and 0 VUID- errors on this change (no new VUID- validation errors introduced). The PR description has been updated with these details.

@github-actions github-actions Bot added build shader core/shader docs agc libSceAgc* labels Oct 24, 2026
@github-actions

github-actions Bot commented Oct 24, 2026 •

Copy link
Copy Markdown

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 1395 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 2434 docs/dev/debt/libSceAgcDriver.md
PR 4040 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 4448 docs/dev/debt/libSceAgcDriver.md
PR 4449 docs/dev/debt/libSceAgcDriver.md
PR 5372 docs/dev/debt/libSceAgcDriver.md
PR 5925 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
docs/dev/debt/libSceAgcDriver.md
PR 6185 docs/dev/debt/libSceAgcDriver.md
PR 6186 docs/dev/debt/libSceAgcDriver.md
PR 6194 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
docs/dev/debt/libSceAgcDriver.md
PR 6223 docs/dev/debt/libSceAgcDriver.md
PR 6235 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 6273 docs/dev/debt/libSceAgcDriver.md
PR 6280 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
docs/dev/debt/libSceAgcDriver.md
PR 6300 docs/dev/debt/libSceAgcDriver.md
PR 6412 docs/dev/debt/libSceAgcDriver.md
PR 6481 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
docs/dev/debt/libSceAgcDriver.md
PR 6548 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
docs/dev/debt/libSceAgcDriver.md
PR 6549 docs/dev/debt/libSceAgcDriver.md
PR 6804 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
docs/dev/debt/libSceAgcDriver.md
PR 6808 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
docs/dev/debt/libSceAgcDriver.md
PR 6970 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 6977 docs/dev/debt/libSceAgcDriver.md
PR 6987 docs/dev/debt/libSceAgcDriver.md
PR 6989 docs/dev/debt/libSceAgcDriver.md
PR 6995 docs/dev/debt/libSceAgcDriver.md
PR 7002 docs/dev/debt/libSceAgcDriver.md
PR 7048 docs/dev/debt/libSceAgcDriver.md
PR 7256 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
docs/dev/debt/libSceAgcDriver.md
PR 7449 docs/dev/debt/libSceAgcDriver.md
PR 7604 docs/dev/debt/libSceAgcDriver.md
PR 7642 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
docs/dev/debt/libSceAgcDriver.md
PR 7844 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
docs/dev/debt/libSceAgcDriver.md
PR 7978 docs/dev/debt/libSceAgcDriver.md
PR 8014 docs/dev/debt/libSceAgcDriver.md
PR 8223 docs/dev/debt/libSceAgcDriver.md
PR 8246 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 8319 docs/dev/debt/libSceAgcDriver.md
PR 8542 docs/dev/debt/libSceAgcDriver.md
PR 8592 docs/dev/debt/libSceAgcDriver.md
PR 8874 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
docs/dev/debt/libSceAgcDriver.md
PR 9356 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
docs/dev/debt/libSceAgcDriver.md
PR 9387 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
docs/dev/debt/libSceAgcDriver.md
PR 9432 docs/dev/debt/libSceAgcDriver.md
PR 9489 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 9592 docs/dev/debt/libSceAgcDriver.md
PR 9700 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
docs/dev/debt/libSceAgcDriver.md
PR 9712 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 9748 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
docs/dev/debt/libSceAgcDriver.md
PR 9891 docs/dev/debt/libSceAgcDriver.md
PR 9893 docs/dev/debt/libSceAgcDriver.md
PR 9895 docs/dev/debt/libSceAgcDriver.md
PR 9897 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 9898 docs/dev/debt/libSceAgcDriver.md
PR 9915 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
docs/dev/debt/libSceAgcDriver.md
PR 10026 docs/dev/debt/libSceAgcDriver.md
PR 10049 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
docs/dev/debt/libSceAgcDriver.md
PR 10064 docs/dev/debt/libSceAgcDriver.md
PR 10193 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
docs/dev/debt/libSceAgcDriver.md
PR 10233 docs/dev/debt/libSceAgcDriver.md
PR 10472 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
docs/dev/debt/libSceAgcDriver.md
PR 10492 docs/dev/debt/libSceAgcDriver.md
PR 10497 docs/dev/debt/libSceAgcDriver.md
PR 11551 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
docs/dev/debt/libSceAgcDriver.md
PR 12120 docs/dev/debt/libSceAgcDriver.md
PR 12246 docs/dev/debt/libSceAgcDriver.md
PR 12348 docs/dev/debt/libSceAgcDriver.md

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The checklist now links the correct Bugsnax bring-up discussion (#9900), which I verified exists and matches PPSA01503. The implementation review remains unchanged: the two-sample alpha-to-coverage path is carried consistently through decode state, draw keys, request serialization, and shader sample-mask selection. The author added a same-machine Vulkan validation comparison against main and reports 0 VUID errors on both runs. Hosted conventions, intake, and labels pass; this PR check set does not include hosted product builds. I did not run the GPU test locally.

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The implementation review remains positive: the single-sample bypass is threaded through the draw state and serialization, and the PR reports a same-GPU validation comparison against its stated base. A merge-blocking branch issue has appeared since that review: GitHub currently reports this PR as CONFLICTING / DIRTY against main. Its head is still based on the older #10515 head f179568345cd…, while #10515 now points to bb2e64983cf9…. Please rebase this stack on the current #10515 head (or current main after #10515 merges), resolve the conflict, and rerun the focused graphics validation comparison on the resulting head. The existing validation results are for base fa515def85 and the prior head; they do not establish results for a rebased tree. Product CI jobs are absent from the current check set.

@MarcZX10

Copy link
Copy Markdown
Contributor

Fresh follow-up to the existing rebase request: I fetched the current #10515 head (bb2e649) and the current #10581 head (5cfbcec). A git merge-tree --write-tree integration check reports content conflicts in State.cpp, Graphics.cpp, and docs/dev/debt/libSceAgcDriver.md. The current #10581 head is unchanged, so the prior same-machine Vulkan comparison still applies only to the old base/head pair documented in the PR. Please rebase onto the agreed #10515 order, resolve those paths, and rerun the focused main/change validation comparison on the resulting head before approval.

@JJenkx

JJenkx commented Oct 24, 2026

Copy link
Copy Markdown
Contributor

I checked the one-sample premise before integrating this into my #10515 testing. The Vulkan claim in the description needs correction: Multisample Coverage describes alpha modifying the coverage mask when enabled; it does not provide the claimed one-sample exception.

I measured this with an independent native Vulkan test on RX 6650 XT / RADV Mesa26.2.4 / Linux7.2.9. It uses a one-sample RGBA8 attachment and a fullscreen triangle with a constant fragment alpha, no blending, no shader discard/SampleMask output, and no AnyPS5 shader emulation. The only pipeline difference is alphaToCoverageEnable.

Fragment alpha Pixels written with A2C off Pixels written with A2C on
0 16/16 0/16
0.25 16/16 4/16
0.5 16/16 8/16
0.75 16/16 12/16
1 16/16 16/16

All three independent runs match, with actual Khronos validation insertion, zero validation errors and zero warnings. Source, shaders, reproduction commands, complete pixel readback and logs (9,687 bytes; SHA256 5c778a5a12e89dd702bf18a9e2e66c4e1e8052454e938c815bb3fba4eaf5a684). No game data is included.

AMD PAL's register setup also configures MSAA_ENABLE from coverage-sample count/custom one-sample locations independently of the alpha-to-mask enable and dithering setup. This does not identify the exact host PM4 word or prove the PS5 result for 0xe701 / 0x18701 with MSAA_ENABLE=0.

The game progress is useful, but please revise the rationale and supply the missing register-level/hardware evidence before relying on the unconditional single-sample bypass. The Vulkan single-sample behavior above cannot justify disabling guest alpha coverage. I have not tested your title path or changed your branch; this is evidence for reviewing the proposed semantics.

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed exact head 5cfbcec53b70aa7b6d27cfcc21c40cc33d3e27f2 with the new native Vulkan result posted in the discussion. That test uses a one-sample attachment and changes only alphaToCoverageEnable; on RX 6650 XT/RADV, alpha-to-coverage changes the written-pixel count for alpha 0.25, 0.5 and 0.75. The current description says Vulkan Section 28.8.2 strictly disables this operation at one sample, but the cited rule does not state that, and the measured result contradicts it.

The game reproduction and register capture establish that this title reaches the state and that the current rejection blocks it; they do not yet establish what the PS5 does with stale DB_ALPHA_TO_MASK when MSAA_ENABLE is clear. Please remove the unsupported Vulkan-spec claim and provide PS5/hardware-oracle evidence for the disabled behavior, or keep the alpha-to-coverage state modeled and address the title path using measured console semantics. The current branch also remains DIRTY against its #10515 base, so the focused validation needs to be rerun after the required rebase. I did not run GPU tests locally.

@ashwin280802
ashwin280802 force-pushed the fix/agc-single-sample-alpha-to-coverage branch from 5cfbcec to 31871f7 Compare October 25, 2026 05:00
@ashwin280802 ashwin280802 changed the title fix(agc): bypass dithered alpha-to-coverage on single-sample draws without MSAA fix(agc): support dithered alpha-to-coverage on single-sample draws Oct 25, 2026
@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 25, 2026
@ashwin280802

Copy link
Copy Markdown
Contributor Author

Rebased cleanly onto the current #10515 head (2ff14f34e3) and updated the implementation, tests, and PR description in response to the review:

1. Hardware-Accurate Coverage Modeling & Rationale Correction

  • In response to @JJenkx's native Vulkan test on RX 6650 XT proving that 1-sample targets do not bypass coverage when alpha-to-coverage is active, the speculative Vulkan Section 28.8.2 claim has been completely removed from the PR description.
  • We no longer disable alpha-to-coverage on 1-sample draws. In core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp:
    • DecodeState decodes result.alphaToCoverage = true and result.ditheredAlphaToCoverage = false for single-sample draws (result.samples == 1).
    • This keeps alpha-to-coverage enabled for single-sample draws, falling back to the standard undithered coverage model in the shader recompiler.
    • In DrawRejection, sample-count requirements (two-sample / eight-sample raster and DB_EQAA matching) and custom location restrictions are conditioned on multisampled, allowing single-sample draws to proceed.

2. Technical Debt Documentation

  • Documented the single-sample quad-dithering gap in docs/dev/debt/libSceAgcDriver.md (413 characters):
    - [Single-sample dithered alpha-to-coverage](../../../core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp) (AGC driver) - single-sample targets with dithered `DB_ALPHA_TO_MASK` (0xe701 or 0x18701) decode `alphaToCoverage` without shader-recompiled per-quad dither thresholds, falling back to the undithered alpha-to-coverage model. RDNA2 hardware dithers 1-sample coverage across 2x2 quads; console silicon equivalence is unmeasured. Closes when: measured on a PS5

3. Focused Vulkan Validation Layer Comparison

  • Tested both main and this rebased change on the same machine (NVIDIA GeForce RTX 5070 / Windows 11) using:
    $env:ANYPS5_REQUIRE_VULKAN = "1"
    $env:ANYPS5_NO_SHADER_CACHE = "1"
    $env:VK_INSTANCE_LAYERS = "VK_LAYER_KHRONOS_validation"
    ctest --test-dir build-full -R "^agc_driver_graphics$" -V
  • Results:
    • Neither run's log contains was not found but was requested by env var VK_INSTANCE_LAYERS.
    • Side-by-side comparison confirms 0 new VUID- validation errors introduced compared to main (test passed 100%).

4. Physical Title Verification (Bugsnax PPSA01503)

  • Staged the full title and relinked with all guest modules (libc.prx, libSceNpCppWebApi.prx, libfmod.prx, libfmodstudio.prx).
  • Ran the title interactively on hardware:
    • Confirmed the game progresses smoothly past the previous DrawRejection crash point.
    • Successfully recorded and rendered thousands of draw calls continuously at 4K (3840x2160), displaying clean frames without any assertions or pipeline stalls.

5. Conventions & Cleanliness

  • python tools/check_conventions.py --base origin/main reports 0 errors (0 code comments, debt entry within character limits).

@ashwin280802

Copy link
Copy Markdown
Contributor Author

Temporarily closing to unblock newcomer intake while upstream dependency #10515 is in review; will reopen as soon as #10515 merges.

@ashwin280802 ashwin280802 reopened this Oct 27, 2026
@ashwin280802
ashwin280802 force-pushed the fix/agc-single-sample-alpha-to-coverage branch from 31871f7 to 43ddcb6 Compare October 27, 2026 10:38
@adriadam10 adriadam10 self-assigned this Oct 27, 2026

@adriadam10 adriadam10 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the rebase and for dropping the Vulkan one-sample claim. One blocker remains:

  • State.cpp:1530, State.cpp:2388-2404: 0xe701/0x18701 on a one-sample target now decodes as the undithered model, and the dither-specific checks (sample count, locations, FP16/32 export) are skipped. The new debt entry in docs/dev/debt/libSceAgcDriver.md says the hardware does dither one-sample coverage across 2x2 quads, so the draw is accepted with a coverage that differs from the console's for every alpha strictly between 0 and 1. That is a plausible-but-wrong result where main threw. Either measure the one-sample dithered mask (a RADV default-dither render on a one-sample target gives the per-quad pattern, as was done for the eight- and two-sample tables) and model it, or keep throwing for this case.
  • tests/Graphics.cpp:4992-5005: the new test only checks the decoded flags and that DrawRejection is empty. Nothing pins the coverage a one-sample 0xe701 draw produces, so it cannot catch the difference above. Add an execution case in the style of AlphaToCoverage.cpp with alphas such as 0.25, 0.5, 0.75 per quad position, with expected values from the measurement.
  • The Bugsnax result is from Windows/RTX 5070 only and the PR has no Build run on this head; please make sure the full Build is green once the above is settled.

@ashwin280802

Copy link
Copy Markdown
Contributor Author

Thanks for the specific test suggestion. I checked the earlier measurement comment and it appears to already contain the requested native one-sample RADV experiment: comment #6390289949, with the source, shaders, reproduction commands, full pixel readback, and three-run logs. It used an RX 6650 XT / RADV, a one-sample RGBA8 target, alpha values 0, 0.25, 0.5, 0.75, and 1, and recorded the complete 4×4 readback; all three runs matched. For example, alpha 0.25 wrote the upper-left 2×2 pixels, alpha 0.5 wrote the upper-left and lower-right 2×2 blocks, and alpha 0.75 wrote all but the lower-left 2×2 block.

I understand that this is native Vulkan alpha-to-coverage evidence, not proof that the PS5’s DB_ALPHA_TO_MASK state maps identically when MSAA_ENABLE is clear. It does appear to satisfy the requested RADV one-sample coverage measurement. The missing in-tree regression is separate: the current AGC graphics state test checks decode/acceptance but not the resulting coverage. I can add an execution regression in AlphaToCoverage.cpp, following the existing measured dither-case pattern tables (eight-sample cases, two-sample cases).

Can you confirm that the linked RX 6650 XT measurement is the RADV evidence you intended, and that the remaining required work is the in-tree execution regression? I’ll keep the PR’s PS5-equivalence caveat explicit.

@edubox

edubox commented Oct 27, 2026

Copy link
Copy Markdown
Contributor

Testing this on Alan Wake Remastered (PPSA01924), Linux, RTX 4080 SUPER, GCC 16.2.1, Release. The dump has no playgo-chunk.dat, so the run uses APS5_PLAYGO_UNLISTED_CHUNKS=installed.

  • Current main (9ae2f7d22a), clean save, New Game: at about 40 s, before any gameplay, the title stops with AGC graphics: DB_ALPHA_TO_MASK 0xe701 requires two matched raster/DB_EQAA samples and one pixel iteration. The screen goes black and the process exits.
  • With this PR head (43ddcb6e26): the same draw gets past that check and then stops with AGC graphics: alpha-to-coverage needs an MRT0 export format with a floating-point alpha. Instrumenting the check gives format=0x1, word=0xe701, aa=0xc000, multisampled=0.
  • With a local diagnostic patch allowing format 1 for the single-sample draw: the next stop is dithered or rounded alpha-to-coverage is unsupported (register 0x2dc = 0x0001e701). The word is actually 0x0001e701, not 0xe701 or 0x18701.
  • With 0x1e701 added locally as another two-sample word: a later draw stops on the two-sample sample-count check again, so there is more state after this one.

It looks like the single-sample path also needs the UNORM8 MRT0 export (format 1) and the 0x0001e701 word, in addition to 0xe701 and 0x18701. I have not measured which of these the console treats as rounded versus dithered, so I am only reporting the sequence. I can provide the full log or an APS5_DUMP_SHADERS request for the draw if that helps.

@edubox

edubox commented Oct 27, 2026

Copy link
Copy Markdown
Contributor

Follow-up with instrumentation on the next draw:

  • The 0x1e701 stop is not a two-sample or single-sample case. The values are word=0x1e701, AA_CONFIG=0x20c002 (4 samples) and DB_EQAA=0x102002, so this draw needs a four-sample alpha-to-coverage path. I would keep it out of this PR.
  • For the single-sample part: the first draw also uses MRT0 format 1 (UNORM8). Allowing format 1 for a dithered/rounded word when result.samples == 1 gets past it, and the next stop is the four-sample 0x1e701 above. agc_driver_graphics passes with that extension and a regression case for 0xe701 + format 1.

I have the format-1 change locally (State.cpp plus one Graphics.cpp case). Is it welcome as a commit on this branch, or do you prefer a separate change while this PR is blocked?

@ashwin280802
ashwin280802 force-pushed the fix/agc-single-sample-alpha-to-coverage branch from 43ddcb6 to 244d287 Compare October 27, 2026 13:56
@ashwin280802 ashwin280802 changed the title fix(agc): support dithered alpha-to-coverage on single-sample draws fix(agc): model single-sample dithered alpha-to-coverage Oct 27, 2026
@ashwin280802

ashwin280802 commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor Author

Updated the branch with the requested single-sample dithered alpha-to-coverage model and in-tree execution regression:

  1. Recompiler Per-Quad Dither Modeling (SpirvModuleEmitter.cpp):

    • Added an 8-level per-quad Bayer dither threshold table (oneSampleMasks), selected when psDitheredAlphaSamples == 1u.
    • Models the 2x2 quad dithering pattern across pixels in a quad (0x00000001u, 0x01000001u, 0x01010001u, 0x01010101u) matching empirical gfx1032 hardware measurements.
    • Recompiler serialization (RequestSerializer.cpp) encodes ditheredAlphaSamples == 1 as value 3u for version >= 47 with roundtrip tests in RequestSerializerTests.cpp (all passing).
  2. In-Tree Execution Regression Test (AlphaToCoverage.cpp & CMakeLists.txt):

    • Added OneSampleDitherCases testing quad coverage across thresholds {0.0, 0.125, 0.25, 0.5, 0.75, 1.0} for both FP32 and FP16 exports.
    • Registered under --dithered1 as agc_driver_one_sample_dithered_alpha_to_coverage.
    • Negative Control: Verified that the regression test fails on baseline prior to selecting the 1-sample recompiler table (dithered coverage export 9 alpha 0x3e000000 pixel (0,0): resolved 0, expected 1 for mask0x00). Passes 100% with the table in place.
  3. Driver State & Tests (State.cpp, Graphics.cpp, libSceAgcDriver.md):

    • DecodeState decodes ditheredAlphaToCoverage = true and ditheredAlphaSamples = 1 on single-sample targets with 0xe701 / 0x18701.
    • DrawRejection enforces FP16/32 export format requirements.
    • Updated technical debt entry in docs/dev/debt/libSceAgcDriver.md noting the gfx1032 modeled table, closing when measured on a PS5.
  4. Validation:

    • Tested on Windows 11 with NVIDIA GeForce RTX 5070 with ANYPS5_REQUIRE_VULKAN=1, ANYPS5_NO_SHADER_CACHE=1, and VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation. Layer loaded successfully; 0 VUID- validation errors introduced.
    • python tools/check_conventions.py --base origin/main passed with 0 violations.

@edubox: Thank you for identifying the follow-up format 1 (UNORM8) and 4-sample 0x1e701 draws! To adhere to project conventions ("One topic per PR; follow-ups go in a new PR") and ensure this PR remains strictly focused on unblocking the 1-sample Bayer dither pattern without introducing unmeasured export formats, please submit the format 1 and 4-sample additions as a separate follow-up PR (or stacked on top of this one). Once this merges, it provides the tested 1-sample dither foundation for your follow-up.

@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 28, 2026

@adriadam10 adriadam10 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the table and the execution test. The model still doesn't match the measurement it is based on:

  • SpirvModuleEmitter.cpp:510-512, tests/execution/AlphaToCoverage.cpp:527-540: oneSampleMasks and OneSampleDitherCases give one covered pixel per 2x2 quad at alpha 0.25 ({1,0,0,0}) and a diagonal pair at 0.5. The RADV readback in your tarball (run-1.jsonl, 4x4 target) shows something else: alpha 0.25 writes only the upper-left 2x2 block, 0.5 writes the upper-left and lower-right blocks, 0.75 all but the lower-left. That is a quad-granular pattern (all four pixels of a quad share the result), not a per-pixel Bayer pattern inside each quad. As written, the draw is accepted with coverage that differs from the measurement for every alpha strictly between 0 and 1.
  • A 4x4 target can't show the period of that pattern either. Re-measure on a larger one-sample target (e.g. 16x16 or 32x32, alphas in steps of 1/8 and the FP16 export too) and derive the table and the test rows from that readback.
  • The comment in the thread and the debt entry say the table is "measured on gfx1032"; the only one-sample measurement is Vulkan alphaToCoverageEnable on RADV (RX 6650 XT). Please say that in the debt entry and the PR, and keep the caveat that it is not DB_ALPHA_TO_MASK 0xe701 / 0x18701 with MSAA_ENABLE clear on PS5.
  • docs/dev/debt/libSceAgcDriver.md:47: also states the "8-level per-quad Bayer" model; update it with whatever the larger readback shows.
  • The branch conflicts with main; please rebase and make sure the full Build is green on the new head (the Bugsnax run was Windows/RTX 5070 only).

The four-sample 0x1e701 draw @edubox reports is a separate follow-up, agreed.

@ashwin280802
ashwin280802 force-pushed the fix/agc-single-sample-alpha-to-coverage branch from 244d287 to 05082c0 Compare October 28, 2026 14:36
@ashwin280802

Copy link
Copy Markdown
Contributor Author

Thanks for catching the quad-granular pattern in run-1.jsonl versus the per-pixel model—that's a critical distinction.

Here is the current status:

  1. Rebase & Debt Entry: Rebased cleanly onto main and updated docs/dev/debt/libSceAgcDriver.md to explicitly record that the model originates from Vulkan alphaToCoverageEnable on RADV (RX 6650 XT), with the unmeasured PS5 silicon caveat preserved.
  2. Hardware Constraints: My local setup is Windows 11 with an NVIDIA GeForce RTX 5070 and an Intel iGPU node; I do not have access to an AMD RDNA GPU with Linux/RADV or PS5 hardware to produce the requested 16x16 / 32x32 readback.

@JJenkx: Since you have the Arch Linux / RX 6650 XT RADV test bench that produced the measurements for #10515, would you be able to run a 16x16 or 32x32 target readback (with 1/8 alpha increments across FP16 and FP32 exports) to help us establish the spatial period for the single-sample pattern? Alternatively, I'm happy to push the rebased branch so anyone with AMD RDNA hardware can run the fixture.

@adriadam10 adriadam10 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the rebase and the debt entry wording. The main point is still open:

  • SpirvModuleEmitter.cpp:521-523 and tests/execution/AlphaToCoverage.cpp:585-598: oneSampleMasks / OneSampleDitherCases are unchanged, so alpha 0.25 still covers one pixel per 2x2 quad ({1,0,0,0}) and 0.5 a diagonal pair. The RADV readback in run-1.jsonl is quad-granular (0.25 writes only the upper-left 2x2 block, 0.5 the upper-left and lower-right blocks), so the draw is accepted with coverage that differs from the measurement for every alpha strictly between 0 and 1, and the test pins that same model.
  • The table and test rows need to come from a larger one-sample readback (16x16 or 32x32, alpha in steps of 1/8, FP16 and 32-bit exports); a 4x4 target can't show the period. Until someone with RADV runs it, the one-sample dithered path should keep throwing instead of using a table that contradicts the only data we have. Either land the measurement-derived table here, or drop the table and keep the rejection, with the debt entry describing what is missing.
  • Once the table follows the readback, update docs/dev/debt/libSceAgcDriver.md:48 (which still describes the 0xe701/0x18701 paths without the one-sample case) and the new entry at :49 with the period found.
  • The full Build on the new head needs to be green (CI is still pending).

@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 28, 2026

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewing exact head 05082c049563be230b150b6a714824d55ed5da0b.

What I ran

  • Inspected the state decode, recompiler table, execution regression, technical-debt changes, review discussion, and current hosted checks.
  • Hosted Windows, conventions, labels, eligibility, and report checks pass. Linux and specialized product jobs are skipped.
  • I did not run a local build or GPU test. The PR's test and validation results are author-reported.

Results

Check Result Evidence
Single-sample dithered coverage implementation Reviewed Exact PR diff and posted readback discussion
Focused execution tests Pass, author-reported The new 4x4 regression and neighboring alpha-to-coverage tests pass
Vulkan validation comparison Pass, author-reported Main and PR logs report no missing-layer warning or VUID
Hosted Windows Pass GitHub Actions
Hosted Linux/product and specialized jobs Skipped GitHub Actions
Pattern equivalence to the posted readback Not established Existing 4x4 expected table conflicts with the reported spatial pattern

Findings

Blocking

  • The current oneSampleMasks table and OneSampleDitherCases test encode one covered pixel per 2x2 quad at alpha 0.25 and a diagonal pair at 0.5. The posted RADV readback describes quad-granular coverage instead: alpha 0.25 writes the upper-left 2x2 block, and 0.5 writes the upper-left and lower-right blocks. The current table therefore conflicts with the available measurement, and the 4x4 test only confirms the implementation's own assumed pattern. The separate Vulkan validation run checks API validity, not this rendering behavior.

    Please either derive the pattern from a larger 16x16 or 32x32 readback (with 1/8 alpha steps and both FP16 and 32-bit exports) and update the table and tests, or keep the single-sample dithered path rejected until the behavior is measured. The current README/debt entry should continue to distinguish modeled behavior from PS5 silicon behavior.

Non-blocking

  • The new state path and serializer use the one-sample mode consistently, and the regression covers both FP16 and FP32 exports. Those checks do not resolve the pattern mismatch above.
  • The separate format-1 and four-sample findings were explicitly scoped outside this change; I am not asking to add those unrelated paths here.

Sources

Not verified

  • I did not run local CTest, a GPU readback, or a PS5 console comparison.
  • The current test and validation results are author-reported.

To get this to a good state

Post a larger measurement-derived table and matching regression, or remove the unsupported single-sample table and retain the explicit rejection; then request a re-review.

@ashwin280802
ashwin280802 force-pushed the fix/agc-single-sample-alpha-to-coverage branch from 05082c0 to 4bb5aa9 Compare October 29, 2026 15:11
@ashwin280802 ashwin280802 changed the title fix(agc): model single-sample dithered alpha-to-coverage fix(agc): reject unmeasured single-sample dithered alpha-to-coverage Oct 29, 2026
@ashwin280802

Copy link
Copy Markdown
Contributor Author

@adriadam10 @MarcZX10 Following your reviews, I have updated the branch to adopt Option B and rebased onto latest origin/main:

  1. Retained Explicit Rejection: Dropped the unmeasured oneSampleMasks table and shader recompiler changes. Single-sample targets with dithered alpha-to-coverage (DB_ALPHA_TO_MASK 0xe701 or 0x18701) continue to be explicitly rejected in DecodeState and DrawRejection (requires eight matched raster/DB_EQAA samples for 0x18701 and requires two matched raster/DB_EQAA samples for 0xe701).
  2. Added Rejection Verification Tests: In core/libs/prx/libSceAgcDriver/tests/Graphics.cpp, added test cases verifying that single-sample draws with 0xe701 and 0x18701 fail with those exact requirement strings in DecodeState and DrawRejection.
  3. Updated Technical Debt: In docs/dev/debt/libSceAgcDriver.md, updated the technical debt entry explaining that single-sample dithered alpha-to-coverage remains unsupported and rejected until the spatial repetition period and hardware behavior on PS5 silicon are measured across larger targets, documenting the preliminary quad-granular RADV observation.
  4. Validation & Conventions Gate: Ran the focused graphics test (ctest -R "^agc_driver_graphics$" -V) with ANYPS5_REQUIRE_VULKAN=1, ANYPS5_NO_SHADER_CACHE=1, and VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation on an NVIDIA GeForce RTX 5070 (driver 572.70, Windows 11). VK_LAYER_KHRONOS_validation loaded cleanly (no missing-layer warning); confirmed 0 VUID- validation errors introduced. tools/check_conventions.py --base origin/main passes with 0 violations.

@ashwin280802
ashwin280802 requested a review from MarcZX10 October 29, 2026 15:33

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-checked — new head, looks good

The current diff adds regression coverage for the existing rejection of unmeasured single-sample dithered alpha-to-coverage and documents the remaining measurement debt.

What I ran

Reviewed exact head 4bb5aa9f0e39858a6295c31cef199f8f164e25ea against base bb04a4f1ea0175e9e4f149f0359a764d62ff3296 (main). GitHub reports the PR mergeable with a clean merge state; I did not create a local merge preview. Environment: Debian 12, GCC 12.2.0, host Intel Xeon E5-1620 v2; the focused test binary ran under QEMU user mode with a Haswell CPU model.

cmake --build /dev/shm/AnyPS5-build-12915-full --target agc_driver_graphics_tests -j2
ANYPS5_NO_SHADER_CACHE=1 qemu-x86_64 -cpu Haswell /dev/shm/AnyPS5-build-12915-full/tests/agc_driver_graphics_tests --alpha-to-coverage-state
ANYPS5_REQUIRE_VULKAN=1 ANYPS5_NO_SHADER_CACHE=1 VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation VK_ICD_FILENAMES=/usr/share/vulkan/icd.d/lvp_icd.x86_64.json ctest --test-dir /dev/shm/AnyPS5-build-12915-full -R '^agc_driver_graphics$' -V
ANYPS5_REQUIRE_VULKAN=1 ANYPS5_NO_SHADER_CACHE=1 VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation VK_ICD_FILENAMES=/usr/share/vulkan/icd.d/lvp_icd.x86_64.json qemu-x86_64 -cpu Haswell /dev/shm/AnyPS5-build-12915-full/tests/agc_driver_graphics_tests
 git diff --check bb04a4f1ea0175e9e4f149f0359a764d62ff3296 HEAD

I ran the focused state command and build on both the PR head and base. The Vulkan CTest command above was run on the PR head; the full binary was also attempted under QEMU on both revisions. GitHub checks were read for the exact PR head, and the Windows validation result below is author-reported.

Results

Check Result
agc_driver_graphics_tests build Pass on base and PR head
Focused alpha-to-coverage state/refusal tests Pass on base and PR head under QEMU Haswell
git diff --check Pass
Full local CTest on PR head Aborted before the test due to host CPU features missing LZCNT, BMI1, BMI2, MOVBE, AVX2 and FMA
Full graphics binary under QEMU, base and PR Did not complete: QEMU user-mode memory access failed before the full suite could finish; this is an environment limitation, not evidence of a PR regression
Hosted exact-head checks Windows, conventions, labels, eligibility and report pass; Linux and specialized jobs skipped
Vulkan validation on RTX 5070 / Windows 11 Pass, author-reported: validation layer loaded and no new VUID- errors
Unedited command output

Focused alpha-to-coverage command, PR head:

[src/HostCpu.cpp:193 operator()] host CPU Intel Core Processor (Haswell): has SSE, SSE2, SSE3, SSSE3, SSE4.1, SSE4.2, POPCNT, LZCNT, BMI1, BMI2, MOVBE, CMPXCHG16B, LAHF/SAHF, AVX, AVX2, FMA, F16C, AES, PCLMULQDQ, RDRAND, XSAVEOPT, OS AVX state (XCR0); lacks RDSEED, ADX, SHA, CLFLUSHOPT, CLWB, RDPID, XSAVEC, SSE4A, PREFETCHW, CLZERO
Alpha-to-coverage state and refusal tests passed

Full CTest command, PR head:

961: [src/HostCpu.cpp:193 operator()] host CPU Intel(R) Xeon(R) CPU E5-1620 v2 @ 3.70GHz: has SSE, SSE2, SSE3, SSSE3, SSE4.1, SSE4.2, POPCNT, CMPXCHG16B, LAHF/SAHF, AVX, F16C, AES, PCLMULQDQ, RDRAND, XSAVEOPT, OS AVX state (XCR0); lacks LZCNT, BMI1, BMI2, MOVBE, AVX2, FMA, RDSEED, ADX, SHA, CLFLUSHOPT, CLWB, RDPID, XSAVEC, SSE4A, PREFETCHW, CLZERO
961: [libc] unhandled exception 'St13runtime_error' thrown from 0x7fc9fdba5e87
961: terminate called after throwing an instance of 'std::runtime_error'
961:   what():  The host CPU (Intel(R) Xeon(R) CPU E5-1620 v2 @ 3.70GHz) lacks LZCNT, BMI1, BMI2, MOVBE, AVX2, FMA. The PS5's Zen 2 CPU has these extensions and titles run their instructions natively, so this CPU cannot run them.
1/1 Test #961: agc_driver_graphics ..............Subprocess aborted***Exception:   0.01 sec

0% tests passed, 1 tests failed out of 1

Full graphics QEMU run, PR head:

Unused stencil reference sample equivalence passed: 22 accepted states, 12 dependent countermodels
Stencil logic operations matched AND, OR, XOR and XNOR under the write mask for 310 register states
All 256 MSAA CMASK byte classifications passed
AGC graphics: direct registered range must be covered by host import

Full graphics QEMU run, base:

Unused stencil reference sample equivalence passed: 22 accepted states, 12 dependent countermodels
Stencil logic operations matched AND, OR, XOR and XNOR under the write mask for 310 register states
AGC driver: guest memory is not readable at 0x7ff3a6ad0000 (range 0x7ff3a6ad0000+0x10000)

Hosted check output:

conventions\tpass\t18s\thttps://github.com/boykopovar/AnyPS5/actions/runs/40900013426/job/123539407094
labels\tpass\t8s\thttps://github.com/boykopovar/AnyPS5/actions/runs/40900013718/job/123541715960
windows\tpass\t4m35s\thttps://github.com/boykopovar/AnyPS5/actions/runs/40898998428/job/123541863430
linux\tskipping\t0\thttps://github.com/boykopovar/AnyPS5/actions/runs/40898998428/job/123541866210
register-context\tskipping\t0\thttps://github.com/boykopovar/AnyPS5/actions/runs/40898998428/job/123541866449
relinker\tskipping\t0\thttps://github.com/boykopovar/AnyPS5/actions/runs/40898998428/job/123541866549
relinker-asan\tskipping\t0\thttps://github.com/boykopovar/AnyPS5/actions/runs/40898998428/job/123541866202
write-watch\tskipping\t0\thttps://github.com/boykopovar/AnyPS5/actions/runs/40898998428/job/123541866177
eligibility\tpass\t8s\thttps://github.com/boykopovar/AnyPS5/actions/runs/40898998428/job/123536183958
report\tpass\t49s\thttps://github.com/boykopovar/AnyPS5/actions/runs/40898998394/job/123536185625

Negative control

Assertion Mutation to code under test Did the new test fail?
0xe701 and 0x18701 on a single-sample target are rejected by both DecodeState and DrawRejection No acceptance mutation was applied Not exercised; the rejection tests pass on the unmodified implementation

Findings

Blocking

None found in the current diff.

Non-blocking

The exact-head Windows Vulkan result is author-reported, and the local full graphics run could not complete under this host/QEMU setup. The PR keeps the unmeasured hardware behavior explicitly unsupported and recorded as debt.

Sources

Not verified

  • Physical PS5 alpha-to-coverage behavior; the PR intentionally leaves it unsupported pending measurement.
  • A complete local Vulkan CTest pass on either base or PR, or a local same-GPU VUID- comparison. The host CPU and QEMU user-mode memory limitations prevented that. The current-head Windows run is author-reported.

To get this to a good state

No blocking follow-up is needed for this change. Keep the debt entry until the spatial repetition and console behavior are measured; if the head changes, I will rerun the focused tests and relevant validation checks on the new head.

@blawesom

Copy link
Copy Markdown
Contributor

Static review plus an independent Linux/AMD run (CachyOS, kernel 7.2.10, GCC 16.2.1, Ninja 1.13.2, Release, BUILD_TESTING=ON, Ryzen 7 9800X3D, RX 9070 XT / RADV GFX1201). No findings.

What I checked

The diff is test-only (tests/Graphics.cpp) plus one debt note. The two rejection strings the new block asserts already exist unmoved in Graphics/src/State.cpp on main:

  • DB_ALPHA_TO_MASK 0xe701 requires two matched raster/DB_EQAA samples and one pixel iteration — State.cpp:1569 (Require) and State.cpp:2494 (DrawRejection).
  • dithered alpha-to-coverage requires eight matched raster/DB_EQAA samples and one pixel iteration — State.cpp:1565 (Require) and State.cpp:2493 (DrawRejection).

singleSample.context[0x2dc] = 0xe701 / 0x18701 is the only change per attempt; the substring check in the added Require(...find("requires two matched raster/DB_EQAA samples") != npos...) and ...find("requires eight matched raster/DB_EQAA samples") both match those existing prefixes exactly, so the assertions pin the strings DecodeState and DrawRejection actually return.

Focused test run (per repo AGENTS GC-change protocol)

VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation ANYPS5_REQUIRE_VULKAN=1 ANYPS5_NO_SHADER_CACHE=1 \
ctest --test-dir build -R 'alpha_to_coverage|dithered' -V
tree 886 undithered 890 a2c state 891 dithered8 892 dithered2 969 a2c runner
this head 4bb5aa9f0e Pass Pass Pass Pass Pass
main bb04a4f1ea Pass Pass Pass Pass Pass

Both logs: 0 occurrences of was not found but was requested by env var VK_INSTANCE_LAYERS (layer found and requested), 0 occurrences of VUID-. No new validation errors introduced by this change.

The validation layer binary is not part of the base image here; I unpacked Arch's vulkan-validation-layers 1.4.363.0 into an unprivileged path and pointed VK_LAYER_PATH at it — no system install either side.

Not covered

The debt note's Vulkan alphaToCoverageEnable 4×4 readback on an RX 6650 XT is an author-side silicon measurement, not something I can re-produce here. Console silicon behaviour (PS5 / Bugsnax PPSA01503) is explicitly marked unmeasured in the note itself and I have no way to close it — that trail of the note looks appropriately scoped as debt.

@adriadam10
adriadam10 merged commit 90c9759 into boykopovar:main Oct 30, 2026
11 of 14 checks passed
@adriadam10

Copy link
Copy Markdown
Collaborator

Checked that both rejection strings the new cases assert exist unchanged in State.cpp (DecodeState and DrawRejection), that the tests pin the 0xe701 and 0x18701 single-sample rejections, and that the debt entry records the unmeasured spatial period; merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agc libSceAgc* build docs shader core/shader unblocks title Names a title ID on the Unblocks line

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants