[frankea]

Replaces #9804, which the open PR limit check closed after its merged-PR search returned 0 for one run (I have 2 PRs merged into main and 10 others open).

What

The tessellation lowering evaluates every ring and LDS address as vertex * control point + relative * relative patch + constant. A v_and_b32 with a constant was accepted only when the mask kept the low bits, like 0xff or 0x1fff. A hull program's v1 packs the control point into bits 8-12, above the relative patch in bits 0-7. A mask like 0x1f00, which keeps only the control point, was refused with the tessellation ring or LDS address is not affine in the control point and relative patch.

• A mask that keeps one contiguous run of bits, from bit lo up to bit hi, is now evaluated as (x mod 2^hi) - (x mod 2^lo).
  • Both remainders come from the existing split. It proves each one exact: the part below the boundary must stay inside the boundary for every control point and relative patch.
  • So v1 & 0x1f00 is control point * 256, as long as the relative patch stays below 256 and the control point below 32.
• A mask that cuts through a term is still refused. For example, 0x1e00 drops the control point's low bit, and 0x1f0f has a gap.

Why: Gran Turismo 7 (PPSA01317) has nine hull programs that address their control-point records in the off-chip ring this way: 0x1280676200, 0x1280ada900, 0x1280c98400, 0x1280d71100, 0x1280ed4e00, 0x1280ed8d00, 0x1280f63d00, 0x12810bce00 and 0x12810d5900. Their control-point records are 256 bytes.

• The address is (v1 & 0x1f00) + relative patch * patch stride. The patch stride is << 10 (4 control points) in eight of them and * 0x300 (3 control points) in 0x1280ed8d00.
• In 0x1280676200, for example: v_and_b32 v0, 0x1f00, v1, then v_lshl_add_u32 v0, v46, 10, v0 with v46 = v1 & 0xff, then buffer_store_dwordx4 ... offen offset:0x68 at pc 0x2d0.
• GT7 creates their pipelines on the way past fix(shader): read other control points' off-chip outputs in hull programs #9743's stop, and the driver deferred all nine with this refusal.

Tested

• Refreshed onto main 0db47b2 (head 2945108; no conflicts), RTX 4090 (NVIDIA 616.92, Windows 11): the 52 ctest entries of the focused tessellation set below pass 52/52, each test run three times, and 52/52 under the validation layer (VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation, ANYPS5_NO_SHADER_CACHE=1, ANYPS5_REQUIRE_VULKAN=1, ctest -V), with no VUID- line; the layer reports nothing on this head. main 0db47b2 passes the union of the refreshed sets, 115/115, under the same layer settings with no VUID- line.
  Gran Turismo 7, on a local stack of main with the open PRs it needs before these hulls (fix(agc): write a replace blend into a packed color target without blending #7243, feat(agc): render layered draws into a depth view of several slices #7256, fix(shader): answer image_get_resinfo on any T# dimension #7550, fix(agc): raise no end-of-pipe event from flips and carry the release's interrupt context #7978, feat(agc): merge partial writes into 4_4_4_4 UNORM color targets #8564, feat(agc): blend into 11_11_10 float color targets in the pixel shader #9020, feat(agc): draw alpha-tested packed-target merges that write depth as a depth pass and an EQUAL merge pass #9335, fix(shader): link table callees that together exceed s_call_b64 range #9068, feat(agc): draw EQAA color targets at their coverage sample count #9700, fix(agc): accept DB_EQAA INTERPOLATE_COMP_Z for a one-sample depth target without MSAA_ENABLE #9721, fix(shader): read other control points' off-chip outputs in hull programs #9743) and this commit:

• The nine hull programs, replayed with that stack's agc_shader_replay from the requests the driver dumped when it deferred them (APS5_DUMP_SHADERS).

  • Without this commit, each one is refused with the message above at its first ring store (pc 0x2c8 to 0x340), whose address masks v1 with 0x1f00.
  • With it, --prepare succeeds for all nine, and the recompiled SPIR-V passes spirv-val --target-env vulkan1.2.
  • They also read control point records back, which fix(shader): read other control points' off-chip outputs in hull programs #9743 handles. In each of them the barrier fix(shader): read other control points' off-chip outputs in hull programs #9743 inserts comes after every control-point output store and before every read-back load.

• GT7 runs with scripted pad presses, with its diagnostic switches that skip refused compute dispatches:

  • With fix(shader): read other control points' off-chip outputs in hull programs #9743 alone (one run), the World Map load gets past fix(shader): read other control points' off-chip outputs in hull programs #9743's stop. On the way, GT7 creates these nine pipelines, and the driver defers each one with this refusal.
  • With this commit as well (two runs), one run takes the same path: it logs the same 29 skipped compute dispatches, and no hull program is refused or deferred. It stops on the same next refusal, a texture sampled with unnormalized coordinates (guest texture sampled with unnormalized coordinates is not a single-layer 1D or 2D view ...).
  • The other run stops earlier, on the one-sample depth rule's other conditions (... PS_ITER_SAMPLE, a sample index or coverage input, a sample-mask export ...), before GT7 creates any tessellation pipeline.

Windows 11 (build 26200), NVIDIA GeForce RTX 4090 (driver 616.92), MinGW GCC Release build with BUILD_TESTING=ON, all on one machine.

New tests:

• agc_tessellation_lowering has three new IR cases.
  • Lowered: a hull LDS read at ((packed & 0x1f00) >> 3) + relative patch * stride, where packed is control point << 8 | relative patch as in v1, reads the invocation's own input control point.
  • Refused: packed & 0x1e00, which drops the control point's low bit, and packed & 0x1f0f, which has a gap.
• agc_driver_tessellation (GPU) adds HullMaskedControlPoint. It is HullCode with its off-chip record address 16 * control point replaced by (v1 & 0x1f00) >> 4. The domain program takes its positions from those records, so a wrong address moves or drops the patches. The test draws it and checks each patch like HullCode.

Runs:

• Focused set: the 52 ctest entries of 8 targets: agc_tessellation_lowering_tests, agc_tessellation_factor_ring_tests, agc_driver_tessellation_tests, agc_driver_graphics_tests, agc_vector_window_tests, agc_vertex_input_validation_tests, agc_driver_bda_device_tests and agc_shader_memory_tests.
• This PR (c46b559, which merges main 7e69be6): 52/52 passed, each test run three times (ctest --repeat until-fail:3).
• Validation layer: VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation (Vulkan SDK 1.4.363.0), ANYPS5_NO_SHADER_CACHE=1 and ANYPS5_REQUIRE_VULKAN=1 with ctest -V, on main 83462cc (7e69be6 plus eight commits to tools/ only) (52/52) and on this PR (52/52). Neither log has was not found but was requested by env var VK_INSTANCE_LAYERS, and neither has a VUID- line.
• Before the main merge (d39285d on main 8b3a6ba): 52/52 three times, and 52/52 under the layer with no VUID- line. main control there: main 8b3a6ba with only this PR's two test files. Both tests fail. The new lowered case is refused with not affine in the control point and relative patch, and the GPU test stops on HullMaskedControlPoint's store at pc 0x60 with the same refusal. The two refusal cases pass on both.
• Linux (lavapipe), by @Dextroid17 on this PR merged with main 9e568a3: the focused set passes 53/53, plain and under the layer, with no VUID- line. The main control (main's lowering with this PR's test files) fails the same two tests as on Windows, the GPU test at pc 0x60, also under the layer with no VUID- line: agc_driver_tessellation_tests creates its Vulkan instance before the refusal, so the comparison is 0 against 0.
• tools/check_conventions.py --base origin/main reports no findings.

Checklist

• Based on current main; no other open PR implements the same functions. No open PR changes the address evaluator's mask.
• One topic per PR; follow-ups go in a new PR
• No comments in code except technical debt
• Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
• No notes, investigation .md files or images added to the repository (attach them to this PR)
• New third-party code is a submodule built from source
• Generic behaviour, not specific to one title
• Depends on: none. Merge order: this PR goes after fix(shader): read other control points' off-chip outputs in hull programs #9743. Both add a helper before Hull::merge and cases at the end of tests/TessellationLowering.cpp; once fix(shader): read other control points' off-chip outputs in hull programs #9743 lands, this PR merges main and keeps both.
• Fixes: none
• Unblocks: PPSA01317 Gran Turismo 7 (PPSA01317) #4697 (with fix(shader): read other control points' off-chip outputs in hull programs #9743, GT7 creates these nine hull pipelines; its next stop is the texture refusal under Tested)
• GPUs tested: NVIDIA GeForce RTX 4090, driver 616.92, Windows 11 build 26200
• AI-assisted: yes (Claude Code)

[MarcZX10]

The masked-field lowering is narrowly constrained to a contiguous bit run, and the two refusal tests cover masks that drop a control-point bit or contain a gap. The report gives useful GT7 shader replays, output tests, negative controls, and exact-head conventions/diff checks passed locally here.

Two repository-readiness items remain:

1. The PR itself identifies open #9743 as touching the same TessellationLowering.cpp and TessellationLowering.cpp test file, but says Depends on: none and does not record an agreed merge order. Please coordinate with #9743’s author and record which branch goes first (or confirm the explicit no-dependency resolution while preserving both changes), as the overlap rule requires.
2. For the shader recompiler change, CONTRIBUTING requires the same focused Vulkan tests on main and the change with the validation layer enabled, and a comparison showing no new VUID- errors. The description reports the validation-layer run on this PR and a main negative-control run without saying that the main control used the layer or reporting its VUID comparison. Please add that baseline result, or explain if the control exits before Vulkan is initialized.

I did not run the C++/Vulkan suite locally; there is no configured build tree in this checkout. Hosted product tests are skipped. The remaining review requests are limited to overlap coordination and the required baseline validation evidence.

[MarcZX10]

Small wording correction to my review: the overlapping files are core/shader/recompiler/Optimization/src/TessellationLowering.cpp and core/libs/prx/libSceAgcDriver/tests/TessellationLowering.cpp (the lowering implementation and its test file). The requested coordination and validation items are unchanged.

[github-actions]

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request	Shared files not compared
PR 7642	core/libs/prx/libSceAgcDriver/tests/execution/Tessellation.cpp
PR 9508	core/libs/prx/libSceAgcDriver/tests/execution/Tessellation.cpp
PR 10193	core/libs/prx/libSceAgcDriver/tests/execution/Tessellation.cpp
PR 10249	core/shader/recompiler/Optimization/src/TessellationLowering.cpp

[Dextroid17]

Linux run, since the body says "Not run on Linux": llvmpipe (LLVM 23.1.1, 256 bits) via lavapipe, Mesa 26.2.4, GALLIUM_OVERRIDE_CPU_CAPS=avx, i9-11900K, CachyOS, GCC 16.2.1, CMake 4.4.3, Release, ANYPS5_REQUIRE_VULKAN=1, ANYPS5_NO_SHADER_CACHE=1, -j1; d39285d7c merged with main 9e568a389 (your base 8b3a6ba88 is 216 commits behind it; local merge 0aebc2265, the 3 files of the diff only). The merge is not clean: tests/TessellationLowering.cpp conflicts in two hunks because e4de56f3c (the domain clamp, on main since today) also adds a helper before run (EmittedEvaluationControlPoint, next to your OwnControlPoint) and appends cases at the end of the file, the spot your checklist already names for #9743; I kept both blocks in both hunks, main's first.

Head. The three tessellation tests (agc_tessellation_lowering, agc_tessellation_factor_ring, agc_driver_tessellation) pass 6 of 6 runs (3 plain, 3 under VK_LAYER_KHRONOS_validation 1.4.363, the layer inserted at both of agc_driver_tessellation_tests' instance creations per run, the other two tests load no Vulkan, 0 VUID-). Your focused set as the ctest entries those 8 targets register here (53, not 52: agc_driver_graphics_maps_fallback is Linux-only) passes 53 of 53 plain and 53 of 53 under the layer, 0 VUID-. agc_driver_tessellation on this device skips the two BDA subtests ("llvmpipe runs tessellation-control loads that the check of the unmapped address skips") and runs HullMaskedControlPoint: nothing in the diff prints on success, so the evidence that it ran is the negative control.

Negative control. core/shader/recompiler/Optimization/src/TessellationLowering.cpp from 9e568a389, your two test files kept, rebuilt: agc_tessellation_lowering fails 6 of 6 with only the lowered case refused (refused: tessellation lowering: the tessellation ring or LDS address is not affine in the control point and relative patch at pc 0x40, then failed: a hull read at its control point masked out of the packed control point and relative patch reads its own input control point); the two refusal cases (0x1e00, 0x1f0f) pass on both arms. agc_driver_tessellation fails 6 of 6 at HullMaskedControlPoint with ShaderRecompiler::Recompile: tessellation lowering: the tessellation ring or LDS address is not affine in the control point and relative patch at pc 0x60, the pc your control names; that abort comes before the BDA subtests, so the control arm never reaches them. Both match your Windows control; agc_tessellation_factor_ring and the other 50 entries still pass. 0 VUID- on either arm. On @MarcZX10's item 2: the control does not exit before Vulkan is initialized. In each of its 3 layer runs agc_driver_tessellation_tests inserts the layer once and selects the device before the refusal at pc 0x60 (one instance creation on the control, two on head), so the VUID- comparison is 0 against 0.

Overlap: git merge-tree --write-tree of 9c033796c (#9743) and d39285d7c also conflicts in tests/TessellationLowering.cpp, in both spots your checklist names: the helper before Hull::merge and the cases at the end of the file.

AI-assisted: yes

[frankea]

@Dextroid17 thank you for the Linux run, and for resolving the e4de56f3c conflict the way I did (main's blocks first). The branch now merges main 7e69be6 (c46b559, conflict resolutions only).

@MarcZX10 both items:

1. Order: this PR goes after fix(shader): read other control points' off-chip outputs in hull programs #9743; both are mine. Once fix(shader): read other control points' off-chip outputs in hull programs #9743 lands, this branch merges main and keeps both helpers and both test blocks. The checklist says so.
2. Validation baseline: on the RTX 4090, main 83462cc passes the focused set under the layer (52/52), and this head passes 52/52, with no was not found but was requested by env var VK_INSTANCE_LAYERS and no VUID- line in either log. Dextroid17's run above also covers the control: it creates its Vulkan instance before the refusal at pc 0x60, and logs 0 VUID- lines, as does the head.

AI-assisted: yes (Claude Code)

[jotitadev]

First run on Apple silicon.

Setup: Apple M2 Max (32 GB), macOS 27.2, x86_64 build under Rosetta, Release, BUILD_TESTING=ON. Method: #8658 (feat/macos-port) @ dc0c8db500 plus this PR's diff (head c46b559897), not main, since main does not build on macOS yet. Two drivers: Vulkan SDK 1.4.363.0 MoltenVK 1.4.2, and MoltenVK main with SPIRV-Cross #2704. ANYPS5_REQUIRE_VULKAN=1 ANYPS5_NO_SHADER_CACHE=1, ctest -R tessellation -j1 --timeout 30 -V.

	SDK MoltenVK 1.4.2	MoltenVK main + #2704
test files only (red)	agc_tessellation_lowering fails (the new masked case is refused as not affine); agc_driver_tessellation stops at the new draw with the tessellation ring or LDS address is not affine in the control point and relative patch at pc 0x60	same
this PR (green)	agc_tessellation_lowering and agc_tessellation_factor_ring pass; agc_driver_tessellation passes the new HullMaskedControlPoint draw	same

agc_driver_tessellation still fails on Apple with this PR, later in main(), at merged hull program, unculled: the level-1 patch was not drawn as one flat triangle, centre (16, 24, 40, 255). That case fails the same way on the #8658 base without this PR, so it is not from this change.

GPUs tested: Apple M2 Max (MoltenVK 1.4.2 from Vulkan SDK 1.4.363.0, and MoltenVK main + SPIRV-Cross #2704), macOS 27.2.

Written with AI assistance (Claude Code). I ran the builds and tests above on my machine.

[MarcZX10]

The latest description resolves the two issues in my prior review: it records that #9806 follows #9743 and reports the same 52-test validation-layer set on main and the change, with no missing-layer warning or VUID- lines; the Linux lavapipe comparison also reports 0 new VUIDs. The reported Windows and Linux GPU test coverage is substantial, and the local diff check, conventions, and test-registration checks pass. I could not independently run the Vulkan tests in this host environment.

One repository-readiness blocker remains: tools/check_main_sync.py --base upstream/main --head <head> reports this branch 2 commits ahead / 194 behind current upstream/main (merge base 7e69be679), though it merges cleanly. CONTRIBUTING requires branches to be based on and kept up to date with main. Please merge current main, preserve the declared ordering with #9743, then rerun the focused tessellation suite and validation-layer comparison on the combined head. This is a freshness request only, not an intake or queue-size objection.

[frankea]

@MarcZX10 this branch now merges main 0db47b2 (head 2945108, no conflicts), and the tessellation set and the paired validation-layer comparison are rerun on it: the first bullet under Tested. The order with #9743 is unchanged: this PR goes after it.

AI-assisted: yes (Claude Code)

[wachi35676]

AMD run, on Windows next to NVIDIA, on current main. Windows 11 10.0.26300, WinLibs GCC 15.2.0, Release; main aeff26b and this PR (2945108) merged onto it (clean); ANYPS5_REQUIRE_VULKAN=1 ANYPS5_NO_SHADER_CACHE=1 (no validation layer installed here); NVIDIA RTX 5070 Ti (621.01) and AMD Radeon RDNA2 iGPU (AMD proprietary 2026.04.17).

• The tessellation set (tessell|hull|domain|patch_|control_point, 19 entries): 19/19 on both GPUs.
• Control: with main's TessellationLowering.cpp, agc_tessellation_lowering (not affine in the control point and relative patch at pc 0x40) and agc_driver_tessellation fail on both.
• A wider set of 248 agc_ entries has the same failures with and without the change, none in tessellation.
• Merged with fix(shader): read other control points' off-chip outputs in hull programs #9743, the only conflict is in tests/TessellationLowering.cpp, as your merge-order note says.

AI-assisted: yes (Claude Code); results above were run on this machine.

Dismissing a stale review: the requested changes were addressed after it (new head) and the merge queue blocks on it. Please request changes again if anything is still open.

[adriadam10]

Rebased onto current main (#9743 is merged, so the order is met): the only conflict was textual in tests/TessellationLowering.cpp, kept both sides' helpers and cases; the diff is still +52 -1. Checked AddressEvaluator::mask: for a contiguous run of bits 0x1f00 over control point << 8 | relative patch it returns control point * 256, with both splits proving the parts exact. 0x1e00 (cuts the control point) and 0x1f0f (gap) are still refused, and scalar and low-mask paths are unchanged. The three new lowering cases and HullMaskedControlPoint fail on main without the change (author's control, repeated by @Dextroid17 and @wachi35676); the branch builds and passes ctest after the rebase; merged.