Skip to content

fix(shader): lower hulls with one input or one output control point - #5457

Merged
oneandonlydean merged 5 commits into
boykopovar:mainfrom
EagleBearerYS:fix/shader-hull-single-control-point-on-4222
Oct 14, 2026
Merged

oneandonlydean merged 5 commits into
boykopovar:mainfrom
EagleBearerYS:fix/shader-hull-single-control-point-on-4222

Conversation

@EagleBearerYS

@EagleBearerYS EagleBearerYS commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

What

Resident Evil Village (PPSA01556, content 01.210.000) tessellates quad patches with one input and one output control point. While it creates its PSO cache, its hull programs stop in tessellation lowering, first with

shader 0x207e47c500 stage 2: ShaderRecompiler::Recompile: tessellation lowering: the hull shader reads no LDS per control point, so the local stride is unknown at pc 0x58

and, once that is fixed, with

shader 0x207e897900 stage 2: ShaderRecompiler::Recompile: tessellation lowering: the hull shader writes off-chip memory outside its control points and patch constants at pc 0x8c

#5234 reports the first message for Resident Evil 2 (PPSA04288) too, after integer partitioning, with the same VGT_TF_PARAM 0x40042 (quad domain, integer). I haven't run that title.

This builds on Celegans12's #4201 and #4222; #4172, the first PR of that chain, has merged. Only the last two commits are this pull request:

  • One input control point (e25d561). derive() learned the LS-to-HS stride from the control-point term of hull LDS reads, or from the patch term of reads at a computed control-point index. A hull with one input control point reads it at constant offsets, so neither applies. Every hull read already has a patch term of exactly inputControlPoints * localStride, which the check after it requires, so the stride is now also taken from the patch term of reads without a control-point term.
  • One output control point (1243f06). Such a hull writes its control point at the patch record offset alone, so the write has no control-point term and was taken for a patch constant with no control stride. With one output control point, a write that fits in one record (its offset is at least 0 and its end within the patch term) is now the control point, and the control stride comes from its patch term. That is the rule fix(shader): record BDA faults in hull programs instead of refusing them #4222 applies to control-point reads in domain programs. Patch-constant records lie outside that range: below the top of the off-chip block, or above every control point. With one invocation per patch, per-patch outputs also need no control-point-0 guard in requireFirstInvocation.

Tested

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: fix(shader): record BDA faults in hull programs instead of refusing them #4222
  • Unblocks: PPSA01556 Resident Evil Village (PPSA01556) #5169
  • GPUs tested: NVIDIA GeForce RTX 3070 Laptop GPU, driver 617.42, Windows 11; lavapipe (Mesa llvmpipe) on Ubuntu 24.04 under WSL2
  • AI-assisted: yes

@tibzejoker

Copy link
Copy Markdown
Contributor

Intel run for this PR: Intel UHD Graphics 770 (Raptor Lake iGPU of an i9-13900K), driver 32.0.101.6129, Windows 11, MinGW-w64 GCC 15.2.0 Release, device picked with ANYPS5_GPU=Intel, at head 391f1b32b.

  • agc_driver_tessellation, agc_shader_memory and agc_driver_graphics pass.
  • ctest -R "agc_driver|agc_masked|agc_shader" -j4: no failures out of 377 (349 pass, 28 skipped). These are the same 28 skips as main has on this GPU (f64, 64-bit and float atomics, fp16 interpolation and dot, mesh, shader clock and the like, which it does not support), and no new ones.

AI-assisted (Claude Code).

A wave64 hull program narrows EXEC with s_and_saveexec_b64 and restores it
later, so its exec words reach the lane-bit test as ANDs with the host
ballot, as loop or join phis, and through separate ballots for the two
halves. Tessellation lowering refused them as "uses a lane mask other
than its own lane's bit".

The own-lane bit of a mask pair is now rebuilt from the words' operations
(ballots, bitwise ops, selects, phis, constants); both halves must have
the same shape and ballot the same predicate. Dead hull cycles such as a
saved-exec loop are removed before the remaining-ballot check, so a mask
whose other lanes are read (s_bcnt1_i32_b64 exec) is still refused. The
words are replaced through identities and block conditions are resolved,
so a branch on a compare copied to VCC keeps its condition. Local and
domain programs keep their exec words for the half-wave matcher.

Block conditions fix and HullCompareBranch test from EagleBearerYS.
… data

Hull programs whose relative patch index (v1 bits 7:0) reached anything
but a tessellation ring or LDS address threw, because the lowering runs
every Vulkan patch as relative patch 0. The quad terrain hull 0x80002fc700
in Ghost of Yotei is one, and so is the domain 0x8068fbd900 (v7).

Such hulls and domains now take (PrimitiveID mod GE_CNTL.PRIM_GRP_SIZE)
mod VGT_LS_HS_CONFIG.NUM_PATCHES, as if hull threadgroups took NUM_PATCHES
patches at a time out of each primitive group; both stages see the same
PrimitiveID for a patch. This holds only when every instance starts a new
threadgroup, so it needs GE_CNTL.BREAK_WAVE_AT_EOI (bit 18); with the bit
clear the instances share threadgroups and such programs still throw.
PRIM_GRP_SIZE and BREAK_WAVE_AT_EOI join the tessellation configuration
(draw keys, hull and domain cache keys, request version 24). A missing
GE_CNTL throws, and so do local programs, whose relative vertex index
depends on vertex reuse that Vulkan does not expose.
Every hull program with a BDA access was refused with "BDA fault
termination requires a barrier-safe tessellation-control execution
protocol". The refusal predates BdaInvocationsMayStop: a program with
s_barrier no longer stops a faulting invocation; the fault is recorded,
the read returns zero and every invocation reaches the barrier. In a
hull, s_barrier is the only control barrier the backend emits after the
guest code starts, so that rule already covers it, and a hull without
one can stop an invocation without stranding a barrier.

The tessellation test draws hulls that scale their levels by a BDA read,
with and without s_barrier, and requires an unmapped read to report a
tessellation-control fault (not on llvmpipe, which runs hull loads that
the unmapped-address check skips). The contract test now accepts a hull
with s_barrier.
@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 14, 2026
…eads one control point

derive() learned the LS-to-HS stride from the control-point term of hull LDS reads, or from the patch term of reads at a computed control-point index. A hull with one input control point reads it at constant offsets, so neither applies and the stride stays unknown ('the hull shader reads no LDS per control point, so the local stride is unknown'). Every hull read already has a patch term of exactly inputControlPoints * localStride, so take the stride from it for reads without a control-point term too. Resident Evil Village's quad patches have one input control point.
A hull with one output control point writes its control point at the
patch record offset alone, so the write has no control-point term and
was taken for a patch constant with no control stride ('the hull shader
writes off-chip memory outside its control points and patch constants').
Take the control stride from the patch term of such writes when the
write fits in one record (its offset is at least 0 and its end within
the patch term), the rule domain programs already apply to control-point
reads; patch-constant records lie outside that, below the top of the
off-chip block or above every control point. With one invocation per
patch, per-patch outputs also need no control-point-0 guard. Resident
Evil Village's quad patches have one output control point.
@EagleBearerYS
EagleBearerYS force-pushed the fix/shader-hull-single-control-point-on-4222 branch from 5589f92 to 1243f06 Compare October 14, 2026 03:45
@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 14, 2026
RINNECODER added a commit to RINNECODER/AnyPS5 that referenced this pull request Oct 14, 2026
* fix(agc): accept integer tessellation partitioning

Ported from upstream PR boykopovar#5234, adapted to our triangle/fractional-odd
only tessellation path: VGT_TF_PARAM.PARTITIONING 0 decodes, emits equal
spacing and bounds factors at 64. Metal still refuses guest tessellation
draws in MetalRenderState.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DgW4n3qYgi5jAAohEmXRQj

* fix(agc): accept fractional-even tessellation partitioning

Ported from upstream PR boykopovar#5402 (its integer-partitioning base commit is
the previous commit), adapted to our triangle-only tessellation path:
VGT_TF_PARAM.PARTITIONING 3 decodes, emits fractional-even spacing and
bounds factors at 64. Metal still refuses guest tessellation draws.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DgW4n3qYgi5jAAohEmXRQj

* fix(shader): place hull input components by their offset in the control-point slot

Ported from upstream PR boykopovar#4723 (shader hunk; its Vulkan execution test is
not ported).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DgW4n3qYgi5jAAohEmXRQj

* fix(shader): lower hulls with one input or one output control point (partial)

Ported from upstream PR boykopovar#5457, commits 1, 4 and 5 of 5 (hull exec masks
restored through phis, input stride from the patch term, one output
control point per patch). Vulkan execution-test hunks are not ported.

Skipped from boykopovar#5457, with reasons:
- relative patch index rebuilt as data: needs GE_CNTL.PRIM_GRP_SIZE and
  BREAK_WAVE_AT_EOI in TessellationConfiguration, VGT_LS_HS_CONFIG
  patchesPerGroup and the request key/serializer additions. None of
  these exist on main, which only lowers triangle fractional-odd hulls.
- recording BDA faults in hull programs: drops a refusal whose safety
  rests on upstream hull work we lack; the refusal stays.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DgW4n3qYgi5jAAohEmXRQj

* fix(shader): honour SPI_BARYC_CNTL.FRONT_FACE_ALL_BITS for the front-face input

Ported from upstream PR boykopovar#5314. Tracks SPI_BARYC_CNTL in the draw key and
shader request key, and bumps the request serialization to version 24
(shared with the following slice-offset port, which adds no second bump).

Not ported: the ShaderMemory serializer-version assertions of the PR,
which build on upstream test scaffolding absent from main.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DgW4n3qYgi5jAAohEmXRQj

* fix(shader): propagate color view slice to pixel ancillary input

Ported from upstream PR boykopovar#5500, adapted to main: the draw decode sets the
color view slice as layerBase (the layer-export decode this PR also
touches does not exist on main, so its diagnostic and layers hunks are
skipped). layerBase is added to the pixel info, request key, shader info,
disk cache and pipeline specialization constants. The request
serialization stays at version 24, bumped once by the front-face commit.
The shader disk cache format is bumped to 36.

Not ported: unrelated hunks the upstream commit carried in Recompiler.cpp
(dot-product capability option) and its ShaderMemory serializer tests.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DgW4n3qYgi5jAAohEmXRQj

* test(shader): cover tessellation partitioning, hull slot placement, layerBase and serializer v24

Fix the stale serialization version expectation in the shader memory test,
add pixel ancillary rows with a non-zero layerBase, extract PixelLayerBase
for a derivation test, and point the FrontFace note at TechnicalDebt.md.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DgW4n3qYgi5jAAohEmXRQj

---------

Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-authored-by: XairRrR <dev910382@gmail.com>
@xXPBSXx

xXPBSXx commented Oct 14, 2026

Copy link
Copy Markdown
Contributor

Title evidence for this PR: PPSA02530 (PRAGMATA v01.200.000) reaches the same first error as the Resident Evil Village one in the description.

Windows 10 (19045), MinGW-w64 GCC 15.2.0, Release, all libs/*.prx rebuilt from the tree. PPSA02530 converted with relinker --windows --registry --trace-imports, 90 s runs:

  • main e33258a64 + fix(relinker): load Windows executables at a fixed low base #4237 (needed so that the title gets past its start-up allocator, see PRAGMATA (PPSA02530 v01.200.000) on Windows: the title's libc rejects its allocator's mspace buffer in the ASLR-relocated image above 1 TiB, so the first malloc returns NULL and start-up faults #3043) + a local log stub that makes sceConvertKeycodeGetCharacter return an error (it throws on main; the stub is not a proposal): the run ends with

    what(): shader 0x2112056900 stage 2: ShaderRecompiler::Recompile: tessellation lowering: the hull shader reads no LDS per control point, so the local stride is unknown at pc 0x58
    

    (run_063225.err, line 115).

  • The same tree + this PR's head 1243f067b (merge into the tree above; git merge-tree of this head against main e33258a64 reports a content conflict in core/libs/prx/libSceAgcDriver/tests/Graphics.cpp, which I resolved with -X theirs for this run only, the test file does not affect the title): that message does not appear any more, the title continues, and the run ends later with

    what(): shader 0x2107d5e500 stage 5: ShaderRecompiler::Recompile: GetImageResource dword 0 is not a valid runtime value; chain: ReadConstBuffer@pc=0x00000568(GetBufferResource(...
    

    (run_063830.err, line 209; stage 5 is a different shader and a different error).

Not tested: this PR's own tests (conflict in the test file as above), Linux, a console; I did not check whether the second error is covered by another open pull request.

AI-assisted (Claude Code).

@EagleBearerYS

Copy link
Copy Markdown
Contributor Author

Thanks for running PRAGMATA on it. That's a second title past the same hull error.

Your next stop, GetImageResource dword 0 is not a valid runtime value; chain: ReadConstBuffer@...(GetBufferResource(..., looks like the case #5188 handles: an image descriptor read from a constant buffer at a uniform offset. If you have time, it would be worth trying #5188 on top as well.

@xXPBSXx

xXPBSXx commented Oct 14, 2026

Copy link
Copy Markdown
Contributor

Correction to my comment above: I named this PR's head as 1243f067b. The run was made on head 391f1b32b (the head that was fetched for the test); I added the current head to the text without having tested it. The result described (the hull-shader message no longer appears, the title continues to the GetImageResource dword 0 error at stage 5) is for 391f1b32b. I will re-run on the current head and report if the result differs. Also, the .prx and relinker file dates of those runs were overwritten by later builds, so I can only give the log times of the 126-library builds (trace_vrr5, trace_vrr6), not the file dates.

AI-assisted (Claude Code).

@xXPBSXx

xXPBSXx commented Oct 14, 2026

Copy link
Copy Markdown
Contributor

Tested on head 1243f067b (the current head of this pull request on GitHub, same SHA; the first comment of mine was for the earlier head 391f1b32b, and this run supersedes it).

PPSA02530 (PRAGMATA v01.200.000), Windows 10 (19045), MinGW-w64 GCC 15.2.0, Release, 90 s run (PPSA02530_rO/run_070451.err, log trace_vrr8_PPSA02530.txt).

Result: with this head the hull-shader stop tessellation lowering: the hull shader reads no LDS per control point ... does not appear (0 matches in the run log; it did appear on the tree without this pull request, run_063225.err line 115). The run goes on and ends later with

what(): shader 0x211273e200 stage 5: ShaderRecompiler::Recompile: GetImageResource dword 0 is not a valid runtime value; chain: ReadConstBuffer@pc=0x00000568(GetBufferResource(...

(line 209; a different shader and a different error). I did not check whether another open pull request covers that error.

Test conditions

  • Tree: main 75690a117 + fix(relinker): load Windows executables at a fixed low base #4237 head bf6b6c0a1 + this head 1243f067b (merge into the tree, -X theirs because core/libs/prx/libSceAgcDriver/tests/Graphics.cpp conflicts with main; the test file does not affect the title run). Merged tree HEAD becd064af. A local log stub makes sceConvertKeycodeGetCharacter return an error (it throws on main); not a proposal.
  • Libraries: ninja -C build -j4 relinker plus all 126 core/libs/libs/*.prx targets from that tree, finished at 07:03:36 local time: 128 .prx files in core/libs/libs, oldest 2026-10-13 05:53:26 (a library without changes in this tree, not rebuilt), newest 2026-10-14 07:03:36; libSceAgcDriver.prx 07:03:35, libkernel.prx 06:47:15 (ninja found it up to date, so it was not rebuilt). relinker.exe 2026-10-14 06:55:28 (ninja found it up to date for this tree).
  • Relink: relink_title_trace.ps1 -Id PPSA02530 -ElfDir I:\AnyPS5\Games\PPSA02530_elf -Suffix O -Repo K:\AnyPS5\wt-sockets = relinker.exe --windows --registry --trace-imports in\eboot.elf app.exe; no -ModuleDir, no --windows-fixed-base. Run: run_title.ps1 -Seconds 90, APS5_TRACE_CALLS=all.

Not tested: this pull request's own tests (conflict in the test file), Linux, a console.

AI-assisted (Claude Code).

@Celegans12

Copy link
Copy Markdown
Contributor

I ran this against Ghost of Yōtei's hull and domain programs to check it doesn't change anything for a title that uses 3 and 4 control points, and it doesn't: every request gives the same result and byte-identical SPIR-V with and without your two commits. Yōtei's terrain hull (0x80002fc700, quads, 4 in / 4 out) still lowers to the same 386,580-word module, and the 14 triangle hull requests (3 in / 3 out) produce the same modules too. None of them has a single control point, so the new paths in derive() never change the outcome. The constant-offset stride rule isn't limited to one input control point, so it does run on these hulls, but there it just agrees with the stride the other reads already give.

agc_driver_tessellation (with your new RequireSingleControlPointHulls), agc_shader_memory and agc_driver_graphics pass on Windows with an RTX 3080 Laptop GPU.

One heads-up: after #5234 landed, I rebased the chain onto main f265cd5. The new heads are #4172 866ed445b9, #4201 2eda9f893c and #4222 c1e4a0ba8b, and they're pushed now. Your two commits cherry-pick cleanly onto c1e4a0ba8b with an identical diff, and the results above are the same there. So you may want to rebase onto them. I'm happy to re-run this on the rebased branch.

Details

Heads tested (Windows 11, WinLibs GCC 15.2.0, Release, NVIDIA driver 610.62):

build commit
#4222 head (base of this PR) f33872bec3
this PR 5589f924ef
#4222 rebased onto main f265cd5 c1e4a0ba8b
this PR cherry-picked onto it c1e4a0ba8b + f8c06b748b + 5589f924ef

ctest -R '^(agc_driver_tessellation|agc_shader_memory|agc_driver_graphics)$' with ANYPS5_REQUIRE_VULKAN=1: 3/3 pass on both PR builds (nothing skipped).

I replayed the requests with agc_shader_replay --spv and checked each module with spirv-val --target-env vulkan1.3. There were 35 captured requests: the terrain hull (three captures of the same program), 14 triangle hulls, 10 domain programs and 8 LS programs. Same results on all four builds:

requests result SPIR-V words spirv-val
terrain hull 0x80002fc700 (4/4, quad) compiles 386,580 ok
14 triangle hulls (3/3) compile 22,739 / 22,857 / 23,281 ok
2 LS programs (quad) compile 13,705 ok
10 domain + 6 LS programs stop in ResourceTracker (GetImageResource dword 0 is not a valid runtime value) after tessellation lowering n/a n/a

The domain programs (including the terrain domain shader_c7_domain) get past tessellation lowering and then stop in resource tracking, because the old captures lack some SRT memory. For those 10, the APS5_DUMP_IR=all dump after the srt pass is byte-identical with and without the PR.

The captures predate request version 24: they're v16, v18 and v19 written by our game branch, whose local fields differ from upstream's. So I re-serialized them to upstream v24 with a small script. It parses each capture to the last byte, keeps every captured field and adds defaults for the fields upstream gained since. The terrain hull uses its relative patch index as data. Older captures don't record GE_CNTL.PRIM_GRP_SIZE, so I set it to the patches per group (63, which matches the newer capture). I also set BREAK_WAVE_AT_EOI. Without those it stops at #4201's checks, identically on all builds. Every build got the same inputs.

@EagleBearerYS

Copy link
Copy Markdown
Contributor Author

Thanks for running it against Yōtei's hulls. Good to know the 3- and 4-point hulls come out byte-identical.

#5457 is already rebased onto c1e4a0b (head 1243f06), with the same two commits as your cherry-pick, so your results there apply to the PR as it is.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agc libSceAgc* docs shader core/shader

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants