Skip to content

fix(shader): keep clip distances in the blocks between the barycentric tessellation stages - #7487

Merged
oneandonlydean merged 3 commits into
boykopovar:mainfrom
DavidSamir:fix/shader-barycentric-tessellation-clip-moltenvk
Oct 18, 2026
Merged

oneandonlydean merged 3 commits into
boykopovar:mainfrom
DavidSamir:fix/shader-barycentric-tessellation-clip-moltenvk

Conversation

@DavidSamir

Copy link
Copy Markdown
Contributor

What

#7330 fixed #7328: our generated tessellation stages declared ClipDistance inside their gl_PerVertex input block, while the vertex shader writes it as a separate built-in array, so the validation layer reported VUID-RuntimeSpirv-OpVariable-08746. Its fix passes ClipDistance as separate built-in arrays through both generated stages.

That broke the route these stages exist for: Apple GPUs under MoltenVK, which have no geometry shaders. SPIRV-Cross, which MoltenVK uses to translate SPIR-V to Metal, cannot handle a separate ClipDistance array in a tessellation evaluation stage:

program_source:134:39: error: no member named 'gl_ClipDistance' in 'main0_out'
    spvArrayCopyFromDeviceToStack(out.gl_ClipDistance, gl_in[_50].elements);
[mvk-error] VK_ERROR_INITIALIZATION_FAILED: Tessellation evaluation shader function could not be compiled into pipeline.

As a result, every pipeline on the barycentric tessellation route fails with vkCreateGraphicsPipelines: Vulkan result -3 on macOS.

This PR keeps #7330's fix where it applies and goes back to the block where it doesn't. VUID-08746 concerns the vertex-to-control interface, so:

Without vertex clip distances, the stages are unchanged. The unit test now checks both: the separate array on the control stage's input, the block member on its output, and only the block member on the evaluation stage. The shader.md entry explains why.

Tested

Apple M2, macOS 26.7.2, MoltenVK 1.4.2 (no geometryShader), x86-64 under Rosetta. The test stack is current main (e5c2b3c) plus mugurc's macOS port branches (ci/macos-series-2), with our open macOS PRs including #7426, and with #6535 so the titles below get past the depth-range check first. Release build, ANYPS5_NO_SHADER_CACHE=1. The changed files also compile on this Mac against plain main, and the conventions check passes.

  • agc_driver_pixel_barycentric_emulation: fails on main at the first tessellation pipeline (MoltenVK's errors above). With this PR it passes the tessellation route: 932 perspective and 932 linear I/J pixels, raw vertex values, host interpolation, face culling, and flat inputs in first and last provoking vertex mode.
  • Validation layer (VK_LAYER_KHRONOS_validation 1.4.363) with this PR: no VUID-RuntimeSpirv-OpVariable-08746. The only message is the unrelated VUID-VkImageCreateInfo-usage-00968, which main also reports. So Barycentric tessellation helper mismatches vertex BuiltIn blocks (VUID 08746) #7328 stays fixed.
  • agc_driver_graphics_tests: passes, including the updated clip-interface checks.

Titles (homebrew catalog, checksum-checked, relinked for macOS, 45 s each, same build with and without this PR):

Title main This PR
EVO Player (PPSA99039) aborts at 37 s, vkCreateGraphicsPipelines: Vulkan result -3 (evaluation stage) runs 45 s
Nuvio PS5 (PPSA99997) aborts at 15 s, same error runs 45 s
Jelly5 (PPSA99505) aborts at 14 s, same error runs 45 s
Twispero (PPSA99474) aborts at 6 s, same error runs 45 s
CD Player (PPSA98427) aborts at 4 s, same error runs 45 s
ProsperoStore (PPSA99000) aborts at 16 s, same error runs 45 s

Without #6535, these titles stop earlier on current main at viewport depth outside [0, 1] requires VK_EXT_depth_range_unrestricted. CD Player and ProsperoStore also need #7426 (last provoking vertex), which is in the stack.

Device limits of the M2 that stop titles here, not tested past them: no geometryShader, shaderFloat64, VK_EXT_provoking_vertex or VK_EXT_depth_range_unrestricted.

Not tested: Windows and Linux. There, devices with geometry shaders don't use this route; the validation result above covers the interface rule #7328 is about.

@MarcZX100 @Amoenus FYI, since this adjusts #7330 / #7328.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none (the titles also need fix(agc): normalize unused viewport depth ranges #6535 to reach this draw on current main)
  • Fixes: none
  • Unblocks: PPSA99039, PPSA99997, PPSA99505, PPSA99474, PPSA98427, PPSA99000 on macOS (macOS support #1)
  • GPUs tested: Apple M2, MoltenVK 1.4.2, macOS 26.7.2 (x86-64 under Rosetta)
  • AI-assisted: yes

…c tessellation stages

boykopovar#7330 passes the vertex clip distances as separate ClipDistance arrays so
the control stage's input matches the vertex shader's output. SPIRV-Cross,
which MoltenVK uses, cannot translate a separate ClipDistance array in a
tessellation evaluation stage, so on MoltenVK every pipeline of the
barycentric tessellation route failed to compile. The control stage now
reads the separate array the vertex shader writes and passes it on inside
its per-vertex block, which the evaluation stage reads and writes.
@DavidSamir

Copy link
Copy Markdown
Contributor Author

Overlap note: this PR and #7426 (last provoking vertex through the same stages) edit adjacent lines of BarycentricGeometryShader.cpp and the same shader.md entry, so whichever merges second needs a small merge. The combination, with #7426's flat-value line next to this PR's clip line, is what ran the titles above. I'll push that merge as soon as the first one lands.

AI-assisted: yes

@github-actions

github-actions Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 1395 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 4040 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 4650 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 5021 docs/dev/debt/shader.md
PR 5371 docs/dev/debt/shader.md
PR 5426 docs/dev/debt/shader.md
PR 5441 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 5499 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 5500 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
docs/dev/debt/shader.md
PR 5925 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 5941 docs/dev/debt/shader.md
PR 5958 docs/dev/debt/shader.md
PR 5999 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 6009 docs/dev/debt/shader.md
PR 6054 docs/dev/debt/shader.md
PR 6094 docs/dev/debt/shader.md
PR 6188 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 6194 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 6222 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
docs/dev/debt/shader.md
PR 6235 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
docs/dev/debt/shader.md
PR 6297 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 6320 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 6412 docs/dev/debt/shader.md
PR 6472 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 6481 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 6488 docs/dev/debt/shader.md
PR 6529 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 6535 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 6548 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 6563 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 6674 docs/dev/debt/shader.md
PR 6783 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 6808 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 6972 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
PR 6977 docs/dev/debt/shader.md
PR 7223 docs/dev/debt/shader.md
PR 7332 core/libs/prx/libSceAgcDriver/tests/Graphics.cpp
docs/dev/debt/shader.md
PR 7395 docs/dev/debt/shader.md

@github-actions github-actions Bot added shader core/shader docs agc libSceAgc* fix error Closes an issue (Fixes #N) labels Oct 18, 2026
@oneandonlydean oneandonlydean self-assigned this Oct 18, 2026
@oneandonlydean

Copy link
Copy Markdown
Collaborator

The change looks right to me: separate array only at VS→TCS, block from the TCS output on. One gap: agc_driver_pixel_barycentric_emulation draws without clip distances, so its validation-layer run covers the unchanged unclipped path, and nothing creates a pipeline for the clipped tessellation route under the layer. Could the emulation test add a case whose vertex shader writes clip distances (e.g. one plane that cuts the triangle, checked in the readback), run through both routes? That would pin the MoltenVK translation and the VUID-08746 rule on the exact layout this PR changes, on lavapipe in CI as well.

@oneandonlydean
oneandonlydean merged commit 721f29c into boykopovar:main Oct 18, 2026
10 checks passed
ycarogabriel-lgtm added a commit to ycarogabriel-lgtm/AnyPS5 that referenced this pull request Oct 18, 2026
Keeps the macOS port current with 50 upstream commits.

- Upstream's boykopovar#7487 keeps the barycentric tessellation stages' clip
  distances in the blocks between the stages, which MoltenVK's
  SPIRV-Cross translates; it replaces the fork's MoltenVK-only switch
  (clipDistancesInPerVertexBlock), so the switch, its device query and
  its debt note are removed and every driver takes upstream's stages.
- The provoking vertex debt entry takes upstream's geometry program and
  strip facts and keeps the fork's rule for devices without
  VK_EXT_provoking_vertex (boykopovar#6741).
- The f16 interpolation debt entries keep the fork's 64-bit integer
  text and take upstream's clip distance description.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ycarogabriel-lgtm added a commit to ycarogabriel-lgtm/AnyPS5 that referenced this pull request Oct 18, 2026
Brings the pull request up to the upstream commits that made it
conflict, with the resolutions already tested on the fork's main:

- upstream's boykopovar#7487 keeps the barycentric tessellation clip distances in
  the blocks between the stages, which MoltenVK translates;
- SpirvTarget keeps fragmentBarycentricBuiltins beside upstream's
  robustBufferAccess2, and Time.cpp keeps both platforms' includes;
- the force-TLS-destructor test stays off macOS beside upstream's new
  inet address test, and the new flexible memory tests run everywhere;
- GuestMath's Bessel references come from libm's jn and yn on macOS,
  whose libc++ has no C++17 special math functions;
- the provoking vertex and f16 interpolation debt entries keep both
  sides' facts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ycarogabriel-lgtm added a commit to ycarogabriel-lgtm/AnyPS5 that referenced this pull request Oct 25, 2026
Keeps the macOS port current with 50 upstream commits.

- Upstream's boykopovar#7487 keeps the barycentric tessellation stages' clip
  distances in the blocks between the stages, which MoltenVK's
  SPIRV-Cross translates; it replaces the fork's MoltenVK-only switch
  (clipDistancesInPerVertexBlock), so the switch, its device query and
  its debt note are removed and every driver takes upstream's stages.
- The provoking vertex debt entry takes upstream's geometry program and
  strip facts and keeps the fork's rule for devices without
  VK_EXT_provoking_vertex (boykopovar#6741).
- The f16 interpolation debt entries keep the fork's 64-bit integer
  text and take upstream's clip distance description.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agc libSceAgc* docs fix error Closes an issue (Fixes #N) shader core/shader

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Barycentric tessellation helper mismatches vertex BuiltIn blocks (VUID 08746)

2 participants