Skip to content

perf(agc): defer pipeline artifacts with first-use preparation - #6743

Draft
JJenkx wants to merge 3 commits into
boykopovar:mainfrom
JJenkx:fix/lazy-pipeline-abi
Draft

JJenkx wants to merge 3 commits into
boykopovar:mainfrom
JJenkx:fix/lazy-pipeline-abi

Conversation

@JJenkx

@JJenkx JJenkx commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

What

Draft: matched-title performance benefit remains unproven.

With APS5_LAZY_SHADER_PREPARE=1, defer artifact preparation in the graphics-stage and rectangle pipeline ABI resolvers until first use, after registered-header and device validation. Default eager behavior is unchanged. Main's existing single-stage ABI planning, saved deferred ABI state and synchronization are preserved; the production change is two early-return guards.

Refreshed as two linear commits on main cf79276fee2f042feb6a7c71d0d6a27e9cfe2592; head 496b51b41ad5161568b089e7e75de281119b033c. The new ABI regression now uses main's per-test CMake fragment layout. No private timing probe or title integration is part of this branch.

Tested

Linux Release/GCC 16.2.1, AMD RX 6650 XT/RADV Mesa 26.2.6, Linux 7.2.11. Both full SDKs/relinkers and focused executables built, with matching dependency gitlinks and patched/unpatched ELF build IDs. Matched focused checks ran through the shared FIFO with ANYPS5_REQUIRE_VULKAN=1, ANYPS5_NO_SHADER_CACHE=1 and actual Khronos validation loading verified:

  • Main: the same 16 prepared-graphics/prepared-shader groups pass.
  • Change: those 16 plus the new lazy pipeline ABI/first rectangle draw regression pass.
  • No skips, VUIDs or missing-layer diagnostics. The focused matrix includes CPU preparation-budget checks as well as GPU execution fixtures; it is not a claim that every group exercises the GPU.
  • Frozen SDK/test hashes remained intact. Conventions and whitespace pass.

Exact source patch, dependency pins, builds, verbose logs, JUnit and terminal receipts, SHA-256 7748e20d19a542d2b31fb8e4fb097a95118807d8a1389e30faf4cc1f2e1f5d3f (uploaded/downloaded and internal hashes verified).

No new Windows, NVIDIA or title speedup is claimed by these checks. The Smurfs – Dreams PPSA21607 01.002.000 cold-start pair is complete. Control 62bbc741fb2f is main cf79276fee2f plus the same #6959/#13934/#6782/#6913 prerequisites and a private common timing probe; change a7cae4a3bed6 adds this PR. Normal flip scheduling, lazy flag enabled in both arms, fresh private AnyPS5/Mesa shader caches disabled, identical initial writable hashes, same scheduled input, 3840×2160. Both exit0 in about98s, restore kernel settings and leave original inputs/SDK hashes intact.

Observed startup metric Control Change
Process start to first driver Present entry (10ms process tick resolution; not display confirmation) 2532.36ms 2395.77ms
Variant compilations completed by first-present+60s 218 209
Summed variant compilation wall time by +60s 256.43ms 246.02ms
Summed shader preparation wall time by +60s 596.68ms 545.36ms
Summed graphics-stage ABI wall time by +60s 225.08ms 0.49ms

No reliable title loading or gameplay FPS benefit is established by this single pair. Inclusive phase sums overlap and include concurrent work; deferred ABI time is not all removed. Only1.29ms of the control's graphics-stage ABI work precedes its first Present, so the136.58ms process-start difference cannot be attributed to that work. Both reviewed90s captures show save selection; the80s captures differ, and inactive-window capture freshness could not be verified. Mixed startup scene histories make the0:60 trace frame distributions unsuitable for a steady-scene FPS claim. The private measurement source is identical between arms and is not in this PR.

Both official traces/summaries, reviewed captures, source-isolation proof, exact private probe patch, reports and timing counters, SHA-256 35a5f2e433624fd81f590e882ac9743fb55745fd161252bae302d881489f2fac (download/internal hashes verified).

The historical Resident Evil Village cold-start pair, main a6dcea9279b5 versus 22cee06b0105, did not establish a loading gain. Its startup scene histories differ, so its frame distributions do not support a gameplay speed comparison.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none; perf(agc): leave a lazily registered shader's resolved ABI to its first use #7644 is merged into main
  • Fixes: none
  • Unblocks: none; performance benefit remains unproven
  • GPUs tested: AMD Radeon RX 6650 XT, RADV Mesa 26.2.6, Linux 7.2.11 (current focused execution checks)
  • AI-assisted: yes

@github-actions

github-actions Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request Conflicting files (lines on main) Same exports Same new files
PR 14265 docs/dev/ARCHITECTURE.md L148-149

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 2434 docs/dev/debt/libSceAgcDriver.md
PR 5925 docs/dev/ARCHITECTURE.md
docs/dev/debt/libSceAgcDriver.md
PR 6186 docs/dev/debt/libSceAgcDriver.md
PR 6188 core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp
PR 6194 docs/dev/debt/libSceAgcDriver.md
PR 6223 docs/dev/debt/libSceAgcDriver.md
PR 6273 docs/dev/debt/libSceAgcDriver.md
PR 6280 docs/dev/debt/libSceAgcDriver.md
PR 6412 docs/dev/ARCHITECTURE.md
docs/dev/debt/libSceAgcDriver.md
PR 6481 docs/dev/debt/libSceAgcDriver.md
PR 6548 docs/dev/debt/libSceAgcDriver.md
PR 6804 docs/dev/debt/libSceAgcDriver.md
PR 6808 docs/dev/debt/libSceAgcDriver.md
PR 6987 docs/dev/debt/libSceAgcDriver.md
PR 6989 docs/dev/debt/libSceAgcDriver.md
PR 6995 docs/dev/debt/libSceAgcDriver.md
PR 7002 docs/dev/debt/libSceAgcDriver.md
PR 7085 docs/dev/debt/libSceAgcDriver.md
PR 7256 docs/dev/debt/libSceAgcDriver.md
PR 7449 docs/dev/debt/libSceAgcDriver.md
PR 7604 docs/dev/debt/libSceAgcDriver.md
PR 7642 core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp
core/libs/prx/libSceAgcDriver/tests/execution/PreparedShaders.cpp
docs/dev/ARCHITECTURE.md
docs/dev/debt/libSceAgcDriver.md
PR 7668 docs/dev/debt/libSceAgcDriver.md
PR 7682 core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp
PR 7684 docs/dev/ARCHITECTURE.md
PR 8592 core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp
docs/dev/debt/libSceAgcDriver.md
PR 8707 docs/dev/debt/libSceAgcDriver.md
PR 9020 docs/dev/debt/libSceAgcDriver.md
PR 9318 docs/dev/debt/libSceAgcDriver.md
PR 9335 docs/dev/debt/libSceAgcDriver.md
PR 9356 docs/dev/ARCHITECTURE.md
docs/dev/debt/libSceAgcDriver.md
PR 9387 docs/dev/ARCHITECTURE.md
docs/dev/debt/libSceAgcDriver.md
PR 9592 docs/dev/debt/libSceAgcDriver.md
PR 9672 docs/dev/debt/libSceAgcDriver.md
PR 9673 docs/dev/debt/libSceAgcDriver.md
PR 9675 docs/dev/debt/libSceAgcDriver.md
PR 9700 docs/dev/debt/libSceAgcDriver.md
PR 9748 docs/dev/debt/libSceAgcDriver.md
PR 9895 docs/dev/debt/libSceAgcDriver.md
PR 9898 docs/dev/debt/libSceAgcDriver.md
PR 9915 docs/dev/debt/libSceAgcDriver.md
PR 9950 docs/dev/debt/libSceAgcDriver.md
PR 10049 docs/dev/debt/libSceAgcDriver.md
PR 10193 core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp
core/libs/prx/libSceAgcDriver/tests/execution/PreparedShaders.cpp
docs/dev/ARCHITECTURE.md
docs/dev/debt/libSceAgcDriver.md
PR 10233 docs/dev/debt/libSceAgcDriver.md
PR 10256 docs/dev/debt/libSceAgcDriver.md
PR 10359 docs/dev/debt/libSceAgcDriver.md
PR 10743 docs/dev/debt/libSceAgcDriver.md
PR 11351 docs/dev/ARCHITECTURE.md
PR 11442 core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp
PR 11825 docs/dev/debt/libSceAgcDriver.md
PR 11998 docs/dev/debt/libSceAgcDriver.md
PR 12120 docs/dev/debt/libSceAgcDriver.md
PR 12565 docs/dev/debt/libSceAgcDriver.md
PR 12567 docs/dev/debt/libSceAgcDriver.md
PR 13599 docs/dev/debt/libSceAgcDriver.md
PR 13707 docs/dev/debt/libSceAgcDriver.md
PR 13863 docs/dev/debt/libSceAgcDriver.md
PR 13920 docs/dev/debt/libSceAgcDriver.md
PR 13928 core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp
core/libs/prx/libSceAgcDriver/tests/execution/PreparedShaders.cpp
docs/dev/ARCHITECTURE.md
docs/dev/debt/libSceAgcDriver.md
PR 13929 core/libs/prx/libSceAgcDriver/Execution/src/Driver/Shaders/ShaderRegistry.cpp
core/libs/prx/libSceAgcDriver/tests/execution/PreparedShaders.cpp
docs/dev/ARCHITECTURE.md
docs/dev/debt/libSceAgcDriver.md
PR 13963 docs/dev/debt/libSceAgcDriver.md
PR 13994 docs/dev/debt/libSceAgcDriver.md
PR 14005 docs/dev/debt/libSceAgcDriver.md
PR 14212 docs/dev/debt/libSceAgcDriver.md
PR 14264 docs/dev/ARCHITECTURE.md
PR 14302 docs/dev/debt/libSceAgcDriver.md
PR 14311 docs/dev/debt/libSceAgcDriver.md
PR 14344 docs/dev/debt/libSceAgcDriver.md
PR 14621 docs/dev/debt/libSceAgcDriver.md
PR 14943 docs/dev/debt/libSceAgcDriver.md
PR 14944 docs/dev/debt/libSceAgcDriver.md
PR 14976 docs/dev/debt/libSceAgcDriver.md

@JJenkx

JJenkx commented Oct 16, 2026

Copy link
Copy Markdown
Contributor Author

Temporarily closing this draft to free one of my 20 open-PR slots for a reproduced cache-header arithmetic correction following #6296. This first-use pipeline work remains on fix/lazy-pipeline-abi at 3bac469, and its prior testing integrations are preserved. It still needs an isolated main-versus-change title performance comparison before becoming ready. Reopen when a slot is available; no branch deletion.

@JJenkx JJenkx closed this Oct 16, 2026
@JJenkx JJenkx reopened this Oct 19, 2026
@github-actions github-actions Bot added perf Performance change unblocks title Names a title ID on the Unblocks line labels Oct 19, 2026
@JJenkx
JJenkx force-pushed the fix/lazy-pipeline-abi branch from 3bac469 to d5ec29e Compare October 19, 2026 02:24
@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 21, 2026
@JJenkx
JJenkx force-pushed the fix/lazy-pipeline-abi branch from d5ec29e to 22cee06 Compare October 22, 2026 00:53
@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 22, 2026
@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 23, 2026
@JJenkx
JJenkx force-pushed the fix/lazy-pipeline-abi branch from 22cee06 to c8b4a44 Compare October 23, 2026 16:06
@JJenkx

JJenkx commented Oct 23, 2026

Copy link
Copy Markdown
Contributor Author

Rebased without merge commits to c8b4a44e6090455554e4c5fcd6646197d1a30d17 on main 16858c04c6c0c63d42943753e9f1d2f491655838; main 15e71dbe1116517913c5fbf309aa86d102285da3 merges cleanly. The semantic conflict with the now-merged #7644 is resolved by preserving upstream ResolveShaderAbi, its saved ABI state and synchronization. The remaining production change is only the two graphics-stage/rectangle lazy guards.

Both main and change pass the same eight focused Vulkan checks, including the upstream deferred compute and concurrent lazy cases, with validation verified loaded and zero VUIDs. The added first-draw lazy pipeline fixture passes too. Full Release libraries build; conventions and whitespace pass. Pinned receipts and logs.

The existing technical draft remains because the required matched-title performance evidence is still outstanding. These focused checks do not establish a title loading or FPS gain.

@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 27, 2026
@JJenkx
JJenkx force-pushed the fix/lazy-pipeline-abi branch from 13843b1 to 5fe8ec8 Compare October 27, 2026 10:41
@JJenkx

JJenkx commented Oct 27, 2026

Copy link
Copy Markdown
Contributor Author

Current-main conflict refresh, 2026-10-27 10:18 UTC pass: exact head 5fe8ec8fae6ac03d94fd611ead6c769e0d56f9c3 on main 3fea2a937ba9fbab2f06c51691fe0c5763659aa3. The two opt-in early-return guards and main's newer ABI planning and prepared-state behavior are retained. This remains a technical draft: these correctness gates do not establish a matched-title loading or performance benefit. Every submodule pointer matches the pinned main.

Linux Release, RX 6650 XT/RADV Mesa 26.2.4, matched SPIRV-Tools-disabled builds. Matched main passes 14 groups × three repetitions (42 passes). Candidate passes 15 × three (45), and all 45 pass again after restoration. With only main's production ShaderRegistry.cpp and candidate tests, the 15 groups run once: 14 pass and only lazy-pipeline-ABI fails at the expected eager-preparation assertion. Restored source matches the candidate and the AGC driver is byte-identical. Actual Khronos insertions: 84 main, 90 initial candidate, 30 negative, 90 restored.

Every physical fixture has actual RX 6650 XT selection and Khronos validation loading verified; CPU/mock checks have no GPU-execution claim. Required Vulkan and shader caches disabled; zero skips, VUIDs or missing-layer failures in the accepted runs. Conventions and whitespace pass. All complete test harnesses and cleanup ran through the shared FIFO in service-owned PTYs and finished normally. No new title, console, Windows, visual, readback, FPS or performance result is claimed. Older observations below retain their original exact pins.

Exact patches/pins, integration and source proofs, builds, verbose matched tests, validation review, hashes and terminal service/FIFO receipts. SHA-256: b246260130e123632f707daeace137d321f738b18df1f47e77a1065435a27341; downloaded copy verified.

AI-assisted: yes.

@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 27, 2026
@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 28, 2026
@JJenkx
JJenkx force-pushed the fix/lazy-pipeline-abi branch from 5fe8ec8 to b3ce8e1 Compare October 28, 2026 12:41
@JJenkx

JJenkx commented Oct 28, 2026

Copy link
Copy Markdown
Contributor Author

Refreshed on main 2227f1366188cda5699dd20e2e756a18006a5d4b as b3ce8e1bd9ef620d1a6548e8d13f6dd26607b69b. Original topic code and tests are preserved, both debt entries retained, and gitlinks match main. Linux Release with bundled SPIRV-Tools enabled, RX 6650 XT/RADV, Khronos validation requested, Vulkan required and shader caches disabled:

  • Matched main: 14 groups ×3, 42 passes.
  • Candidate: 15 groups ×3, 45 passes.
  • Main ShaderRegistry.cpp production with candidate tests: 14/15 pass; only the added lazy-ABI assertion fails as expected.
  • Restored candidate: 45 passes, with source and driver library byte-identical to the initial candidate.

Loader logs verify validation insertion (84 main, 90 initial candidate, 30 control, 90 restored), the selected physical GPU, zero VUIDs, missing-layer diagnostics or skips. The complete harnesses, including source control/restoration, ran through the shared FIFO under service-owned PTYs. Conventions and whitespace pass. This remains a technical draft because a matched-title loading/performance benefit is still unqualified; no new title, Windows or speed result is claimed.

Exact source/dependency pins, source proof, builds, service/FIFO receipts, controls, logs and JUnit, SHA-256 760f4c075c659cb6073fd732f852947543dea35484f03e24e41c036df57ed44f.

AI-assisted: yes.

@JJenkx

JJenkx commented Oct 29, 2026

Copy link
Copy Markdown
Contributor Author

Refreshed head cf0094f04ba894a5d1f0331ff7ad0535fcb7fdc7 integrates pinned main 5454cd2e455994d992aca39335ce60507eeda1eb.

Both production early-lazy guards are retained. Main’s new budget-only fixture never requests an ABI, so the transplanted ResolveShaderAbi setup is now conditional on !budgetOnly; its setup and assertions are otherwise preserved. The initial budget-fixture failure is included. Main-production control passes16/17 groups and fails exactly the lazy-ABI assertion; restoration passes51 executions and restores an identical driver binary. This remains a technical draft pending matched-title performance evidence; these correctness checks do not establish a performance benefit.

Fresh matched Linux Release builds on AMD Radeon RX6650XT / RADV, with SPIRV-Tools enabled: main 48 passes and change 51 passes (three repetitions of each focused group), zero skips and zero VUIDs. Actual Khronos validation-layer instance insertions are 96 on main and 102 on the change. CPU-only groups are not counted as GPU coverage. Conventions, whitespace and submodule-pointer checks pass. Later-main merge previews were clean; those preview trees were not executed. Every GPU harness used the shared reservation and launch queue.

Evidence: raw logs, JUnit and source receipts; SHA-256 20e96798dd8a9efb18a99124ff0c30bd18ce5cdea87e2287c77880384b63c424. Queue and cleanup receipts. No new Windows, macOS, native-title or performance qualification is claimed. Earlier measurements retain their original source pins.

AI-assisted: yes (Codex).

@github-actions github-actions Bot added needs rebase Conflicts with main, rebase needed and removed needs rebase Conflicts with main, rebase needed labels Oct 29, 2026
@JJenkx
JJenkx force-pushed the fix/lazy-pipeline-abi branch from cf0094f to 496b51b Compare November 1, 2026 03:27
@JJenkx

JJenkx commented Nov 1, 2026

Copy link
Copy Markdown
Contributor Author

Refreshed on main cf79276fee2f as 496b51b41ad5: two linear topic commits, no merge commits. Matched focused checks pass main16/change16 plus the new first-use regression, no skips or VUIDs; exact logs/builds are linked in the description. The separate Smurfs cold-title pair is now also attached with compiler timing counters and both official traces. It shows deferred startup work, but a single pair with differing startup scene histories does not establish a title loading/FPS gain, so the specific performance-evidence draft blocker remains. No private timing instrumentation or title prerequisites were pushed in this branch.

@github-actions github-actions Bot added perf numbers missing perf pull request without a before and after summary line and removed needs rebase Conflicts with main, rebase needed unblocks title Names a title ID on the Unblocks line labels Nov 1, 2026
@github-actions

github-actions Bot commented Nov 1, 2026

Copy link
Copy Markdown

This pull request is labeled perf, so it needs the summary line of a capture on main and one with the change, from the same title, scene and machine, as the Performance report form asks. Please paste both aps5-summary.jsonl lines, attach the two traces and add the output of tools/trace_digest.py.

@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Nov 4, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@JJenkx

JJenkx commented Nov 4, 2026

Copy link
Copy Markdown
Contributor Author

Cold shader cache A/B (Linux), at head 615095c51b. Head 615095c51b merges main b611e634f7. The control is main b611e634f7 itself.

Setup:

  • Fobia – St. Dinfna Hotel (PPSA05626), RX 6650 XT (RADV Mesa 26.2.6), Release.
  • Both arms run with APS5_LAZY_SHADER_PREPARE=1, because the variable already exists on main and this PR extends it.
  • The title's shader and pipeline cache is deleted before every run.
  • Five alternating pairs of 125 s runs, with the same Cross inputs.
  • FPS comes from the window title in 2 s samples, over the 52–90 s scene where a cold cache stalls.
Pair main median (min) FPS this PR median (min) FPS
1 19.9 (5.1) 23.9 (6.5)
2 2.7 (2.1) 25.2 (10.8)
3 24.5 (2.9) 19.5 (11.8)
4 25.3 (10.1) 24.9 (8.8)
5 23.4 (4.0) 24.8 (5.5)
  • Median FPS: apart from main's pair 2, the scene medians overlap: main 23.4 and this PR 24.8 over the five pairs. In pair 2, main collapsed to 2–3 FPS, which also happens on main without lazy preparation on a cold cache.
  • Minimum FPS: the scene minimum is higher with this PR in 4 of 5 pairs, a median of 8.8 against 4.0.
  • Exits: all ten runs exit 0. Before 52 s, both arms hold about 60 FPS.

That is a hint that deferring graphics-stage ABI preparation eases the worst cold-cache stalls. Five pairs and 2 s samples don't establish it. The PR stays a draft until a larger or trace-based measurement does.

AI-assisted: yes

@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Nov 4, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agc libSceAgc* build docs perf numbers missing perf pull request without a before and after summary line perf Performance change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants