Skip to content

perf(agc): prebuilt fragment libraries that match the draw (push-free offset, depth formats) - #3

Closed
Pipodi wants to merge 4 commits into
Celegans12:yotei/bestfrom
Pipodi:perf/prebuild-draw-depth-format-parts-ybest
Closed

Pipodi wants to merge 4 commits into
Celegans12:yotei/bestfrom
Pipodi:perf/prebuild-draw-depth-format-parts-ybest

Conversation

@Pipodi

@Pipodi Pipodi commented Oct 26, 2026 •

Copy link
Copy Markdown

Two changes to the background fragment-library prebuild in yotei/best. Before them, a prebuilt fragment library never matched a draw whose shader has no push constants or whose target has a depth or stencil attachment, so that draw still built its own library on the draw path.

  1. Push offset for stages without push constants (ShaderRegistry.cpp). The draw builds a stage with no push constants with push offset 0. The registration prebuild always passed the fixed fragment push offset. The offset is part of the fragment library key, so the prebuilt library never matched and the draw built another one. The prebuild now uses offset 0 when the stage has no push constants.
  2. Depth and stencil formats (PipelineLibrary, VulkanDevice.cpp). Fragment libraries are keyed on the depth and stencil attachment formats, but the prebuild only built colour-only libraries. A draw now records the format pair of each fragment library it builds, and PrebuildGraphicsLibraries builds once more for every recorded pair.

The debt entry "Prebuilt pipeline libraries" in docs/dev/debt/libSceAgcDriver.md is extended:

  • a format is covered only after one draw used it;
  • each recorded pair multiplies the fragment libraries built per shader;
  • the recorded pairs are never pruned;
  • the memory cost is unmeasured.

Tests

  • agc_driver_prepared_graphics gains two cases:
    • a push-constant-free fragment shader prebuilt at registration is found by the draw, which builds no further fragment library;
    • a fragment shader prebuilt after a draw on a D32S8 target draws on that target without building a fragment library.
  • Each case fails with its fix reverted.
  • ctest on yotei/best 358458a and on this branch (rebased 2026-10-30): the same 12 failures on both (direct_image_refusal, gpu_buffer_format, sampled_cache_budget, sampled_cache_budget_without_storage_image_multisample, sampled_cache_later_use_refusal, draw_unsupported_buffer_type_draw_pipeline, draw_shader_failure_materialization, draw_shader_failure_materialization_correlation_disabled, depth_plane_partial_store, host_import_cut_lifetime, prx_thread_local_destructors, public_headers). The timings below were measured before the rebase, on 703b139.
  • Khronos validation (-R "prepared_graphics|prepared_shaders|depth", REQUIRE_VULKAN, NO_SHADER_CACHE): the same 4 VUID-VkWriteDescriptorSet-descriptorType-00337 lines on the base and on the change, all inside the already failing sample_order_recipe_depth_locations.

Ghost of Yōtei A/B (cold)

  • Hardware: RTX 4090 (NVIDIA 617.42), i9-13900K, 32 GB RAM, Windows 11.
  • Setup:
    • fresh profile;
    • empty shader cache and empty NVIDIA DXCache/GLCache;
    • one run per arm, stopped at 2,700 s;
    • the same environment in both arms (in the summary lines below).
  • The graphics-pipeline wall during the loading cinematic is 21 s shorter (161 s → 140 s), and frame 1456 arrives 43 s earlier. Pipeline-cache misses don't change (1,187 / 1,193), because the change makes each complete pipeline build cheaper rather than avoiding it.
  • Graphics-pipeline time per miss inside the wall goes from 334 ms to 301 ms.
  • No extra prebuild cost shows in the trace: the two prebuild threads take 34 ms per frame against 35 ms.
  • This is one pair of runs. The compute walls, whose code is the same in both arms, vary by 20–40 s between runs, so the end-to-end gain is about the size of that noise. The graphics-pipeline wall is the number this change targets.
base 703b139 this PR 6e83d3a
time to frame 1000 1,158 s 1,112 s
time to frame 1456 2,708 s 2,665 s
graphics-pipeline wall (frame) 161 s (1382) 140 s (1346)
draws' pipeline phase in that wall 140 s, 419 misses 126 s, 418 misses
draws' pipeline phase, whole run 401 s 389 s
[pipecache] misses, whole run 1,187 1,193
compute compiles, whole run 1,665 s 1,593 s
compute walls (unchanged code) 302 s (812), 711 s (1119) 281 s (808) + 40 s (807), 691 s (1083)
Trace summary lines (APS5_TRACE_FRAMES=620:720)

base 703b139: frames 620:720 from present 620: 720 presents, frame p50 507.6 / p90 1565.3 / p99 34542.7 / max 738856.4 ms, GPU busy 440.2 ms per present
NVIDIA GeForce RTX 4090, NVIDIA 617.42, Windows 10.0.26200, 3840x2160, commit 703b139, dropped: yes

this PR 6e83d3a (built as f2eef06385, the same tree; only the commit messages changed since): frames 620:720 from present 620: 720 presents, frame p50 468.9 / p90 1509.3 / p99 33378.2 / max 726692.2 ms, GPU busy 491.1 ms per present
NVIDIA GeForce RTX 4090, NVIDIA 617.42, Windows 10.0.26200, 3840x2160, commit f2eef0638524, dropped: yes

Environment, the same in both arms: APS5_ASYNC_LANE=51 APS5_BINDLESS_IMAGES=1 APS5_BINDLESS_LAZY=1 APS5_DEFER_REFRESH_FLUSH=1 APS5_GPU_CHECKPOINTS=1 APS5_HOST_IMPORT_MIB=16384 APS5_NO_ADDRESS_DRAW_REUSE=1 APS5_NO_DRAW_AHEAD=1 APS5_NO_PIPELINE_JOURNAL=1 APS5_NO_RESULT_MEMO=1 APS5_NO_SNAPSHOT_CHECK=1 APS5_PIPELINE_DISPATCH=1 APS5_PIPELINE_INDIRECT=1 APS5_PREBUILD_THREADS=2 APS5_PREPARED_SPIRV_MIB=6144 APS5_PROFILE_DRAW=1 APS5_RESOURCE_CACHE_ENTRIES=1024 APS5_SPANNED_WAVE=1 APS5_STAGING_DIFF=1 APS5_STAGING_POOL_MIB=2048 APS5_TRACE_EVENTS=1048576 APS5_TRACE_FRAMES=620:720

Trace digest, frames 620:720 (ms per frame)
base this PR
window 2,098 s 2,047 s
frame p50 / p90 507.6 / 1,565.3 468.9 / 1,509.3
DrawPipeline.commit 525.7 514.9
DrawPipeline.drain.packet 446.2 437.2
Vulkan.ComputePipeline 2,450.8 2,345.2
Shader.PrepareArtifact 1,217.3 1,191.4
Vulkan.GraphicsPipelinePrebuild (2 threads) 35.3 + 34.4 34.0 + 33.0
slowest frame 738.9 s (present 1118) 726.7 s (present 1083)

The slowest frames in both runs are the compute-giant wall: three builds of 0x800014e300 (~1.42M words, ~425–433 s together), plus 0x8000173b00 and 0x807b30a100 at ~160 s each. This PR doesn't touch them.

Main

Main 3d2b7c0 has the same two gaps (ShaderRegistry.cpp:1334 passes the fixed push offset, and there is no depth-format prebuild). The same change rebased on main is ready as perf/prebuild-draw-depth-format-parts (08c6ea862f). On main it merges cleanly with boykopovar#9563 and boykopovar#10388. Following the plan in discussion boykopovar#675, I can open it as a standalone PR on main whenever you want it in a stack.

🤖 Generated with Claude Code

Pipodi and others added 4 commits October 30, 2026 11:13
…he stage has no push constants

The draw builds a CompiledShader with offset 0 when the shader has no push
constants, but the registration prebuild always passed the fixed fragment
push offset. The offset is part of the fragment library key, so the prebuilt
part never matched and the draw built its own.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…he draw

Expose PrebuildStageLibrary and check that after the registration prebuild of
a fragment shader without push constants, the draw-side lookup with offset 0
builds no further fragment library.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s use

Graphics pipeline library fragment parts are keyed on the depth and stencil
attachment formats, but the registration prebuild only built colour-only parts.
A draw now records the format pair of each fragment part it builds, and
PrebuildGraphicsLibraries prebuilds once more for every recorded pair.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Draw a fragment shader on a D32S8 target, prebuild a second one, and require
that the second shader draws without building a fragment library.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Pipodi
Pipodi force-pushed the perf/prebuild-draw-depth-format-parts-ybest branch from 6e83d3a to c6e47f9 Compare October 30, 2026 10:30
@Celegans12

Copy link
Copy Markdown
Owner

@Pipodi thanks, this is in yotei/best now as of 32558fef16. I cherry-picked the four commits with you as the author (1fdf781482, f3f0f1dbb5, 259adb338c, 27171091f2 on our tree), so I'm closing this PR instead of merging it. The branch had moved on since your rebase.

On our side the agc_driver_prepared_graphics tests pass and the 60-frame title check still matches at 4K and at 1080p. I haven't done a cold run with it, or a timed warm one, so the README credits your cold numbers (the 161 → 140 s graphics-pipeline wall) as yours.

If you open the main version (perf/prebuild-draw-depth-format-parts) upstream, I'm happy to run it on Yōtei here and post the numbers there.

@Celegans12 Celegans12 closed this Oct 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants