Repository navigation
Conversation
github-actions
Bot
added
agc
JJenkx
force-pushed
the
validation/pipeline-journal-fa
branch
2 times, most recently
from
October 15, 2026 08:41
b508a4f to
80d4ae4
Compare
… next one A graphics pipeline the store builds or a compute pipeline a dispatch builds is appended to pipelines.journal next to the shader cache entries, as the materialized result of each stage (the specialized SPIR-V, bindings and push constants) plus the recipe the pipeline was built from: the stages' identities, vertex input layout, descriptor set layout keys and the state fields the Pipeline constructor reads. When a VulkanDevice is created with a window (a headless device waits for its first draw or dispatch, since the window's device usually replaces it first) the journal is loaded and its pipelines are built on a pool of threads while the title loads, so the first draw or dispatch of a configuration seen on an earlier run finds its pipeline resident instead of compiling it on the queue worker. The pipeline stores key on RecompileResult::Identity, a hash of the module words, specialization constants, subgroup width and bindings that is the same on every run (SharedSpirv::ContentHash, computed once per shared module), rather than on the process-lifetime variant ids; a generated stage is keyed like any other. CachedPipeline builds outside the store's mutex and the library parts and link are built outside the library store's mutex, so replay threads and queue workers build in parallel; a concurrent build of the same key keeps the first inserted. A replayed pipeline enters the store at its least recently used end and a full store takes none, and its optimized relink waits for the first draw that binds it, so the title's own pipelines come first in both. The compute pipeline creation moves into VulkanDevice::createComputePipeline, shared by dispatches and the replay. The store key also covers the color component mapping, the depth target's sample count and a depth bias clamp, which the constructor reads. The file names the device it was written for (the properties the driver's pipeline cache file is named by) and starts over for another; a torn tail is cut at a record boundary; two runs sharing the directory take turns on a lock file and keep each other's records; a read failure never starts the file over. A device cancels its replay before it is replaced or destroyed, and a replaced device destroys its pipelines then rather than at the driver's stop. APS5_NO_PIPELINE_JOURNAL=1 disables it; APS5_PIPELINE_JOURNAL_THREADS, APS5_PIPELINE_JOURNAL_ENTRIES (the first recorded), APS5_PIPELINE_STORE_ENTRIES and APS5_PIPELINE_JOURNAL_MAX_MIB bound it; APS5_PROFILE_DRAW prints a [pipeline-journal] summary. APS5_VERIFY_SET_LAYOUTS=1 now also checks a dispatch's set layout against the one described from its shader. The lavapipe test suite, which shares one cache directory, runs with the journal off except for the journal's own tests. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P6hw3kRAuQdfcU9YCZgaoK
…a pipeline boykopovar#5861 added both to the pipeline state and its store key; the recipe dropped them, so a replayed pipeline keyed another configuration. Co-Authored-By: Claude <noreply@anthropic.com>
JJenkx
force-pushed
the
validation/pipeline-journal-fa
branch
from
October 15, 2026 09:14
80d4ae4 to
bf371e0
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Refresh Adrià Franch’s upstream pipeline journal #5875, source head
569ff84ea258, on main0afa6a607ed08a1e3d131c80a943c4485c71c7f7; current public head isbf371e0f17d14455dd5ecf73d5d38850a87138c2. Both original commits retain their author. This is a fork validation draft, not a replacement upstream submission or a recommended Astro runtime.Keep main’s futures-based compute pipeline preparation outside the GPU mutex. Replay and live dispatch use that shared builder and stable content identities; replay creation is counted separately without re-recording its own entries. Preserve current stencil artifacts and the compact shared-module representation in the shader-cache codec guards.
Extend the journal recipe codec for current instanced-tessellation fields and dynamic stencil-reference state. Preserve the dynamic-state presence and normalize its per-draw reference value. The latest update preserves the effective host sample mask and bumps the journal file format to 3, rejecting older recipes; inactive mask bits and shader-only early-Z do not split identical fixed pipelines. Existing shader cache format 35 remains intact. Update the outside-GPU-mutex fixture to produce different SPIR-V: its old NOP-only difference correctly shares a pipeline under content identities.
Earlier validation
Linux x86-64 / GCC Release / renderer SPIRV-Tools enabled. Full patched
libsbuild succeeds. Eight selected CTest cases pass with no skips on both RX 6650 XT / RADV and lavapipe, Mesa 26.2.4: pipeline cache, pipeline libraries (normal/off/fast-only), shader disk cache, mock journal codec/lifetime, end-to-end graphics/compute journal replay, and compute pipeline creation while another thread holds the GPU mutex.The expanded codec fixture fails first for missing instanced-tessellation fields; after fixing those it fails for missing dynamic stencil-reference state. The final codec passes. Four initial pipeline-case failures used a recorder test executable that had not been rebuilt after the shared-module representation changed; those stale results are discarded. The final passing runs rebuild every selected test executable. Current conventions and whitespace pass.
Synthetic Radeon benchmark, 32 compute programs, two replay threads: the cold application pipeline-cache run builds 32 pipelines at dispatch and takes 864.7 ms; the warm driver-cache run still creates 32 pipelines but takes only 2.9 ms; journal replay creates all 32 ahead (1.7 ms replay work, 20.7 ms from device opening), creates zero at dispatch, and takes 2.4 ms at dispatch. These are one synthetic capture, with the driver’s disk cache left enabled, not first-time-driver or title measurements. They establish the replay mechanism; they do not establish a practical warm-load or FPS improvement on Radeon.
No title run for this draft yet. Hosted validation is recorded below. Do not infer Astro gains from this benchmark.
Checklist
Previous full Linux/Windows validation completed for old head
22644b65e47fon mainfa539dd9bafa. Linux full CI: 1,156 passed, 57 skipped, zero failures (1,213 total). Windows builds, then reports 782 passed, 443 skipped and 10 failed (1,235 total). Failures include guest shader-link/helpers, PRX thread-local destructors and seven VideoOut cases. Several report a C++ system_error during teardown with missing unwind data in libSceAgcDriver.prx. The cause remains under investigation; this draft is not ready for adoption.Current refresh and lifetime fixes
Rebased the four commits onto current main
eea428ca855fb59a3edd394bab9abf99165590f1; the only manual conflict was additive usage documentation. Original authorship is retained. Current head isb508a4f1c2d1d254e333889fee13b25869c93cb1.Use the existing
HostThreadLocalmechanism for the graphics pipeline key scratch vector, avoiding the emulated-TLS destructor rejected by the Windows lifetime checker and implicated in the upstream Windows title report. Flushing an unused journal now checks for an already-created instance rather than constructing a writer during shutdown. An expanded mock fixture sets an invalid journal-only option before the unused flush: before the fix it fails because the flush initializes/parses the journal; afterward it passes, clears the option and exercises normal record/replay. No parser validation was weakened.Full Linux libraries and every selected executable target rebuild successfully. Selected current-head tests: Radeon 17 passed, zero skipped (6.42 s); lavapipe 16 passed, one skipped (6.74 s; pipeline-cache-file case). These cover journal mock/codec, graphics/compute replay, pipeline cache/libraries, shader disk cache, compute preparation outside the GPU mutex, shader-link/helpers and VideoOut teardown fixtures. Current conventions, whitespace and test registration checks pass. These Linux runs do not validate the Windows TLS path.
New hosted CI could not be dispatched: GitHub returned HTTP 422, “Actions has been disabled for this repository,” despite the repository Actions permissions reporting enabled and the Build workflow active. One retry after checking those settings returned the same rejection. Windows and the full suite remain pending for this new head; the old failure totals above are historical. Keep this draft; no title performance gain is claimed.
Prior db819 validation
Current public head
80d4ae462rebases the published branch onto maindb81992c554f1b395b4c04987cfbdc316a03d69b. The additive shader-cache guard-list conflict retains bothShaderStageand the journal's reviewed symbols. Before the compatibility fix, the rebase changes consist only of intervening main changes; the original author and topic commits remain.A Linux build then exposed the intended State layout guard: main added quad sample masks and shader-only early-Z, increasing State from 704 to 720 bytes. The fix updates canonical fixed state to the effective host sample mask, serializes those masks, bumps the journal file format to 3 and adds registered codec regressions. The tests cover partial masks, ignored inactive bits, shader-handled nonuniform masks, shader-only early-Z, distinct effective pipeline masks, round-trip decoding and truncated old-layout payloads.
The full Release runtime-library target and graphics/recorder/journal test executables build at private integration
bbe2763cb. Its only tree difference from the public head is the disclosed private SPIRV-Tools teardown repair. Eight focused CTests pass, zero skips, on RX 6650 XT / RADV: actual graphics/compute journal replay, mock recipe/store replay, cache, cache file, library normal/off/fast-only, and main's draw-program trace. Removing the new mask handling while leaving the updated size guard in place makes the registered end-to-end journal case fail withpipeline recipe lost the effective sample mask; restoring the code makes all eight pass again. Current-base conventions and whitespace pass. Earlier CI, lavapipe and benchmark results above remain tied to their recorded revisions; no new title speedup, Windows build or independent official-vendor build is claimed.Tested
Current-main depth-metadata refresh: Linux compilation of the clean rebase failed its State guard with
752 == 720after main boykopovar#5496 added resource metadata to DepthTarget. The new fields are not read by Pipeline construction or fixed pipeline keys; the canonical depth recipe still preserves format and sample count. Updating the reviewed size guard to 752 leaves journal format 3 and its encoded recipe unchanged.At private integration
1e7a2029e, the Release runtimelibstarget and rebuilt graphics, recorder, standalone journal and draw-trace executables succeed. Its only tree difference from the public head is the previously disclosed private SPIRV-Tools teardown repair. All eight focused Radeon CTests pass with zero skips:agc_driver_pipeline_journal(real graphics/compute replay),agc_pipeline_journal(mock), cache, cache file, library normal/off/fast-only and draw-program trace. Current conventions and whitespace pass. Historical Windows, lavapipe and synthetic measurements above were not rerun for this refresh; no title performance gain or official-vendor build is claimed.