Repository navigation
Conversation
JJenkx
force-pushed
the
validation/draw-ahead-760
branch
2 times, most recently
from
October 15, 2026 05:03
3e9fba4 to
0eb0d89
Compare
JJenkx
changed the title
JJenkx
force-pushed
the
validation/draw-ahead-760
branch
3 times, most recently
from
October 15, 2026 10:02
60d1893 to
a6a868d
Compare
PrepareDrawBindings gave every read-only input it snapshotted per draw a pooled Buffer of its own. When the unwritten-snapshot map missed, it collected the range's writes and probed the reuse cache, and on a miss it allocated a Buffer, copied, and recorded the Buffer in both caches. Inputs of up to 4 KiB now probe only the map of unwritten slices: a hit binds the slice an earlier draw of the same collect epoch and tracker generation copied, and a miss copies the input into a slice of a 4 MiB host-visible chunk owned by the recorder (a bump allocation and one copy) and notes it there under the same rule as the unwritten snapshots. Without a collect epoch (APS5_NO_COLLECT_MEMO), which alone bounds that rule for CPU stores nothing collected, small inputs are copied every time. The descriptor points at the slice's offset and range. Larger inputs and moved shader data keep their reuse caches. Offsets only grow within a chunk and a full chunk is replaced, never rewound, so no two slices share bytes; the draw's bindings hold the chunk until their batch completed, and replacing the chunk forgets the noted slices, so the map pins at most the current chunk. Each new slice charges the open batch the chunk space it consumed, alignment padding included, and the slice that replaces a chunk also charges the tail it leaves unused, so a chunk's whole capacity is charged once and a batch's kept bytes cover the chunks it retains (at most one chunk shared with an earlier, completed batch is not counted). A new slice counts as kept by its batch, so reusing it there charges nothing; a slice reused by a later batch is charged once by that batch, as a reused snapshot is. The descriptor tests compared snapshot descriptors against offset 0 of a buffer of their own; they now compare against the snapshot's range, and the reuse test uses an input above the slice limit. A new test covers slice placement, alignment, disjointness across chunks, the charge of a chunk filled by 4-byte slices, the reuse of an unchanged slice within a collect epoch without a second charge (and its copy every time with collect memoization off, agc_draw_snapshots_no_collect_memo) and its replacement after a store and a new epoch, and the chunk's lifetime; agc_draw_snapshots runs the draw snapshot tests.
VulkanDevice copied the cached Graphics::Context (device functions, limits, the buffer pool reference) for every KnownDrawRejection, Draw and DrawFromRecipe call, twice per draw on the queue-0 thread. These calls now take a reference to the context the device built at creation; only APS5_NO_CONTEXT_CACHE builds one per call, as before.
Every queue-0 draw resolved its parameters, ran the register precheck and built the draw key on the recording thread (with the incremental register fingerprints the key still hashes the user words and looks up the five shader addresses in the registry). A scan thread now executes each queue-0 submission on a copy of the queue state while queue 0 records it, and publishes per draw packet, in order, the key with its shape, the parameters and the precheck outcome. Queue 0 takes a record instead of computing them when it is certain and still matches. A record is exact for the state the scan executed. Packets queue 0 may skip (COND_EXEC ranges, predicated packets while predication is armed) or whose value comes from memory (COPY_DATA into a register) mark the registers they write as divergent; an unconditional write ends the divergence, CLEAR_STATE, CONTEXT_STATE and a reset carry it like the state itself, and a whole-state change under a condition loses the scan until an unconditional reset. A certain record lists at most 64 divergent registers with the values the scan had, and queue 0 uses it only when these, INDEX_BASE, the index type, the instance count, the device and the submission all match its own state; any other draw takes the full path. Indirect draws are never published. The precheck's register part becomes PrecheckDrawState so both threads share it; the metadata pass and the trace stay on queue 0. The scan notes a dispatch's initiator and group counts in the shader registers as queue 0 does. Pm4 exposes the register offset decode and the registers a COPY_DATA writes.
Draws whose register key is new (per-frame ring addresses in user data make most keys new every frame) captured and recompiled their stages on the queue-0 thread, or, when their shape has an entry, re-read its user words and recompiled the stages whose captured reads changed. The draw-ahead scan now queues every such draw it publishes with a Proceed precheck to worker threads (a quarter of the hardware threads, one to four), which decode the draw from the scan state, capture and materialize its stages with the same request the queue-0 miss path builds (DrawStageRequest, DrawStageVariant, shared with it, with the position export layout the miss path gives the vertex stage, and the pure data positions of each stage) and insert the variants into the draw cache under the draw's key and shape. Queue 0 then finds an entry and validates it as it validates any cached variant, against current memory. The key holds registers only, and validation compares captured memory, so a worker compiles only draws whose compile request depends on nothing else (DrawAhead::CompilesAhead). A mesh draw takes the descriptor of its index buffer from its packet into its hidden user words, and an instanced tessellation draw numbers its hull patches by its packet's counts (queue 0 does not cache those): a skipped draw of one buffer would otherwise leave its compile under the key an executed draw of another buffer finds. Both are left to queue 0, as indirect draws are. Admission. The driver caches a key at its second sight, or at its first when its shape was seen before and has no entry, so that a key that never comes back does not push a live entry out. A key the workers compile is consumed by queue 0 at its own draw, so the scan compiles keys on their first sight too. An entry inserted ahead is no sighting: it goes on probation, whatever the key's earlier sightings, since queue 0 may skip its draw. It lives in the draw cache beside the others, in its own list bounded by the cache's size, so probation entries push out only each other and never an admitted entry. A queue-0 draw that found a probation entry counts as a sighting under the same rule at the point where the driver sights an absent key, so only a draw it would consider caching sights it (no instanced tessellation draw, no indirect draw drawn on the CPU), once per draw (the entry itself does not count as its shape's entry); the sighting that admits the key promotes it into the cache's order, so a key is admitted at the execution that would have admitted it without compile-ahead. The sighting and the decision to insert queue 0's own compile are one step under the cache mutex (admitDrawKey), so a probation entry a worker inserts while queue 0 compiles the same key counts this execution's sighting as the absent key would have. A worker never inserts into an existing entry, and it builds its entry before it takes the cache mutex, so queue 0's lookups wait for no more than the map and list update. An insert builds the entry and its list node before it changes the cache, and traces and reserves the evictions that follow before it, so it either completes, evictions included, or leaves the map, the lists, the shapes and the variant accounting as they were. APS5_TRACE_DRAW_CACHE, whose eviction trace can still throw, keeps the workers out. Entries an insert evicts or replaces are released after the cache mutex is, so queue 0's lookups do not wait for their destruction. Speculative reads. A worker may read guest memory before queue 0 passes the waits that order it after CPU or GPU stores, and memory a skipped or divergent draw names may be unmapped meanwhile. Under a GuestMemory::SpeculativeReadScope, Read copies through the kernel (process_vm_readv on the own process, ReadProcessMemory on Windows) into the capture's private buffer: no C++ evaluation of the worker accesses guest memory, so a concurrent store is not a data race, and an unmapped page fails the copy instead of faulting. No flush hook runs, and every other guest memory access (compares, committed reads, stores, FlushGpuWrites) throws in that scope. The compile reads only the private copy, and the variant records exactly the words it was compiled from; queue 0 compares them with memory after its waits and flushes, so a capture holding stale or torn words is used only when memory then holds exactly those words. Without a working kernel copy no workers run. Lifetime. Every guest read of a job and its cache insertion are pinned to the job's draw. Queue 0 passes a draw before executing it and retires its submission when it ends (also on an exception); passing waits for pinned sections at or before that position, later pins are refused, and retiring drops the submission's queued jobs and scans and stops its scan; a scan checks for retirement before it allocates its records. So no worker reads guest memory for, or inserts an entry of, a draw queue 0 passed or a submission that retired, including draws inside a false COND_EXEC. The scan decodes ahead only with registered shaders: raw programs would be read from memory and NoteRawGraphicsDraw, which refuses later shader registration, must only follow a draw queue 0 executes. Submissions without a direct draw are not posted, nor is a submission that arrives while the scanner is still busy with an earlier one, so queue 0 copies a submission for the scanner only when the scanner can start on it. A debug probe or APS5_INSERT_COMPARE keeps the workers out. The print-once flags in DecodeState become atomics now that two threads decode. Driver::DrawAheadStatistics counts published records, records queue 0 used, compiled and dropped jobs (a draw whose decode fails ahead is dropped), and APS5_PROFILE_DRAW reports them per 10 s ([draw-ahead]). DrawCacheReuse counts queue 0's own admissions, data hits and inserts, so it turns compile-ahead off (DrawAhead::SetCompileAhead). The new test holds queue 0 behind a WAIT_REG_MEM and requires every draw behind it to be compiled ahead, its published record used, reused without misses and to render the pixels of the uncached path (debug probe), then to keep their entries through their second and third sight; a draw executed once, then compiled ahead inside a false COND_EXEC, stays on probation and is admitted at its next execution; two executions of an indirect draw the queue draws on the CPU do not promote the probation entry of a skipped draw with the same registers, and the direct draw is then admitted at its second execution, as without compile-ahead; mesh, instanced tessellation and indirect draws are not compiled ahead; a submission whose 1024 draws lie in a false COND_EXEC inserts nothing after it retired; a table read by the fragment shader, stored by the CPU while queue 0 waits or by WRITE_DATA before the draw, renders the stored value. It runs again under APS5_VERIFY_DRAW_ENTRIES, where queue 0 recompiles each hit and compares its capture and its compilation (SPIR-V module, variant, bindings, push constants) with the workers'. DrawAheadRaw requires that a raw draw the queue skips leaves shader registration open.
A draw that misses its recipe (every draw of an entry the workers just compiled) validated its shaders against the state, looked up or built the vertex input layout and built the pipeline stage mask and the resource content key on queue 0. These depend only on the decoded state and the compiled stages, so the draw-ahead worker computes them once (Graphics::PrepareDraw) and stores them on the entry, tied to the exact variants it compiled (weak references, so a released or replaced variant never matches). Queue 0 passes them to Graphics::Draw when its hit matched those variants, refreshed no stage data words, is not indirect and the stage count is unchanged; otherwise, and for rect lists and emulated barycentrics, it computes them as before. The prepared vertex input layout is shared with the draw like a retained one. Rotated and replaced entries keep the record. A speculative failure never surfaces. An exception in the scan, a draw's resolve or decode, or a worker's capture and compile means only that the draw has no precomputed result (for a scan exception, the rest of the submission has none): no record is published and no entry inserted. An exception in PrepareDraw drops only the prepared state: the compiled entry is still inserted and queue 0 prepares the draw itself. PrepareDraw's shader validation keeps its failure to itself: only queue 0 records one in the failures KnownValidationFailure reports, since the draw that fails it ahead may be one queue 0 skips or draws nothing for (a draw of zero instances returns before its shaders are validated). Queue 0 then performs the draw exactly as without draw-ahead, on current memory. This is not a silent fallback: that serial path is the authoritative implementation and throws on a genuine error, while what the speculation saw (stale, torn or not yet mapped memory, a context stack it cannot follow past a skipped packet, a vertex layout the draw never builds) is not an error of the draw. A guest read refused to a speculative reader, whose draw the queue has already passed or whose range is inaccessible, throws GuestMemory::SpeculativeReadRefused. The test renders, as the uncached path does, a draw whose buffer descriptor is invalid while the worker compiles (the capture records the invalid words) and repaired before the draw, a draw whose table is mapped only after the worker's read, a draw after a CONTEXT_STATE push in a skipped COND_EXEC (the scan executes it, fails on the next push and publishes nothing more), and an indexed draw of restart indices only whose vertex stride exceeds the device limit (its prepared layout fails, the draw builds none). An indexed draw of fetched vertices compiled ahead draws through its prepared vertex input layout and writes the pixels of the uncached path, and a zero-instance indexed draw whose vertex program exports no position, compiled and prepared ahead, draws nothing as without draw-ahead; a speculative read of an inaccessible range is refused. Each pass sets NUM_INSTANCES.
Every cache hit validates each stage's captured runs, and two of its checks scanned linearly: StorageTexture::AnyPendingOverlaps compared every run with every pending image, and AnyShadowedOverlaps took the shadow registry's mutex once per run. With compile-ahead almost every draw is a hit, and these checks were two thirds of a stage validation in ASTRO BOT's crash-site hub. AnyPendingOverlaps now asks the pending list's sorted, merged range index (PendingList::MayOverlap, the one PendingStorageOverlaps uses), which holds exactly the union of the pending images' guest ranges, so its answer is unchanged; images being flushed are still compared one by one. AnyShadowedOverlaps takes the registry mutex once for all runs. The unit shadow test and the store-over-untracked test check a list of runs that overlaps only through its last run, and one that does not overlap at all.
JJenkx
changed the title
JJenkx
force-pushed
the
validation/draw-ahead-760
branch
from
October 15, 2026 11:16
a6a868d to
bf8fd65
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Integrate mitul72's six-commit upstream #6188 draw-ahead preparation with main's now-default queue-0 draw pipeline (boykopovar#1201/boykopovar#1202), preserving authorship. Eleven linear commits on main
8276a1d0594313a5d037fd902957f6cacf943bdc; public headbf8fd65b8324c50896e7f603b471b3f07842c858.The integration retains the pipeline drain before color metadata passes and indirect draws. Queued commits keep their matched draw entry and compiled stages alive and consume its prepared record, falling back to normal preparation if the device changes. Submission completion is enqueued only after draw-ahead retirement, so idle cannot precede the last speculative read/publication.
Main's newer lazy shader-memory pages attempt a flush that speculative readers refuse. Speculative page captures now use the existing pinned, kernel-mediated
GuestMemory::Read; ordinary captures retain main's lazy path. Raw shader cache comparisons and fresh code reads now drain overlapping queued writes. The existing variant-validation drain is retained for captured descriptor/table ranges.The previous watched-snapshot fixture correction remains: shared-backed Linux direct allocations exercise watched imports; new CPU contents are checked after advancing the collect epoch. Driver-store checks retain immediate
MarkWrittenassertions. No unsafe import-policy override is used.Validation: Linux x86-64 GCC Release full runtime libraries and all selected test executables built. All 21 selected CTest cases passed with zero skips on Radeon RX 6650 XT / RADV Mesa 26.2.4 and lavapipe. Coverage includes actual rendered-pixel comparisons, prepared-record use, inline and pipelined draws, validation mode, cache eviction, retirement/pinned reads, raw-code and captured-table writes, snapshots, PM4 failures, metadata, draw-write ranges, pipelined DMA/fills and predication. Conventions and whitespace pass.
Controls establish that the regressions detect the integration errors:
All temporary faults were removed and the full runtime rebuilt before the final positive runs. The private validation tree differs from this public head only by the previously disclosed SPIRV-Tools teardown-repair gitlink. That dependency repair is excluded; these runs do not independently validate the official pin. NVIDIA and Windows retests, full-suite results and title gameplay validation remain pending.
Completed title comparison on this exact base: main
8276a1dversus this integration, same Linux/Radeon host and idle Astro Crash Site scene, first-present-relative300:60trace. Main: 230 presents /60.200989 s = 3.821 FPS, p50 263.672 ms, GPU busy 180.453 ms/present. Integration: 321 /60.022567 s = 5.348 FPS, p50 177.557 ms, GPU busy 152.091 ms/present. +39.98% throughput in one pair, not repeated statistical evidence or broad gameplay coverage. Main has advanced since this measured base.Both used initially byte-identical independent warm-cache copies (1946 shader hits, zero misses/writes, 1659 resource-plan hits), the same private dependency repair, inactive windows, no input, and no concurrent builds/tests. Both traces report no dropped events. Compositor images at 325/415 s show the same idle camera/HUD with different animation poses; both runs ended at their scheduled 425 s watchdog, not normal title shutdown. The committer CPU thread was not registered, so those spans are absent despite no overflow; follow-up upstream #6341 addresses that gap. Inclusive phase times are not global CPU work.
Full timing evidence and protocol is now linked in the upstream Astro discussion. The queued raw-code read fix has been split out as upstream #6326. This integration draft retains the parent feature's authorship and is not a duplicate upstream feature submission. The older
042eb7c8result (3.388 → 4.869 FPS, +43.7%) remains separately archived; results on different bases are not pooled.0eb0d8903b0b2eaa9317d74edadbe8353851e693)