Repository navigation
perf(agc): keep draw snapshots in resizable BAR memory - #9712
panicDa3m0n wants to merge 10 commits into
Conversation
|
Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.
|
|
Traces of the four runs whose summary lines are in the description, in run order (main, this PR, main, this PR). GitHub did not take the .json.gz files as attachments here, so they are stored unchanged in one zip; trace_digest.py and performance_compare.py read them as they are.
|
|
Preparing a Windows 007 cross-test of b5b8d4e, not a duplicate implementation. Source-only integration merges cleanly on current main 4777cac with our existing bring-up stack; conventions pass and original authorship is retained: UNBUILT/UNTESTED here so far; our pinned same-build input-policy control is running first. Planned checks are the same native draw/vertex/index/snapshot selection on the base and candidate, actual Khronos layer, no shader cache, plus the disabled-BAR control before a title comparison. Hardware is Windows 11 / RTX 5070 Ti 617.42 / 7800X3D / 32 GiB. One concern from your stated limitation: our 007 runs are already memory-constrained. Before I stress the title, are you planning the Buffer/slab host-memory fallback in this branch? I will not attribute an allocation failure under VRAM pressure to the title itself. No speed claim yet; raw title code/captures stay local. AI-assisted: Codex. |
|
One source-review question before my native run: HostReadableInput takes a second GuestMemory::Read for BAR-backed index buffers, after CopyDrawInput already populated the draw snapshot. If the guest writes those indices between the copy and the highest-index/restart-table scan, the CPU can scan different indices from the snapshot bound for this draw. The host-memory path scans copy.buffer itself, so it preserves that relationship. Could we retain a host shadow of the exact index snapshot (or read back the snapshot for these scans), rather than re-read mutable guest memory? A focused regression could copy indices, mutate the guest source, then check that the derived highest index / restart table still match the bound snapshot. This is a source-review concern; I have not reproduced it in a title or run that regression yet. AI-assisted: Codex. |
|
Thanks, both points are right.
I will push both today with their tests, the validation runs on |
MarcZX10
left a comment
MarcZX10
left a comment
There was a problem hiding this comment.
I reviewed exact head b5b8d4e. Two blocking issues remain in this head: the BAR index path scans guest memory again after creating the GPU-bound snapshot, and BAR-backed Buffer allocations have no host-memory fallback when the selected heap cannot satisfy them. Both were already raised in the discussion, and the author acknowledged both with a plan to address them; those fixes are not present in the reviewed head. Please keep the index snapshot and all derived index metadata from the same byte copy, and add the agreed allocation fallback before treating this as ready. The branch is also 15 commits behind current upstream/main, so update it and rerun the focused draw/index tests plus the documented main-vs-change validation run on the updated tree. The current GitHub check list contains successful conventions/intake/label checks but no product build or test job; I could not run a local build because there is no configured build tree and the filesystem is full. I have not independently reproduced the performance figures. Please also coordinate the already-disclosed interaction with open #8756 and confirm the landing/rebase order.
2befd75 to
8231521
Compare
|
Rebased onto current @Scramblrrrr, both of your points are fixed:
@MarcZX10, both blocking points are fixed in those two commits. On the updated tree:
On the order with #8756: whichever is merged first stays as it is, and the other rebases its neighbouring lines in The description has the new performance numbers for the head, measured against |
|
Traces of the six runs in the description (rounds 1, 2 and 3 there), each with its aps5-traces-pr9712-round1.zip
|
There was a problem hiding this comment.
The PR contains title-specific Astro Bot performance measurements, so I am not treating it as a title-agnostic change. However, the baseline uses main 3ca07d0, not current main, and current main has since changed AGC draw code. Please rerun the comparison on current main so the reported performance evidence can be evaluated against the current code.
The branch is 19 commits behind main. Please update it and rerun the focused snapshot, draw-input and buffer-pool tests plus the main-vs-change performance run on the updated tree. Current hosted checks show conventions and labels passing, with no product build/test run. The author-reported AGC tests and Astro Bot A/B results have not been independently reproduced here.
The review also found a portability issue in Context::MappedBufferProperties: snapshot memory selection uses device-wide physical properties, but the buffer’s VkMemoryRequirements::memoryTypeBits may exclude the selected BAR type while allowing a host-visible type. In that case MemoryType(mask, flags) throws rather than falling back to HostBufferProperties. Please add a regression where the BAR type is excluded but a compatible host type remains. I syntax-checked the relevant production and test sources, and git diff --check and current-main conventions pass. The current hosted checks have no product build/test run.
|
Follow-up: I reproduced the per-buffer compatibility case in a small executable built against the exact PR-head Context.hpp. The physical properties exposed both a host-visible/coherent host type and a host-visible/coherent/device-local BAR type, while the buffer memoryTypeBits mask contained only the host type. Calling MappedBufferProperties(mask, MappedDeviceProperties, 4096) throws “AGC graphics: required Vulkan memory type is unavailable” instead of selecting the compatible host type. |
|
Thanks for reproducing it. I'll make |
Draw snapshots of read-only storage buffers and the vertex and index copies of draws now live in device-local host-visible memory when the device exposes it through a resizable BAR (a heap of at least 4 GiB), and in host memory otherwise. The CPU still fills them with memcpy, but the GPU reads them from VRAM instead of over PCIe. The CPU never reads that memory back: refreshing a stale snapshot rewrites its changed blocks without comparing them, and the highest index of a copied index buffer is scanned from a host copy. APS5_NO_BAR_SNAPSHOTS keeps the previous host memory. On an RTX 4070 Ti SUPER (driver 616.64, PCIe 4.0 x16), shaders read host memory at 6-8 GB/s sequentially and 1.4 GB/s scattered, against 650 and 92 GB/s from VRAM. AI-assisted: yes.
The CPU scans an index copy for its highest index and, for mesh strips, its restart table. With the copy in device-local memory the scans read the guest's index buffer a second time, so a store between the copy and the scan gave a highest index or restart table that did not match the bound copy. Index copies now stay in host memory and the scans read the copy itself; vertex copies stay in BAR memory. agc_draw_input_memory copies indices, stores to the guest source, and checks that the copy is in host memory and keeps its highest index.
A Buffer asked for device-local host-visible memory now takes host memory when the reported budget of the device-local heap has no room for it, as DeviceMemoryType places device buffers. It also takes host memory when its allocation, or the allocation of its slab block, is still refused after the pool trim and the cache reclaim, as device buffers and images do through RefusedDeviceMemoryFallback. Properties() then reports host memory, so refreshing such a snapshot compares its blocks again. MappedDevicePlacement in tests/BufferPool.cpp checks both placements for a buffer and for a slab.
…ry types is mapped device-local MappedBufferProperties looked up the BAR type with MemoryType(mask, flags), which throws when the buffer's memoryTypeBits exclude it, and returned the flags unchanged without VK_EXT_memory_budget, so BufferMemoryType threw the same way. The buffer's own memory types are checked first now: a mask with no mapped device-local type takes HostBufferProperties, with a rate-limited line. MappedDevicePlacement checks a buffer and a slab whose mask excludes the BAR type and keeps the host type, with and without the budget extension.
|
I’ve noticed that asking you to hold this PR was unnecessary. It’s fine to keep submitting PRs; maintainers will handle them in due course. |
8231521 to
b8663c5
Compare
|
Rebased onto current @MarcZX10, thanks for the reproduction. On the updated tree:
|
|
Traces of the six runs in the description, one zip per round, each with its aps5-traces-pr9712-v4-round1.zip
|
…ativeFrame main The line carried the first BAR snapshot port with HostReadableInput, which read index bytes back from the guest after the copy. It now has the reviewed pull request: index copies in host memory scanned from the copy the draw binds, MatchingMemoryType and the host fallbacks of mapped buffers and slabs. The stack's snapshot shadow stays: storage snapshots still fill through FillDrawSnapshot, and a device-local snapshot with a shadow is still compared on refresh.
|
@savvaniss, this pull request also lists only NVIDIA. If you have time for a run like the one you did on #8756, your RX 9070 XT would cover AMD here. What would help most:
No title run is needed. Thanks either way. |
|
@panicDa3m0n thanks for the TakeSlot fix. I ran a same-machine pair of head 539e1d1 against its main base (01cd4a4) on an RTX 4090 (resizable BAR on, Linux, driver 610.57.04), twice on 26 Oct and again twice on a quiet box on 27 Oct (no builds or other jobs running). Both times the PR is slower on the title screen and in the menu, by the same amount:
That is about +2 ms (6-8%) per frame, well outside the control spread. Intro and level are equal, peak VRAM is 11.0 / 11.0 GB vs 10.8 / 10.3 GB on main, and there are no faults. So on this NVIDIA card the BAR path costs time where it should save it, while @BunnyFl's and @QDarkRex's AMD numbers show it saving ~1.4 ms of GPU time. Could you look at what the title/menu frames do differently on the BAR path (for example many small snapshots per frame, or reads back from BAR memory)? If the gain is AMD-specific, an option is to enable it only where it measures faster, with the default staying as on main. A same-machine main-vs-head run of the title screen on your side would help too. |
MarcZX10
left a comment
MarcZX10
left a comment
There was a problem hiding this comment.
I re-reviewed the latest head 4e93e34d8e0929f29032c4a81712e4dd899e65f7. The earlier code concerns about keeping index metadata with its copy, memoryTypeBits fallback, and fallback slab indexing have been addressed in later changes and the reported agc_buffer_pool regression checks are useful. The reported AMD measurements show 1.4–1.5 ms lower GPU-busy time on two Radeon systems, with little frame-time change on the CPU-bound RX 9060 XT run.
A new blocking result is the RTX 4090 A/B reported against the same main base: title/menu p50 worsened by about 2 ms (6–8%) in repeated quiet runs, while intro/level were unchanged. The current implementation enables BAR snapshots for any qualifying device, so the PR's current evidence shows a material regression on NVIDIA alongside the AMD gain. Please investigate the title/menu path and either address the regression or narrow the default enablement to hardware where the benefit is demonstrated; then rerun the same-machine main/head comparison and focused tests on the resolved head. GitHub currently reports this PR DIRTY / non-mergeable against current main, and the current head is 1,319 commits behind. The hosted checks shown are conventions and labels only; no product build/test is reported here. I did not run a local GPU build or performance test.
|
@panicDa3m0n a nudge on this one. Two things are open:
Once that's in, we'll run one more 4090 pair on the new head and merge on a pass with green CI. |
|
@oneandonlydean, the title-screen A/B on AMD that the NVIDIA result calls for. On an RX 9060 XT the BAR path does not slow the title screen: in all three rounds it is 0.3–0.5 ms per frame faster than the host-memory path. So the cost you see on the RTX 4090 looks NVIDIA-specific, at least on this scene.
AI-assisted: yes (Claude Code). The runs were made on my machine. |
|
@panicDa3m0n #15570 builds on this PR. It places the other CPU-written, shader-read driver buffers through In case it helps here, from Ghost of Tsushima (PPSA02225) on an RTX 4070 Ti with a GPU profile and both placements together: GPU time per present went from 227 to 180 ms, and its instanced decal draws from 88 to 48 ms. Your snapshot refresh without the compare also mattered there: without it the committer threads spent most of their time reading BAR memory back, at about 95 MB/s. AI-assisted: yes (Claude Code) |
|
NVIDIA data on Windows, as asked in the review above. On an RTX 5090 the BAR path does not slow the title screen: over three rounds the frame p50 is 56.8 ms with it and 57.1 ms without it, and GPU busy per present is 41.3 ms against 41.6 ms. At Crash Site it is faster in every round, by 3.9-4.8 ms of frame p50 and 10.4-10.8 ms of GPU busy per present. So the +2 ms on the RTX 4090 does not show on this NVIDIA/Windows machine. This data cannot tell whether it comes from Linux or from that machine.
Title screen:
Crash Site:
AI-assisted: yes (Claude Code) |
XairRrR
left a comment
XairRrR
left a comment
There was a problem hiding this comment.
Reviewed — changes requested
This pull request does not merge cleanly into main (1c8c0ef): head 4e93e34 conflicts in
core/libs/prx/libSceAgcDriver/CMakeLists.txtcore/libs/prx/libSceAgcDriver/Graphics/src/BufferPool.cppcore/libs/prx/libSceAgcDriver/Graphics/src/Resources.cppcore/libs/prx/libSceAgcDriver/tests/BufferPool.cpp
Please rebase it onto current main. I will review the change once it merges cleanly.
Device-local snapshot buffers skipped the byte compare to avoid reading resizable BAR memory back, so every CPU store of identical bytes copied the draw input again. Keep a host copy of each device-local storage snapshot, compare against it, and write changed blocks to both; BAR is never read.
|
I checked head The resolutions preserve main's retained-part memory accounting and staging-headroom gate, keep your selected slab memory type and mapped-memory fallback, unify the mock memory-type mask fields, retain both sets of tests, and register Linux 7.2.11-arch1-1, GCC16.2.1, RX6650XT8GB with full8GB PCI BAR, RADV Mesa26.2.6-arch1.1:
Logs, exact pins, patches and per-file hashes (27,619 bytes; uploaded/downloaded SHA-256 verified): This is scoped compatibility/regression evidence, not title performance evidence, a new negative-control run, or a claim that the original branch is rebased/mergeable. No native game was launched with this integration. The larger live-buffer allocations in my separate plain-main Trek memory diagnostic also mean I will measure memory pressure before assuming additional BAR placement helps that title. |
✅ AMD GPU test: passedPR head
Unstable on main (1, these also failed in recent main runs on this machine; not counted against this PR)Failing on main too (8, not counted against this PR)Validated but not compared (2, not in the main baseline's validation pass)Validation IDs already present on main (4 IDs in 556 tests)Skipped (67)ReproduceAutomated build and test on a contributor's AMD machine; it does not replace upstream CI. A new comment is posted for each new push. Full logs. |
✅ NVIDIA GPU test: passedPR head
Unstable on main (1, these also failed in recent main runs on this machine; not counted against this PR)Failing on main too (3, not counted against this PR)Validated but not compared (1, not in the main baseline's validation pass)Validation IDs already present on main (9 IDs in 550 tests)Skipped (55)ReproduceAutomated build and test on a contributor's NVIDIA machine; it does not replace upstream CI. A new comment is posted for each new push. Full logs. |
|
Hi @panicDa3m0n, thanks for this. To get it merged we pushed a merge of current
Thanks also to @JJenkx, whose separate integration of this PR onto We also pushed one small fix on top, in
Tested on Linux with an RTX 4090: |
The RTX 4090 title/menu regression does not reproduce on the current head. Paired same-machine A/B, RTX 4090, empty NVIDIA shader cache per arm: main 0540ef0 vs head 601e9e9. Title: 31.1 vs 31.2 fps, p50 32.0 vs 31.9 ms, p99 35.1 vs 34.5 ms. Menu: 30.3 vs 30.2 fps, p50 32.7 vs 32.6 ms, p99 39.4 vs 39.8 ms. Differences are at most 0.6 ms, within run-to-run noise, with no 2 ms / 6-8% slowdown. The PR is also mergeable against current main now.
What
Summary. Draw snapshots of read-only storage buffers and vertex copies go to resizable BAR memory when the device exposes one, and stay in host memory otherwise or with
APS5_NO_BAR_SNAPSHOTS=1; index copies stay in host memory. On an RTX 4070 Ti SUPER at Astro Bot's Crash Site, three alternating rounds againstmain8499ea5 give a frame p50 of 152.8-160.3 ms instead of 174.5-174.6 ms, and GPU time per present of 91.6-96.4 ms instead of about 152 ms. New tests cover the memory choice, the index copies and the mapped-buffer fallbacks, and each fails on a deliberate mistake.Draw snapshots of read-only storage buffers and the vertex copies of draws now live in device-local host-visible memory when the device exposes a resizable BAR over its device-local heap (a host-visible device-local type on a heap of at least 4 GiB), and in host memory otherwise. The CPU still fills them with
memcpy, but the GPU reads them from VRAM instead of over PCIe. Index copies stay in host memory: the CPU scans them for the highest index and the mesh restart table, and those scans read the same copy the draw binds.The CPU never reads BAR memory back on the draw path: refreshing a stale snapshot rewrites its changed blocks without comparing them.
APS5_NO_BAR_SNAPSHOTS=1keeps the snapshots and vertex copies in host memory as before.A mapped device-local
Buffertakes host memory in three cases:memoryTypeBitsis mapped device-local, since Vulkan reports the supported types per resource and they may exclude the BAR type the device lists;DeviceMemoryTypeplaces device buffers;RefusedDeviceMemoryFallback.Properties()then reports host memory, so refreshing such a snapshot compares its blocks again. The kept snapshots stay within the existing budgets: 1 GiB for storage snapshots and 1 GiB for draw inputs.Why: on an RTX 4070 Ti SUPER (driver 616.64, PCIe 4.0 x16) shaders read host memory at 6-8 GB/s sequentially and 1.4 GB/s scattered, against 650 and 92 GB/s from VRAM.
Tests:
snapshotMemoryTestsintests/Graphics.cppchecks the choice of memory:agc_draw_input_memorycopies indices, stores to the guest source, and checks that the copy is in host memory and keeps its highest index.MappedDevicePlacementintests/BufferPool.cppplaces a mapped device-local buffer and a slab within the budget, over it, after a refused allocation, and with the BAR type excluded from theirmemoryTypeBits, with and withoutVK_EXT_memory_budget.Related:
PrepareDrawBindings, so the second to land needs a small rebase.Context::HeapFitsfor the budget check thatDeviceMemoryTypeandMappedBufferPropertiesboth write inline, and takes a host type only when its heap has room. Whichever of the two lands second can use it inMappedBufferPropertiesandRefusedMappedMemoryFallback.CopyDrawInputas this pull request: this one picks the copy's memory, perf(agc): copy only the vertex records a draw fetches #9875 the bytes it reads. Merged together they conflict there, inCMakeLists.txtand intests/Recorder.cpp, and the second to land keeps both.add_library_test. If it lands first,agc_draw_input_memorymoves to that helper and the Depends line below names it.verifySnapshotscheck reads snapshots back on the CPU. With this change that read is uncached on device memory, slower but still correct.Tested
Windows 11 (10.0.26200), NVIDIA GeForce RTX 4070 Ti SUPER (driver 616.64, PCIe 4.0 x16), i7-11700K, 32 GB; Astro Bot (PPSA21567, 01.019.000) at Crash Site with its default settings, 3840x2160.
mainbelow is 8499ea5, the base of this PR.snapshotMemoryTests, without the heap-size check ofSnapshotMemoryProperties:a 256 MiB BAR window was taken. This ran on b5b8d4e, which has the same check and test.agc_draw_input_memory, with index copies in snapshot memory:an index copy left host memory, so its highest index would be scanned from the guest instead of the copy.agc_buffer_pool(MappedDevicePlacement):a mapped device-local buffer over the device-local budget was not placed in host memory;Buffer: the refused allocation throws (vkAllocateMemory buffer ... Vulkan result -2);vkAllocateMemory buffer slab ... Vulkan result -2);a mapped device-local slab whose block exceeds the device-local budget was not placed in host memory;VK_EXT_memory_budget: the buffer whose mask excludes the BAR type throws (AGC graphics: required Vulkan memory type is unavailable);MemoryTypelookup in place of the mask check: the same throw with the budget extension.ctest -R ^agc_on this PR's head (895 tests): all pass except four, and three skip on this device (sparse host import, dma-buf import, stencil reference export).agc_driver_sample_order,agc_driver_sample_order_grid_referenceandagc_driver_sample_order_recipe_depth_locationsfail the same way onmain's build at 3ca07d0 on this machine ("the slopes did not leave depth both at and below the clear value").agc_compute_pipeline_capture_optional_byte_captimed out in the parallel run: it writes 256 MiB to this machine's slow temporary folder.VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation,ANYPS5_NO_SHADER_CACHE=1,ANYPS5_REQUIRE_VULKAN=1andctest -V:agc_driver_draw_*,agc_driver_vertex_*,agc_driver_rect_list_*,agc_driver_mesh_index_cache,agc_driver_graphics,agc_driver_zero_stride_vertex_fetch), plusagc_draw_input_memoryon the head.main's build at 8499ea5 (26 tests), on this PR's head (27) and on the head withAPS5_NO_BAR_SNAPSHOTS=1(27).VUID-error or awas not found but was requested by env var VK_INSTANCE_LAYERSline.Undefined-Value-ShaderOutputNotConsumed-DynamicRenderingand 1Undefined-Value-ShaderOutputNotConsumed.APS5_TRACE_SECONDS=150:5and no otherAPS5_TRACE_*orAPS5_PROFILE_*switch, with warm shader caches. Builds alternatemainand this PR in one session.Start-Process -RedirectStandardOutput), the same for all runs.APS5_HOST_IMPORT_MIB=13312. On this 32 GB machine the default host import budget is 8149 MiB, below what Crash Site's largest heaps need, so they fall back to copies. fix(agc): keep half the host heap out of the host import budget #6472 set that budget; fix(agc): keep at most 4 GiB of the host heap from the host import budget #7844 changes it.performance_compare.pylistswindow.first_presentas the only configuration difference. The absolute times include the trace's own cost.Over the three rounds, frame p50 goes from 174.5-174.6 ms on
mainto 152.8-160.3 ms with this PR (−19.2, −14.3 and −21.7 ms per round), and GPU busy time per present from 152.0-152.6 ms to 91.6-96.4 ms. Index copies in host memory account for about 12-15 ms of GPU time per present against the first version of this PR, which kept them in BAR memory (83.5-88.1 ms against 98.3-99.7 ms, both on d6eafa5).8499ea52eb8663c5148499ea52eb8663c5148499ea52eb8663c514Summary lines, in run order:
{"commit":"8499ea52e571","dirty":false,"env":{"APS5_HOST_IMPORT_MIB":"13312","APS5_TRACE_DIR":"<path>","APS5_TRACE_SECONDS":"150:5"},"gpu":"NVIDIA GeForce RTX 4070 Ti SUPER","driver":"NVIDIA 616.64","os":"Windows 10.0.26200","cpu":"11th Gen Intel(R) Core(TM) i7-11700K @ 3.60GHz","cpu_threads":16,"memory_mib":32599,"title_id":"PPSA21567","title":"ASTRO BOT","content_version":"01.019.000","resolution":"3840x2160","window":{"spec":"seconds 150:5","first_present":2938,"ms":5090.897},"presents":29,"frame_ms":{"p50":174.458,"p90":183.441,"p99":189.159,"max":189.159},"gpu_busy_ms_per_present":152.250,"worker_busy_pct":{"queue worker 0x0":98.786,"queue worker 0x40":97.544,"queue worker 0x48":94.148,"queue worker 0x46":10.122},"top_phases_ms_per_frame":{"Driver.Packet":527.312,"Driver.Packet.memory_wait":302.260,"Driver.Packet.draw":91.258,"Driver.Draw":90.090,"Graphics.Draw":83.514,"Graphics.Draw.draw_and_resource_release":66.774,"Driver.Packet.prepare_memory":60.030,"Driver.Draw.shader_compile_and_link":55.787},"trace_dropped":false,"trace":"aps5-trace-8499ea52e571-25288.json.gz"} {"commit":"b8663c514ef5","dirty":false,"env":{"APS5_HOST_IMPORT_MIB":"13312","APS5_TRACE_DIR":"<path>","APS5_TRACE_SECONDS":"150:5"},"gpu":"NVIDIA GeForce RTX 4070 Ti SUPER","driver":"NVIDIA 616.64","os":"Windows 10.0.26200","cpu":"11th Gen Intel(R) Core(TM) i7-11700K @ 3.60GHz","cpu_threads":16,"memory_mib":32599,"title_id":"PPSA21567","title":"ASTRO BOT","content_version":"01.019.000","resolution":"3840x2160","window":{"spec":"seconds 150:5","first_present":2753,"ms":5110.320},"presents":33,"frame_ms":{"p50":155.333,"p90":164.003,"p99":165.208,"max":165.208},"gpu_busy_ms_per_present":96.173,"worker_busy_pct":{"queue worker 0x0":98.684,"queue worker 0x40":97.781,"queue worker 0x48":94.318,"queue worker 0x46":11.173},"top_phases_ms_per_frame":{"Driver.Packet":467.172,"Driver.Packet.memory_wait":275.849,"Driver.Packet.draw":87.205,"Graphics.Draw":86.542,"Driver.Draw":86.075,"Graphics.Draw.draw_and_resource_release":69.994,"Driver.Draw.shader_compile_and_link":52.713,"Driver.Packet.dispatch":43.352},"trace_dropped":false,"trace":"aps5-trace-b8663c514ef5-26736.json.gz"} {"commit":"8499ea52e571","dirty":false,"env":{"APS5_HOST_IMPORT_MIB":"13312","APS5_TRACE_DIR":"<path>","APS5_TRACE_SECONDS":"150:5"},"gpu":"NVIDIA GeForce RTX 4070 Ti SUPER","driver":"NVIDIA 616.64","os":"Windows 10.0.26200","cpu":"11th Gen Intel(R) Core(TM) i7-11700K @ 3.60GHz","cpu_threads":16,"memory_mib":32599,"title_id":"PPSA21567","title":"ASTRO BOT","content_version":"01.019.000","resolution":"3840x2160","window":{"spec":"seconds 150:5","first_present":2660,"ms":5017.386},"presents":29,"frame_ms":{"p50":174.619,"p90":182.895,"p99":185.561,"max":185.561},"gpu_busy_ms_per_present":152.577,"worker_busy_pct":{"queue worker 0x0":98.804,"queue worker 0x40":97.803,"queue worker 0x48":93.909,"queue worker 0x46":9.464},"top_phases_ms_per_frame":{"Driver.Packet":518.590,"Driver.Packet.memory_wait":298.946,"Driver.Packet.draw":86.549,"Driver.Draw":85.458,"Graphics.Draw":81.895,"Driver.Packet.prepare_memory":65.500,"Graphics.Draw.draw_and_resource_release":65.075,"Driver.Draw.shader_compile_and_link":52.003},"trace_dropped":false,"trace":"aps5-trace-8499ea52e571-12684.json.gz"} {"commit":"b8663c514ef5","dirty":false,"env":{"APS5_HOST_IMPORT_MIB":"13312","APS5_TRACE_DIR":"<path>","APS5_TRACE_SECONDS":"150:5"},"gpu":"NVIDIA GeForce RTX 4070 Ti SUPER","driver":"NVIDIA 616.64","os":"Windows 10.0.26200","cpu":"11th Gen Intel(R) Core(TM) i7-11700K @ 3.60GHz","cpu_threads":16,"memory_mib":32599,"title_id":"PPSA21567","title":"ASTRO BOT","content_version":"01.019.000","resolution":"3840x2160","window":{"spec":"seconds 150:5","first_present":4310,"ms":5118.256},"presents":32,"frame_ms":{"p50":160.333,"p90":167.439,"p99":172.874,"max":172.874},"gpu_busy_ms_per_present":91.612,"worker_busy_pct":{"queue worker 0x0":98.652,"queue worker 0x40":97.934,"queue worker 0x48":94.517,"queue worker 0x46":9.894},"top_phases_ms_per_frame":{"Driver.Packet":481.041,"Driver.Packet.memory_wait":286.595,"Driver.Packet.draw":92.170,"Graphics.Draw":91.690,"Driver.Draw":90.991,"Graphics.Draw.draw_and_resource_release":75.211,"Driver.Draw.shader_compile_and_link":55.081,"Shader.DrawStage":44.516},"trace_dropped":false,"trace":"aps5-trace-b8663c514ef5-3760.json.gz"} {"commit":"8499ea52e571","dirty":false,"env":{"APS5_HOST_IMPORT_MIB":"13312","APS5_TRACE_DIR":"<path>","APS5_TRACE_SECONDS":"150:5"},"gpu":"NVIDIA GeForce RTX 4070 Ti SUPER","driver":"NVIDIA 616.64","os":"Windows 10.0.26200","cpu":"11th Gen Intel(R) Core(TM) i7-11700K @ 3.60GHz","cpu_threads":16,"memory_mib":32599,"title_id":"PPSA21567","title":"ASTRO BOT","content_version":"01.019.000","resolution":"3840x2160","window":{"spec":"seconds 150:5","first_present":2658,"ms":5045.459},"presents":29,"frame_ms":{"p50":174.497,"p90":183.493,"p99":184.907,"max":184.907},"gpu_busy_ms_per_present":151.991,"worker_busy_pct":{"queue worker 0x0":98.787,"queue worker 0x40":97.445,"queue worker 0x48":94.411,"queue worker 0x46":9.031},"top_phases_ms_per_frame":{"Driver.Packet":520.917,"Driver.Packet.memory_wait":298.770,"Driver.Packet.draw":91.749,"Driver.Draw":90.597,"Graphics.Draw":83.471,"Graphics.Draw.draw_and_resource_release":66.106,"Driver.Packet.prepare_memory":55.606,"Driver.Draw.shader_compile_and_link":53.222},"trace_dropped":false,"trace":"aps5-trace-8499ea52e571-25028.json.gz"} {"commit":"b8663c514ef5","dirty":false,"env":{"APS5_HOST_IMPORT_MIB":"13312","APS5_TRACE_DIR":"<path>","APS5_TRACE_SECONDS":"150:5"},"gpu":"NVIDIA GeForce RTX 4070 Ti SUPER","driver":"NVIDIA 616.64","os":"Windows 10.0.26200","cpu":"11th Gen Intel(R) Core(TM) i7-11700K @ 3.60GHz","cpu_threads":16,"memory_mib":32599,"title_id":"PPSA21567","title":"ASTRO BOT","content_version":"01.019.000","resolution":"3840x2160","window":{"spec":"seconds 150:5","first_present":2779,"ms":5089.771},"presents":33,"frame_ms":{"p50":152.754,"p90":161.061,"p99":166.455,"max":166.455},"gpu_busy_ms_per_present":96.415,"worker_busy_pct":{"queue worker 0x0":98.647,"queue worker 0x40":97.920,"queue worker 0x48":94.465,"queue worker 0x46":11.271},"top_phases_ms_per_frame":{"Driver.Packet":465.854,"Driver.Packet.memory_wait":274.814,"Driver.Packet.draw":87.031,"Graphics.Draw":86.515,"Driver.Draw":85.879,"Graphics.Draw.draw_and_resource_release":70.117,"Driver.Draw.shader_compile_and_link":51.299,"Driver.Packet.dispatch":43.309},"trace_dropped":false,"trace":"aps5-trace-b8663c514ef5-27560.json.gz"}performance_compare.py, round 1
Performance comparison
Configuration differences
Frame and GPU times (ms)
Top phases (ms per frame)
performance_compare.py, round 2
Performance comparison
Configuration differences
Frame and GPU times (ms)
Top phases (ms per frame)
performance_compare.py, round 3
Performance comparison
Configuration differences
Frame and GPU times (ms)
Top phases (ms per frame)
trace_digest.py, run 1 (main)
seconds 150:5 from present 2938: 29 presents, frame p50 174.5 / p90 183.4 / p99 189.2 / max 189.2 ms, GPU busy 152.2 ms per present
NVIDIA GeForce RTX 4070 Ti SUPER, NVIDIA 616.64, Windows 10.0.26200, 3840x2160, commit 8499ea5, dropped: no
{"commit":"8499ea52e571","dirty":false,"env":{"APS5_HOST_IMPORT_MIB":"13312","APS5_TRACE_DIR":"<path>","APS5_TRACE_SECONDS":"150:5"},"gpu":"NVIDIA GeForce RTX 4070 Ti SUPER","driver":"NVIDIA 616.64","os":"Windows 10.0.26200","cpu":"11th Gen Intel(R) Core(TM) i7-11700K @ 3.60GHz","cpu_threads":16,"memory_mib":32599,"title_id":"PPSA21567","title":"ASTRO BOT","content_version":"01.019.000","resolution":"3840x2160","window":{"spec":"seconds 150:5","first_present":2938,"ms":5090.897},"presents":29,"frame_ms":{"p50":174.458,"p90":183.441,"p99":189.159,"max":189.159},"gpu_busy_ms_per_present":152.25,"worker_busy_pct":{"queue worker 0x0":98.786,"queue worker 0x40":97.544,"queue worker 0x48":94.148,"queue worker 0x46":10.122},"top_phases_ms_per_frame":{"Driver.Packet":527.312,"Driver.Packet.memory_wait":302.26,"Driver.Packet.draw":91.258,"Driver.Draw":90.09,"Graphics.Draw":83.514,"Graphics.Draw.draw_and_resource_release":66.774,"Driver.Packet.prepare_memory":60.03,"Driver.Draw.shader_compile_and_link":55.787},"trace_dropped":false,"trace":"aps5-trace-8499ea52e571-25288.json.gz"}Slowest frames (largest slices inside each, inclusive)
Threads (ms per frame)
Largest slices (ms per frame, inclusive, all threads): Driver.Packet 506.8, Driver.Packet.memory_wait 302.3, Driver.WorkerWait 156.6, batch 152.1, Driver.WorkerWait.dequeue_order 144.8, Driver.Packet.draw 78.7, Driver.Draw 77.5, Graphics.Draw 70.5, Driver.Packet.prepare_memory 58.7, Graphics.Draw.draw_and_resource_release 56.9
Longest waits
Largest GPU batches (GPU ranges inside, and the CPU slices that recorded each, inclusive)
trace_digest.py, run 2 (this PR)
seconds 150:5 from present 2753: 33 presents, frame p50 155.3 / p90 164.0 / p99 165.2 / max 165.2 ms, GPU busy 96.2 ms per present
NVIDIA GeForce RTX 4070 Ti SUPER, NVIDIA 616.64, Windows 10.0.26200, 3840x2160, commit b8663c5, dropped: no
{"commit":"b8663c514ef5","dirty":false,"env":{"APS5_HOST_IMPORT_MIB":"13312","APS5_TRACE_DIR":"<path>","APS5_TRACE_SECONDS":"150:5"},"gpu":"NVIDIA GeForce RTX 4070 Ti SUPER","driver":"NVIDIA 616.64","os":"Windows 10.0.26200","cpu":"11th Gen Intel(R) Core(TM) i7-11700K @ 3.60GHz","cpu_threads":16,"memory_mib":32599,"title_id":"PPSA21567","title":"ASTRO BOT","content_version":"01.019.000","resolution":"3840x2160","window":{"spec":"seconds 150:5","first_present":2753,"ms":5110.32},"presents":33,"frame_ms":{"p50":155.333,"p90":164.003,"p99":165.208,"max":165.208},"gpu_busy_ms_per_present":96.173,"worker_busy_pct":{"queue worker 0x0":98.684,"queue worker 0x40":97.781,"queue worker 0x48":94.318,"queue worker 0x46":11.173},"top_phases_ms_per_frame":{"Driver.Packet":467.172,"Driver.Packet.memory_wait":275.849,"Driver.Packet.draw":87.205,"Graphics.Draw":86.542,"Driver.Draw":86.075,"Graphics.Draw.draw_and_resource_release":69.994,"Driver.Draw.shader_compile_and_link":52.713,"Driver.Packet.dispatch":43.352},"trace_dropped":false,"trace":"aps5-trace-b8663c514ef5-26736.json.gz"}Slowest frames (largest slices inside each, inclusive)
Threads (ms per frame)
Largest slices (ms per frame, inclusive, all threads): Driver.Packet 446.9, Driver.Packet.memory_wait 275.8, Driver.WorkerWait 137.4, Driver.WorkerWait.dequeue_order 128.4, batch 96.2, Driver.Packet.draw 74.6, Graphics.Draw 73.5, Driver.Draw 73.4, Graphics.Draw.draw_and_resource_release 59.9, Driver.Packet.dispatch 43.2
Longest waits
Largest GPU batches (GPU ranges inside, and the CPU slices that recorded each, inclusive)
trace_digest.py, run 3 (main)
seconds 150:5 from present 2660: 29 presents, frame p50 174.6 / p90 182.9 / p99 185.6 / max 185.6 ms, GPU busy 152.6 ms per present
NVIDIA GeForce RTX 4070 Ti SUPER, NVIDIA 616.64, Windows 10.0.26200, 3840x2160, commit 8499ea5, dropped: no
{"commit":"8499ea52e571","dirty":false,"env":{"APS5_HOST_IMPORT_MIB":"13312","APS5_TRACE_DIR":"<path>","APS5_TRACE_SECONDS":"150:5"},"gpu":"NVIDIA GeForce RTX 4070 Ti SUPER","driver":"NVIDIA 616.64","os":"Windows 10.0.26200","cpu":"11th Gen Intel(R) Core(TM) i7-11700K @ 3.60GHz","cpu_threads":16,"memory_mib":32599,"title_id":"PPSA21567","title":"ASTRO BOT","content_version":"01.019.000","resolution":"3840x2160","window":{"spec":"seconds 150:5","first_present":2660,"ms":5017.386},"presents":29,"frame_ms":{"p50":174.619,"p90":182.895,"p99":185.561,"max":185.561},"gpu_busy_ms_per_present":152.577,"worker_busy_pct":{"queue worker 0x0":98.804,"queue worker 0x40":97.803,"queue worker 0x48":93.909,"queue worker 0x46":9.464},"top_phases_ms_per_frame":{"Driver.Packet":518.59,"Driver.Packet.memory_wait":298.946,"Driver.Packet.draw":86.549,"Driver.Draw":85.458,"Graphics.Draw":81.895,"Driver.Packet.prepare_memory":65.5,"Graphics.Draw.draw_and_resource_release":65.075,"Driver.Draw.shader_compile_and_link":52.003},"trace_dropped":false,"trace":"aps5-trace-8499ea52e571-12684.json.gz"}Slowest frames (largest slices inside each, inclusive)
Threads (ms per frame)
Largest slices (ms per frame, inclusive, all threads): Driver.Packet 497.9, Driver.Packet.memory_wait 298.9, Driver.WorkerWait 153.5, batch 152.4, Driver.WorkerWait.dequeue_order 138.5, Driver.Packet.draw 73.7, Driver.Draw 72.6, Graphics.Draw 68.9, Driver.Packet.prepare_memory 64.2, Graphics.Draw.draw_and_resource_release 55.0
Longest waits
Largest GPU batches (GPU ranges inside, and the CPU slices that recorded each, inclusive)
trace_digest.py, run 4 (this PR)
seconds 150:5 from present 4310: 32 presents, frame p50 160.3 / p90 167.4 / p99 172.9 / max 172.9 ms, GPU busy 91.6 ms per present
NVIDIA GeForce RTX 4070 Ti SUPER, NVIDIA 616.64, Windows 10.0.26200, 3840x2160, commit b8663c5, dropped: no
{"commit":"b8663c514ef5","dirty":false,"env":{"APS5_HOST_IMPORT_MIB":"13312","APS5_TRACE_DIR":"<path>","APS5_TRACE_SECONDS":"150:5"},"gpu":"NVIDIA GeForce RTX 4070 Ti SUPER","driver":"NVIDIA 616.64","os":"Windows 10.0.26200","cpu":"11th Gen Intel(R) Core(TM) i7-11700K @ 3.60GHz","cpu_threads":16,"memory_mib":32599,"title_id":"PPSA21567","title":"ASTRO BOT","content_version":"01.019.000","resolution":"3840x2160","window":{"spec":"seconds 150:5","first_present":4310,"ms":5118.256},"presents":32,"frame_ms":{"p50":160.333,"p90":167.439,"p99":172.874,"max":172.874},"gpu_busy_ms_per_present":91.612,"worker_busy_pct":{"queue worker 0x0":98.652,"queue worker 0x40":97.934,"queue worker 0x48":94.517,"queue worker 0x46":9.894},"top_phases_ms_per_frame":{"Driver.Packet":481.041,"Driver.Packet.memory_wait":286.595,"Driver.Packet.draw":92.17,"Graphics.Draw":91.69,"Driver.Draw":90.991,"Graphics.Draw.draw_and_resource_release":75.211,"Driver.Draw.shader_compile_and_link":55.081,"Shader.DrawStage":44.516},"trace_dropped":false,"trace":"aps5-trace-b8663c514ef5-3760.json.gz"}Slowest frames (largest slices inside each, inclusive)
Threads (ms per frame)
Largest slices (ms per frame, inclusive, all threads): Driver.Packet 460.9, Driver.Packet.memory_wait 286.6, Driver.WorkerWait 144.0, Driver.WorkerWait.dequeue_order 134.5, batch 94.7, Driver.Packet.draw 79.7, Graphics.Draw 78.8, Driver.Draw 78.5, Graphics.Draw.draw_and_resource_release 65.5, Driver.Draw.shader_compile_and_link 44.4
Longest waits
Largest GPU batches (GPU ranges inside, and the CPU slices that recorded each, inclusive)
trace_digest.py, run 5 (main)
seconds 150:5 from present 2658: 29 presents, frame p50 174.5 / p90 183.5 / p99 184.9 / max 184.9 ms, GPU busy 152.0 ms per present
NVIDIA GeForce RTX 4070 Ti SUPER, NVIDIA 616.64, Windows 10.0.26200, 3840x2160, commit 8499ea5, dropped: no
{"commit":"8499ea52e571","dirty":false,"env":{"APS5_HOST_IMPORT_MIB":"13312","APS5_TRACE_DIR":"<path>","APS5_TRACE_SECONDS":"150:5"},"gpu":"NVIDIA GeForce RTX 4070 Ti SUPER","driver":"NVIDIA 616.64","os":"Windows 10.0.26200","cpu":"11th Gen Intel(R) Core(TM) i7-11700K @ 3.60GHz","cpu_threads":16,"memory_mib":32599,"title_id":"PPSA21567","title":"ASTRO BOT","content_version":"01.019.000","resolution":"3840x2160","window":{"spec":"seconds 150:5","first_present":2658,"ms":5045.459},"presents":29,"frame_ms":{"p50":174.497,"p90":183.493,"p99":184.907,"max":184.907},"gpu_busy_ms_per_present":151.991,"worker_busy_pct":{"queue worker 0x0":98.787,"queue worker 0x40":97.445,"queue worker 0x48":94.411,"queue worker 0x46":9.031},"top_phases_ms_per_frame":{"Driver.Packet":520.917,"Driver.Packet.memory_wait":298.77,"Driver.Packet.draw":91.749,"Driver.Draw":90.597,"Graphics.Draw":83.471,"Graphics.Draw.draw_and_resource_release":66.106,"Driver.Packet.prepare_memory":55.606,"Driver.Draw.shader_compile_and_link":53.222},"trace_dropped":false,"trace":"aps5-trace-8499ea52e571-25028.json.gz"}Slowest frames (largest slices inside each, inclusive)
Threads (ms per frame)
Largest slices (ms per frame, inclusive, all threads): Driver.Packet 500.3, Driver.Packet.memory_wait 298.8, Driver.WorkerWait 157.2, batch 151.0, Driver.WorkerWait.dequeue_order 143.9, Driver.Packet.draw 79.0, Driver.Draw 77.9, Graphics.Draw 70.5, Graphics.Draw.draw_and_resource_release 56.1, Driver.Packet.prepare_memory 54.3
Longest waits
Largest GPU batches (GPU ranges inside, and the CPU slices that recorded each, inclusive)
trace_digest.py, run 6 (this PR)
seconds 150:5 from present 2779: 33 presents, frame p50 152.8 / p90 161.1 / p99 166.5 / max 166.5 ms, GPU busy 96.4 ms per present
NVIDIA GeForce RTX 4070 Ti SUPER, NVIDIA 616.64, Windows 10.0.26200, 3840x2160, commit b8663c5, dropped: no
{"commit":"b8663c514ef5","dirty":false,"env":{"APS5_HOST_IMPORT_MIB":"13312","APS5_TRACE_DIR":"<path>","APS5_TRACE_SECONDS":"150:5"},"gpu":"NVIDIA GeForce RTX 4070 Ti SUPER","driver":"NVIDIA 616.64","os":"Windows 10.0.26200","cpu":"11th Gen Intel(R) Core(TM) i7-11700K @ 3.60GHz","cpu_threads":16,"memory_mib":32599,"title_id":"PPSA21567","title":"ASTRO BOT","content_version":"01.019.000","resolution":"3840x2160","window":{"spec":"seconds 150:5","first_present":2779,"ms":5089.771},"presents":33,"frame_ms":{"p50":152.754,"p90":161.061,"p99":166.455,"max":166.455},"gpu_busy_ms_per_present":96.415,"worker_busy_pct":{"queue worker 0x0":98.647,"queue worker 0x40":97.92,"queue worker 0x48":94.465,"queue worker 0x46":11.271},"top_phases_ms_per_frame":{"Driver.Packet":465.854,"Driver.Packet.memory_wait":274.814,"Driver.Packet.draw":87.031,"Graphics.Draw":86.515,"Driver.Draw":85.879,"Graphics.Draw.draw_and_resource_release":70.117,"Driver.Draw.shader_compile_and_link":51.299,"Driver.Packet.dispatch":43.309},"trace_dropped":false,"trace":"aps5-trace-b8663c514ef5-27560.json.gz"}Slowest frames (largest slices inside each, inclusive)
Threads (ms per frame)
Largest slices (ms per frame, inclusive, all threads): Driver.Packet 445.3, Driver.Packet.memory_wait 274.8, Driver.WorkerWait 136.7, Driver.WorkerWait.dequeue_order 127.8, batch 96.5, Driver.Packet.draw 74.1, Graphics.Draw 73.1, Driver.Draw 72.8, Graphics.Draw.draw_and_resource_release 60.0, Driver.Packet.dispatch 43.1
Longest waits
Largest GPU batches (GPU ranges inside, and the CPU slices that recorded each, inclusive)
Checklist
main; no other open PR implements the same functionsNotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file.mdfiles or images added to the repository (attach them to this PR)