Skip to content

fix(agc): let a kept descriptor set's table record hold its textures weakly - #7802

Merged
oneandonlydean merged 2 commits into
boykopovar:mainfrom
ionut27001:fix/agc-kept-set-weak-textures
Oct 19, 2026
Merged

oneandonlydean merged 2 commits into
boykopovar:mainfrom
ionut27001:fix/agc-kept-set-weak-textures

Conversation

@ionut27001

@ionut27001 ionut27001 commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

What

Since #6454, a freed dedicated descriptor set kept for reuse (APS5_KEPT_DESCRIPTOR_SETS, 8 by default) carries a TableDescriptors record whose textures is every sampled texture of the ShaderResources that freed it: keptTable moves textures into it. ReleaseTextureCaches counts an evicted sampled entry as freed only when it held the last reference, so the textures these records hold leave the cache but keep their memory. On Marvel's Wolverine, heap 0 overruns its budget within a minute, allocations move to system memory, and vkAllocateMemory fails with -2 (bisect and logs in #7801).

The record now holds std::weak_ptr<Texture>. The build compares a kept table's views only while every recorded texture is still alive. Two live textures cannot share a view handle, so an equal handle is still the same view, which is what the strong references were there to guarantee. When a recorded texture has died, the table is written in full, as for any other change. The rest of the kept-set path is unchanged.

ImageTableReuse.cpp gains two steps after its existing ones, as asked in review. A full ReleaseTextureCaches(~std::uint64_t{0}) must leave SampledTextureMemory() at 0. Then a table is built twice, and the second build must keep its table write, so the set it leaves holds a record that matches while its textures live. A full release then destroys every texture (checked again through SampledTextureMemory()), and the next build of the table must make one table write and keep none.

Tested

NVIDIA GeForce RTX 4080 SUPER 16 GB (driver 617.42), Windows 11 Pro 26300, Ryzen 7 5800X3D, WinLibs GCC 15.2.0 r7, Release.

  • ctest -R '^agc_driver_(graphics|image_table_reuse)$' passes with this change on main fb0dc11 and on 676d423 (both with feat(libs): restore libSceNpCppWebApi for dumps that ship no copy #6901 and fix(agc): run the unwatch hook when a watched import leaves the write watch #6308 merged, as for the game). These are the descriptor cache's kept-set unit test and the image table reuse execution test, run on the GPU.
  • This change with the new test steps, on main 6030b31 (this branch): agc_driver_image_table_reuse passes in 3 of 3 runs, and in 3 of 3 with APS5_KEPT_DESCRIPTOR_SETS=0; agc_driver_graphics passes. With ShaderResources.cpp and ShaderResources.hpp reverted to 6030b31 and the new steps kept, agc_driver_image_table_reuse fails in 3 of 3 runs: "96256 bytes of sampled textures outlived a full release of the texture caches after the kept-table steps".
  • With only the heldAlive check removed (the record still weak), the new steps pass in 3 of 3 runs on this card: the second build kept its write, and the build after the textures were destroyed made one table write and kept none, so it was written in full anyway. Without the check the step can fail only on a driver that gives every new view of the table the handle that the destroyed view in the same slot had; that did not happen here, so on this card the step does not catch a missing check.
  • python tools/check_conventions.py passes.
  • Marvel's Wolverine PPSA03671 (01.001.005), the "Sony Interactive Entertainment presents" card, no input, default budgets, with feat(libs): restore libSceNpCppWebApi for dumps that ship no copy #6901 (needed to load the title) and fix(agc): run the unwatch hook when a watched import leaves the write watch #6308 (removes an unrelated later abort) merged on top, with and without this change. On fb0dc11 the first runs spend 35 s and more on each new compute pipeline until the Vulkan pipeline cache holds them; the fb0dc11 rows are runs after that.
Runs Result Heap 0 use, worst budget line Peak VRAM Peak private bytes
676d423 2 × 120 s abort at 32 s and 38 s (vkAllocateMemory ... memoryType=0 ... initialResult=-2) 20745, 19118 MiB 15.9 GB 36.7-41.3 GiB
676d423 + this change 2 × 120 s, 1 × 300 s no abort, no cache release logged 10463, 10337, 10608 MiB 11.6-11.9 GB 25.7-27.4 GiB
676d423, APS5_KEPT_DESCRIPTOR_SETS=0 1 × 120 s no abort 10457 MiB 11.5 GB 26.5 GiB
fb0dc11 1 × 180 s, 1 × 120 s abort at 40 s; the other run had heap 0 over budget 18153, 20380 MiB 15.9 GB 38.7-39.3 GiB
fb0dc11 + this change 1 × 180 s no abort 10236 MiB 11.7 GB 26.6 GiB

Frame times on the card with this change are the same as with APS5_KEPT_DESCRIPTOR_SETS=0 and as before #6454: on 676d423, about 5.8 s per present once warm (31 presents in the 252 s after the first one, in the 300 s run).

Not checked: Linux, lavapipe and other vendors; #6454's lavapipe timings with this change; how often the comparison now falls back to a full write (the kept and made counters are only exposed to tests).

Checklist

…weakly

A freed dedicated set kept for reuse recorded every texture of the resources that freed it and kept
them alive, so the kept handles could not be reused while the set referred to them. Up to eight kept
sets held textures the caches had already evicted: their memory stayed in use outside the caches'
budgets, and a title with large image tables ran out of device memory.

The record now holds the textures weakly. A build compares the kept table's views only while every
recorded texture is still alive, so a matching handle is still the same view; when one has died the
table is written in full, as for any other change.
@github-actions github-actions Bot added the unblocks title Names a title ID on the Unblocks line label Oct 19, 2026
@github-actions

github-actions Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 6177 core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 6188 core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp
core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 6192 core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp
core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 6194 core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp
core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 6202 core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp
core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 6223 core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp
core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
core/libs/prx/libSceAgcDriver/tests/execution/ImageTableReuse.cpp
PR 6292 core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp
core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 6340 core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
core/libs/prx/libSceAgcDriver/tests/execution/ImageTableReuse.cpp
PR 6615 core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp
core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 6636 core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp
core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 6661 core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 6718 core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp
core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 6820 core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp
core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 6937 core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
core/libs/prx/libSceAgcDriver/tests/execution/ImageTableReuse.cpp
PR 6946 core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp
core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 6947 core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp
core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 6970 core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 6972 core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp
core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 7002 core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp
core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 7332 core/libs/prx/libSceAgcDriver/tests/execution/ImageTableReuse.cpp
PR 7484 core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp
core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 7642 core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp
core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 7689 core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp
core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp
PR 7800 core/libs/prx/libSceAgcDriver/Graphics/include/ShaderResources.hpp
core/libs/prx/libSceAgcDriver/Graphics/src/ShaderResources.cpp

@github-actions github-actions Bot added the agc libSceAgc* label Oct 19, 2026
@oneandonlydean

Copy link
Copy Markdown
Collaborator

Thanks, the cause and the fix both check out here.

One ask: nothing in the PR fails on main yet. Both of these fit in tests/execution/ImageTableReuse.cpp:

  1. After the kept-table steps, ReleaseTextureCaches(~std::uint64_t{0}) should leave SampledTextureMemory() == 0. With a print at the end of main() on an RTX 4090, main still counts 96256 of 98304 sampled bytes after a full release, and this PR counts 0.
  2. The new expired branch: let a build's set be kept, drop every texture its record names (no other holder), then build the same table again and expect one more table write made and none kept. Today the tests still pass with the heldAlive check removed.

@oneandonlydean oneandonlydean self-assigned this Oct 19, 2026
@github-actions github-actions Bot added the fix error Closes an issue (Fixes #N) label Oct 19, 2026
… a kept table whose textures died is written again
@ionut27001

Copy link
Copy Markdown
Contributor Author

Thanks. Both are in ImageTableReuse.cpp now, after the existing steps:

  1. A full ReleaseTextureCaches(~std::uint64_t{0}) must leave SampledTextureMemory() == 0. With ShaderResources.cpp and .hpp reverted to main 6030b31, it fails in 3 of 3 runs on an RTX 4080 SUPER (617.42, Windows): "96256 bytes of sampled textures outlived a full release of the texture caches after the kept-table steps". With this PR it passes.
  2. The table is built twice, and the second build must keep its table write. A full release then destroys every texture (the step checks SampledTextureMemory() == 0, so nothing else holds them), and the next build must make one table write and keep none. It passes with this PR, also with APS5_KEPT_DESCRIPTOR_SETS=0.

On this card, step 2 also passes with the heldAlive check removed (3 of 3): the rebuilt table never matched the kept record, so it was written in full anyway. Without the check it could fail only on a driver that gives every new view the handle of the destroyed view in the same slot, so on its own it doesn't catch the removal. If you want a test that does, I can count the full writes made because a recorded texture expired, next to the kept and made counters, and expect one there.

The runs are in the Tested section.

@fogydes fogydes left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the updated head addressing @oneandonlydean's feedback:

  • Holding textures weakly in kept descriptor table records prevents stale descriptor sets from indefinitely locking evicted VRAM allocations during ReleaseTextureCaches, directly resolving the device memory exhaustion observed in Marvel's Wolverine (Issue #7801).
  • The two requested regression steps are now present in ImageTableReuse.cpp:
    1. Confirming SampledTextureMemory() == 0 after ReleaseTextureCaches(~0ull) (which failed with 96,256 outliving bytes on main).
    2. Verifying that expired weak texture references invalidate the kept table and force a table write.
  • Zero comments added in code; Conventional Commits title; all CI checks pass.

Ready to merge.

AI-assisted: yes

@savvaniss

Copy link
Copy Markdown
Contributor

AMD and Intel Windows data: with this PR's test change, agc_driver_image_table_reuse fails on main and passes with the PR, on an AMD RX 9070 XT and an Intel UHD 770.

Windows 11 Pro 26300, WinLibs GCC 15.2.0 r7, Release, every execution test built, ANYPS5_NO_SHADER_CACHE=1, serial CTest. Intel UHD Graphics 770 (8086:a780), driver 32.0.101.7088. AMD Radeon RX 9070 XT (1002:7550), driver 32.0.31041.1004, with VkCache emptied first. This head 15c6c9ff1 merged into main 85029a00f. "Without" means only this PR's diff to tests/execution/ImageTableReuse.cpp, applied to main 85029a00f.

Tests, ×3 UHD 770 RX 9070 XT
This head: agc_driver_image_table_reuse passes passes
Without fails: image table reuse: 385024 bytes of sampled textures outlived a full release of the texture caches after the kept-table steps fails the same way with 96256 bytes

AI-assisted: yes (Claude Code)

@oneandonlydean
oneandonlydean merged commit 7b18fa8 into boykopovar:main Oct 19, 2026
11 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agc libSceAgc* fix error Closes an issue (Fixes #N) unblocks title Names a title ID on the Unblocks line

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Marvel's Wolverine runs out of device memory since #6454: kept descriptor sets keep evicted textures alive

4 participants