Skip to content

fix(tests): build the frame limit failure messages only when a check fails - #14995

Merged
oneandonlydean merged 1 commit into
boykopovar:mainfrom
xXPBSXx:fix/agc-frame-limit-test-messages
Nov 3, 2026
Merged

oneandonlydean merged 1 commit into
boykopovar:mainfrom
xXPBSXx:fix/agc-frame-limit-test-messages

Conversation

@xXPBSXx

@xXPBSXx xXPBSXx commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

What

agc_frame_limit timed out in the nightly Sanitizers run on 2026-11-02 (run 41384606155, main 926f7dd492): ***Timeout 30.06 sec, and the run's summary lists no sanitizer report for it. The same test passed in 1.07 s on Linux in the normal build of 4569fe654a (run 41395467000).

The test spends its time building failure messages that are never shown. require(condition, reason) takes the message as a std::string, so every call builds it before the check, including the calls inside the loops: 8 limits × 1,000,000 frames in the deadline test, 100,000 frames in the overshoot test and 3 × 10,000 in the slow-game test, each message made of several std::to_string calls and concatenations. Under AddressSanitizer each of those allocations is instrumented.

This adds a require overload that takes a callable and builds the message only when the check fails, and uses it for the three checks inside those loops. The checks themselves, the messages and the TIMEOUT 30 are unchanged.

Changed: core/libs/prx/libSceAgcDriver/tests/FrameLimit.cpp only. No product code, no CMake change.

Rechecked on 2026-11-02 at 08:54 UTC: this head merges with main 88e804b0a7 without a conflict, and none of the 11 pull requests opened since 07:30 UTC (#14979 to #14993) changes FrameLimit.cpp.

Tested

The test executable built on its own with the flags the Sanitizers workflow gives it (g++ 13.3.0, -O3 -DNDEBUG -std=c++20 -g1 -fsanitize=undefined -fno-sanitize=vptr -fsanitize=address -fno-omit-frame-pointer -DANYPS5_ENABLE_ASAN=1) and run with the workflow's ASAN_OPTIONS, UBSAN_OPTIONS and APS5_SKIP_TIMING_TESTS, three runs each, on 2026-11-02 between 07:26 UTC and 07:31 UTC:

Build Before (main 4569fe654a) After
ASan and UBSan 17.63, 12.45, 15.24 s; again 12.22, 15.07, 12.05 s 0.20, 0.08, 0.08 s
No sanitizer 0.94, 0.95 s 0.03 s

All runs print frame limit tests passed. These local runs stay under the nightly's 30 s, so they do not reproduce the timeout; the CI runner, which runs four tests at once (ctest --parallel 4), was not measured. What they show is that building the messages is where the time goes: the change makes the sanitizer run 60 to 220 times faster (12.05 s / 0.20 s to 17.63 s / 0.08 s).

The test still fails when the pacer is wrong. With the extra nanosecond that FramePacer::step() adds for the carried remainder removed (++length), it fails both before and after this change with the same message, frame 3 at limit 30000 is presented at 99999999 instead of 100000000, now built by the lazy overload. With next set from the current time plus the wait instead of from the schedule, it fails both before and after in the late-frame test. Both mutations were made on copies; FrameLimit.hpp is unchanged.

tools/check_conventions.py --base upstream/main passes.

Overlap

All 1914 open PRs were checked on 2026-11-02 at 07:32 UTC against main 4569fe654a for FrameLimit.cpp, FrameLimit.hpp and agc_frame_limit_tests.cmake. Four list these files (#10351, #9588, #8630, #7694), each only through a merge of main: none of them changes the files against main, and none conflicts with this change. No open PR changes the test; #11566, which comes up in a search for agc_frame_limit, changes only tests/Graphics.cpp and tests/cmake/agc_driver_graphics_tests.cmake. It merges cleanly into main 88e804b0a7 (2026-11-02 08:07 UTC), which does not change these files or the Sanitizers workflow.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: none
  • Unblocks: none
  • GPUs tested: n/a
  • AI-assisted: yes

AI-assisted (Claude Code).

🤖 Generated with Claude Code

…fails

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@TechLevelZero

Copy link
Copy Markdown
Contributor

✅ NVIDIA GPU test: passed

PR head bf74e82 merged onto main 2517aa0 · tested 2026-11-02 09:29 UTC · 22 min

GPU NVIDIA RTX 2000 Ada Generation, driver 595.99.02, Vulkan 1.4.329
Host Ubuntu 26.04.1 LTS, kernel 7.0.0-38-generic, container ubuntu:24.04
Logs build and test logs for this run
Build ✅ passed (Release, -DANYPS5_ENABLE_SPIRV_TOOLS=ON)
Tests ✅ 837 passed · 2 failed (0 new, 1 also fail on main, 1 unstable on main) · 0 skipped (ANYPS5_REQUIRE_VULKAN=1)
Test setup CPU-only tests without a GPU; GPU tests (and any CPU-only test that failed or skipped without one) once on the GPU with VK_LAYER_KHRONOS_validation and ANYPS5_NO_SHADER_CACHE=1; main ran the same way
Test selection 839 of 3,174 tests run: the ones this change can reach, chosen by tools/select_tests.py from main's build graph
Vulkan validation ✅ no new validation IDs vs main (408 tests run on the GPU)
Slow tests (>30 s, GPU tests timed with validation layers) ⚠️ agc_driver_pipeline_journal 120.1 s
Unstable on main (1, these also failed in recent main runs on this machine; not counted against this PR)
agc_driver_pipeline_journal: failed in 2 of the last 10 main runs
Failing on main too (1, not counted against this PR)
agc_driver_image_table
Validation IDs already present on main (5 IDs in 346 tests)
VUID-VkDeviceCreateInfo-pNext-pNext (346 tests): agc_driver_address_passes, agc_driver_alias_copy_label, agc_driver_alpha_to_coverage, agc_driver_array_layer_target, agc_driver_bc_block_copy, agc_driver_bda_anchor_reads, agc_driver_bda_span_reads, agc_driver_bindless_images, and 338 more
VUID-vkCmdCopyImage-srcImage-01548 (7 tests): agc_driver_depth_clear_hle, agc_driver_depth_plane_buffer_memory, agc_driver_depth_plane_partial_store, agc_driver_depth_plane_sampling, agc_driver_depth_storage_dispatch, agc_driver_depth_surface_from_storage, agc_driver_dispatch_move_depth
VUID-vkCmdCopyImage-srcImage-01551 (7 tests): agc_driver_depth_clear_hle, agc_driver_depth_plane_buffer_memory, agc_driver_depth_plane_partial_store, agc_driver_depth_plane_sampling, agc_driver_depth_storage_dispatch, agc_driver_depth_surface_from_storage, agc_driver_dispatch_move_depth
VUID-VkPipelineColorBlendStateCreateInfo-pAttachments-00605 (1 test): agc_driver_prepared_exports
VUID-VkShaderModuleCreateInfo-pCode-08737 (1 test): agc_driver_image_sample_derivatives
Reproduce
git fetch origin bf74e8264110186e5db851da79d15cfe2911d72c 2517aa02f78e00b8b06ae5ff4206a8c04e22a1cb
git checkout 2517aa02f78e00b8b06ae5ff4206a8c04e22a1cb && git merge --no-ff bf74e8264110186e5db851da79d15cfe2911d72c
git submodule update --init --recursive
cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=ON -DANYPS5_ENABLE_SPIRV_TOOLS=ON
cmake --build build && cmake --build build --target libs
VK_DRIVER_FILES=/nonexistent/icd.json ctest --test-dir build -E '^(agc|shader|recompiler|videoout|video_out|guest_video_out|vulkan)|gpu' --timeout 120 --parallel 32 --output-on-failure
ANYPS5_REQUIRE_VULKAN=1 ANYPS5_NO_SHADER_CACHE=1 VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation \
  ctest --test-dir build -R '^(agc|shader|recompiler|videoout|video_out|guest_video_out|vulkan)|gpu' --timeout 120 --parallel 4 --output-on-failure
# failures are then re-run alone: --timeout 360 --parallel 2; new validation IDs: the test alone, twice, --parallel 1

Automated build and test on a contributor's NVIDIA machine; it does not replace upstream CI. A new comment is posted for each new push. Full logs.

@github-actions github-actions Bot added the agc libSceAgc* label Nov 2, 2026

@BunnyFl BunnyFl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks right to me. The std::invocable overload only matches the three lambdas. The existing calls pass a std::string or a literal, which isn't invocable, so they still pick the original overload, and the messages and conditions are unchanged. pacer.Admit(now) in the slow-game loop is still evaluated once, in the condition.

Tested on Windows 11, WinLibs GCC 15.2.0, Release (no sanitizer), with head bf74e82641 merged onto main: agc_frame_limit passes in 0.15 s, against 1.46 s with main's FrameLimit.cpp. Even without ASan the eager message building dominates, which supports your Sanitizers timeout analysis. I didn't run the ASan/UBSan build itself.

@maskym

maskym commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

Windows + NVIDIA result for head bf74e82641, merged alone into main 9090972b7e (the PR's Tested section names no Windows run). Correctness only; nothing here speaks to a speed claim.

Host: Windows 11 26300, Intel i7-12700K, NVIDIA GeForce RTX 2070 SUPER, WinLibs GCC 15.2.0 (x86_64-ucrt-posix-seh, r6), Release, BUILD_TESTING=ON, Ninja. Build clean.

Same machine, both trees, with ANYPS5_REQUIRE_VULKAN=1, ANYPS5_NO_SHADER_CACHE=1, VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation (Khronos VVL 1.4.366) and ctest --test-dir build -R '^(agc|guest_video_out|video_out|pad_sdl)' --parallel 4 -V:

Tree Result Layer inserted was not found but was requested Distinct VUID-
main 9090972b7e 1332/1333 1159 0 VkPipelineColorBlendStateCreateInfo-pAttachments-00605 ×18 (#11990)
this PR 1332/1333 1159 0 the same, ×18

No new validation errors. agc_frame_limit, the test this PR extends, passes. The one failure on both trees, agc_driver_direct_write_completion ("the completion of a buffer write dropped image results recorded after it"), fails on plain main too and 0 of 3 times rerun alone, so it is not this PR's. Not run on a PS5.

AI-assisted: yes (Claude Code), the runs and the numbers above are from this machine.

@oneandonlydean oneandonlydean self-assigned this Nov 3, 2026
@oneandonlydean
oneandonlydean merged commit 0be89a5 into boykopovar:main Nov 3, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agc libSceAgc*

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants