Skip to content

fix(libc): keep the unwinder's frame caches alive through static destruction - #16313

Merged
oneandonlydean merged 1 commit into
boykopovar:mainfrom
SandstromPL:fix/libc-unwind-caches-outlive-exit
Nov 6, 2026
Merged

oneandonlydean merged 1 commit into
boykopovar:mainfrom
SandstromPL:fix/libc-unwind-caches-outlive-exit

Conversation

@SandstromPL

Copy link
Copy Markdown
Contributor

What

The nightly Sanitizers job of 11-06 (42158260477, head ea3ae18) fails five tests with AddressSanitizer: heap-use-after-free at core/libs/prx/libc/src/exception/Unwind.cpp:233:

  • download_data_mount_invalid
  • guest_flexible_memory_invalid
  • guest_convert_keycode_title_malformed-json
  • guest_convert_keycode_title_oversized-id
  • guest_convert_keycode_title_oversized-title

None of them failed in the 11-05 nightly (head 1882108).

FindFrameFast (Linux, glibc 2.35 and later) caches each image's text and data bases in a function-local static std::map guarded by a static std::mutex. Function-local statics are destroyed at exit. In the report, the map is freed on the main thread by __cxa_finalize when _dl_fini finalizes libc.prx, and a thread that is still throwing then reads it.

#14235 (merged 11-05) added that thread. RunReport::Collector::RefreshGame reads the title metadata on a detached thread. In these tests the metadata is invalid, so copyToFixedBuffer in AppMetadata.cpp throws there while the process exits:

READ of size 8 ... thread T3
    #9  LibcUnwind::FindFrameFast(LibcUnwind::Lookup&) Unwind.cpp:233
    #13 _Unwind_RaiseException_nid_postfix Unwind.cpp:722
    #15 __cxa_throw_nid_postfix Runtime.cpp:301
    #16 copyToFixedBuffer AppMetadata.cpp:40
    #19 ProvideGame RunReport.cpp:1630
    #24 operator() RunReport.cpp:1310
freed by thread T0 here:
    #9  __cxa_finalize
    #10 libc.prx (fini) <- _dl_fini <- exit

The run report only exposes the bug. Any guest thread that throws while the title exits can read the freed map. IndexFrames, the Windows path, has the same pattern: a static map and mutex, plus a returned reference into the map that is used after the lock is released.

This PR allocates both caches and their mutexes and never destroys them. libkernel already does this for its unwind images (Module.cpp:42), its frame table cache (ModuleInfo.cpp:259) and its thread stacks and their lock (#9272, #9960). Changes one file, core/libs/prx/libc/src/exception/Unwind.cpp (4 lines).

Tested

Ubuntu 24.04, GCC 13.3, main bbd85f7, configured with the Sanitizers workflow's flags (Release, -DANYPS5_ENABLE_ASAN=ON, -g1 -fsanitize=undefined -fno-sanitize=vptr), run with its environment (ASAN_OPTIONS=detect_leaks=0, UBSAN_OPTIONS=print_stacktrace=1:halt_on_error=1). I ran every test of the three executables, 12 tests in all:

  • guest_convert_keycode_title_*
  • download_data_mount_*
  • guest_flexible_memory_invalid

Results:

  • main (red): the same five tests fail in 3 of 3 runs. Each prints the use-after-free above: read at Unwind.cpp:233, allocated at Unwind.cpp:245, freed by __cxa_finalize in libc.prx's fini, on the RunReport.cpp:1310 thread.
  • This PR (green): 12 of 12 pass in 5 of 5 runs, with no AddressSanitizer or UBSan report.
  • The 44 libc unwind and exception tests I built pass under the same ASan build. They come from exception_runtime, exception_personality, exception_compact_unwind, exception_c_cleanup, unwind_callback_abi, unwind_record_bounds, guest_unwind_module_info(_failure), guest_raise_exception, guest_exception_handler, guest_sce_v2_exceptions, guest_libc_internal_exception_destructors, guest_throw_bad_array_new_length, guest_cxa_guard and guest_json_initialization_exceptions.
  • python3 -I tools/check_conventions.py --base origin/main passes and git diff --check is clean.

Not run:

  • Windows. IndexFrames is only compiled for _WIN32 and I have no Windows toolchain, so CI is its first compile.
  • macOS. Neither function is compiled there.

Overlap

I checked the file lists of all 1884 open PRs (2026-11-06). 13 of them list more than 100 files and were truncated; I didn't check those further. Seven change Unwind.cpp:

No open PR changes the run report's detached thread for this. #15973 keeps .detach().

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: none
  • Unblocks: none
  • GPUs tested: n/a
  • AI-assisted: yes (Claude Code)

🤖 Generated with Claude Code

…ruction

FindFrameFast (Linux) and IndexFrames (Windows) cache each image's data in a
function-local static std::map guarded by a static std::mutex. Both are
destroyed at exit when libc.prx is finalized, while another thread can still
be throwing. Since boykopovar#14235 the run report reads the title metadata on a
detached thread, and in the ASan nightly the throw from AppMetadata's
copyToFixedBuffer in that thread reads the freed map (heap-use-after-free at
Unwind.cpp:233) in download_data_mount_invalid, guest_flexible_memory_invalid
and three guest_convert_keycode_title tests.

Allocate the caches and their mutexes and never destroy them, as libkernel
does for its unwind images, its frame table cache and its thread stacks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@oneandonlydean oneandonlydean self-assigned this Nov 6, 2026
@github-actions github-actions Bot added the libc label Nov 6, 2026
@JJenkx

JJenkx commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

Windows check of head a6643dea78, merged onto main 7920389640 (eb1a8735c3, clean merge).

Setup: Windows 11 Enterprise 22621, WinLibs GCC 15.2 UCRT, Release. This is not an ASan build: MinGW has no AddressSanitizer, so the heap-use-after-free itself isn't observable here. The hosted Windows check does not run these tests on pull requests.

  • The 8 exception and unwind tests from your list that are registered on Windows pass in 3 of 3 runs, none skipped:
    • exception_personality, guest_cxa_guard, guest_exception_handler, guest_libc_internal_exception_destructors;
    • guest_raise_exception, guest_sce_v2_exceptions, guest_throw_bad_array_new_length, unwind_record_bounds.
  • exception_c_cleanup, exception_compact_unwind and exception_runtime aren't registered in the Windows build, so they didn't run.

So keeping the frame caches alive through static destruction doesn't regress the unwinder on Windows.

AI-assisted: yes.

@oneandonlydean
oneandonlydean merged commit 150b7d2 into boykopovar:main Nov 6, 2026
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants