Skip to content

feat(libc): record traced system library calls on Linux - #7411

Merged
3 commits merged into
boykopovar:mainfrom
IshmamDC217:feat/libc-call-trace-linux
Oct 20, 2026
Merged

3 commits merged into
boykopovar:mainfrom
IshmamDC217:feat/libc-call-trace-linux

Conversation

@IshmamDC217

Copy link
Copy Markdown
Contributor

What

The call tracer behind --trace-imports and APS5_TRACE_CALLS (#4546) was built for Windows only. This makes the libc side work on Linux too, as the first half of #3815's Linux support. The relinker change that emits the TraceImport calls into Linux executables follows in a separate PR that depends on this one.

  • CallTrace.cpp moves from specifics/windows to specifics/x86_64, since the thunk and the register-saving entry are x86-64 code rather than Windows code. It builds on Windows and Linux; macOS is left out.
  • On Linux the thunk area is a memfd mapped twice, once writable and once executable, matching the two views of the Windows file mapping. Threads are identified by gettid.
  • Linux has no crash report, so the first traced import installs a SIGABRT handler. It prints the same two sections as the Windows report (last 64 calls with the caller as module+offset from dladdr, then the 32 most called imports) and passes the signal on to the previous handler, or re-raises it with the default action. This covers both guest abort() and host std::abort().
  • Fatal signals such as SIGSEGV do not print the trace on Linux. libc's guest write watch handles SIGSEGV and SIGBUS itself, so a handler in front of it would report faults it recovers from. This is recorded in docs/dev/debt/libc.md.

Tested

Linux x86-64, GCC 13, full Release build with BUILD_TESTING=ON.

  • guest_call_trace and guest_call_trace_abort are now registered on Linux as well as Windows, and both pass. They passed 20 runs in a row, including the 4-thread × 20000-call ring check.
  • In abort mode on Linux, the test installs its own SIGABRT handler, which exits with status 3 like the Windows CRT. The test therefore also checks that the report chains to the previous handler, and CTest judges the output against the existing regular expression instead of treating the signal as a crash. The executable suffix in that expression now comes from CMAKE_EXECUTABLE_SUFFIX.
  • Linux output from the abort test:
[libc] SIGABRT
  last traced system library calls (APS5_TRACE_CALLS):
    #1 thread 7713 libTraced.prx Integers from 0x0000564192c7b268 guest_call_trace_tests+0x3268
    #2 thread 7713 libOther.prx Blocks from 0x0000564192c7b2bb guest_call_trace_tests+0x32bb
  most called traced imports (2 calls in total):
               1 libTraced.prx Integers
               1 libOther.prx Blocks

Windows and macOS were not built. The Windows code paths are unchanged apart from being moved behind #ifdef _WIN32.

python3 tools/check_conventions.py --base origin/main passes.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Unblocks: Linux --trace-imports in the relinker (follow-up PR)
  • GPUs tested: n/a
  • AI-assisted: yes

@github-actions

github-actions Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 3965 core/libs/prx/libc/CMakeLists.txt
docs/dev/debt/libc.md
PR 3987 docs/dev/debt/libc.md
PR 4627 core/libs/prx/libc/CMakeLists.txt
PR 5021 core/libs/prx/libc/CMakeLists.txt
docs/dev/debt/libc.md
PR 5303 docs/dev/debt/libc.md
PR 5431 docs/dev/ENVIRONMENT.md
PR 6044 core/libs/prx/libc/CMakeLists.txt
core/libs/tests/GuestCallTrace.cpp
PR 6084 core/libs/prx/libc/CMakeLists.txt
PR 6085 core/libs/prx/libc/CMakeLists.txt
PR 6086 core/libs/prx/libc/CMakeLists.txt
PR 6226 docs/dev/debt/libc.md
PR 6469 docs/dev/debt/libc.md
PR 6565 docs/dev/debt/libc.md
PR 6636 docs/dev/ENVIRONMENT.md
PR 6655 docs/dev/ENVIRONMENT.md
PR 6713 core/libs/prx/libc/CMakeLists.txt
docs/dev/debt/libc.md
PR 6819 core/libs/prx/libc/CMakeLists.txt
docs/dev/debt/libc.md
PR 6879 docs/dev/ENVIRONMENT.md
PR 6924 docs/dev/ENVIRONMENT.md
PR 6946 docs/dev/ENVIRONMENT.md
PR 6947 docs/dev/ENVIRONMENT.md
PR 6948 docs/dev/ENVIRONMENT.md
PR 6951 docs/dev/ENVIRONMENT.md
PR 7081 core/libs/prx/libc/CMakeLists.txt
docs/dev/debt/libc.md
PR 7201 core/libs/prx/libc/CMakeLists.txt
PR 7307 docs/dev/ENVIRONMENT.md
PR 7386 docs/dev/debt/libc.md
PR 7427 docs/dev/ENVIRONMENT.md
PR 7447 core/libs/prx/libc/CMakeLists.txt
docs/dev/debt/libc.md
PR 7457 docs/dev/ENVIRONMENT.md
PR 7464 docs/dev/ENVIRONMENT.md
PR 7470 core/libs/prx/libc/CMakeLists.txt
PR 7473 docs/dev/debt/libc.md
PR 7480 docs/dev/ENVIRONMENT.md
PR 7484 docs/dev/ENVIRONMENT.md
PR 7525 core/libs/prx/libc/CMakeLists.txt
docs/dev/debt/libc.md
PR 7544 docs/dev/debt/libc.md
PR 7604 docs/dev/ENVIRONMENT.md
PR 7642 docs/dev/ENVIRONMENT.md
PR 7643 docs/dev/ENVIRONMENT.md
PR 7648 docs/dev/debt/libc.md
PR 7705 docs/dev/ENVIRONMENT.md
PR 7715 core/libs/prx/libc/CMakeLists.txt
docs/dev/debt/libc.md
PR 7841 core/libs/prx/libc/CMakeLists.txt
PR 7851 core/libs/prx/libc/CMakeLists.txt
PR 7870 docs/dev/ENVIRONMENT.md
PR 7920 docs/dev/ENVIRONMENT.md
PR 7922 docs/dev/ENVIRONMENT.md
PR 7930 docs/dev/ENVIRONMENT.md
PR 7952 docs/dev/debt/libc.md
PR 8004 docs/dev/ENVIRONMENT.md
PR 8066 docs/dev/ENVIRONMENT.md
PR 8075 docs/dev/ENVIRONMENT.md
PR 8077 docs/dev/ENVIRONMENT.md
PR 8087 core/libs/prx/libc/CMakeLists.txt
docs/dev/debt/libc.md
PR 8104 docs/dev/ENVIRONMENT.md
PR 8137 core/libs/prx/libc/CMakeLists.txt
docs/dev/debt/libc.md
PR 8150 core/libs/prx/libc/CMakeLists.txt
docs/dev/debt/libc.md
PR 8160 docs/dev/ENVIRONMENT.md
PR 8161 docs/dev/ENVIRONMENT.md
PR 8206 docs/dev/ENVIRONMENT.md
PR 8226 docs/dev/debt/libc.md
PR 8246 docs/dev/ENVIRONMENT.md
PR 8279 docs/dev/debt/libc.md
PR 8284 docs/dev/debt/libc.md
PR 8287 docs/dev/debt/libc.md
PR 8306 docs/dev/debt/libc.md
PR 8311 core/libs/prx/libc/CMakeLists.txt
PR 8352 core/libs/prx/libc/CMakeLists.txt
docs/dev/debt/libc.md
PR 8376 core/libs/prx/libc/CMakeLists.txt
PR 8384 core/libs/prx/libc/CMakeLists.txt
PR 8388 docs/dev/debt/libc.md
PR 8397 core/libs/prx/libc/CMakeLists.txt
PR 8417 docs/dev/debt/libc.md
PR 8427 docs/dev/debt/libc.md
PR 8444 docs/dev/debt/libc.md
PR 8449 docs/dev/debt/libc.md
PR 8479 core/libs/prx/libc/CMakeLists.txt
docs/dev/debt/libc.md
PR 8484 docs/dev/ENVIRONMENT.md
PR 8542 docs/dev/ENVIRONMENT.md
PR 8546 docs/dev/debt/libc.md
PR 8550 docs/dev/debt/libc.md
PR 8600 docs/dev/debt/libc.md
PR 8626 docs/dev/debt/libc.md
PR 8635 core/libs/prx/libc/CMakeLists.txt
PR 8636 docs/dev/debt/libc.md

}

void ReportOnAbort(int number, siginfo_t* info, void* context) {
std::fprintf(stderr, "[libc] SIGABRT\n");

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up to the same signal-context concern I raised on stacked #7417: this PR introduces the handler that #7417 exercises. ReportOnAbort calls fprintf/fflush, then ReportCallTrace uses dladdr/snprintf and reads the live trace state. These are not async-signal-safe. If SIGABRT arrives while a thread holds a stdio, loader, or trace-related lock, the handler can deadlock before the previous/default action runs, turning the abort into a hang. POSIX restricts signal handlers to async-signal-safe functions (signal actions). Could the handler hand off to normal context or use only a signal-safe, preformatted write path? The end-to-end abort test confirms the report is printed on its normal path, but not when the signal interrupts one of these routines.

@IshmamDC217 IshmamDC217 Oct 18, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cheers, good spot. You're right, and I managed to reproduce it: with another thread holding the stderr lock, the old handler just sat there until the test timed out.

Sorted in f670141:

  • the handler now only uses write(2) on a stack buffer, with no stdio;
  • callers are named from a module list taken outside the handler, so there's no dladdr;
  • counts are read without a lock.

I've also added a regression test that holds the stderr lock from another thread during the abort. It timed out before and passes now. #7417 is rebased on top.

@IshmamDC217
IshmamDC217 force-pushed the feat/libc-call-trace-linux branch from f670141 to ecf408c Compare October 18, 2026 22:36

@jotitadev jotitadev left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Linux run of ecf408c7c, with two abort races on top of the existing tests. With one abort the report holds up under load. With two threads aborting at the same time, the report is cut off or missing in every run.

Machine: Linux x86_64, i7-8700, Ubuntu 26.04, GCC 15.2, Release, BUILD_TESTING=ON, full build.
Head ecf408c7c merged onto main a373e85e5 (clean merge).

  • Builds. guest_call_trace and guest_call_trace_abort pass 50/50 with --repeat until-fail:50, and 20/20 pinned to one core with taskset -c 0. The non-GPU suite (ctest -E "^agc|gpu|vulkan|video_out" -j7) is 747/747. main is 745/745, and the difference is the two tests this PR registers on Linux.

  • Probe, added to GuestCallTrace.cpp as a race mode: install a SIGABRT handler that _exit(3)s, as the abort test does, and trace Integers. Then 4 threads call the traced import in a loop, and after 20 ms the main thread calls abort(). Each variant ran 300 times with a 10 s timeout:

    exit 3 report header "most called" section 64 entries entries out of order or with a wrong name hangs
    one abort() 300 300 300 151 (the rest 60–63) 0 0
    two threads call abort() together 300 35 0 0 0 0

    One abort while other threads keep writing the ring is fine. A few slots are skipped because they are mid-write, which is what the sequence check is for. With two aborts, the thread that loses g_reporting.exchange(true) goes straight to the previous handler while the winner is still writing. Here that handler is _exit(3). In a real run it is usually SIG_DFL, and the sigaction(SIG_DFL) plus raise() kills the process the same way. So the report is lost exactly when a crash spreads to a second thread. One way to fix it is to have the losing thread wait until the winner has finished writing before it chains, for example with a second g_reported flag and nanosleep, which is async-signal-safe. A bounded wait would avoid hanging if the winner never finishes. I tried exactly that: the winner sets g_reported after ReportCallTrace(), and the loser polls it with a 1 ms nanosleep for at most 2 s before chaining. With that change the two-abort variant printed the full report in 300/300 runs (header and "most called" section in every run, 60–64 entries). The one-abort variant stayed at 300/300, and both call trace tests still pass.

#7417 carries the same CallTrace.cpp, so this applies there too.

Written with AI assistance (Claude Code). I ran the builds and tests above on my machine.

@IshmamDC217

Copy link
Copy Markdown
Contributor Author

@jotitadev Cheers, brilliant testing. You're spot on: the thread that lost the exchange chained straight away and took the report down with it. Sorted in 6c7bfaa, essentially your fix. The winner sets g_reported once it's written, and the loser polls it with a 1 ms nanosleep for at most 2 s before chaining. There's also a new guest_call_trace_abort_race test that aborts from two threads at once after 100 extra traced calls, and checks the whole report. It failed 35/50 here without the wait and passes 100/100 with it. #7417 is rebased on top.

@jotitadev jotitadev left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Linux rerun of 6c7bfaa29 with the same race probe as my earlier report on ecf408c7c (4 threads calling the traced Integers in a loop, then abort() after 20 ms, SIGABRT handler that _exit(3)s, 300 runs per variant).

Machine: Linux x86_64, i7-8700, Ubuntu 26.04, GCC 15.2, Release, BUILD_TESTING=ON, full build. Head 6c7bfaa29 merged onto main 8816fd772 (clean merge).

ecf408c7c 6c7bfaa29
one abort() header and "most called" 300/300, 60-64 entries header and "most called" 300/300, 60-64 entries
two threads call abort() together header 35/300, "most called" 0/300 header and "most called" 300/300, 60-64 entries

No hangs, no entries out of order or with a wrong name, exit 3 in every run. The call trace tests pass (3/3, including the new guest_call_trace_abort_race), and the non-GPU suite is 778/778 (main 775/775 plus the tests this PR registers on Linux).

Written with AI assistance (Claude Code). I ran the builds and tests above on my machine.

@adriadam10 adriadam10 self-assigned this Oct 19, 2026
Build the call tracer for Linux as well as Windows. The thunk area is a
memfd mapped twice, writable and executable, and threads are named by
gettid. Linux has no crash report, so the first traced import installs a
SIGABRT handler that prints the last calls and the most called imports,
then passes the signal on to the previous handler.

guest_call_trace and guest_call_trace_abort now run on Linux too.
The SIGABRT handler used stdio and dladdr, so an abort raised while another thread held the stderr lock or the loader lock hung instead of ending. The report is now formatted into a stack buffer and written with write(2). Callers are named from a module list taken with dl_iterate_phdr when imports are traced, and import counts are read without a lock from a fixed record array. The abort test now holds the stderr lock in another thread, which timed out before this change.
A thread that lost the race to write the report passed SIGABRT on straight away, and the default action ended the process while the first thread was still writing. It now waits for the report to finish, polling with nanosleep for at most 2 seconds, before it chains. guest_call_trace_abort_race aborts from two threads at once and checks the whole report; it failed 35 times in 50 without the wait. Reported by jotitadev.
@IshmamDC217
IshmamDC217 force-pushed the feat/libc-call-trace-linux branch from 6c7bfaa to 0bad015 Compare October 19, 2026 21:59
@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 20, 2026
@oneandonlydean oneandonlydean closed this pull request by merging all changes into boykopovar:main in 29e4c46 Oct 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

build docs libc needs rebase Conflicts with main, rebase needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants