Skip to content

feat(relinker): trace function imports on macOS - #9408

Merged
adriadam10 merged 2 commits into
boykopovar:mainfrom
rursache:feat/macos-trace-imports
Oct 26, 2026
Merged

adriadam10 merged 2 commits into
boykopovar:mainfrom
rursache:feat/macos-trace-imports

Conversation

@rursache

Copy link
Copy Markdown
Contributor

What

--trace-imports with --macos walks each function import before the guest entry and passes the resolved pointer through TraceImport_nid_no_patch, as Linux and Windows already do. A PLT slot, or a GOT slot whose symbol is a function, is traced. A data import, a weak import, a nonzero addend, a lookup through sceKernelDlsym, and an import of a module loaded at run time are not. The macOS slot stays in a writable segment, so the stub stores the thunk and does not change the page protection.

An x86-64 build of libc records the call and prints the trace on SIGABRT. An arm64 build of libc does not include the recorder.

python3 tools/pr_overlap.py --repo boykopovar/AnyPS5 --base main --branch feat/macos-trace-imports reports shared lines with #8658 (header padding on the same entry-stub placement in MacOsImage.cpp) and with #8973 and #9332 (the usage rows beside --trace-imports). None of them implement import tracing. #7706 also edits MacOsImage.cpp, at the GS-trap loop, and conflicts with main, so that file was not compared. This change does not build on any of them.

Tested

macOS, Apple Silicon, AppleClang 21.0.0, relinker-only Release. Rosetta is not installed, so an x86-64 Mach-O does not run on this machine.

cmake -S . -B build-relinker -G Ninja -DCMAKE_BUILD_TYPE=Release -DANYPS5_RELINKER_ONLY=ON -DBUILD_TESTING=ON
cmake --build build-relinker
ctest --test-dir build-relinker -R 'trace_imports|macos_output|macos_fixed_base|macos_executable_exports' --output-on-failure
ctest --test-dir build-relinker -R 'macos_' --output-on-failure
python3 tools/check_conventions.py --base origin/main
xcrun clang++ -arch x86_64 -std=c++20 -c core/libs/prx/libc/src/specifics/x86_64/CallTrace.cpp -I core/libs

Passed: windows_trace_imports, linux_trace_imports, macos_trace_imports, macos_output, macos_tls_alu, macos_empty_tls, macos_tls_overlap, macos_gs_traps, macos_fixed_base, macos_executable_exports, macos_guest_weak_imports. Conventions passed. CallTrace.cpp compiled for x86-64.

macos_trace_imports relinks the traced call, the fixed-base call, a data import, a weak import, a libc that does not export TraceImport_nid_no_patch, and an executable that does not need libc.prx. The process run is skipped here (Bad CPU type in executable), which is the same skip the other macOS execution tests use when Rosetta is absent. The traced Mach-O was checked from the file: LC_MAIN jumps to the walk, the walk calls TraceImport_nid_no_patch, and it jumps back to the existing entry stub.

Not tested: a title, Windows, a Linux run of this change beyond the existing Linux relink checks, and execution of the traced Mach-O under Rosetta. The x86-64 recorder was compiled and not linked or run.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: none
  • Unblocks: none
  • GPUs tested: n/a
  • AI-assisted: yes

@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request Conflicting files (lines on main) Same exports Same new files
PR 9772 docs/dev/debt/libc.md L175
PR 11284 core/relinker/elfpatcher/src/macos/MacOsImage.cpp L819-820

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 3786 docs/user/USAGE.md
PR 5021 core/relinker/CMakeLists.txt
docs/dev/debt/libc.md
docs/dev/debt/relinker.md
docs/user/USAGE.md
PR 5303 docs/dev/debt/libc.md
PR 5925 docs/user/USAGE.md
PR 6171 docs/user/USAGE.md
PR 6713 core/relinker/CMakeLists.txt
docs/dev/debt/libc.md
PR 6924 docs/user/USAGE.md
PR 7081 docs/dev/debt/libc.md
PR 7386 docs/dev/debt/libc.md
PR 7447 docs/dev/debt/libc.md
PR 7544 docs/dev/debt/libc.md
PR 7604 docs/dev/TRACING.md
PR 7642 docs/user/USAGE.md
PR 7648 docs/dev/debt/libc.md
PR 7715 docs/dev/debt/libc.md
PR 8150 docs/dev/debt/libc.md
PR 8161 docs/dev/TRACING.md
PR 8417 docs/dev/debt/libc.md
PR 9345 docs/user/USAGE.md
PR 9371 docs/dev/debt/libc.md
PR 9447 docs/dev/debt/libc.md
PR 9468 docs/dev/debt/libc.md
PR 9508 docs/user/USAGE.md
PR 9526 docs/dev/debt/libc.md
PR 9698 docs/dev/debt/libc.md
PR 9735 docs/dev/debt/libc.md
PR 9761 docs/dev/debt/libc.md
PR 9867 docs/dev/debt/libc.md
PR 10002 docs/dev/debt/libc.md
PR 10007 docs/dev/debt/libc.md
PR 10158 docs/dev/debt/libc.md
PR 10165 docs/dev/debt/libc.md
PR 10193 docs/user/USAGE.md
PR 10200 docs/dev/debt/libc.md
PR 10222 docs/dev/debt/libc.md
PR 10229 docs/dev/debt/libc.md
PR 10234 docs/dev/debt/libc.md
PR 10249 core/relinker/elfpatcher/src/macos/MacOsImage.cpp
PR 10315 docs/dev/debt/libc.md
PR 10317 docs/dev/debt/libc.md
PR 10321 docs/dev/debt/libc.md
PR 10325 docs/dev/debt/libc.md
PR 10328 docs/dev/debt/libc.md
PR 10345 docs/dev/debt/libc.md
PR 10405 docs/dev/debt/libc.md
PR 10467 docs/dev/debt/libc.md
PR 10468 docs/dev/debt/libc.md
PR 10506 docs/dev/debt/libc.md
PR 10510 docs/dev/debt/libc.md
PR 10512 docs/dev/debt/libc.md
PR 10529 docs/dev/debt/libc.md
PR 10565 docs/dev/debt/libc.md
PR 10792 core/relinker/CMakeLists.txt
docs/dev/debt/relinker.md
docs/user/USAGE.md
PR 10861 core/relinker/CMakeLists.txt
core/relinker/elfpatcher/include/elfpatcher/macos/MacOsImage.hpp
core/relinker/elfpatcher/src/macos/MacOsImage.cpp
core/relinker/elfpatcher/src/macos/MacOsMachOPatcher.cpp
docs/dev/debt/relinker.md
PR 11139 docs/dev/debt/libc.md
PR 11270 docs/dev/debt/libc.md

@jotitadev jotitadev left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Validated on Apple Silicon with Rosetta 2, which complements your run (your machine has no Rosetta, so the converted x86-64 Mach-O was not executed there).

  • Machine: Apple M2 Max, macOS 27.2, AppleClang, relinker-only Release.
  • Head: 989866c, on top of current main a6dcea9 (merged clean).

macos_trace_imports goes through its full execution path here: it builds the x86-64 dylibs, relinks each input with --macos, and runs the converted Mach-O under Rosetta (/usr/bin/arch -x86_64). All import-tracing exit codes are exercised end to end:

  • untraced → 11 (guest shared runs), traced → 77 (redirected through TraceImport_nid_no_patch),
  • --macos-fixed-base 0x400000 → 77 (tracing survives the fixed base),
  • data import (symbol_info 0x11) → 11 and weak import (0x22) → 11 (neither is traced),
  • unsupported libc (no recorder) → nonzero with TraceImport_nid_no_patch in stderr,
  • --trace-imports without libc.prx → exit 2 --trace-imports requires libc.prx; with --lazy-binding → conflict error.

The writable-segment stub (store the thunk, no page-protection change) holds up: the traced binaries actually redirect at run time under Rosetta, not just at relink time.

Full relinker-only CTest, both targets on this machine:

  • arm64: 230/230 .
  • x86-64 under Rosetta (-DCMAKE_OSX_ARCHITECTURES=x86_64 -DPython3_EXECUTABLE=/usr/bin/python3, Python 3.9.6): 230/230 .

Written with AI assistance (Claude Code). I ran the builds and tests above on my machine.

@wachi35676

Copy link
Copy Markdown
Contributor

Windows and Linux non-regression of 989866c merged onto main a6dcea9 (merges cleanly), which complements the macOS runs above. Windows 11 Home 10.0.26300, Ryzen 5 9600X, WinLibs MinGW-w64 GCC 15.2.0, Release relinker-only BUILD_TESTING=ON; Linux: WSL Ubuntu 24.04, GCC 13.3.0, same configuration.

  • ctest (relinker-only): Windows 234/234 and Linux 239/239, both including the new macos_trace_imports, which converts its fixtures on these hosts and skips execution.
  • Negative control: test_macos_trace_imports.py and the updated test_linux_trace_imports.py both fail against main's relinker (FAIL: --trace-imports cannot be used with --macos) and pass with this PR (Windows host).
  • CallTrace.cpp: built for Windows (MinGW GCC 15.2.0) and Linux (GCC 13.3.0) at -O2, main against this PR. On both hosts the preprocessed source differs only in "call " "CallTraceRecord_nid_no_patch" "\n", and the object files are byte-identical (Windows sha256 25d95159…, Linux b5072aee…). The Windows and Linux recorders are therefore unchanged.
  • Four homebrew (PPSA99005, PPSA47402, PPSA99581, PPSA99995) × --windows, --macos --to-rosetta and Linux output, without --trace-imports: all 12 output trees byte-identical to main.
  • With --macos --to-rosetta --trace-imports, PPSA47402, PPSA99005 and PPSA99581 convert with exit code 0, and the executables grow by 8–16 KiB over the untraced ones. They were not run, because there is no macOS host here.

AI-assisted: yes (Claude Code); results above were run on this machine.

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The implementation has strong independent macOS/Rosetta runtime reports, and I independently built the relinker-only target from this exact head on Linux. python3 tools/check_conventions.py --base upstream/main passed, and ctest --test-dir build-review -R '^(macos_.*|linux_trace_imports)$' --output-on-failure passed 10/10, including macOS image generation and Linux trace-import regression. On this Linux host, macos_trace_imports exercised conversion only; its x86-64 Mach-O runtime path is platform-gated and was not run here. The relinker build emitted GCC optimizer warnings in unchanged Linux/test sources, with no build errors.

The branch is still based on 0c6a5bd5938f, 423 commits behind current main (da54c652e30b), and GitHub reports DIRTY; a read-only merge-tree check against current main identifies a docs/user/USAGE.md conflict. Please rebase, resolve the current documentation conflicts, rerun conventions and the focused relinker tests, and update the checklist's current-main claim.

There is also a new live overlap with #9439: both branches change CallTrace.cpp, docs/dev/TRACING.md, and docs/user/USAGE.md. A merge-tree check of the two heads reports conflicts in the two docs (the implementation additions in CallTrace.cpp merge together). Please coordinate with #9439's author on ordering, refresh the overlap report after rebasing, and record any prerequisite in Depends on. The existing overlap report also notes older uncomparable/conflicting relinker PRs; re-evaluate those on the rebased branch. No new duplicate implementation was established, but the ordering/doc conflicts are unresolved.

AI assistance was used to prepare this review.

@EagleBearerYS

Copy link
Copy Markdown
Contributor

@rursache a note on merge order: #10043, an opt-in FSR 1 scaler for the presented image, adds an "Upscaling" section to docs/user/USAGE.md next to the section this PR changes. It declares that it goes after this PR (Depends on: #9408, merge order only), and it will keep your section when it merges main. Nothing is needed from you.

@rursache
rursache requested a review from MarcZX10 October 24, 2026 14:53
@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 24, 2026
@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 25, 2026
@adriadam10 adriadam10 self-assigned this Oct 25, 2026
@adriadam10
adriadam10 force-pushed the feat/macos-trace-imports branch from a56c23b to af74b35 Compare October 25, 2026 12:44
@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 25, 2026
@adriadam10
adriadam10 dismissed MarcZX10’s stale review October 25, 2026 12:51

Dismissing a stale review: the requested changes were addressed after it (new head) and the merge queue blocks on it. Please request changes again if anything is still open.

@maximilianfv maximilianfv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Request changes: on macOS the new import walk calls TraceImport_nid_no_patch with the stack misaligned by 8 bytes (measured below). Everything else I ran works, and the x86-64 recorder logs a traced guest call at run time.

Tested on Apple M1 Pro, macOS 27.0.1 (x86_64 build under Rosetta 2)

Applied at af74b35f8 to mugurc's macOS series (27e968974) merged with main at 2742004ed2, with other open macOS fixes, then toggled only this PR. The docs/dev/debt/relinker.md hunk was left out in that tree because its context differs there (the series adds an entry after that line); it is documentation only, and the PR merges cleanly with main. Full x86_64 build, so the relinker, libc.prx and every target linking librelinker_core were rebuilt in both states. Each test set ran twice per state.

trace_imports|macos_ tests (2 runs) Direct run: guest calls shared in a.prx, which aborts, APS5_TRACE_CALLS=all, real libc.prx
Without this PR 10/10 passed (macos_trace_imports not registered); libc.prx exports none of the call-trace symbols untraced: SIGABRT, no trace; --trace-imports --macos refused: --trace-imports cannot be used with --macos
With this PR 11/11 passed, including macos_trace_imports (its Rosetta execution path ran) untraced: SIGABRT, no trace; traced: the abort report lists #1 thread … a.prx shared from … output+0x10006 and 1 a.prx shared; same with --macos-fixed-base 0x400000 (from 0x400006)

Stack probe: the same fixture with a libc.prx whose TraceImport_nid_no_patch is an -O0 function printing __builtin_frame_address(0) % 16. It reads 8 with --trace-imports, with and without --macos-fixed-base. A correct SysV call reads 0, and the Aps5StartGuest_nid_no_patch stub in the same process reads 0.

Findings

  1. core/relinker/elfpatcher/src/macos/MacOsImage.cpp:105 and :128: the walk pushes 5 registers and subtracts 8 (48 bytes) before call [rip+slot] at :119. LC_MAIN is entered like a called function (rsp ≡ 8 mod 16), so the call goes out with rsp ≡ 8 and TraceImport_nid_no_patch starts misaligned. The Linux TracePrologue this follows (3 pushes + sub rsp,8, 32 bytes) is aligned only because ELF _start begins with rsp ≡ 0. Under Rosetta the real recorder survived this here. On an Intel Mac, any 16-byte-aligned stack access in the recorder's first call (mapping, new, unordered_map) would fault; I could not run that. The walk also overwrites r12 (lea r12 at :114) without saving it. Fix: save r12 as Linux does, which also restores alignment (6 pushes + 8 = 56 bytes). Add 0x41, 0x54 after 0x53 at :105, and 0x41, 0x5c before 0x5b at :128. In test_macos_trace_imports.py, make HOOK check the alignment, for example if (((unsigned long)__builtin_frame_address(0) & 15) != 0) _exit(13);. The test builds its dylibs without -O, so that check is reliable, and it would have caught this.
  2. core/libs/prx/libc/CMakeLists.txt:777: guest_call_trace, guest_call_trace_abort and guest_call_trace_abort_race still run only on Windows and Linux. As a result, the new Apple paths in CallTrace.cpp (mach_vm_remap thunk area, dyld module snapshot, abort report) have no test in the repository. My direct run above is the only run-time check of them. Enabling the test for x86-64 macOS needs one change: core/libs/tests/GuestCallTrace.cpp does not compile with Apple clang as it is, because __builtin_sysv_va_list is an unknown type there (tried here against this PR's libc.prx), so the variadic case needs plain va_list off Windows. If that is out of scope, add a full-build test instead (macos_trace_imports also runs in relinker-only builds, where libc.prx is not built): run one traced case against the built libc.prx with APS5_TRACE_CALLS=all and a provider that aborts, and match last traced system library calls. The run above shows this works on Rosetta hosts.
  3. core/libs/prx/libc/src/specifics/x86_64/CallTrace.cpp:186: the macOS SnapshotModules records every segment with nonzero initprot into the 256-entry ModuleMap (:155) and stops silently when it is full. An x86-64 C++ program under Rosetta here loads 45 images with 198 such segments before any .prx. A title with a few dozen modules (about 4 segments each) therefore fills the map, and callers in the modules that do not fit print without a module name. Only return addresses are resolved, so record only segments with VM_PROT_EXECUTE (45 in that process), or skip images that have MH_DYLIB_IN_CACHE.
  4. Nit, core/relinker/elfpatcher/src/macos/MacOsImage.cpp:530: (input.TraceImports ? traceFunctionSlot : startSlot) + 8 is traceFunctionSlot + 8 in both cases.

AI-assisted: yes (Claude Code applied the PR, ran the tests and the probes, read the code and wrote this review).

rursache and others added 2 commits October 25, 2026 18:37
Walk each macOS function import before the guest entry and pass it through TraceImport_nid_no_patch, as Linux and Windows already do. An x86-64 libc build records the call and prints it on SIGABRT.
…utable segments

Save r12 in the entry walk so TraceImport_nid_no_patch is called with an aligned stack, record only executable Mach-O segments in the module map, and make the test hook check the alignment.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@adriadam10
adriadam10 force-pushed the feat/macos-trace-imports branch from af74b35 to 44ea38f Compare October 25, 2026 17:40
@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 25, 2026

@maximilianfv maximilianfv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve: at 44ea38fc2 the macOS import walk calls TraceImport_nid_no_patch with a 16-byte-aligned stack (measured below), and macos_trace_imports now fails when the r12 save is removed. Finding 2 can be a follow-up; please squash the two commits (see 5).

Tested on Apple M1 Pro, macOS 27.0.1 (x86_64 build under Rosetta 2)

Applied at 44ea38fc2 to mugurc's macOS series (27e968974) merged with main at 2742004ed2, with other open macOS fixes and #9439 (this head builds on it; the CallTrace.cpp hunks need it), then toggled only this PR. The docs/dev/debt/relinker.md hunk was left out because its context differs in that tree. Full x86_64 build in both states. Each test set ran twice per state.

trace_imports|macos_ tests (2 runs) frame % 16 in TraceImport_nid_no_patch / Aps5StartGuest_nid_no_patch, with and without --macos-fixed-base 0x400000
Without this PR 10/10 passed (macos_trace_imports not registered) not applicable (no macOS import walk)
With this PR 11/11 passed, including macos_trace_imports 0 / 0
Control: this PR without the r12 push and pop, relinker rebuilt macos_trace_imports fails: AssertionError: ('traced', 13, '', '') 8 / 0

Direct run with the built libc.prx, APS5_TRACE_CALLS=all and a provider that aborts: the SIGABRT report lists #1 thread … a.prx shared from 0x0000000140010006 output+0x10006 and 1 a.prx shared. With --macos-fixed-base 0x400000 it lists the same call from 0x0000000000400006 output+0x10006.

Findings from the previous review (at af74b35f8)

  1. Fixed. core/relinker/elfpatcher/src/macos/MacOsImage.cpp:105 pushes r12 after rbx and :128 pops it, so the frame is 6 pushes plus 8 (56 bytes). test_macos_trace_imports.py:18 exits 13 on a misaligned frame, and the control above shows the test catches the old layout.
  2. Open, fine as a follow-up. core/libs/prx/libc/CMakeLists.txt:777 still registers guest_call_trace only on Windows and Linux, and core/libs/tests/GuestCallTrace.cpp:48 still uses __builtin_sysv_va_list. The Apple paths in CallTrace.cpp have no test in the repository; the direct run above is the only run-time check.
  3. Fixed. core/libs/prx/libc/src/specifics/x86_64/CallTrace.cpp:225 records only VM_PROT_EXECUTE segments.
  4. Fixed. MacOsImage.cpp:530 is now traceFunctionSlot + 8.
  5. New nit: commit 66ca5c044 adds a conflict marker at docs/dev/TRACING.md:14 (>>>>>>> af74b35f84 (feat(relinker): trace function imports on macOS)), and 44ea38fc2 removes it. Please squash the two commits so the marker does not land in main's history.

AI-assisted: yes (Claude Code applied the PR, ran the tests, the probe and the control, read the code and wrote this review).

@adriadam10
adriadam10 merged commit cbf76e7 into boykopovar:main Oct 26, 2026
14 checks passed
@adriadam10

Copy link
Copy Markdown
Collaborator

Checked the r12 save/restore (stack 16-byte aligned at TraceImport_nid_no_patch), the executable-only module map and the tlsExports offset against @maximilianfv's review: every point is met at 44ea38fc2, and the approval is on this head. macos_trace_imports fails without the r12 fix (control in that review); conventions and the other checks are green; merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants