Skip to content

feat(libc): write the called imports to APS5_TRACE_CALLS_FILE every 10 seconds - #9439

Merged
adriadam10 merged 1 commit into
boykopovar:mainfrom
milescoviello:feat/trace-calls-file
Oct 25, 2026
Merged

adriadam10 merged 1 commit into
boykopovar:mainfrom
milescoviello:feat/trace-calls-file

Conversation

@milescoviello

@milescoviello milescoviello commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

What

An executable converted with --trace-imports counts its calls to the libraries in APS5_TRACE_CALLS, but only a crash or an abort report prints them, and then only the 32 most called imports. A run that ends normally, or is killed at a timeout, leaves no record of what it called.

With APS5_TRACE_CALLS_FILE set to a path, the first traced import starts a thread that rewrites that file every 10 seconds. It holds one line per import called so far, most called first: the number of calls, the library and the import name. A run that is killed keeps its last rewrite.

245557 libc.prx gQX+4GDQjpM#guest
3866 libSceAgc.prx 57labkp+rSQ

I used it to list what the 10 retail games on my test host actually call (200 to 476 different imports in 2 to 5 minutes each) and to check which of them reach a function that still has a NotImplemented_nid_no_patch path. The crash report couldn't answer that, because none of those runs crashed in a library.

How

  • CallTrace.cpp: WriteCallCounts takes every import record with a nonzero count, sorts the records by count (stable, so equal counts keep their order), and writes the lines through a std::ofstream on a std::filesystem::path.
  • StartCallCountWriter runs once, when the first traced import sets up the trace. It reads APS5_TRACE_CALLS_FILE, through _wgetenv on Windows as the other path variables do, and starts a detached thread that rewrites the file every 10 seconds.
  • WriteCallCounts_nid_no_patch(const char* utf8Path) writes the same file on demand. It returns false for a null or empty path, or when the file can't be written. The test uses it.
  • ENVIRONMENT.md, TRACING.md and the tracing section of USAGE.md describe the variable.

Tested

  • Rebased onto main 17f5335, with no conflicts, as head ba73390965. guest_call_trace and guest_call_trace_abort pass 3 of 3 on Linux (Ubuntu 26.04.1, GCC 15.2, Release), and 3 of 3 on the Windows 11 VM (WinLibs GCC 15.2.0, Release) on this head and on its merge base. Conventions are clean, and the overlap scan shows no line-level overlap. The Windows runs cover the direct path-writing call; the 10-second writer thread is still exercised only by the Linux title runs below. The other results are from pre-rebase heads.

Linux (Ubuntu 26.04.1, GCC 15.2, Release), on main 0c6a5bd, then rebased onto a6dcea9. That merge doesn't touch these files.

  • guest_call_trace, guest_call_trace_abort and guest_call_trace_abort_race pass 3 of 3.

  • The new checks in guest_call_trace run after the existing calls and compare the written file exactly: 80001 Integers, 3 Blocks, 2 Doubles, 1 Mixed, 1 Variadic, in that order, each with the library name it was traced under. They also check that a path under a regular file, a null path and an empty path return false.

  • On main the test doesn't build, because WriteCallCounts_nid_no_patch is new.

  • Titles: Steins;Gate, Ghost of Tsushima, Forspoken, MGS4, Astro Bot, Crimson Desert, MXGP 26, Super Meat Boy 3D, CoD Black Ops II and Borderlands 4, converted with --trace-imports, APS5_TRACE_CALLS=all and APS5_TRACE_CALLS_FILE=calls.txt, for 120 to 300 s each. Every run left its file (204 to 476 lines), including the two that were killed (Ghost of Tsushima at 54 s, Astro Bot at 75 s). These runs used an earlier local version of this change that wrote the same lines unsorted.

  • Windows 11 VM, WinLibs GCC 15.2.0, Release, head f1a1c05 and its merge base a6dcea9: guest_call_trace and guest_call_trace_abort (the two that run on Windows) pass 3 of 3 on both. So the new checks, the exact file contents included, pass there.

Not tested: macOS, and the 10-second writer thread on Windows (the test calls WriteCallCounts_nid_no_patch directly; the titles above ran on Linux).

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: none
  • Unblocks: none
  • GPUs tested: none needed (libc only)
  • AI-assisted: yes (Claude Code)

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request Conflicting files (lines on main) Same exports Same new files
PR 9408 docs/dev/TRACING.md L14

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 1896 docs/dev/ENVIRONMENT.md
PR 3786 docs/user/USAGE.md
PR 5021 docs/user/USAGE.md
PR 5925 docs/user/USAGE.md
PR 6171 docs/user/USAGE.md
PR 6924 docs/dev/ENVIRONMENT.md
docs/user/USAGE.md
PR 7449 docs/dev/ENVIRONMENT.md
PR 7457 docs/dev/ENVIRONMENT.md
PR 7514 docs/dev/ENVIRONMENT.md
PR 7604 docs/dev/ENVIRONMENT.md
docs/dev/TRACING.md
PR 7642 docs/dev/ENVIRONMENT.md
docs/user/USAGE.md
PR 8004 docs/dev/ENVIRONMENT.md
PR 8077 docs/dev/ENVIRONMENT.md
PR 8246 docs/dev/ENVIRONMENT.md
PR 8484 docs/dev/ENVIRONMENT.md
PR 8896 docs/dev/ENVIRONMENT.md
PR 9248 docs/dev/ENVIRONMENT.md
PR 9345 docs/dev/ENVIRONMENT.md
docs/user/USAGE.md
PR 9356 docs/dev/ENVIRONMENT.md
PR 9385 docs/dev/ENVIRONMENT.md
PR 9387 docs/dev/ENVIRONMENT.md
PR 9400 docs/dev/ENVIRONMENT.md
PR 9669 docs/dev/ENVIRONMENT.md
PR 9676 docs/dev/ENVIRONMENT.md
PR 9898 docs/dev/ENVIRONMENT.md
PR 10193 docs/dev/ENVIRONMENT.md
docs/user/USAGE.md
PR 10442 docs/dev/ENVIRONMENT.md
PR 10547 docs/dev/ENVIRONMENT.md

@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 22, 2026
@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 22, 2026

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fixed-capacity import records remain stable while counts are sampled atomically, the output sorts every nonzero record with deterministic tie ordering, and the Linux report covers both the on-demand API and periodic file snapshots during real title runs (including runs later killed). Windows validates the direct path-writing API, while the periodic thread itself is explicitly untested there.

The branch was rebased onto 6ef5a1c4c9, but current upstream main has since advanced to 17f5335562. Please update to current main and rerun the focused guest_call_trace tests plus conventions and overlap. Since the file-path/environment handling has a Windows-specific _wgetenv branch, please also run the updated Windows focused test or state why that environment is unavailable. Hosted product jobs remain skipped. I did not run local tests.

@milescoviello

Copy link
Copy Markdown
Contributor Author

@MarcZX10 rebased onto main 17f5335 as ba73390965, with no conflicts. guest_call_trace and guest_call_trace_abort pass 3 of 3 on Linux, and 3 of 3 on the Windows 11 VM on this head and on its merge base. Conventions are clean, and the overlap scan has no line-level entry.

On Windows the tests cover WriteCallCounts_nid_no_patch, the direct path-writing call that the writer thread repeats. The thread itself only starts when a traced title sets up its imports, and writes after 10 s. No test runs that long, and my traced title runs are on Linux, so the Windows thread remains untested, as the Tested section says.

AI-assisted: yes (Claude Code)

@klosixgaming-arch

Copy link
Copy Markdown
Contributor

Used this branch (main 1a745ae38e + #9439, APS5_TRACE_CALLS=all, about 2 min per title, 15 titles) to see which census stubs and absent imports titles actually call, rather than only link.

Stubs actually called (call count in parentheses):

  • sceSystemServiceParamGetInt: all 13 titles that boot far enough (1 to 8 calls each)
  • kevent (nid_postfix): fc27 (1)
  • _Thrd_id (nid_postfix): ds (2)
  • sceAudioOut2MasteringInit: shsm (1)

Absent imports actually called:

Not covered: runs were not played past title/boot, so anything only reached in gameplay is missing. Titles that abort before tracing (ufc5 at sceConvertKeycodeGetImeKeyboardType, ds without APS5_PLAYGO_UNLISTED_CHUNKS) produced no or short traces. Raw per-title files available.

AI-assisted: yes. Written with AI assistance and reviewed by me.

@milescoviello

Copy link
Copy Markdown
Contributor Author

@klosixgaming-arch thanks, that is exactly the use this was for: what titles call rather than what they link. I added it to Tested.

AI-assisted: yes (Claude Code)

…0 seconds

A traced run that ends without a crash or an abort never reports which
imports it called. With APS5_TRACE_CALLS_FILE set, the first traced import
starts a thread that rewrites that file every 10 seconds with every import
called so far, most called first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@milescoviello

Copy link
Copy Markdown
Contributor Author

@MarcZX10 rebased onto current main cf528e7 as you asked; head a938dd8f20, one commit, no conflicts.

AI-assisted: yes (Claude Code)

@adriadam10 adriadam10 self-assigned this Oct 25, 2026
@adriadam10
adriadam10 dismissed MarcZX10’s stale review October 25, 2026 13:11

Dismissing a stale review: the requested changes were addressed after it (new head) and the merge queue blocks on it. Please request changes again if anything is still open.

@adriadam10
adriadam10 merged commit 7b7d445 into boykopovar:main Oct 25, 2026
17 checks passed
@adriadam10

Copy link
Copy Markdown
Collaborator

Checked the writer against CopyCallCounts_nid_no_patch (same fixed-capacity records, acquire loads, atomic counts), the exact-file checks in guest_call_trace (they cannot build without WriteCallCounts_nid_no_patch), the null/empty/unwritable path cases and the three docs entries. The rebase onto current main that was requested is in; Conventions, relinker and write-watch are green, and the full Build runs in the merge queue; merged.

@milescoviello

Copy link
Copy Markdown
Contributor Author

Thanks for checking the writer and the exact-file tests, and for merging, @adriadam10.

AI-assisted: yes (Claude Code)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants