Skip to content

feat(libc): export the num_get and time_get locale objects as data - #8880

Merged
adriadam10 merged 9 commits into
boykopovar:mainfrom
ycbuzi:feat/libc-num-get-objects
Oct 22, 2026
Merged

adriadam10 merged 9 commits into
boykopovar:mainfrom
ycbuzi:feat/libc-num-get-objects

Conversation

@ycbuzi

@ycbuzi ycbuzi commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

What

The public SDK's libSceLibcInternal table declares num_get<char>::id (-mLzBSk-VGs), time_get<char>::id (a54t8+k7KpY) and the num_get<char> vtable (KfcTPbeaOqg) as objects. libSceAmpr exported those NIDs as int(void) functions calling NotImplemented, which a data consumer reads as code bytes rather than a trap (#8875).

libc now defines the three the way it keeps the other facet ids and the num_put and codecvt vtables: the ids as zero uint64, the vtable as 17 zeroed uintptr_t slots, the 136-byte size of KfcTPbeaOqg that mazen320 read from the console's libSceLibcInternal symbol table. mazen320 mapped its relocated slots: the offset-to-top and typeinfo entries, the two destructors, locale::facet::_Incref and _Decref, and the eleven do_get that Dinkumware's src/xlocnum at crt.vc12 f1d47126 declares, the reference the other libc entries cite. libSceAmpr drops the three functions. An import qualified to libSceAmpr.prx is searched there and then in libc (WindowsEntryStubBuilder.cpp), so each object has one identity across the two modules, which is what the report asked about. A debt entry says what the objects are and that a facet call through the vtable faults.

Not changed: the time_get functions libSceAmpr keeps (get, _Getcat) are functions in the SDK table and stay as they are.

Tested

Windows 11, MinGW GCC 15.2.0, Release, BUILD_TESTING=ON, on main 7ab4b2e:

  • guest_locale, extended: both ids read zero, take a value and read it back, and are reset; every vtable slot reads zero. Passes. On main the executable cannot link, since libc has no such symbols there.
  • tools/import_audit.py with the three NIDs qualified to libSceLibcInternal.prx: implemented 3, absent 0 (exit 1 only to note that --libs has no file of that name, which libc.prx serves). Qualified to libSceAmpr.prx: implemented 3, absent 0, exit 0, through the fallback to libc.
  • windows_import_identity (new, Windows): two probes built with the entry stub builder and run beside the patched libraries. same.exe imports the three NIDs qualified to libSceAmpr.prx and to libc.prx and exits 0 only if each pair resolved to one address; different.exe compares the first with getpagesize and must exit 1. Passes (0.2 s).
  • entry_stub (extended by MarcZX10's cherry-picked 7157ebb): the generated search order of a libSceAmpr.prx-qualified import is Ampr then libc, and a libc.prx-qualified one goes to libc directly. Passes on Windows; passes on Linux per the author.
  • python tools/check_conventions.py --base origin/main --head HEAD passes.
  • Not run: a title.
  • Overlap: feat(libc): export _Thrd_yield #8352 adds two other facet ids at the same lines of LocaleSupport.cpp and fix(libs): share Sony CRT handler and lconv state between libc and libSceAmpr #7081 touches other lines of the Ampr file; both conflict with main already, and whichever lands second re-adds a few lines.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: libSceAmpr: three locale data NIDs are exported as throwing functions #8875
  • Unblocks: none
  • GPUs tested: n/a
  • AI-assisted: yes

🤖 Generated with Claude Code

The public SDK's libSceLibcInternal table declares num_get<char>::id, time_get<char>::id and the
num_get<char> vtable as objects; libSceAmpr exported their NIDs as functions that throw, which a
data consumer reads as code (boykopovar#8875). libc now defines them the way the other facet ids and the
num_put and codecvt vtables are kept: the ids zero, the vtable 15 zeroed slots (two header
entries, the two destructor entries and Dinkumware's eleven do_get), and libSceAmpr drops the
functions. An import qualified to libSceAmpr.prx falls back to libc, so each object has one
identity.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Progress report (8c26eac → 2a796cb)

📈 System libraries: 42.34% (+0.01%, -3 removed functions)

🗑️ 3 removed
Library Name
libSceAmpr _ZNSt7num_getIcSt19istreambuf_iteratorIcSt11char_traitsIcEEE2idE_nid_postfix
libSceAmpr _ZNSt8time_getIcSt19istreambuf_iteratorIcSt11char_traitsIcEEE2idE_nid_postfix
libSceAmpr _ZTVSt7num_getIcSt19istreambuf_iteratorIcSt11char_traitsIcEEE_nid_postfix

_ZNSt8time_getIcSt19istreambuf_iteratorIcSt11char_traitsIcEEE2idE_nid_postfix = 9;
Require(_ZNSt7num_getIcSt19istreambuf_iteratorIcSt11char_traitsIcEEE2idE_nid_postfix == 7 && _ZNSt8time_getIcSt19istreambuf_iteratorIcSt11char_traitsIcEEE2idE_nid_postfix == 9);
_ZNSt7num_getIcSt19istreambuf_iteratorIcSt11char_traitsIcEEE2idE_nid_postfix = 0;
_ZNSt8time_getIcSt19istreambuf_iteratorIcSt11char_traitsIcEEE2idE_nid_postfix = 0;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR description relies on libSceAmpr.prx-qualified imports falling back to the same libc object, but these tests only check the objects through the direct libc-linked test and import_audit provider inventory. Could you add a resolver/entry-stub regression with a scoped libSceAmpr.prx import that verifies its search order reaches libc (or a Windows runtime probe showing both qualified imports resolve to the same address)? That would pin the cross-module identity claim independently of the current resolver implementation.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I checked the current fallback path: on Windows a host import qualified to libSceAmpr.prx searches that module and then libc.prx, so the intended shared provider looks plausible by inspection. A narrow regression could use WindowsEntryStubBuilder with libraries = {"libSceAmpr.prx", "libc.prx"} and one of these NIDs qualified to Ampr, then assert the search table contains both indexes in that order; the libc-qualified case should resolve directly to the same libc index. That would exercise the resolver plan without needing a proprietary title.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added the resolver-plan regression on my fork at MarcZX10:codex/pr-8880-ampr-libc-resolver-test (commit babde2b5bd). It asserts that an Ampr-qualified num_get::id import searches Ampr then libc, while the libc-qualified import searches libc directly. On this PR head, entry_stub_tests builds and passes on Linux; python tools/check_conventions.py --base upstream/main --head HEAD and git diff --check also pass. This verifies the generated search order, not a runtime address comparison; the branch is available to cherry-pick since direct push to your PR branch is unavailable.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both are in now. Your search-order regression is cherry-picked as 7157ebb with your authorship; on this head entry_stub_tests passes on Windows as well. The runtime side is 2657a81: windows_import_identity (core/relinker/elfpatcher/tests/WindowsImportIdentityTests.cpp, registered next to windows_dependency_diagnostics). It generates two probes with WindowsEntryStubBuilder, WindowsImportBuilder and WindowsPeWriter, the way #8853's generator and the diagnostics test do, and runs them beside the patched libraries (core/libs/libs, so it depends on libs):

  • same.exe imports the three NIDs twice each, qualified to libSceAmpr.prx and to libc.prx, into six GOT slots, and its entry compares each pair (mov rax,[a]; cmp rax,[b]; jne fail), exiting 0 only if all three resolved to one address.
  • different.exe compares the first NID from libSceAmpr.prx with getpagesize from libkernel.prx and must exit 1, so a comparison that could not tell addresses apart would fail the test.

A failed probe's stub output lands in the test's message (an empty library folder gives Failed to load module: ...libSceAmpr.prx, GetLastError: 126). Qualifying the second import to libSceLibcInternal.prx is not possible at this layer, the stub builder only accepts modules from its load list, and the relinker adds libc.prx for that name before this stage (GuestModuleBuilder.cpp), so libc.prx is the direct reference here.

Result on this PC (Windows 11, MinGW GCC 15.2.0, Release): windows_import_identity passes (ctest -R windows_import_identity, 0.2 s) after ninja windows_import_identity_tests, which builds libs first. The description's Tested section lists both.

@ycbuzi

ycbuzi commented Oct 21, 2026

Copy link
Copy Markdown
Contributor Author

On the progress report's "3 removed functions": those are libSceAmpr's int(void) stubs for num_get<char>::id, time_get<char>::id and the num_get<char> vtable, which threw NotImplemented when called but were never callable in the first place: the SDK declares the three NIDs as objects, so a guest reads the address as data, and resolving it to code gave neither a trap nor an object (#8875). They are replaced by data objects in libc, which the report does not count as functions, and the NIDs still resolve for both qualifications: tools/import_audit.py reports implemented 3, absent 0 with the registry qualified to libSceLibcInternal.prx and, through the fallback to libc, also qualified to libSceAmpr.prx.

@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request Conflicting files (lines on main) Same exports Same new files
PR 6565 docs/dev/debt/libc.md L136
PR 9001 docs/dev/debt/libc.md L136
PR 9038 core/libs/prx/libc/src/LocaleSupport.cpp L300
core/libs/tests/GuestLocale.cpp L36
PR 9133 docs/dev/debt/libc.md L136
PR 9170 docs/dev/debt/libc.md L136
PR 9447 docs/dev/debt/libc.md L136
PR 9456 core/libs/prx/libc/src/LocaleSupport.cpp L300
core/libs/tests/GuestLocale.cpp L36, L251
PR 9464 core/libs/prx/libc/src/LocaleSupport.cpp L300
core/libs/tests/GuestLocale.cpp L36, L251
PR 9526 docs/dev/debt/libc.md L136
PR 9527 docs/dev/debt/libc.md L136
PR 9669 docs/dev/debt/libc.md L136
PR 9698 core/libs/prx/libc/src/LocaleSupport.cpp L300
core/libs/tests/GuestLocale.cpp L36, L251
PR 9735 core/libs/tests/GuestLocale.cpp L36, L251
PR 9761 core/libs/tests/GuestLocale.cpp L36, L251

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 3965 docs/dev/debt/libc.md
PR 5021 core/relinker/CMakeLists.txt
docs/dev/debt/libc.md
PR 5303 docs/dev/debt/libc.md
PR 6044 core/libs/tests/GuestLocale.cpp
core/relinker/CMakeLists.txt
PR 6469 docs/dev/debt/libc.md
PR 6879 core/relinker/CMakeLists.txt
docs/dev/debt/libc.md
PR 7081 core/libs/prx/libSceAmpr/Export.cpp
core/libs/prx/libc/src/LocaleSupport.cpp
docs/dev/debt/libc.md
PR 7447 docs/dev/debt/libc.md
PR 7594 core/relinker/elfpatcher/tests/EntryStubTests.cpp
PR 7648 core/libs/prx/libSceAmpr/Export.cpp
docs/dev/debt/libc.md
PR 7715 docs/dev/debt/libc.md
PR 8150 docs/dev/debt/libc.md
PR 8965 docs/dev/debt/libc.md
PR 9408 core/relinker/CMakeLists.txt
docs/dev/debt/libc.md
PR 9770 core/relinker/CMakeLists.txt

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I manually checked the incomparable overlaps the bot listed. #6044 only relocates GuestLocale.cpp and its CMake/test registration as part of suite organization; it does not change the test logic in this PR. #7081 changes shared Sony CRT handler/localeconv plumbing, #7648 changes thread error dispatch, #8352 adds different locale facets/exports (, , ), and #8711 changes string/thread ABI; none duplicates these three , , or vtable objects. #6565's debt entry is unrelated; both documentation entries need to be retained if rebasing/merging. These look like independent changes, though Git may require routine conflict resolution around shared files.

…bjects against libc's

windows_import_identity generates two probes with the entry stub builder, as the diagnostics
test does, and runs them next to the patched libraries: same.exe imports the three locale objects
of boykopovar#8875 twice, qualified to libSceAmpr.prx and to libc.prx, and exits 0 only if each pair
resolved to one address; different.exe compares the first with libkernel's getpagesize and must
exit 1. That pins the fallback from a host module to libc at run time, independently of how the
stub builds its search order.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I manually checked the incomparable overlaps the bot listed. #6044 only relocates GuestLocale.cpp and its CMake/test registration as part of suite organization; it does not change the test logic in this PR. #7081 changes shared Sony CRT handler/localeconv plumbing, #7648 changes thread error dispatch, #8352 adds different locale facets/exports (codecvt, numpunct, time_put), and #8711 changes string/thread ABI; none duplicates these three num_get::id, time_get::id, or num_get vtable objects. #6565's debt entry is unrelated; both documentation entries need to be retained if rebasing/merging. These look like independent changes, though Git may require routine conflict resolution around shared files.

@MarcZX10

Copy link
Copy Markdown
Contributor

The new Windows relinker CI job currently fails during CMake generation: with ANYPS5_RELINKER_ONLY=ON, top-level CMakeLists.txt returns before adding core/libs, so the new add_dependencies(windows_import_identity_tests libs) at core/relinker/CMakeLists.txt references a nonexistent target. The logged failure is The dependency target "libs" ... does not exist before compilation. Could this test be gated out of relinker-only mode, or be restructured to consume built patched libraries only in the full build while keeping the pure stub-builder coverage in the relinker-only test? The standalone entry_stub assertion is useful but does not exercise actual object-address identity.

…ies are built

A relinker-only configuration has no libs target to depend on, so the Windows relinker job could
not generate; the test needs the patched libraries and is left out there.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@ycbuzi
ycbuzi force-pushed the feat/libc-num-get-objects branch from 5144b14 to 0c8fb06 Compare October 21, 2026 04:42
@ycbuzi

ycbuzi commented Oct 21, 2026

Copy link
Copy Markdown
Contributor Author

Gated out in 0c8fb06: the windows_import_identity target and test are registered only when ANYPS5_RELINKER_ONLY is off, since they need the patched libraries that a relinker-only tree never builds. A relinker-only configure with the job's options (-DANYPS5_RELINKER_ONLY=ON -DBUILD_TESTING=ON, Ninja, g++) now generates here, and the full build keeps the runtime test (ctest -R windows_import_identity passes). The address identity itself stays covered by that test in the full build; entry_stub keeps the search-order assertion in both configurations.

@MarcZX10

Copy link
Copy Markdown
Contributor

Thanks, that addresses the relinker-only configure failure and keeps the actual address-identity test in the full-build configuration. The rerun has passed the Linux/macOS register-context jobs so far; I will check the remaining CI results before treating the PR checks as clean.

@github-actions github-actions Bot added the fix error Closes an issue (Fixes #N) label Oct 21, 2026
@MarcZX10

Copy link
Copy Markdown
Contributor

The rerun is now fully green for the applicable jobs: conventions, eligibility/labels, Debug and Release register-context on Ubuntu and macOS, relinker on Linux/macOS/Windows, relinker ASan, and write-watch on both macOS runners all pass. Linux and Windows broad build jobs are skipped by this workflow configuration.

@MarcZX10

Copy link
Copy Markdown
Contributor

I checked the current head and found the Ampr-qualified locale-object fallback regression already included (windowsLibSceAmprObjectImportsFallBackToTheLibcProvider, added in the updated PR history). I configured a fresh Linux relinker-only build on head 0c8fb06573, built entry_stub_tests, and ran ctest --test-dir build -R "^entry_stub$" --output-on-failure: 1/1 passed. The conventions checker and git diff --check upstream/main...HEAD also pass. No additional commit is needed for that coverage.

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The earlier relinker-only configure failure is fixed, and I independently built and ran entry_stub on Linux (1/1 pass). Current applicable CI checks are green. The PR covers the three data exports identified by #8875 and keeps the unverified facet-call behavior documented as debt. Approved.

Resolves the conflict with boykopovar#8867 in EntryStubTests.cpp: keep its two
kernel-variant tests and the BufferUtils include, and this branch's
libSceAmpr-to-libc fallback test, with all three called from main.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 21, 2026
@mazen320

Copy link
Copy Markdown
Contributor

Console measurement (jailbroken PS5 FTP 192.168.0.144, libSceLibcInternal.sprx SHA-256 a2d70cc64c3abc3927662cbe0c3765b74b2385b4b2ad80eb7b942a891f9259fb, parsed with the module's own DT_SYMTAB / DT_STRTAB; no Ghidra).

NID Symbol (SDK) st_info bind st_size storage
-mLzBSk-VGs num_get<char>::id STT_OBJECT STB_WEAK 8 BSS (zero-init; not in file)
a54t8+k7KpY time_get<char>::id STT_OBJECT STB_WEAK 8 BSS (zero-init)
KfcTPbeaOqg num_get<char> vtable STT_OBJECT STB_WEAK 136 file-backed, all zeros (17×8)

libSceAmpr.sprx from the same console has no dynsym entries for these three NIDs (Ampr is 152 STT_FUNC + 2 unrelated STT_OBJECT). That matches moving the providers into libc and dropping the Ampr int(void) stubs.

Mismatch with this PR head: the vtable is exported as 15 uintptr_t slots (120 bytes). On the console the symbol size is 136 (17 slots). Ids of size 8 and zero-init match what you already have. Suggest sizing _ZTVSt7num_get... to [17] so st_size matches the console export; a 15-slot object is short by two pointers relative to the firmware symbol.

REA: Ghidra/Hopper not usable for these ELF SPRX on Windows here; sizes come from the ELF symbol table only. No title run.

mazen320 read KfcTPbeaOqg from the console's libSceLibcInternal.sprx
symbol table: an object of 136 bytes, 17 pointer slots. Export that
size instead of the 15 derived from Dinkumware's crt.vc12 xlocnum, and
say in the debt entry that which two virtual functions are added is
not known.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ycbuzi

ycbuzi commented Oct 21, 2026

Copy link
Copy Markdown
Contributor Author

@mazen320 thanks, that settles the size. 428e9cb exports the num_get<char> vtable (KfcTPbeaOqg) as 17 zeroed slots, the 136 bytes you read, and the debt entry and the description now give that size and its source instead of the 15 I had derived from Dinkumware's crt.vc12 xlocnum. Which two virtual functions the console's build adds is not known, so the entry doesn't name them. guest_locale (which reads every slot), windows_import_identity and entry_stub pass on Windows 11 on both repeats.

If you still have the module at hand: the sizes of the num_put<char> vtable (_ZTVSt7num_putIcSt19ostreambuf_iteratorIcSt11char_traitsIcEEE) and the codecvt<char, char, _Mbstatet> vtable (_ZTVSt7codecvtIcc9_MbstatetE) would show whether the two extra slots are common to every facet. libc on main keeps those as 12 and 16 slots, and 12 is what the same crt.vc12 count gives for num_put.

@mazen320

Copy link
Copy Markdown
Contributor

Follow-up after installing Ghidra 12.1.4 and re-checking the same console libSceLibcInternal.sprx:

  • Raw file bytes at KfcTPbeaOqg are all zero; R_X86_64_RELATIVE fixups fill the object. After applying DT_RELA, the symbol is still st_size 136 and expands to 17 qwords:
    • [0]=0 (offset-to-top)
    • [1]=0x1af350 (typeinfo)
    • [2]..[16] = 15 method pointers
  • So an export of uintptr_t[15] is two slots short of the firmware symbol. uintptr_t[17]{} matches st_size (headers + 15 methods). Slot contents can stay zeroed as debt until facet methods are wired; sizing should still match the console export.

@ycbuzi

ycbuzi commented Oct 21, 2026

Copy link
Copy Markdown
Contributor Author

Thanks, that matches 428e9cb (17 zeroed slots). If the 15 relocation targets in [2]..[16] resolve to symbol names, they would show which two virtual functions the console's num_get<char> adds beyond Dinkumware's two destructors and eleven do_get; the debt entry leaves that open.

@mazen320

Copy link
Copy Markdown
Contributor

@ycbuzi Mapped the 15 relocation targets in [2]..[16] (plus typeinfo at [1]) on the same console libSceLibcInternal.sprx after applying R_X86_64_RELATIVE. Slot [0] is the Itanium offset-to-top word (0); a dynsym entry also sits at VA 0 and is not part of this vtable.

slot role NID demangled hint
1 typeinfo 6ddOFPDvuCo typeinfo for std::num_get<char, …>
2 complete dtor kAay0hfgDJs num_get<…>::~num_get()
3 deleting dtor -1G1iE3KyGI num_get<…>::~num_get() (deleting)
4 facet -EgSegeAKl4 locale::facet::_Incref
5 facet AgxEl+HeWRQ locale::facet::_Decref
6–16 do_get ×11 SmtBNDda5qU … VKdXFE7ualw bool, unsigned short, unsigned, long, unsigned long, long long, unsigned long long, float, double, long double, void*

So the console layout is 2 header words + 4 non-do_get methods + 11 do_get overloads = 17 qwords. There are not two extra do_gets beyond eleven; the earlier “15 slots total” count was collapsing the headers. Names from tools/nid_names.py against the relocated targets; no title run.

mazen320 mapped the relocated slots of KfcTPbeaOqg in the console's
libSceLibcInternal: offset-to-top, typeinfo, the two destructors,
locale::facet::_Incref and _Decref, and the eleven do_get. The debt
entry now says so instead of leaving the two extra entries unknown.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ycbuzi

ycbuzi commented Oct 21, 2026

Copy link
Copy Markdown
Contributor Author

Thanks, that answers it: locale::facet::_Incref and _Decref. I checked the five non-do_get NIDs by hash (6ddOFPDvuCo, kAay0hfgDJs, -1G1iE3KyGI, -EgSegeAKl4, AgxEl+HeWRQ) and they match those names. 6287af9 names the slots in the debt entry and the description. If the console's other facets carry the same two entries, libc's 12-slot num_put vtable on main, counted from crt.vc12 the same way as my 15, may be two short as well; its symbol size would settle that.

@adriadam10
adriadam10 merged commit 9062485 into boykopovar:main Oct 22, 2026
17 checks passed
@adriadam10

Copy link
Copy Markdown
Collaborator

Checked the 17-slot num_get<char> vtable against the console size reported in the thread, the guest_locale check over every slot, the entry_stub fallback order, the three data exports and the debt entry; merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

libSceAmpr: three locale data NIDs are exported as throwing functions

4 participants