Repository navigation
feat(libc): export the num_get and time_get locale objects as data - #8880
Conversation
The public SDK's libSceLibcInternal table declares num_get<char>::id, time_get<char>::id and the num_get<char> vtable as objects; libSceAmpr exported their NIDs as functions that throw, which a data consumer reads as code (boykopovar#8875). libc now defines them the way the other facet ids and the num_put and codecvt vtables are kept: the ids zero, the vtable 15 zeroed slots (two header entries, the two destructor entries and Dinkumware's eleven do_get), and libSceAmpr drops the functions. An import qualified to libSceAmpr.prx falls back to libc, so each object has one identity. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Progress report (8c26eac → 2a796cb)📈 System libraries: 42.34% (+0.01%, -3 removed functions) 🗑️ 3 removed
|
| _ZNSt8time_getIcSt19istreambuf_iteratorIcSt11char_traitsIcEEE2idE_nid_postfix = 9; | ||
| Require(_ZNSt7num_getIcSt19istreambuf_iteratorIcSt11char_traitsIcEEE2idE_nid_postfix == 7 && _ZNSt8time_getIcSt19istreambuf_iteratorIcSt11char_traitsIcEEE2idE_nid_postfix == 9); | ||
| _ZNSt7num_getIcSt19istreambuf_iteratorIcSt11char_traitsIcEEE2idE_nid_postfix = 0; | ||
| _ZNSt8time_getIcSt19istreambuf_iteratorIcSt11char_traitsIcEEE2idE_nid_postfix = 0; |
There was a problem hiding this comment.
The PR description relies on libSceAmpr.prx-qualified imports falling back to the same libc object, but these tests only check the objects through the direct libc-linked test and import_audit provider inventory. Could you add a resolver/entry-stub regression with a scoped libSceAmpr.prx import that verifies its search order reaches libc (or a Windows runtime probe showing both qualified imports resolve to the same address)? That would pin the cross-module identity claim independently of the current resolver implementation.
There was a problem hiding this comment.
I checked the current fallback path: on Windows a host import qualified to libSceAmpr.prx searches that module and then libc.prx, so the intended shared provider looks plausible by inspection. A narrow regression could use WindowsEntryStubBuilder with libraries = {"libSceAmpr.prx", "libc.prx"} and one of these NIDs qualified to Ampr, then assert the search table contains both indexes in that order; the libc-qualified case should resolve directly to the same libc index. That would exercise the resolver plan without needing a proprietary title.
There was a problem hiding this comment.
I added the resolver-plan regression on my fork at MarcZX10:codex/pr-8880-ampr-libc-resolver-test (commit babde2b5bd). It asserts that an Ampr-qualified num_get::id import searches Ampr then libc, while the libc-qualified import searches libc directly. On this PR head, entry_stub_tests builds and passes on Linux; python tools/check_conventions.py --base upstream/main --head HEAD and git diff --check also pass. This verifies the generated search order, not a runtime address comparison; the branch is available to cherry-pick since direct push to your PR branch is unavailable.
There was a problem hiding this comment.
Both are in now. Your search-order regression is cherry-picked as 7157ebb with your authorship; on this head entry_stub_tests passes on Windows as well. The runtime side is 2657a81: windows_import_identity (core/relinker/elfpatcher/tests/WindowsImportIdentityTests.cpp, registered next to windows_dependency_diagnostics). It generates two probes with WindowsEntryStubBuilder, WindowsImportBuilder and WindowsPeWriter, the way #8853's generator and the diagnostics test do, and runs them beside the patched libraries (core/libs/libs, so it depends on libs):
same.exeimports the three NIDs twice each, qualified tolibSceAmpr.prxand tolibc.prx, into six GOT slots, and its entry compares each pair (mov rax,[a]; cmp rax,[b]; jne fail), exiting 0 only if all three resolved to one address.different.execompares the first NID fromlibSceAmpr.prxwithgetpagesizefromlibkernel.prxand must exit 1, so a comparison that could not tell addresses apart would fail the test.
A failed probe's stub output lands in the test's message (an empty library folder gives Failed to load module: ...libSceAmpr.prx, GetLastError: 126). Qualifying the second import to libSceLibcInternal.prx is not possible at this layer, the stub builder only accepts modules from its load list, and the relinker adds libc.prx for that name before this stage (GuestModuleBuilder.cpp), so libc.prx is the direct reference here.
Result on this PC (Windows 11, MinGW GCC 15.2.0, Release): windows_import_identity passes (ctest -R windows_import_identity, 0.2 s) after ninja windows_import_identity_tests, which builds libs first. The description's Tested section lists both.
|
On the progress report's "3 removed functions": those are libSceAmpr's |
|
Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.
Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.
|
MarcZX10
left a comment
There was a problem hiding this comment.
I manually checked the incomparable overlaps the bot listed. #6044 only relocates GuestLocale.cpp and its CMake/test registration as part of suite organization; it does not change the test logic in this PR. #7081 changes shared Sony CRT handler/localeconv plumbing, #7648 changes thread error dispatch, #8352 adds different locale facets/exports (, , ), and #8711 changes string/thread ABI; none duplicates these three , , or vtable objects. #6565's debt entry is unrelated; both documentation entries need to be retained if rebasing/merging. These look like independent changes, though Git may require routine conflict resolution around shared files.
…bjects against libc's windows_import_identity generates two probes with the entry stub builder, as the diagnostics test does, and runs them next to the patched libraries: same.exe imports the three locale objects of boykopovar#8875 twice, qualified to libSceAmpr.prx and to libc.prx, and exits 0 only if each pair resolved to one address; different.exe compares the first with libkernel's getpagesize and must exit 1. That pins the fallback from a host module to libc at run time, independently of how the stub builds its search order. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
MarcZX10
left a comment
There was a problem hiding this comment.
I manually checked the incomparable overlaps the bot listed. #6044 only relocates GuestLocale.cpp and its CMake/test registration as part of suite organization; it does not change the test logic in this PR. #7081 changes shared Sony CRT handler/localeconv plumbing, #7648 changes thread error dispatch, #8352 adds different locale facets/exports (codecvt, numpunct, time_put), and #8711 changes string/thread ABI; none duplicates these three num_get::id, time_get::id, or num_get vtable objects. #6565's debt entry is unrelated; both documentation entries need to be retained if rebasing/merging. These look like independent changes, though Git may require routine conflict resolution around shared files.
(cherry picked from commit babde2b)
|
The new Windows |
…ies are built A relinker-only configuration has no libs target to depend on, so the Windows relinker job could not generate; the test needs the patched libraries and is left out there. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
5144b14 to
0c8fb06
Compare
|
Gated out in 0c8fb06: the |
|
Thanks, that addresses the relinker-only configure failure and keeps the actual address-identity test in the full-build configuration. The rerun has passed the Linux/macOS register-context jobs so far; I will check the remaining CI results before treating the PR checks as clean. |
|
The rerun is now fully green for the applicable jobs: conventions, eligibility/labels, Debug and Release register-context on Ubuntu and macOS, relinker on Linux/macOS/Windows, relinker ASan, and write-watch on both macOS runners all pass. Linux and Windows broad build jobs are skipped by this workflow configuration. |
|
I checked the current head and found the Ampr-qualified locale-object fallback regression already included ( |
MarcZX10
left a comment
There was a problem hiding this comment.
The earlier relinker-only configure failure is fixed, and I independently built and ran entry_stub on Linux (1/1 pass). Current applicable CI checks are green. The PR covers the three data exports identified by #8875 and keeps the unverified facet-call behavior documented as debt. Approved.
Resolves the conflict with boykopovar#8867 in EntryStubTests.cpp: keep its two kernel-variant tests and the BufferUtils include, and this branch's libSceAmpr-to-libc fallback test, with all three called from main. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Console measurement (jailbroken PS5 FTP
Mismatch with this PR head: the vtable is exported as 15 REA: Ghidra/Hopper not usable for these ELF SPRX on Windows here; sizes come from the ELF symbol table only. No title run. |
mazen320 read KfcTPbeaOqg from the console's libSceLibcInternal.sprx symbol table: an object of 136 bytes, 17 pointer slots. Export that size instead of the 15 derived from Dinkumware's crt.vc12 xlocnum, and say in the debt entry that which two virtual functions are added is not known. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@mazen320 thanks, that settles the size. 428e9cb exports the If you still have the module at hand: the sizes of the |
|
Follow-up after installing Ghidra 12.1.4 and re-checking the same console
|
|
Thanks, that matches 428e9cb (17 zeroed slots). If the 15 relocation targets in |
|
@ycbuzi Mapped the 15 relocation targets in
So the console layout is 2 header words + 4 non- |
mazen320 mapped the relocated slots of KfcTPbeaOqg in the console's libSceLibcInternal: offset-to-top, typeinfo, the two destructors, locale::facet::_Incref and _Decref, and the eleven do_get. The debt entry now says so instead of leaving the two extra entries unknown. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks, that answers it: |
|
Checked the 17-slot |
What
The public SDK's libSceLibcInternal table declares
num_get<char>::id(-mLzBSk-VGs),time_get<char>::id(a54t8+k7KpY) and thenum_get<char>vtable (KfcTPbeaOqg) as objects. libSceAmpr exported those NIDs asint(void)functions callingNotImplemented, which a data consumer reads as code bytes rather than a trap (#8875).libc now defines the three the way it keeps the other facet ids and the
num_putandcodecvtvtables: the ids as zerouint64, the vtable as 17 zeroeduintptr_tslots, the 136-byte size ofKfcTPbeaOqgthat mazen320 read from the console's libSceLibcInternal symbol table. mazen320 mapped its relocated slots: the offset-to-top and typeinfo entries, the two destructors,locale::facet::_Increfand_Decref, and the elevendo_getthat Dinkumware'ssrc/xlocnumat crt.vc12f1d47126declares, the reference the other libc entries cite. libSceAmpr drops the three functions. An import qualified tolibSceAmpr.prxis searched there and then in libc (WindowsEntryStubBuilder.cpp), so each object has one identity across the two modules, which is what the report asked about. A debt entry says what the objects are and that a facet call through the vtable faults.Not changed: the
time_getfunctions libSceAmpr keeps (get,_Getcat) are functions in the SDK table and stay as they are.Tested
Windows 11, MinGW GCC 15.2.0, Release,
BUILD_TESTING=ON, onmain7ab4b2e:guest_locale, extended: both ids read zero, take a value and read it back, and are reset; every vtable slot reads zero. Passes. Onmainthe executable cannot link, since libc has no such symbols there.tools/import_audit.pywith the three NIDs qualified tolibSceLibcInternal.prx:implemented 3, absent 0(exit 1 only to note that--libshas no file of that name, whichlibc.prxserves). Qualified tolibSceAmpr.prx:implemented 3, absent 0, exit 0, through the fallback to libc.windows_import_identity(new, Windows): two probes built with the entry stub builder and run beside the patched libraries.same.exeimports the three NIDs qualified tolibSceAmpr.prxand tolibc.prxand exits 0 only if each pair resolved to one address;different.execompares the first withgetpagesizeand must exit 1. Passes (0.2 s).entry_stub(extended by MarcZX10's cherry-picked 7157ebb): the generated search order of alibSceAmpr.prx-qualified import is Ampr then libc, and alibc.prx-qualified one goes to libc directly. Passes on Windows; passes on Linux per the author.python tools/check_conventions.py --base origin/main --head HEADpasses.LocaleSupport.cppand fix(libs): share Sony CRT handler and lconv state between libc and libSceAmpr #7081 touches other lines of the Ampr file; both conflict withmainalready, and whichever lands second re-adds a few lines.Checklist
main; no other open PR implements the same functionsNotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file.mdfiles or images added to the repository (attach them to this PR)🤖 Generated with Claude Code