Skip to content

feat(libc): implement the _Smtx shared mutex calls over libkernel's rwlocks - #12146

Merged
oneandonlydean merged 4 commits into
boykopovar:mainfrom
AlexFigas:feat/libc-shared-mutex
Nov 4, 2026
Merged

oneandonlydean merged 4 commits into
boykopovar:mainfrom
AlexFigas:feat/libc-shared-mutex

Conversation

@AlexFigas

@AlexFigas AlexFigas commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

What

AnyPS5's libc had none of the _Smtx_* calls behind std::shared_mutex and std::shared_timed_mutex. The bundled libc.prx of PPSA21564 (build J03093718) exports all eleven, and this PR adds them as that binary implements them, plus the helper its timed locks call.

Each call is a thin wrapper of a libkernel rwlock call:

Export Binary libkernel call
_Smtx_init 0xc9020 scePthreadRwlockInit(smtx, NULL, "SceLibc_V2Smtx")
_Smtx_init_with_name 0xc8fb0 scePthreadRwlockInit(smtx, NULL, name)
_Smtx_destroy 0xc90d0 scePthreadRwlockDestroy
_Smtx_lock_exclusive, _Smtx_lock_shared 0xc9100, 0xc9130 scePthreadRwlockWrlock, scePthreadRwlockRdlock
_Smtx_try_lock_exclusive, _Smtx_try_lock_shared 0xc9160, 0xc9190 scePthreadRwlockTrywrlock, scePthreadRwlockTryrdlock
_Smtx_timedlock_exclusive, _Smtx_timedlock_shared 0xc91c0, 0xc9210 scePthreadRwlockTimedwrlock, scePthreadRwlockTimedrdlock
_Smtx_unlock_exclusive, _Smtx_unlock_shared 0xc9260, 0xc9290 scePthreadRwlockUnlock
  • Results. 0 stays 0. A kernel code 0x8002xxxx becomes its errno, the low 16 bits. Any other non-zero result becomes 1.
  • Names. _Smtx_init builds SceLibc_V2Smtx with snprintf("SceLibc_%s", "V2Smtx"). _Smtx_init_with_name passes the name only when it is non-null and non-empty (strncmp(name, "", 1)), the rule SyncObjectName already applies to the std::__sce_v2 mutexes. The binary also drops the default name when the process's libc parameter has a non-null field at 0xa0. AnyPS5's scePthreadRwlockInit does not keep names, so that case is not modelled.
  • Timed locks. These pass the absolute timespec through std::__sce_v2::_Timespec64_diff_to_useconds (0xc9600, NID m2SotKzIAw4), which this PR exports too. It reads sceKernelGettimeofday and normalizes the target's nanoseconds by floor division. A target that is not in the future gives 0. Otherwise the result is the difference rounded up to whole microseconds, as seconds * 1000000 + usec in 32 bits, so a wait longer than about 2147 s wraps. The port reads the same realtime microseconds through _Xtime_get_ticks.

libc reaches libkernel through PthreadSyncForwards_nid_no_patch, as the std::__sce_v2::_Mtx_* calls already do. libkernel now registers its nine rwlock functions there. The new entries sit before detach, so #9447's entries appended after yield do not collide with them.

Tested

On Ubuntu 24.04.5 under WSL2 (GCC 13.3, Release), head 7542cc1 (this PR with main 1a5267f merged in; b9bd78e had main 4ac683f and 797464d main 98ddd9c) unless noted. The 72 thread-family tests (guest_libc_shared_mutex, guest_libc_threads, guest_pthread_*, guest_kernel_sys_pthread* and threads*, guest_std_thread*) pass 3 runs on it, guest_libc_shared_mutex passes 20 repeats in a row, and the standalone configure of core/libs/tests/write_watch that the macOS jobs run succeeds now that #13220 is on main. The relinker jobs that failed on b9bd78e failed in check_skips_ctest because main had lost --require-tests in tools/check_skips.py; main has restored it, and check_skips and check_skips_ctest pass on this head.

  • New guest_libc_shared_mutex. It passes 20 runs in a row.
    • Real libkernel rwlocks: exclusive and shared ownership across threads; 11 for a second exclusive lock by its owner, 16 for try-locks that cannot enter, 60 for timeouts, 1 for an unlock without a holder, 22 after destroy. A null handle is created lazily on first lock. scePthreadRwlockTrywrlock on the same handle returns 0x80020010, so the handle is a libkernel rwlock.
    • _Timespec64_diff_to_useconds: past and zero targets give 0; nanoseconds of −2 s and +2 s are normalized; a target 2 s + 1 ns ahead rounds up; 3000 s ahead wraps to 3,000,000,000 as an unsigned 32-bit value; a null pointer throws.
    • Fake forwards: the init name, the null attribute and the timeouts each call passes. Results 0x80020023 → 35, 0x8002ffff → 65535, 0x80550001, −1 and 7 → 1, for every wrapper. Without libkernel registered, a call throws std::runtime_error.
  • Native comparison (on the earlier head cefb2b4; _Timespec64_diff_to_useconds is unchanged since). I mapped the binary's segments into a Linux process, stubbed its sceKernelGettimeofday import and ran its _Timespec64_diff_to_useconds next to this one. They agree on 19,200 edge combinations (nanoseconds at 0, ±1, ±10⁹ − 1, ±10⁹, ±10⁹ + 1, ±2·10⁹, INT64_MIN, INT64_MAX, and so on, against several current times) and on 3,000,000 random targets: 0 mismatches.
  • Thread tests. All 67 pass, main's new guest_pthread_cancel_inactive_wait_* cases included: guest_kernel_sys_pthread*, guest_pthread_*, guest_libc_threads, guest_libc_cnd_*, guest_sce_v2_exceptions and guest_ult_sync_rwlock-*. The built libc.prx exports all twelve NIDs.
  • Conventions. python3 tools/check_conventions.py --base origin/main passes.
  • Overlap. python3 tools/pr_overlap.py --branch feat/libc-shared-mutex lists no pull request with the same exports or conflicting lines.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: none
  • Unblocks: none
  • GPUs tested: n/a
  • AI-assisted: yes

@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Progress report (7feb37e → 7542cc1)

📈 System libraries: 26.39% (+0.02%, +12 implemented functions)

✅ 12 implemented
Library Name
libc _Smtx_destroy_nid_postfix
libc _Smtx_init_nid_postfix
libc _Smtx_init_with_name_nid_postfix
libc _Smtx_lock_exclusive_nid_postfix
libc _Smtx_lock_shared_nid_postfix
libc _Smtx_timedlock_exclusive_nid_postfix
libc _Smtx_timedlock_shared_nid_postfix
libc _Smtx_try_lock_exclusive_nid_postfix
libc _Smtx_try_lock_shared_nid_postfix
libc _Smtx_unlock_exclusive_nid_postfix
libc _Smtx_unlock_shared_nid_postfix
libc _ZNSt8__sce_v228_Timespec64_diff_to_usecondsEPK8timespec_nid_postfix

Changed libraries (implemented exports)

Library Before After Change
libc 1574/1589 (99.06%) 1586/1601 (99.06%) +0.01%

@github-actions

github-actions Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 1835 core/libs/prx/libc/src/Threads.cpp
PR 5021 core/libs/prx/libc/CMakeLists.txt
core/libs/prx/libc/src/Threads.cpp
PR 6819 core/libs/prx/libc/CMakeLists.txt
PR 7081 core/libs/prx/libc/CMakeLists.txt
PR 7135 core/libs/prx/libkernel/Pthread/Posix/Thread.cpp
PR 7447 core/libs/prx/libc/CMakeLists.txt
PR 7645 core/libs/prx/libc/CMakeLists.txt
PR 7715 core/libs/prx/libc/CMakeLists.txt
PR 8150 core/libs/prx/libc/CMakeLists.txt
PR 8965 core/libs/prx/libc/CMakeLists.txt
PR 9447 core/libs/prx/libc/include/General.hpp
core/libs/prx/libc/src/Threads.cpp
core/libs/prx/libkernel/Pthread/Posix/Thread.cpp
PR 9929 core/libs/prx/libc/CMakeLists.txt
core/libs/prx/libc/include/General.hpp
PR 10780 core/libs/prx/libc/include/Api.hpp
PR 11621 core/libs/prx/libc/include/Api.hpp
PR 11664 core/libs/prx/libc/include/Api.hpp
PR 11831 core/libs/prx/libc/CMakeLists.txt
core/libs/prx/libc/include/Api.hpp
PR 13449 core/libs/prx/libc/CMakeLists.txt
PR 13486 core/libs/prx/libc/CMakeLists.txt
core/libs/prx/libc/include/Api.hpp
core/libs/prx/libc/include/General.hpp
core/libs/prx/libkernel/Pthread/Posix/Thread.cpp
PR 13793 core/libs/prx/libc/CMakeLists.txt
PR 13809 core/libs/prx/libc/CMakeLists.txt
PR 14235 core/libs/prx/libc/CMakeLists.txt
core/libs/prx/libc/include/General.hpp

@B-Howell

B-Howell commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Windows result (the description has WSL2 only).

Setup: Windows 11 Pro 26300, AMD Ryzen 7 7800X3D, WinLibs GCC 15.2.0 r7, Ninja Release, BUILD_TESTING=ON, libs rebuilt (so libc.prx and libkernel.prx both carry the new PthreadSyncForwards_nid_no_patch layout). Head cefb2b4ca5ff9b2c135811606a80862044b6e72c merged onto main cccdaa2734a71d27e6c907325c60fdf2d3baf0af (clean).

Test: ctest -R '^guest_libc_shared_mutex$' --repeat until-fail:20: passes all 20 runs (1.4 s total). That covers the cross-thread exclusive and shared cases, the timeouts, the 0x80020010 try-lock on the same handle, and _Timespec64_diff_to_useconds.

No control: the _Smtx* calls and the rwlock forwards are new, so restoring Threads.cpp from main leaves the test unable to link.

Not tested: Linux on this machine (in the description), macOS, contention beyond what the test does, any title.

AI-assisted (Claude Code).


Follow-up, 2026-10-30 — private frozen-main integration qualification: author 3c592e8141a09309b89548efed776fb62c675b27 privately merged onto frozen main 5ecef095b61e7a406b279c97896f3a8a6fbd6353 as 3c32645cca42043d144150366da723799d35e3d4 (tree e7eccb507f680ffc4c79d1cf5e658a140f476e4f).

Windows 11 Pro 26300 / WinLibs GCC 15.2.0 r7 and Ubuntu 22.04.1 / WSL2 / GCC 11.4.0, Ninja Release: on each platform, the actual selected thread family passed 67/67 on main and 67/67 on the private composition, with zero skips; the unchanged shared-mutex fixture passed 20/20 executions (one registration). Linux's first invocation was loader-traced, followed by 19 normal serial repeats. Both libc/libkernel were freshly rebuilt together with the matching forward-table layout, and Release checks use direct abort calls.

Each platform's actual patched production libc exports all 12 NIDs; fixture imports and live module/loader evidence resolve the same-arm unpatched production libc/libkernel. Patched NID export inspection and source-named fixture execution are separate evidence paths. Main plus only the identical fixture/registration produces the expected compile-availability RED for missing declarations/forward fields; this is not runtime RED.

Focused CPU component evidence for the private composition. Follow-up qualification assisted by Codex.


Follow-up, 2026-10-30 — shared macOS CI prerequisite

At exact head 797464dfc282e61031ea9081f308930a3d4bd060, write-watch (macos-15) and write-watch (macos-15-intel) stop during standalone CMake configuration. Both report that write_watch_execute-fault and write_watch_read-fault are already registered (macos-15 log, macos-15-intel log). The duplicate registrations are in the write-watch CMake file at this head.

Existing PR #13220 commit e9f09f54fabb6408d3e2711b8df10c10112ab00e removes only the second five-line registration block. The independently reviewed prior configure-only RED/GREEN check confirms the fixed source generates 18 unique tests, with each fault mode registered once and every case retaining its 30-second timeout. That evidence is pinned to earlier source compositions; it includes no fixture build or CTest execution and is not a new CI result for this head.

Please integrate the existing #13220 fix into this branch, or rerun the macOS jobs after it lands, so CI can continue on a fresh head.

AI-assisted (Claude Code; Codex follow-up).

@JJenkx

JJenkx commented Oct 28, 2026

Copy link
Copy Markdown
Contributor

Windows run of the current head 6d7b03b730 (rebased onto main 2227f13661). B-Howell's Windows run above was on cefb2b4ca5, and the description's thread tests list only WSL2.

Setup: Windows 11 Enterprise 10.0.22621, Ryzen 9 5950X, WinLibs GCC 15.2.0 UCRT r7, Ninja, Release, BUILD_TESTING=ON. Full build of the PR's own head, not merged onto anything else; it builds without errors.

  • ctest -R '^guest_libc_shared_mutex$' --repeat until-fail:20: passes all 20 runs.
  • ctest -R 'guest_kernel_sys_pthread|guest_pthread_|guest_libc_threads|guest_libc_cnd_|guest_sc' (the description's thread set, which on this tree matches 69 tests): 69/69 pass.

I didn't repeat the native _Timespec64_diff_to_useconds comparison. The libc.prx of PPSA07923, PPSA13934 and PPSA05916 don't export the _Smtx calls, so the titles I have can't be compared side by side here.

AI-assisted: yes (Claude Code).

@AlexFigas

Copy link
Copy Markdown
Contributor Author

@MarcZX10 This is ready for review: head 6d7b03b on main 2227f13, which still merges cleanly. The Tested section lists the runs on this head, and JJenkx posted a Windows run of it above.

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed — changes requested

The wrapper mapping, thread tests, and native time-difference comparison are thoroughly documented, but this branch is based on main 2227f13661, while current main is 52c09d567d2fa453494dd720fcf53a98149468e4 (500 commits ahead). CONTRIBUTING.md requires each branch to be based on current main, so the exact-head results do not yet establish that this large thread/runtime change integrates with the current pthread implementation.

What I ran

  • Reviewed exact head 6d7b03b730560c3772b370811f5b8c25d1a81c07 and its declared base.
  • Ran a merge preview and git diff --check against current main.
  • Checked the author’s focused Linux/Windows tests, native comparison, and current hosted checks.

Results

Check Result
Current-main merge preview Clean, but current main is not in the branch history
git diff --check Passed
Focused tests Author reports 20 mutex-test runs and 69 thread tests on the older base; Windows also reports 20 and 69 passes on this head
Native comparison Reported comparison is on an earlier head; the implementation of the time-difference helper is unchanged by the rebase
Current product jobs Skipped by hosted PR CI

Please refresh onto current main and rerun the focused shared-mutex and pthread suites on that exact integrated head. I did not run a local build or the thread tests.

…wlocks

The bundled libc.prx (J03093718) implements std::shared_mutex's _Smtx_* calls as thin wrappers of scePthreadRwlock*: a zero result stays 0, a 0x8002xxxx kernel code becomes its errno and any other result becomes 1. _Smtx_init names the lock SceLibc_V2Smtx, _Smtx_init_with_name passes a non-empty name, and the timed locks turn the absolute timespec into microseconds with std::__sce_v2::_Timespec64_diff_to_useconds, which is exported as well.
@AlexFigas
AlexFigas force-pushed the feat/libc-shared-mutex branch from 6d7b03b to 3c592e8 Compare October 29, 2026 10:17
@AlexFigas

Copy link
Copy Markdown
Contributor Author

@MarcZX10 Rebased onto main 1b580f1: head 3c592e8, which merges cleanly with current main 1d8fd64. On this head guest_libc_shared_mutex passes 20 runs in a row, the 67 pthread and thread tests listed in the description pass, and the built libc.prx exports all twelve NIDs.

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-checked — changes requested

The new report gives 20 runs of guest_libc_shared_mutex, the listed pthread/thread suite, and all twelve export checks on 3c592e8141. The branch is based on 1b580f13cf; current main is ba5b5330b3, and it has changed this PR's libc CMake registration since that base. Please merge current main and rerun the focused mutex/thread tests and export check on the resulting exact head. The existing report does not cover that integration.

@AlexFigas

Copy link
Copy Markdown
Contributor Author

@MarcZX10 Merged main 98ddd9c, which is still current main: head 797464d. On this head guest_libc_shared_mutex passes 20 runs in a row, the 67 pthread and thread tests listed in the description pass, and the built libc.prx exports all twelve NIDs.

@AlexFigas
AlexFigas requested a review from MarcZX10 October 30, 2026 09:49

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed — looks good

I reviewed exact head 797464dfc282e61031ea9081f308930a3d4bd060 against PR base 98ddd9c1792bbab016ceed5db267a93d1d61f00a; its merge preview against current upstream main b7488e1ff53e6cfca82af972df1d09fa71e7051d is clean. Local git diff --check and repository conventions passed, and source/ABI plus CMake test registration were inspected; the author reports the exact-head shared-mutex test passing 20/20 and 67 related thread tests on WSL2, as well as zero mismatches in an earlier native-helper comparison. I did not build or run tests locally: hosted Windows CI passed but explicitly skipped this guest test, Linux CI was skipped, and both macOS write-watch jobs failed at CMake configuration on duplicate test names in the separate write-watch test project. Console/title behavior remains unverified. Reviewed implementation

@AlexFigas

Copy link
Copy Markdown
Contributor Author

@MarcZX10 The head 797464d you approved still merges cleanly into current main 8a4fc62. Main has changed Thread.cpp, General.hpp and core/libs/prx/libc/CMakeLists.txt since, so I built the merge result (6bd165e0b0) separately and did not move this branch: on it the 68 thread-family tests (guest_libc_shared_mutex, guest_libc_threads, guest_pthread_*, guest_kernel_sys_pthread*/threads*, guest_std_thread*) pass 3 runs each, and guest_libc_shared_mutex passes 20 repeats in a row.

@B-Howell Thanks for the Windows and Linux runs. The two macOS write-watch failures are not caused by this PR: main's core/libs/tests/write_watch/CMakeLists.txt registers execute-fault and read-fault twice, which #13220 removes. I'm leaving that out of this branch so it stays one topic; it needs only a rerun once #13220 lands.

@B-Howell

B-Howell commented Nov 1, 2026

Copy link
Copy Markdown
Contributor

@AlexFigas Thanks. #13220 has now landed in main (8728104), so the macOS write-watch duplicate registration is fixed there.

One detail on the rerun: in build.yml the eligibility job reads the current main SHA and the write-watch jobs check that SHA out. The failed run 41029358985 recorded main 98ddd9c, which predates #13220. Re-run failed jobs reuses that recorded SHA and would hit the same add_test given test NAME "write_watch_execute-fault" which already exists error. Re-run all jobs, or any new push, goes through eligibility again and picks up current main.

I checked this branch only statically: head 797464d merges cleanly with main b6e3f68, and the merged core/libs/tests/write_watch/CMakeLists.txt is identical to main's, with execute-fault/read-fault registered once. I did not run anything new; my earlier Windows/Linux results keep their original heads.

@AlexFigas

Copy link
Copy Markdown
Contributor Author

@MarcZX10 #13220 has landed, so I merged main 4ac683f into this branch to rerun the macOS write-watch jobs: head b9bd78e, the approved 797464d plus the merge and nothing else. On this head the 68 thread-family tests pass 3 runs each, guest_libc_shared_mutex passes 20 repeats in a row, and the standalone configure of core/libs/tests/write_watch succeeds. git diff --check and check_conventions.py pass, and each overlapping PR gives the same git merge-tree conflicts with this head as with main.

@B-Howell

B-Howell commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

@AlexFigas The two red relinker jobs at b9bd78e (ubuntu-latest g++, windows-latest g++) come from main, not this branch. Each fails only check_skips_ctest with check_skips.py: error: unrecognized arguments: --require-tests. The run merged this head into main 4ac683f. There #14881 (0a0daa5, from a stale base) had dropped --require-tests from tools/check_skips.py, while tools/tests/test_check_skips_ctest.py still passes it. Main restored the file in 929671a, so a run against current main should no longer hit it. This PR does not touch tools/.

At this head both macOS write-watch jobs now pass (macos-15, macos-15-intel), so the duplicate-registration failure is gone now that the merged main includes #13220. Some macOS relinker and register-context jobs were still queued when I checked.

@AlexFigas

Copy link
Copy Markdown
Contributor Author

@MarcZX10 Head 7542cc1 merges current main 1a5267f into the approved branch and nothing else. @B-Howell, thanks for tracing the red relinker jobs: they failed in check_skips_ctest because of main, which has since restored --require-tests in tools/check_skips.py, and both check_skips tests pass on this head. The 72 thread-family tests pass 3 runs each, guest_libc_shared_mutex passes 20 repeats in a row, the standalone configure of core/libs/tests/write_watch succeeds, and git diff --check and check_conventions.py pass. Each uncomparable overlapping PR gives the same git merge-tree conflicts with this head as with main.

@AlexFigas AlexFigas closed this Nov 2, 2026
@AlexFigas AlexFigas reopened this Nov 2, 2026
@AlexFigas AlexFigas closed this Nov 2, 2026
@AlexFigas AlexFigas reopened this Nov 2, 2026
@AlexFigas AlexFigas closed this Nov 3, 2026
@AlexFigas AlexFigas reopened this Nov 3, 2026
@oneandonlydean
oneandonlydean merged commit b593093 into boykopovar:main Nov 4, 2026
61 of 75 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants