[rooklz]

What

_umtx_op threw for the umutex, condition variable and read-write lock operations. 26 of the 67 title and homebrew executables I have import _umtx_op, and these are the operations FreeBSD's libthr builds its mutexes, condition variables and rwlocks on. This implements them as a payload process on a PS5 (firmware 13.40) answers:

Operation	Console
MUTEX_TRYLOCK (4), free / contested-free	0, owner = thread id (`
MUTEX_TRYLOCK / MUTEX_LOCK (5), owned by the caller	0, owner unchanged (no count: one UNLOCK frees it)
MUTEX_TRYLOCK, owned by another thread	EBUSY
MUTEX_LOCK / MUTEX_WAIT (17), owned by another, 1 ms timeout	ETIMEDOUT, UMUTEX_CONTESTED left set
MUTEX_UNLOCK (6), not the owner / unowned	EPERM
MUTEX_WAIT, unowned	0
MUTEX_WAKE (18), contested, no waiters	0, owner becomes 0
UMUTEX_PRIO_INHERIT trylock/unlock	as a normal mutex
UMUTEX_PRIO_PROTECT trylock	EINVAL
SET_CEILING (7) on a non-PP object	EINVAL
CV_SIGNAL (9) / CV_BROADCAST (10), no waiters	0, c_has_waiters 0
CV_WAIT (8), 1 ms timeout	ETIMEDOUT; the mutex is left unlocked, c_has_waiters 0
RW_RDLOCK (12) / RW_WRLOCK (13) / RW_UNLOCK (14), free	state 1 / 0x80000000 / 0
RW_RDLOCK against a writer, 1 ms timeout	ETIMEDOUT, URWLOCK_READ_WAITERS left set
RW_WRLOCK against a reader, 1 ms timeout	ETIMEDOUT, no waiter bit left, rw_blocked_writers 0
RW_UNLOCK, unlocked	EPERM

The owner a mutex records is scePthreadGetthreadid, which on the console equals thr_self. Two answers differ from FreeBSD 11's kern_umtx.c, and this follows the console: relocking a mutex the caller owns succeeds instead of EBUSY, and a reader that times out leaves URWLOCK_READ_WAITERS set. Blocking, contention and wakeups (who is woken on unlock, UMUTEX_CONTESTED when more than one waiter remains, writers before readers unless URWLOCK_PREFER_READER) follow kern_umtx.c. CV_WAIT queues the waiter before it unlocks the mutex, so a signal sent between the unlock and the sleep is not lost. Its timeout is a timespec in uaddr2, relative unless CVWAIT_ABSTIME, on c_clockid with CVWAIT_CLOCKID.

Robust umutexes, unlocking or setting the ceiling of a priority-protected one, MUTEX_WAKE2 and the SEM2, SHM and robust-list operations still throw. Priority inheritance is not applied, and CVWAIT_CHECK_UNPARKING is ignored. The debt entry records this.

Tested

• PS5 (firmware 13.40), the console's _umtx_op from a payload: the table above.
• guest_umtx_op gains MutexOwnershipFollowsTheConsole, MutexLockWaitsForTheOwnerToUnlock, ConditionWaitReleasesTheMutexAndWakes, RwlockStateFollowsTheConsole and RwlockUnlockWakesTheBlockedWriter; the old check that MUTEX_LOCK throws is replaced by operation 7's EINVAL. Linux x86-64 container (Ubuntu 24.04, GCC 13.3, Release): passes 5 of 5 runs.
• Hosted Linux runner, verify run on my fork (Ubuntu 24.04, GCC, Release, lavapipe): 2189 of 2190 pass; the failure, pr_overlap_encoding, is main's on that runner.
• check_conventions.py passes. feat(libkernel): resolve syscall, equeue, shm and threading stubs #9823 also edits _umtx_op's default: lines (it swaps the NotImplemented_nid_no_patch there for a plain throw). It implements none of these operations, but whichever lands second will need a small rebase there.

Checklist

• Based on current main; no other open PR implements the same functions
• One topic per PR; follow-ups go in a new PR
• No comments in code except technical debt
• Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
• No notes, investigation .md files or images added to the repository (attach them to this PR)
• New third-party code is a submodule built from source
• Generic behaviour, not specific to one title
• Depends on: none
• Fixes: none
• Unblocks: none
• GPUs tested: n/a
• AI-assisted: yes

[MarcZX10]

Reviewed exact head eeb5cf0cc0e8cdf1ff73ee03ed6c1b0abe80c91c. The changed source and test translation units pass GCC 12.2 syntax checks with warnings as errors (suppressing pre-existing unused-parameter/unused-variable warnings); git diff --check and the current-main conventions check pass. I built and ran the exact GuestUmtxOp.cpp test with the changed SyncOnAddress.cpp and real TimedWait.cpp, using temporary host hooks for guest thread ID, guest errno, clock and trace calls: 5/5 runs passed, and the same focused suite passed under ASan+UBSan. The PR's hosted product jobs are path-skipped.

I found a functional omission in the read-lock flags. _umtx_op passes its value to RwlockLock only indirectly today: the dispatcher calls RwlockLock(rwlock, writer, ...) and drops value, while the helper derives reader preference only from rwlock->flags. FreeBSD stable/11's do_rw_rdlock uses the operation's fflag as well as rw_flags when deciding whether URWLOCK_WRITE_WAITERS blocks a reader (source). The PR says reader preference affects writer-before-reader behavior, but none of its tests pass that operation flag.

I reproduced the missing case against the exact PR source: with rwlock.state = URWLOCK_WRITE_WAITERS, rwlock.flags = 0, and a short timeout, _umtx_op(RW_RDLOCK, value = URWLOCK_PREFER_READER, ...) returns ETIMEDOUT and leaves state 0x60000000 instead of taking the read lock. Please thread the read-lock value into the helper and add this regression, or provide a PS5 measurement showing the console ignores this flag. Until then the advertised reader-preference behavior is incomplete.

[github-actions]

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request	Conflicting files (lines on main)	Same exports	Same new files
PR 13336	docs/dev/debt/libkernel.md L87-89

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request	Shared files not compared
PR 4630	docs/dev/debt/libkernel.md
PR 5021	docs/dev/debt/libkernel.md
PR 6936	docs/dev/debt/libkernel.md
PR 7088	docs/dev/debt/libkernel.md
PR 7105	docs/dev/debt/libkernel.md
PR 7121	docs/dev/debt/libkernel.md
PR 7135	docs/dev/debt/libkernel.md
PR 7149	docs/dev/debt/libkernel.md
PR 7192	docs/dev/debt/libkernel.md
PR 7447	docs/dev/debt/libkernel.md
PR 7464	docs/dev/debt/libkernel.md
PR 7804	docs/dev/debt/libkernel.md
PR 9156	docs/dev/debt/libkernel.md
PR 9199	docs/dev/debt/libkernel.md
PR 9293	docs/dev/debt/libkernel.md
PR 9297	docs/dev/debt/libkernel.md
PR 9302	docs/dev/debt/libkernel.md
PR 9389	docs/dev/debt/libkernel.md
PR 9557	docs/dev/debt/libkernel.md
PR 9566	docs/dev/debt/libkernel.md
PR 9843	docs/dev/debt/libkernel.md
PR 9938	docs/dev/debt/libkernel.md
PR 10025	docs/dev/debt/libkernel.md
PR 10319	docs/dev/debt/libkernel.md
PR 10442	docs/dev/debt/libkernel.md
PR 10558	docs/dev/debt/libkernel.md
PR 11896	docs/dev/debt/libkernel.md
PR 11939	docs/dev/debt/libkernel.md
PR 12104	docs/dev/debt/libkernel.md
PR 12560	docs/dev/debt/libkernel.md
PR 12970	docs/dev/debt/libkernel.md
PR 13582	docs/dev/debt/libkernel.md
PR 13587	docs/dev/debt/libkernel.md
PR 14103	docs/dev/debt/libkernel.md
PR 14128	docs/dev/debt/libkernel.md
PR 14212	docs/dev/debt/libkernel.md
PR 14264	docs/dev/debt/libkernel.md
PR 14497	docs/dev/debt/libkernel.md

[rooklz]

@MarcZX10 good catch, thanks. Measured on a PS5 (the console's _umtx_op from a payload, 1 ms timeout):

rw_state before	rw_flags	operation value	result	rw_state after
WRITE_WAITERS	0	0	ETIMEDOUT	0x60000000
WRITE_WAITERS	0	URWLOCK_PREFER_READER	0	0x40000001
WRITE_WAITERS	URWLOCK_PREFER_READER	0	0	0x40000001
WRITE_OWNER	0	URWLOCK_PREFER_READER	ETIMEDOUT	0xa0000000

So the console honours the flag from either place, as do_rw_rdlock does. 2900cecebe passes the operation's value into RwlockLock, so reader preference is rw_flags | value. It also adds RwlockReaderPreferenceFollowsTheConsole with exactly these four cases. guest_umtx_op passes 5 of 5 runs in a Linux x86-64 container (Ubuntu 24.04, GCC 13.3, Release).

[MarcZX10]

Reviewed the updated head 2900cecebe859508f4c51a725e1b8662e2a4bd7c. The implementation now passes the RW_RDLOCK operation flags into RwlockLock and combines URWLOCK_PREFER_READER with the rwlock flags. The regression covers the four reported combinations, including reader preference supplied by the operation and a writer-owner timeout. I rebuilt the changed source, test, and TimedWait translation units with GCC 12.2, linked the focused GuestUmtxOp harness with temporary host hooks, and ran it five times successfully. tools/check_conventions.py and git diff --check also pass against current main. The author reports matching PS5 measurements for the same four cases. Hosted product jobs are skipped by PR workflow policy.

[wachi35676]

Stress check of head 2900cecebe85 merged onto main 90f918fa7. Windows 11 with WinLibs GCC 15.2.0, and WSL Ubuntu 24.04 with GCC 13.3, both Release.

• guest_umtx_op passes (Windows 20/20). With main's SyncOnAddress.cpp it aborts with _umtx_op operation 7 not implemented.

• Mutex under contention is correct: 4 threads × 20,000 MUTEX_LOCK/increment/MUTEX_UNLOCK gives 80,000/80,000, and the owner ends at 0.

• Condition variables are fine too: 2 producers and 2 consumers using CV_WAIT/CV_SIGNAL with the umutex move 40,000/40,000 items, 5/5 runs.

• Rwlock loses a wakeup. With 4 threads × 20,000 rounds (every 4th round RW_WRLOCK, the rest RW_RDLOCK, then RW_UNLOCK), every run stops after roughly 1,000 rwlock operations with all threads asleep on a free lock. That's 5/5 on Windows and 5/5 on Linux:

  rw.state=0x60000000 rw.blockedReaders=3 rw.blockedWriters=1
  

  With 2 threads it's the same (blockedReaders=1 blockedWriters=1). Writers-only completes. gdb shows all four workers waiting inside _umtx_op_nid_postfix.

  Cause: in RwlockLock, the predicate loads state, sees it taken, breaks, then does an unconditional state.fetch_or(waiterBit) before sleeping. RwlockUnlock picks whom to wake from the bits in the value its CAS stored, without taking the waiter's bucket lock. An unlock that lands between the waiter's load and its fetch_or sees no waiter bit and wakes nobody. The waiter then publishes its bit and sleeps on a lock that is already free. Setting the bit with a CAS against the value just checked closes the window, as the mutex path already does for UMUTEX_CONTESTED:

  if (!free) {
      if ((current & waiterBit) == 0 && !state.compare_exchange_weak(current, current | waiterBit)) continue;
      blocked.fetch_add(1);
      return false;
  }

  With that change the same stress completes 5/5 (20,000 writes, no torn reads, state 0), and guest_umtx_op still passes. A regression for this could be the mixed reader/writer loop above with a progress deadline.

AI-assisted: yes (Claude Code); results above were run on this machine.

[oneandonlydean]

@rooklz In addition to the rwlock lost wakeup in #10789 (comment) (still open on 2900cecebe), one more thing in SyncOnAddress.cpp:

ConditionWait (around lines 535-541) puts its stack waiter into the global bucket, drops the lock and then calls MutexUnlock. CheckMutex (367-378) accepts PRIO_PROTECT, and MutexUnlock's PRIO_PROTECT branch calls NotImplemented_nid_no_patch, which throws. Nothing removes the queue entry or clears hasWaiters when that unwinds, so the destroyed stack waiter stays queued and a later ConditionWake uses a dangling pointer.

Could you either reject the unsupported mutex before inserting the waiter, or remove the entry on every exceptional exit? A test that checks the queue is clean and reusable after that failure would cover it, without asserting that the priority-protected form has to throw.

[rooklz]

Both fixed at 4b3467ad41.

@wachi35676's rwlock lost-wakeup: applied your CAS-guarded fix exactly (RwlockLock now retries the freedom check on a failed CAS instead of blindly setting the waiter bit after a stale read). Confirmed it was load-bearing: a first attempt that CAS'd the bit in a tight retry loop without falling back to the outer freedom check still deadlocked under a 4-thread mixed read/write stress test (one thread blocks forever after the lock frees between its check and its CAS). With the continue-back-to-freedom-check structure, 25+ consecutive runs of a 4-thread/2000-round mixed RW stress test pass clean.

@oneandonlydean's dangling ConditionWait waiter: ConditionWait now rejects a priority-protected mutex before the waiter is queued, so MutexUnlock's NotImplemented_nid_no_patch throw can no longer unwind past the bucket cleanup.

Added RwlockDoesNotLoseAWakeupUnderContention (4 threads x 2000 rounds, asserts a clean final state) and ConditionWaitRejectsAPriorityProtectedMutexWithoutLeakingAWaiter (asserts the throw happens before queuing and the condition's waiter bucket stays reusable afterward) to GuestUmtxOp.cpp. guest_umtx_op passes 15/15 consecutive runs; check_conventions.py passes.

[oneandonlydean]

@rooklz Thanks, the CAS-guarded waiter publish in RwlockLock and the early PRIO_PROTECT rejection in ConditionWait both read right. Two follow-ups on 4b3467ad41:

1. ConditionWaitRejectsAPriorityProtectedMutexWithoutLeakingAWaiter requires the call to throw. A condvar wait on a priority-protected mutex is a valid form that is just unsupported here, so the test would fail once it is implemented. Could you catch and ignore any exception there and keep only the hasWaiters == 0 and queue-reuse checks?
2. There is a second lost-wakeup window in RwlockLock (SyncOnAddress.cpp around lines 657-662). After the wait returns, the last waiter does blocked.fetch_sub(1) and then ClearRwlockBits(rwlock, waiterBit) outside the bucket lock, while a new waiter publishes its bit by CAS inside the predicate before its blocked.fetch_add(1). If that new waiter's CAS lands between the sub and the clear, its bit is erased, and the owner's unlock sees no waiter bit and wakes nobody. Please clear the bit under the same bucket lock the predicate runs under, or re-check blocked after clearing and set the bit again if a waiter appeared. If you can, extend the mixed reader/writer stress with many short rounds so the narrow window gets more chances to show up.

[rooklz]

Both fixed at `6ffb86cbe6`.

1. Loosened the condvar test to only check `hasWaiters == 0` and queue reuse; it no longer requires the throw.
2. Re-check `blocked` after `ClearRwlockBits` and restore the bit if a new waiter's CAS landed in that window — matches the mechanism you described exactly.

I couldn't reproduce the second race empirically even after 60 extended runs of the widened stress test (8 threads, 6000 shorter rounds, write eligibility spread across all threads) with the fix reverted — the window is evidently narrower than what this harness's scheduling can reliably hit. The fix is a direct translation of the mechanism you described (re-validate after the unsynchronized clear, same as the first fix re-validated after a failed CAS), and it's cheap on the common path (one extra load, only taken when `left == 0`). With the fix restored, 20/20 consecutive runs of the widened stress test pass, as do the other suites. `check_conventions.py` passes.

[oneandonlydean]

@rooklz Thanks, the condvar test now only checks hasWaiters == 0 and queue reuse, which is right. The new recheck after ClearRwlockBits narrows the rwlock window but doesn't close it (SyncOnAddress.cpp around lines 638-667):

• C owns the write lock. A is a writer whose wait timed out and is cleaning up; A decrements blocked to zero.
• B runs the predicate, sees WRITE_OWNER | WRITE_WAITERS, skips the CAS because the bit is already set, and pauses before blocked.fetch_add(1).
• A clears WRITE_WAITERS, reads blocked == 0, skips the restore and returns.
• B increments blocked, queues and sleeps. C unlocks with no waiter bit set, so nobody wakes B.

There is also a gap when B has already counted: C can unlock after A clears the bit but before A restores it, and restoring the bit after the last unlock sends no wake.

The predicate runs under the bucket mutex but A's decrement, clear and recheck don't. Two ways to close it: do A's fetch_sub, clear and recheck under the same BucketOf(key).mutex, so they're atomic with B's load and fetch_add; or have the predicate do blocked.fetch_add(1) before it loads the state (and undo it on the acquire and EAGAIN returns), so A either sees B in blocked or B sees the cleared bit and publishes it. A barrier-controlled test that forces the timed-out-writer interleaving would cover what the stress test can't reliably hit.

[rooklz]

Good catch, and the barrier-test suggestion is right that the stress test can't reliably force this. Took the first option: the whole cleanup (`fetch_sub`, `ClearRwlockBits`, recheck) now runs under `BucketOf(key).mutex` at `2210ddc367` — the same mutex `WaitOnKeyLocked`'s predicate holds for its own CAS and `blocked.fetch_add`. Since both critical sections now share that lock, there's no window left for them to interleave at all: either the new waiter's full publish (CAS + fetch_add) completes first, so the cleanup's recheck sees it and restores the bit, or the cleanup finishes first (bit cleared, count still 0), so the new waiter runs its predicate against a clean post-clear state and sets its own bit from scratch. No partial interleaving is possible either way, so this closes it structurally rather than narrowing the window further.

25/25 runs of the widened stress test still pass, as do the other suites. `check_conventions.py` passes.

[maskym]

Windows result for head 491fe94270, merged into main f759e165d9 (the PR's Tested section names no Windows run).

Host: Windows 11 26300, Intel i7-12700K, WinLibs GCC 15.2.0 (x86_64-ucrt-posix-seh, r6), Release, BUILD_TESTING=ON, Ninja. Build clean.

The tree also carried the open PRs #11945, #11899, #11859, #11772, #11087, #11627, #11354, all merged cleanly, so the run is a joint one; the tests of this PR are guest_umtx_op (1.50 s), all passed (the batch's 14 tests: 14/14). Not run on a PS5.

AI-assisted: yes (Claude Code), the runs and the numbers above are from this machine.

[rooklz]

@oneandonlydean when you have a moment: the timed-out-writer window from your last comment is addressed in 491fe94270 (the head). The last waiter's fetch_sub, ClearRwlockBits and recheck now run under BucketOf(key).mutex, the same lock the predicate holds for its CAS and blocked.fetch_add, so the two can no longer interleave. The branch merges cleanly with current main and the checks are green. Could you take another look?

[wachi35676]

Re-check of our rwlock lost-wakeup report on head 491fe94270 merged onto main 448450275f (clean). Windows 11 with WinLibs GCC 15.2.0 and WSL Ubuntu 24.04 with GCC 13.3, both Release. (4b3467ad41 from the earlier comment is no longer on the branch after the rebase; the same changes are 0ef9031fc0, ad80ae33b9 and 491fe94270.)

• guest_umtx_op: 10/10 on Windows, 10/10 on Linux.
• The same mixed reader/writer stress that hung every run before (4 threads × 20,000 rounds, every 4th RW_WRLOCK, the rest RW_RDLOCK, then RW_UNLOCK) now completes 10/10 on Windows and 10/10 on Linux: writes=20000 torn=0 errors=0 state=0x0. With 2 and 8 threads it completes 3/3 on both. The mutex (80,000/80,000) and condvar (40,000/40,000, hasWaiters=0) stress still pass.
• Negative control on Windows: with the three fix commits' SyncOnAddress.cpp changes reverse-applied (git diff 4f4383535b 491fe94270) and the new tests kept, guest_umtx_op fails 3/3 (0xc0000409 after ~11.5 s, the new RwlockDoesNotLoseAWakeupUnderContention timing out), and the stress hangs again with rw.state=0x60000000 rw.blockedReaders=3 rw.blockedWriters=1. So the new test catches the original bug on this machine.

From my side the lost wakeup is fixed.

AI-assisted: yes (Claude Code); results above were run on this machine.

[MarcZX10]

Re-reviewed exact head 491fe942705cef60c011a19222ef6650ac9b3646 against live main 4c7d275bef89d309313d16bc2bc05d37a1b50be2.

The three follow-up commits address the reported lost-wakeup race structurally: the wait predicate publishes its waiter bit/count under BucketOf(key).mutex, and last-waiter cleanup now decrements, clears, and rechecks under that same mutex. A new waiter therefore cannot publish between the clear and recheck. I also checked the priority-protected condition-wait guard: it throws before deadline parsing or incrementing hasWaiters, and the regression checks that no waiter state leaks.

Current-main checks:

• git merge-tree --write-tree: clean.
• git diff --check: passed.
• python3 tools/check_conventions.py --base upstream/main --head refs/review/upstream-10789: passed.
• Main has changed SyncOnAddress.cpp since the PR base, but those hunks are in the standalone SyncOnAddress wait/wake API; the umtx wait predicate/cleanup changes reviewed here remain intact.

Validation evidence: wachi35676 reports guest_umtx_op 10/10 on Windows and Linux, the 4-thread lost-wakeup stress 10/10 on both, 2- and 8-thread stress 3/3, and the mutex/condvar stress passing. The Windows negative control with the fix commits reversed fails 3/3 and reproduces the stuck waiter state. I did not rerun those platform tests locally; hosted product jobs are skipped.

No blocking finding remains.

[rooklz]

Rebased onto current main (== 1cac0ca433); the only conflict was the libkernel debt entry, resolved to this PR's implemented _umtx_op wording and kept main's other entries. Full relinker build + ctest -j 8 = 540/540, check_conventions.py --base upstream/main clean. No code change since the approval — could I get a one-line re-approval on the new head 83b1015c4f?

[MarcZX10]

Re-checked — new head, looks good

The rebased head 83b1015c4fb99e527b72f458848d0230094758a1 preserves the previously approved implementation and tests; its current-main preview is clean, conventions pass, and hosted Windows passes. The reported 540/540 CTest run is author-provided; I did not build or run it locally.