Skip to content

fix(libc): follow the console's rand, srand and rand_r - #9648

Open
JJenkx wants to merge 3 commits into
boykopovar:mainfrom
JJenkx:windows/libc-rand-console
Open

JJenkx wants to merge 3 commits into
boykopovar:mainfrom
JJenkx:windows/libc-rand-console

Conversation

@JJenkx

@JJenkx JJenkx commented Oct 22, 2026

Copy link
Copy Markdown
Contributor

What

rand, srand and rand_r now follow the libc.prx that titles ship in sce_module/. I read it from disassembly of the copies of PPSA13934, PPSA07923 and PPSA05916, which have the same code; I didn't measure on a console.

On main, both rand and rand_r use FreeBSD's do_rand (Park and Miller on state % 0x7ffffffe + 1, minus one), so:

  • every seeded sequence differs from the console's
  • rand returns values up to 0x7ffffffd
  • rand() matches rand_r() from the same seed

The console uses two different generators.

rand (cpCOXWMgha0) and srand (VPbJwTCgME0): a 64-bit LCG

movabs $0x5851f42d4c957f2d,%rax
imul   state(%rip),%rax
inc    %rax
mov    %rax,state(%rip)      ; initial value 1
shr    $0x20,%rax
and    $0x3fffffff,%eax
  • srand stores its seed zero-extended (mov %edi,%eax; mov %rax,(state)).
  • rand's results stay at or below 0x3FFFFFFF. This is the part titles are most likely to notice: code that divides by its own RAND_MAX can get values above 1.0 on main.

rand_r (dW3xsu3EgFI): the minimal standard generator

  • It computes state * 16807 (imul $0x41a7) modulo 2^31 − 1 on the signed 32-bit state. From 1 that gives 16807, 282475249, 1622650073 (MINSTD), where FreeBSD gives 33613.
  • A zero state becomes 0x2AE1B767 (mov $0x2ae1b767,%eax).

The code is written from that arithmetic. The console's rand keeps its state without a lock; this one keeps g_randLock. The libc debt file records both.

Tested

  • Platform: Windows 11 (10.0.22621), WinLibs GCC 15.2.0 UCRT r7, Release.

  • guest_math now expects the console's results:

    • the rand_r chain from 1
    • the zero, 0x7FFFFFFF, 0x7FFFFFFE and 0xFFFFFFFF states
    • rand after srand(1), srand(0) and srand(0xFFFFFFFF)
    • rand's range over 1000 calls

    Its previous checks expected FreeBSD's values and rand == rand_r. It passes, and so do the other random tests: guest_posix_random, guest_random_state, guest_random, guest_sandbox_random_word. The whole libc suite passes (143/143).

  • Conventions: tools/check_conventions.py passes.

  • Not tested: no title was run, and nothing was measured on a console. random/srandom are not touched; fix(libc): seed random() and its default state as measured on a PS5 #8979 covers those.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: none
  • Unblocks: none
  • GPUs tested: n/a
  • AI-assisted: yes

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.

Pull request Conflicting files (lines on main) Same exports Same new files
PR 8979 docs/dev/debt/libc.md L36-37

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 3965 docs/dev/debt/libc.md
PR 5021 docs/dev/debt/libc.md
PR 5303 docs/dev/debt/libc.md
PR 6044 core/libs/tests/GuestMath.cpp
PR 6158 core/libs/prx/libc/src/Math.cpp
core/libs/tests/GuestMath.cpp
PR 7081 docs/dev/debt/libc.md
PR 7201 core/libs/prx/libc/src/Math.cpp
PR 7648 docs/dev/debt/libc.md
PR 7715 docs/dev/debt/libc.md
PR 8150 docs/dev/debt/libc.md
PR 8306 docs/dev/debt/libc.md
PR 8965 docs/dev/debt/libc.md
PR 9133 docs/dev/debt/libc.md
PR 9408 docs/dev/debt/libc.md
PR 9447 docs/dev/debt/libc.md
PR 9526 docs/dev/debt/libc.md
PR 9527 docs/dev/debt/libc.md
PR 9552 docs/dev/debt/libc.md
PR 9669 docs/dev/debt/libc.md
PR 9763 docs/dev/debt/libc.md
PR 9929 docs/dev/debt/libc.md

@fogydes

fogydes commented Oct 22, 2026

Copy link
Copy Markdown
Contributor

Reviewed; the change checks out. I re-derived MinimalStandardNext against the MINSTD reference recurrence over the full signed range including the wrap boundaries and the negative-state branches, and the rand pipeline (srand's zero-extended 64-bit state, the 32-bit-order shift) reproduces the claimed console values, including the srand(0)/proc-0x5851F42D4C957F2E derivation and the exact test vectors. Values stay in int range everywhere, so no narrowing UB. The lock-free single-state deviation from libc.prx's locking is recorded in the debt entry with its reasoning, which is the behavior the project wants recorded rather than faked. The removal of the no-longer-needed helper has no live callers.

One coverage nit only: the two rand_r boundary cases (0x7ffffffe, 0xffffffff) assert the returned value but not the post-call state, so a bad state write on that branch would pass. Asserting the state next to the value would pin that path.

@JJenkx

JJenkx commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @fogydes. d933925 adds the state checks to both boundary cases: after rand_r on 0x7ffffffe and on 0xffffffff, the test now also requires randomState == 2147466840u. guest_math passes three times out of three on Windows with it.

@MarcZX10 MarcZX10 left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The added boundary-state assertions address the coverage gap raised by @fogydes. I independently checked the recurrence and vectors in MinimalStandardNext, including the 0x7ffffffe/0xffffffff signed-state paths; the implementation and expected values are internally consistent. The debt entry also records the lock behavior difference and that the disassembly has not been measured on hardware. Validation: exact-head conventions and git diff --check pass; hosted relinker, registration, report, eligibility and labels checks pass. Product Linux/Windows jobs were skipped by path filters. I attempted a local exact-head configure, but it stopped before building because the SDL2 submodule is uninitialized and this host lacks XRandR support; no local runtime result is claimed. The author reports Windows libc suite 143/143 and the updated guest_math 3/3. This is generic behavior with Unblocks: none.

I’ve noticed that holding this PR because of #5239 isn’t necessary. It’s fine for PRs to keep coming in; they’ll be handled in due course. Please disregard my earlier request to hold or defer it.

@JJenkx

JJenkx commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @MarcZX10, for the review and the independent checks. Under the #5239 direction this has no title needing it yet, so I'm converting it to a draft. I'll mark it ready when a title run shows the need, or when the queue is under the threshold.

@JJenkx
JJenkx marked this pull request as draft October 22, 2026 09:30
@JJenkx

JJenkx commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto main 8b3a6ba (head e380d13, the same two commits). On Windows (UCRT GCC 15.2, Release), guest_math, guest_math_exact, guest_random, guest_random_state, guest_posix_random and the libc tests pass in three runs out of three. tools/check_conventions.py passes. It stays a draft under the intake rule.

@JJenkx
JJenkx force-pushed the windows/libc-rand-console branch from d933925 to e380d13 Compare October 22, 2026 12:34

@MarcZX10 MarcZX10 left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am withdrawing that hold as a review concern: current CONTRIBUTING.md sets the per-author cap based on merged-PR count and has no cap from 20 merged PRs; the author is well beyond that tier. I found no code issue in this head and am not asking for additional changes. I rechecked the exact patch: git diff --check and tools/check_conventions.py pass. The hosted register-context, relinker (Linux, Windows and macOS), ASan, write-watch, report, eligibility, conventions and labels checks pass; full Linux/Windows product builds are skipped by path filters. I also compiled and ran a small UBSan-enabled harness using the PR’s exact rand/rand_r implementation and current test vectors; it passed. A full project configure was blocked before compilation because the SDL2 submodule is absent and this host lacks XRandR support, so this is not a local guest_math run. The author reports the Windows libc and guest-math suites pass 3/3. The draft status is the author’s choice.

I’ve noticed that holding this PR because of #5239 isn’t necessary. It’s fine for PRs to keep coming in; they’ll be handled in due course. Please disregard my earlier request to hold or defer it.

@JJenkx
JJenkx marked this pull request as ready for review October 22, 2026 12:48
@JJenkx

JJenkx commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

@MarcZX10 thanks for rechecking it. I've marked this ready for review.

@JJenkx
JJenkx marked this pull request as draft October 22, 2026 16:00
@JJenkx

JJenkx commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

Back to draft: @MarcZX10 clarified that the #5239 intake guidance is still active (#9688 (comment)). While the open queue is above 700, a PR should name a title shown to need the change on current main. This one names none yet, so it waits here. I'll mark it ready once a title run shows the need, or once the queue drops below the threshold. The code and test evidence above are unchanged.

@JJenkx
JJenkx marked this pull request as ready for review October 22, 2026 17:09
@MarcZX10

MarcZX10 commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

I’ve noticed that asking you to hold this PR was unnecessary. It’s fine to keep submitting PRs; maintainers will handle them in due course.

JJenkx and others added 2 commits October 22, 2026 18:15
rand and rand_r both used FreeBSD's do_rand (Park and Miller on
state % 0x7ffffffe + 1, minus one), so rand returned values up to
0x7ffffffd and rand(), rand_r() and every seeded sequence differed
from the console. The libc.prx that titles ship (the same code in the
copies of PPSA13934, PPSA07923 and PPSA05916, read from disassembly)
uses two different generators:

- rand steps a 64-bit state, initially 1, as
  state * 0x5851F42D4C957F2D + 1 and returns (state >> 32) & 0x3FFFFFFF,
  so its results stay at or below 0x3FFFFFFF. srand stores the seed
  zero-extended as that state.
- rand_r is the minimal standard generator, state * 16807 modulo
  2^31 - 1 on the signed 32-bit state (1 gives 16807, 282475249,
  1622650073), and a zero state becomes 0x2AE1B767.

guest_math now expects those results: the rand_r chain from 1, the
zero and 0x7FFFFFFF states, rand after srand(1), srand(0) and
srand(0xFFFFFFFF), and rand's range over 1000 calls. Its previous
checks expected FreeBSD's values and rand matching rand_r.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@JJenkx

JJenkx commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto current main 8964c12 without conflicts; head is now e6047b0. Windows (GCC 15.2.0 UCRT r7, Release) build of that exact head: guest_math and the libc tests (12) pass in three runs out of three, and tools/check_conventions.py --base origin/main passes.

@JJenkx
JJenkx force-pushed the windows/libc-rand-console branch from e380d13 to e6047b0 Compare October 22, 2026 22:20
@JJenkx

JJenkx commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

Runtime check of the rebased head e6047b0, adding to the disassembly reading. A scratch loader runs PPSA07923's decrypted libc.prx natively and calls its rand, srand and rand_r. Then it calls this build's rand_nid_postfix, srand_nid_postfix and rand_r_nid_postfix.

The first five values match exactly in every case, both unseeded and after srand(0), srand(1), srand(42) and srand(0xffffffff):

  • unseeded: 408024109 11635919 196474438 117649705 812669700. That's the same as seed 1, so the initial state is 1.
  • seed 0: 0 408024109 …, confirming that the LCG adds 1 before taking the high word.
  • rand_r from state 7: 117649 1977326743 621132276 452154665, and the final state matches too.

On main every one of those sequences differs; for example, seed 1 gives 33613 564950497 … and rand_r from 7 gives 134455 ….

@ntamas94

Copy link
Copy Markdown
Contributor

Two PS4 data points for this change.

PS4 titles ship the same code. The libc.prx in sce_module/ of two PS4 titles (CUSA04865 and CUSA12435, my own dumps, different SDKs: the files differ in size and hash) contains exactly the code you read from the PPSA copies:

leaq   state(%rip), %rcx
movabsq $0x5851f42d4c957f2d, %rax
imulq  (%rcx), %rax
incq   %rax
movq   %rax, (%rcx)
shrq   $0x20, %rax
andl   $0x3fffffff, %eax        ; rand
...
movl   %edi, %eax
movq   %rax, (%rcx)             ; srand, zero-extended
...
movslq (%rdi), %rcx
movl   $0x2ae1b767, %eax        ; rand_r: zero state
imulq  $0x41a7, %rcx, %rcx      ; MINSTD

and the state's initial value in the data segment is 1 in both. So the model here also covers PS4 titles, which load their bundled libc.prx.

The console's system libc differs. A homebrew probe on a PS4 Pro (firmware 10.71), which has no bundled libc.prx and so uses the system one, gets a different generator: rand returns state >> 33 (values up to 0x7FFFFFFF; 20000 calls after srand(1) peak at 0x7FFC2B41), srand(s) stores (uint32_t)(s - 1), the initial state is 0, and rand_r is a tempered 32-bit LCG (state * 1103515245 + 12345, then the MT-style tempering, halved), as in musl. After srand(1) the sequence is 0, 0x2C28FA16, 0x6058C667, ...; rand_r from 0 gives 0x01A71336 with state 0x3039. That only matters for code that runs without a bundled libc.prx; I mention it so the debt entry can say which libc the model follows.

…system libc

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@JJenkx

JJenkx commented Oct 23, 2026

Copy link
Copy Markdown
Contributor Author

Thanks, both data points are useful.

  • PS4 bundled copies: CUSA04865 and CUSA12435 carry the same rand/srand/rand_r code, with an initial state of 1. So the model covers PS4 titles that load their bundled libc.prx as well.
  • System libc: I added a sentence to the debt entry (23ba036, docs only) saying the console's system libc, which code without a bundled libc.prx uses, has a different generator per your PS4 Pro report and is not modelled. I haven't measured that generator myself, so the entry credits it to this thread rather than describing it.

check_conventions.py and git diff --check pass. The code and tests are unchanged since the last tested head.

AI-assisted: yes (Claude Code).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants