Skip to content

fix(libkernel): give Linux guest threads the stack size they request - #1988

Merged
adriadam10 merged 2 commits into
boykopovar:mainfrom
NevesHenry:fix/libkernel-linux-thread-stack-size
Oct 10, 2026
Merged

adriadam10 merged 2 commits into
boykopovar:mainfrom
NevesHenry:fix/libkernel-linux-thread-stack-size

Conversation

@NevesHenry

Copy link
Copy Markdown
Contributor

What

On Linux, scePthreadCreate ignored the requested stack size. Guest threads were started with std::thread, so every thread got the host default stack (RLIMIT_STACK, 8 MiB on most systems) whatever the title passed to scePthreadAttrSetstacksize / pthread_attr_setstacksize. Windows already honours it (_beginthreadex with the requested size).

  • A thread that asks for more than the host default and uses it crashes with SIGSEGV. A thread that asks for 16 MiB and recurses through ~16 MiB dies on main.
  • After the thread started, RunThread → SetStackFromHost replaced stackSize/stackAddress with the host range. scePthreadAttrGet then reported 8 MiB for every thread. guest_pthread_attr only passes because it usually reads the attributes before the new thread gets that far. Pinned to one CPU (taskset -c 0) it fails 2 times in 1000 on main.
  • The host stack size also depended on the user's ulimit -s.

Change (Linux only, Pthread/src/Thread.cpp):

  • Guest threads are created with pthread_create and pthread_attr_setstacksize. PthreadPrivate::_thr (std::thread) becomes pthread_t hostThread. Join and detach use pthread_join / pthread_detach.
  • glibc carves static TLS and the thread control block out of the top of the thread's stack; FreeBSD allocates TLS separately. Asking glibc for exactly the requested size would leave the guest less than it asked for. With a large static TLS, a small request fails with EINVAL (a 16 KiB stack with 512 KiB of TLS). The host stack is therefore requested + reserve, rounded up to the page size. The reserve is everything glibc keeps above the start routine's frame. It is measured once with a probe thread, because it is constant within a process.
  • The new thread records [top - reserve - size, top - reserve) as its guest stack, as Windows records [top - size, top). The creator waits until the range is set before scePthreadCreate returns, as on Windows. NativeThreadArgs is now shared by both platforms. scePthreadAttrGet and sceKernelIsStack report the requested size, and the race above is gone.
  • Threads created without attributes now get DEFAULT_STACK_SIZE (1 MiB) on Linux, as on Windows, instead of the host default.

Not changed: an address passed with scePthreadAttrSetstack / scePthreadAttrSetstackaddr is still ignored on both platforms. The thread runs on a host-allocated stack of that size. This could be a follow-up.

Tested

Linux: WSL2, Ubuntu 22.04, glibc 2.35, GCC 13.4, Release, ulimit -s 8192. No title run.

  • New guest_pthread_stack:
    • Setup: creates threads that request 16 KiB, 64 KiB, 1 MiB and 16 MiB, in a process with 512 KiB of thread_local data. Each thread checks that scePthreadAttrGet reports the requested size and that its frame is inside the reported range. It then recurses through the range down to 8 KiB above its start.
    • On main: it aborts at the first thread, where a 16 KiB thread reports 8 MiB. A variant without the size checks (16 MiB only) dies with SIGSEGV.
    • With this change: it passes. The usable stack below the thread function is requested - 616 bytes for all four sizes.
  • Mutation: dropping the reserve (stack = requested size) fails with Creating guest thread: Invalid argument on the 16 KiB thread.
  • guest_pthread_attr pinned to one CPU: 2/1000 failures on main, 0/3000 with this change.
  • ctest:
    • All 287 tests outside agc_driver* / agc_write_tracking pass.
    • The thread, stack, mutex, condition and semaphore tests pass with --repeat until-fail:20.
    • On this machine's Vulkan device, agc_driver_emulated_color_compare and agc_driver_shader_clock fail and agc_driver_flat_store and agc_driver_global_vcc_base time out, with or without this change.
  • tools/check_conventions.py --base origin/main and tools/check_test_registration.py build pass.
  • Windows: not built (no MinGW GCC 15 here). The Windows code is unchanged; only the #ifdef around NativeThreadArgs moved. A MinGW GCC 10 syntax check of Thread.cpp gives the same errors as main (all from that toolchain's old headers).

Overlap: tools/pr_overlap.py needs git merge-tree --write-tree (git 2.38+), so I checked the open PRs that touch Thread.cpp / Pthread.hpp by hand with git merge-tree. No open PR changes the Linux stack size.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed in TechnicalDebt
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • AI-assisted: yes (Claude Code)

🤖 Generated with Claude Code

@MarcZX10

MarcZX10 commented Oct 9, 2026

Copy link
Copy Markdown
Contributor

I reviewed the complete four-file diff. The Linux pthread_create stack sizing, startup synchronization, reported guest range, and coverage for 16 KiB–16 MiB requests with large TLS are internally consistent, and the test exercises both stack membership and recursion. I do not see upstream checks attached to this head yet. Please run the standard Build and Conventions workflows before requesting final review; the Windows build is useful here too because the thread startup structure and conditional header changed even though the Windows path is intended to remain behaviorally unchanged.

@popcoder565

Copy link
Copy Markdown
Contributor

Windows build check, since the description says Windows wasn't built. Setup: Windows 10 (19045), MinGW-w64 GCC 15.2.0 (WinLibs r7), Release. Merged onto main 745b775 together with #1926, #1950, #1260 and #1988. Full ctest: 507/507.

It builds, and guest_pthread_stack passes.

🤖 Addressed by Claude Code

@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 9, 2026
@oneandonlydean

Copy link
Copy Markdown
Collaborator

This conflicts with main after today's merges; could you rebase? It passed testing and is otherwise ready.

NevesHenry and others added 2 commits October 10, 2026 08:57
scePthreadCreate on Linux started guest threads with std::thread, so every
thread got the host default stack (RLIMIT_STACK, usually 8 MiB) whatever the
title asked for. A thread that asks for more than that and uses it crashes
with SIGSEGV. Once the thread ran, scePthreadAttrGet also reported the host
size instead of the requested one.

Create the thread with pthread_create and a stack of the requested size plus
what glibc keeps at its top (static TLS, the thread control block and the
start_thread frame). That reserve is measured once with a probe thread.
The thread reports [top - reserve - size, top - reserve) as its stack, as
Windows reports [top - size, top), and the creator waits until the range is
set, as on Windows.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…coverage

Co-Authored-By: Claude <noreply@anthropic.com>
@adriadam10

Copy link
Copy Markdown
Collaborator

Rebased on current main, as requested. Two mechanical changes of ours on top:

  • Thread.cpp: main now keeps the current guest thread in CurrentGuestState().current; adapted RunThread and the Linux ThreadGuard accordingly (conflict with the new CMakeLists.txt test entry resolved by keeping both).
  • PthreadStart.cpp (new on main) used _thr, which no longer exists; it now checks pthread_equal(hostThread, pthread_self()).

Checked: build and the guest pthread/mutex/cond/semaphore tests pass 10 times in a row; guest_pthread_stack fails without the Thread.cpp/Pthread.hpp changes. Windows ctest confirmed earlier by @popcoder565. Merging once CI is green.

@adriadam10
adriadam10 force-pushed the fix/libkernel-linux-thread-stack-size branch from aee35dd to f455b42 Compare October 10, 2026 07:05
@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 10, 2026
@adriadam10
adriadam10 merged commit f9cd694 into boykopovar:main Oct 10, 2026
12 checks passed
SleglJan pushed a commit to SleglJan/AnyPS5 that referenced this pull request Oct 10, 2026
…x-thread-stack-size

fix(libkernel): give Linux guest threads the stack size they request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants