You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
libc: malloc, calloc, realloc and memalign throw through the guest call on failure instead of returning NULL with ENOMEM; fix branch ready #5242
When an allocation fails, the guest malloc, calloc, realloc, memalign, aligned_alloc and reallocalign throw a C++ exception (bad_alloc, length_error, runtime_error or system_error) through the guest call, which ends the process because guest code cannot catch it. FreeBSD 11.4 (jemalloc) returns NULL with errno = ENOMEM, or EINVAL for an alignment that is zero or not a power of two. A guest that checks for NULL (the normal pattern) never gets the chance. Cases that abort today: a size the heap rejects (SIZE_MAX, PTRDIFF_MAX + 1, 1 << 40 and up), a calloc product that overflows, a bad alignment, an exhausted arena, and a size that fits the roughly 1 TB guest arena but exceeds the host's commit limit (malloc(300 GiB) throws "paging file too small").
Fix branch: https://github.com/husseinkad/AnyPS5/tree/fix/libc-alloc-failures (four commits on a recent main: the change, then three follow-ups from review). FileAndHeap.cpp catches the failures and sets errno; GuestHeap.cpp turns an arena runtime_error and the host out-of-memory codes (Windows 8, 14 and 1455, Linux ENOMEM) into bad_alloc, and releases the reserved arena range when a commit fails; GuestArena.cpp is left untouched on purpose so it does not conflict with #5218. New tests GuestAllocatorFailures.cpp and rewritten expectations in GuestHeapRounding.cpp and GuestReallocalign.cpp; debt entries in docs/dev/debt/libc.md.
Checks:
guest_allocator_failures crashes on main and passes with the change; guest_heap_rounding, application_heap, guest_reallocalign, mspace, guest_memory and guest_new_handler pass. The test's leak check was weak in the first version (a reviewer removed the release call and it still passed); it now fails with that mutation.
Probed through the guest exports against main: 1,000 failing 300 GiB mallocs all return NULL/ENOMEM with no throw, private bytes flat and the arena's lowest-free address unchanged; a failed realloc(p, huge) leaves p intact and malloc_usable_size unchanged, reallocf frees it and sets ENOMEM; alignments 0, 3, 24 and SIZE_MAX give EINVAL; errno is untouched on success over 1,000 mixed multi-threaded calls; with the arena filled to 1 MiB free every failing call gave ENOMEM; 6 threads allocating alongside 300 failing huge mallocs showed no corruption or deadlock; zero bytes stay committed after a failed commit. Side effect: posix_memalign and reallocf on arena exhaustion now return ENOMEM where main threw.
Not covered: memalign and reallocalign setting errno is an assumption (FreeBSD has neither); FreeBSD 11.4 resizes realloc(ptr, 0) to a one-byte block, but this repo frees the block and returns NULL and existing tests assert that (recorded in the debt entry). The Linux commit-failure mapping was read, not built or run. Only Windows was built and run.
Carried the allocation failure fix into PR #5289: #5289. Linux guest_allocator_failures and guest_reallocalign pass (2/2), and conventions pass. The host commit-limit failure case was not exercised locally; the Windows 300 GiB exhaustion behavior is from the issue report.
Activity
MarcZX10 commentedon Oct 13, 2026
Carried the allocation failure fix into PR #5289: #5289. Linux
guest_allocator_failuresandguest_reallocalignpass (2/2), and conventions pass. The host commit-limit failure case was not exercised locally; the Windows 300 GiB exhaustion behavior is from the issue report.