Windows: resuming a write fault overwrites the guest red zone when AVX state is live

On Windows, a guest thread that takes a write fault on a write-protected shared view resumes with the bytes directly below rsp overwritten when its AVX state is live. System V leaf functions keep locals in that red zone, so the guest then reads garbage. The code is on main (313de1c5) and is not specific to one title.

Mechanism

Direct memory is mapped as section views. WindowsMappings::Collect write-protects the views it collects (core/libs/prx/libc/include/WindowsMappings.hpp, "arm shared memory write tracking").
The vectored handler in core/libs/prx/libc/src/specifics/windows/CrashReport.cpp calls GuestArenaHandleWrite, which restores the protection, and returns EXCEPTION_CONTINUE_EXECUTION.
Windows delivers the exception on the faulting thread's own stack, directly below rsp; the Windows ABI has no red zone. When the thread's AVX state is live, the kernel writes the upper halves of ymm0-ymm15 (0x100 bytes) at the top of that frame. Measured, they start rsp mod 64 bytes below rsp.
With clean AVX state the first 0x120 to 0x140 bytes below rsp are left alone, which is why most code survives.
The bytes are gone before any handler runs, so the handler cannot restore them.
HandleSse4a in the same handler resumes an exception on the guest stack in the same way.

Evidence

Astro Bot (PPSA21564), Windows 11 Pro 10.0.26200, Core Ultra 9 285K, MinGW-w64 GCC 15.2.0, astrobot branch eeda258c plus #296, #349, #355, #286 and #289. Full notes in discussion #357.

Access violation on SceSndzAudioOutMain about 10 s after start, in 2 of 4 launches. The faulting function (eboot +0xec6090) is a leaf that keeps locals at [rsp-0x08] .. [rsp-0x31] without moving rsp, and stores into a ring buffer in direct memory.
With every resumed exception recorded per thread: in 3 of 3 captured crashes the last resumed write fault on the crashing thread was at the same rsp as the crash, a few instructions earlier (+0xec61e1 then +0xec64ab or +0xec64b5; +0xe46400 then +0xe4636d).
In the +0xec64ab case rsp mod 64 is 8 and [rsp-0x10], the saved rbx, reads back as 0.

Standalone reproduction

No AnyPS5 code. g++ -O1 redzone.cpp with MinGW-w64 GCC 15.2.0.

redzone.cpp

#include <windows.h>
#include <cstdint>
#include <cstdio>

static void* g_page;

static LONG WINAPI Resume(EXCEPTION_POINTERS* info) {
    if (info->ExceptionRecord->ExceptionCode != EXCEPTION_ACCESS_VIOLATION) return EXCEPTION_CONTINUE_SEARCH;
    DWORD previous;
    VirtualProtect(g_page, 4096, PAGE_READWRITE, &previous);
    return EXCEPTION_CONTINUE_EXECUTION;
}

extern "C" void Probe(void* target, std::uint64_t* out, std::uint64_t wide);
__asm__(
    ".globl Probe\n"
    "Probe:\n"
    "    movabs $0x1122334455667788, %rax\n"
    "    mov $512, %r9\n"
    "    lea -8(%rsp), %r10\n"
    "1:  mov %rax, (%r10)\n"
    "    sub $8, %r10\n"
    "    dec %r9\n"
    "    jnz 1b\n"
    "    test %r8, %r8\n"
    "    jz 2f\n"
    "    vpcmpeqd %ymm0, %ymm0, %ymm0\n"
    "    vpcmpeqd %ymm1, %ymm1, %ymm1\n"
    "    vpcmpeqd %ymm2, %ymm2, %ymm2\n"
    "    vpcmpeqd %ymm3, %ymm3, %ymm3\n"
    "    vpcmpeqd %ymm4, %ymm4, %ymm4\n"
    "    vpcmpeqd %ymm5, %ymm5, %ymm5\n"
    "    vmovups %ymm0, (%rcx)\n"
    "    jmp 3f\n"
    "2:  movb $1, (%rcx)\n"
    "3:  mov $512, %r9\n"
    "    lea -8(%rsp), %r10\n"
    "4:  mov (%r10), %rax\n"
    "    mov %rax, (%rdx)\n"
    "    add $8, %rdx\n"
    "    sub $8, %r10\n"
    "    dec %r9\n"
    "    jnz 4b\n"
    "    vzeroupper\n"
    "    ret\n");

int main() {
    AddVectoredExceptionHandler(1, Resume);
    volatile char touch[0x2000];
    touch[0] = 0;
    for (int mode = 0; mode < 3; ++mode) {
        g_page = VirtualAlloc(nullptr, 4096, MEM_RESERVE | MEM_COMMIT, mode ? PAGE_READONLY : PAGE_READWRITE);
        static std::uint64_t slots[512];
        Probe(g_page, slots, mode == 2);
        int nearest = -1;
        int changed = 0;
        for (int i = 0; i < 512; ++i) {
            if (slots[i] == 0x1122334455667788ull) continue;
            if (nearest < 0) nearest = i;
            ++changed;
        }
        const char* name = mode == 0 ? "no fault" : mode == 1 ? "byte write faulted and resumed" : "ymm write faulted and resumed";
        if (nearest < 0) std::printf("%s: nothing below rsp changed\n", name);
        else std::printf("%s: %d of 512 qwords changed, the nearest at rsp-0x%x\n", name, changed, (nearest + 1) * 8);
    }
}

no fault: nothing below rsp changed
byte write faulted and resumed: 197 of 512 qwords changed, the nearest at rsp-0x140
ymm write faulted and resumed: 229 of 512 qwords changed, the nearest at rsp-0x40

Workaround and its cost

APS5_NO_WRITE_WATCH=1. Without write watching the driver never reaches WindowsMappings::Collect, so no view is write-protected and no write fault is resumed on a guest stack. Astro Bot: two launches of 240 s and 498 s, no crash on SceSndzAudioOutMain.

Dreaming Sarah (PPSA02929) on main (e1654a97 plus feat(libkernel): implement sceKernelMlock #359): 60 FPS by default, 38 FPS with the switch.
Astro Bot with the switch: 1.6 to 1.9 FPS, then vkAllocateMemory device buffer: Vulkan result -2 (VK_ERROR_OUT_OF_DEVICE_MEMORY) after 239 to 498 s on a 16 GB RTX 5080. A local counter saw 175 760 device buffer allocations in 239 s, with buffers of 289 MB to 1.99 GB allocated over and over; GPU use swings between 7 and 14 GB.
Hypothesis, not established: those allocations are the whole-range copies made when nothing is write-watched. The frame rate with tracking on was not measured for this title.

Possible directions

None chosen here; this looks like a design decision.

Stop write-protecting shared views. Reporting only the shared views as untracked while private memory stays watched is not enough: tried as a local diagnostic, it renders black frames, in Dreaming Sarah on main too.
Back direct memory with private MEM_WRITE_WATCH memory instead of section views.
Remove red-zone use at relink time.

The first two leave the HandleSse4a path as it is.

Guys, I see everything and I'll get on it. And thank you all so much for helping. But now I still need to review more than a hundred pull requests and at the same time complete the optimization, which will remove the last fundamental extra work compared to the PS5 - I want to transfer the recompilation of the shader from the moment of display to the moment of registration.

@boykopovar

@boykopovar about moving shader recompilation from display to registration: in case it saves you time, I split it into small pull requests. Each one can be reviewed and merged on its own, and none changes the generated SPIR-V (checked byte for byte on the 88 shaders the test suite compiles).

perf(shader): build a shader's front end once on first use #471 runs the front end once. On a shader's first use through the cache, decode → resource tracking ran twice (getSource for the plan, findOrCompileVariant again for the first variant). The plan now owns the values it walks, and the first variant takes the program the entry was built with. First use: 126.6M → 70.4M instructions (callgrind, 1754-word compute shader).
perf(agc): build compute shaders' front end when they are registered #473 (depends on perf(shader): build a shader's front end once on first use #471) is the registration step itself, for compute only. RegisterShader hands compute binaries to a worker that builds the cache entry from the header: SH registers, user SGPR count, wave size from specials->dispatch_modifier bit 15. The first dispatch then only specializes and emits. If a dispatch doesn't match the prediction, it builds its own entry as today. One open question: I couldn't confirm that titles pass dispatch_modifier to sceAgcCbDispatch. If you know, that settles it. Pixel and vertex depend on draw state and are not included.
perf(shader): index ValidateProgram by value and block id #470 indexes ValidateProgram by value/block id instead of pointer-keyed hash maps. It was about 20 % of a recompile; now -32 % there, -7…13 % cycles overall.
test(agc): keep shader memory test tables inside one guest page #472 is test only: agc_shader_memory failed under AddressSanitizer because the test used stack arrays as guest pages.

#470–#472 are green on CI; #473 is still running. I measured on synthetic shaders, not on a title. If you can share a few APS5_DUMP_SHADERS=1 dumps from Dreaming Sarah, I can rerun the numbers on real shaders.

@SP4C3B4R-8

Same standalone result on an AMD Zen 3 without AVX-512 (Ryzen 7 5700X, Windows 11, MinGW-w64 GCC 15.2.0, g++ -O1 redzone.cpp), 3 runs:

no fault: nothing below rsp changed
byte write faulted and resumed: 196 of 512 qwords changed, the nearest at rsp-0x130   (0x120 in one run)
ymm write faulted and resumed: 228 of 512 qwords changed, the nearest at rsp-0x30     (0x20 in one run)

On this host Astro Bot hit B in 2 of 2 write-watched launches (rip eboot+0xec64ab, read of 0x4000, and +0xec64b5), details in #357. The enabled XSAVE area here is 832 bytes (XCR0 = 0x7, CPUID 0xD), so the AVX upper halves are its last 256 bytes, the part next to rsp.

A guess about why B depends on the host, not checked: with AVX-512 enabled the area grows to about 2.7 KB, and the components after AVX (opmask and ZMM) would put the AVX halves about 2 KB further below rsp, out of reach of a leaf function's locals. The Core Ultra 9 285K above has no AVX-512 and hits B; SP4C3B4R-8 did not report it in #357. @SP4C3B4R-8, if you have a minute: which CPU is that, and what does redzone.cpp print there? If the guess holds, a fix verified only on an AVX-512 host could look fine while B still happens on others.

Tested with AI assistance (Claude Code).

@QDarkRex

@QDarkRex your guess holds here. My host is an AMD Ryzen 7 7800X3D (Zen 4, AVX-512 enabled), Windows 11 10.0.26300, MinGW-w64 GCC 15.2.0, g++ -O1 redzone.cpp, 3 runs:

no fault: nothing below rsp changed
byte write faulted and resumed: 180 of 512 qwords changed, the nearest at rsp-0x750   (0x770 in one run)
ymm write faulted and resumed: 212 of 512 qwords changed, the nearest at rsp-0x650    (0x670 in one run)

XCR0 = 0xe7 and CPUID 0xD reports an enabled XSAVE area of 2432 bytes, so the nearest overwritten qword is about 0x650 below rsp instead of 0x20-0x140 on your 5700X and the 285K. That is far below the [rsp-0x08] .. [rsp-0x31] locals of the leaf at +0xec6090, which fits B never showing up in my write-watched runs.

Since the System V red zone is only 128 bytes, an AVX-512 host like mine never shows the bug, so a fix for #361 checked only on such a host proves nothing, and I can't be the one to verify it.

Written with AI assistance (Claude Code).

Third host without AVX-512, same result. AMD Ryzen 7 5800X (Zen 3), Windows 11 10.0.26200, MinGW-w64 GCC 15.2.0 r7, g++ -O1 redzone.cpp, 3 runs:

no fault: nothing below rsp changed
byte write faulted and resumed: 196 of 512 qwords changed, the nearest at rsp-0x120   (0x130 in one run)
ymm write faulted and resumed: 228 of 512 qwords changed, the nearest at rsp-0x20     (0x30 in one run)

So this machine shows the clobber and can verify a fix; the AVX-512 hosts above cannot.

Where the two platforms differ, from main (c72d5ce1):

Direct memory is a PhysicalBacking: a memfd on Linux, a pagefile section on Windows (DirectMemory.cpp), mapped shared at every guest address through GuestArenaMap.
On Linux the write tracking of those shared mappings is userfaultfd write-protect. The fault is serviced on the handler thread; the guest thread only blocks, so nothing is written below its rsp.
On Windows the tracking is WindowsMappings::Collect arming PAGE_READONLY on the views and the vectored handler resuming (HandleWrite). That is the only path where a tracked write is delivered as a user-mode exception on the guest thread, and the kernel frame below rsp comes with it. Nothing in the handler can prevent it.
All other guest memory on Windows is already private MEM_WRITE_WATCH memory (WindowsMappings::Commit), which reports writes without any exception. It is the only Windows primitive that does.

So of the three directions in the issue, the second is the one that removes the exception rather than moving it. A sketch that keeps the section only where it is needed:

PhysicalBacking on Windows allocates nothing up front. The first Map of a physical range replaces the placeholder with private MEM_WRITE_WATCH memory (the existing Commit path), so Collect reaches GetWriteWatch for direct memory too and never arms a view.
A second Map of a physical range that is already mapped is the only case that needs a section. Convert then: create the section, PAGE_NOACCESS the first mapping, copy its pages into the section, replace both ranges with views. A guest write during the copy faults and resumes as today, so the window is the conversion itself, not every tracked write.
The GPU import (GuestBufferMemory createHostPointerImport) can import the guest address of private direct memory directly, since no page of it is ever write-protected. MapAlias stays for converted ranges, as now.

What I could not measure here, having no title: how often a title maps the same physical range twice. APS5_TRACE_MEMORY=1 on Dreaming Sarah or Astro Bot, counting sceKernelMapDirectMemory calls whose physical range overlaps an earlier live mapping, would say whether step 2 is a corner case or the common case. If someone with a title can post that count, it settles the design.

I can prototype 1 to 3 and verify on this host. If a different direction is preferred, say so and I'll follow it.

Investigated with AI assistance (Claude Code).

@SP4C3B4R-8

Two data points for this issue, from a Ryzen 7 7800X3D (Zen 4, AVX-512, the same CPU @SP4C3B4R-8 reported). Windows 11 Pro 26200, MinGW-w64 GCC 15.2.0, NVIDIA GeForce RTX 5070 Ti.

1. AVX-512 hosts are affected too, once zmm16-31 are live

The earlier runs on AVX-512 hosts measured the clobber about 0x650 below rsp, but only with zmm16-31 clean. I filled 8 KiB below rsp with a pattern, dirtied one register group, took a write fault, resumed, and found the nearest overwritten qword:

Register group dirtied before the fault	Nearest clobber below `rsp`
none / `xmm` only	0x768
`ymm0-15` upper halves	0x678
opmask `k1`	0x638
`zmm0-15` upper halves	0x428
`zmm16-31` (Hi16_ZMM)	0x28
`zmm16-31`, then `vzeroupper`	0x28

With Hi16_ZMM live, the kernel writes it at the top of the frame, inside the 128-byte red zone, and vzeroupper doesn't clear zmm16-31. Zeroing them (or XRSTOR to init) puts the group back in its init state, and the clobber moves back to about 0x760.

They do become live on guest threads in practice:

Guest code with runtime CPU dispatch. The guest sees the host's cpuid. RetroArch's eboot (PPSA99169) has 740 instructions using zmm16-31, all in one block; on a PS5 (Zen 2) they never run, here they do.
Vulkan drivers called from guest threads. amdvlk64.dll (loaded for this machine's integrated GPU) has 361 instructions using zmm16-31, nvoglv64.dll 2. No Windows system DLL, CRT, GCC runtime or AnyPS5 library uses them. The audio/video libraries that link FFmpeg are the exception (141 each).

The resulting crash, RetroArch running the Dolphin core with write watch on: zlib's inflate_fast keeps locals at -0x48/-0x60/-0x78(%rsp). A write fault in its copy loop, on the inflate output buffer in direct memory, overwrites them, and inflate later passes a ~190 MB length to adler32. It is deterministic: the same crash site in every run (in adler32, usually at the same fault address), also with the process pinned to one CPU. It goes away when the heap's pages are never write-protected, and when zmm16-31 are reset in the vectored handler before resuming.

Neither mitigation closes the window. The handler reset only protects a thread's later faults; the first fault after the registers are dirtied again still clobbers. Clearing zmm16-31 after every Vulkan call (a local experiment) was not enough either, because the guest's own AVX-512 code dirties them again. So, as on the non-AVX-512 hosts, the exception has to go. That supports @frankea's direction.

2. How often a title maps the same physical range twice

@frankea asked for this. Measured with APS5_TRACE_MEMORY=1, counting maps (map direct and remap fixed) whose physical range overlaps a mapping still live at another address:

Title	Direct maps	Of which alias a live physical range	Distinct physical ranges aliased
ccleste (PPSA59322), 40 s	37	0	0
RetroArch (PPSA99169) + Dolphin core, 70 s	471	86	3

In RetroArch the aliasing is almost all one 32 MiB block, mapped once at 0x404fb0000 and then remapped at a dozen CPU addresses (81 extra mappings over the run). That looks like Dolphin's emulated-RAM arena and its mirrors. Of the rest, one 32 MiB block has 4 extra mappings and one 256 KiB block has 1. That's 64.25 MiB aliased out of 4.1 GiB mapped. So step 2 would be rare for a plain RADV title, but would repeat for an emulator core that mirrors its memory.

A side note for a separate issue if wanted: the guest sees the host's cpuid and xgetbv, so it can choose code paths the PS5 doesn't have (AVX-512 here).

Investigated with AI assistance (Claude Code).

#2054 does step 1 of this: direct memory stays write-watched private pages until it is aliased.

Another title and host that hit this. Unity (PPSA07117), Windows 11 10.0.26300, Intel Core Ultra 7 265K (no AVX-512), RTX 4070, WinLibs GCC 15.2.0 r7.

The crash is deterministic, 3 of 3 runs: eboot+0xdad17a, a null read in a leaf function (Unity input-axis update) that keeps a pointer at [rsp-0x8]. Under x64dbg, the last first-chance exception before it is a resumed write fault in the same loop at +0xdad00d (mov dword [rcx+0x9c], 0).
redzone-style test on this host: with ymm1 and ymm9 dirtied before the fault, 30 of 32 qwords in [rsp-0x100, rsp) were overwritten. The nearest was at rsp-0x18.
Cost of the workaround: with APS5_NO_WRITE_WATCH=1 the title runs, but at 24 fps instead of 60. APS5_PROFILE_DRAW shows the GPU busy 4.7 ms per frame. The [guestmem] compare path took 38.9 s of CPU per 10 s (about 355 GB compared), and draws averaged 1 ms each, mostly readTarget and lookup.

This host can verify #2054 once it is rebased. I can run guest_memory and this title on it.

Investigated with AI assistance (Claude Code).

Another title and host that hit this: Disney Epic Mickey: Rebrushed (PPSA15657, UE4.27). Windows 11 10.0.26300, Intel Core i7-13700K (no AVX-512), RTX 4080, WinLibs GCC 15.2.0 r7, Release.

Crash: FATAL 0xc0000005 at app.exe+0x55fac49 (eboot 0x55eac49), a null write after Continue.
The function at eboot 0x55eaad0 is a leaf (a bounding-box tree refit) that keeps a pointer in the red zone at [rsp-0x18] and then stores through it. The pointer reads back as 0, which matches a fault resumed on that thread overwriting the red zone.
With fix(libkernel): map direct memory as write-watched private pages until it is aliased #2054 (head 630b18f0 cherry-picked onto my branch), the game ran more than 14 minutes past that point without this crash.

Another title and host: DREDGE (PPSA09076, Unity), Windows 10, Ryzen 7 5800X3D (Zen 3, no AVX-512), main a11c8f7. Intermittent 0xc0000005 in Job.Worker 4, 2 of 8 runs of 60 to 180 s, at app.exe+0x17f0796 (mov rax,[r8+0x30] with r8 = 0x600000000) and +0x17efe13 (a read of 0x600000004). The first routine loads r8 from [rsp-0x30] and uses vmovups ymm. With APS5_NO_WRITE_WATCH=1: 8 runs of 60 s, no fault. Details and the other checks (no mapping near 0x600000000) are in #3915. I did not test #2054.

AI-assisted (Claude Code).

A way to resume these faults without touching the guest stack: let a debugger process continue them.

With a debugger attached, Windows reports an exception to the debugger before the user-mode dispatch. If the debugger continues it with DBG_CONTINUE, the thread resumes from the kernel's trap frame, and nothing is written below rsp. Measured on an i5-13600K (Windows 11 26300): a leaf function fills 64 qwords below rsp, then writes to a read-only page.

Fault resumed by	Qwords changed below `rsp` (of 64)	Cost per protect, fault and resume
Vectored handler, clean AVX	23–27, nearest `rsp-0x108`	1.9 µs
Vectored handler, `ymm1`/`ymm9` dirty	55–59, nearest `rsp-0x8`	1.9 µs
Debugger process	0, with or without AVX	15.6 µs

Local prototype

A small helper, aps5-faultd.exe, attaches with DebugActiveProcess.
WindowsMappings marks the pages it arms in a shared table, one byte per 16 KiB page, set before the page is protected.
The helper unprotects those pages and queues them in a ring. WindowsMappings drains the ring under its mutex, as HandleWrite records a fault today.
The debugger only runs while every thread is stopped, so it can't race with the guest.
A fault on a page that another fault already reopened is retried.
Every other exception goes to the vectored handler as now, including HandleSse4a.

Result: Palworld (PPSA23126) used to crash at character creation within 30–60 s, at app.exe+0x575bdc0 or +0x4009f3, each a red-zone local read back as 0. With the helper it ran for over 2 minutes there without a crash. It handled 1,600–12,600 faults per 10 s, at the same frame rate as without it.

Cost: each handled fault stops the whole process for one debug event round trip.

Unlike #2054, it keeps the shared views and their tracking unchanged. Would you take this as a PR, opt-in at first?

AI-assisted (Claude Code)

@parl0v

Data point on the red-zone mechanism (thanks @parl0v for #44). It looks host-CPU dependent.

Windows builds the user-mode exception frame (CONTEXT + XSAVE area) right below the faulting rsp, but it writes the XSAVE area with init optimization: a state component that is still in its initial state is skipped. So whether the 128-byte SysV red zone gets overwritten depends on the CPU's XSAVE layout (which component ends up directly under rsp) and on which vector registers are live when the fault happens.

The repro below fills 4 KiB below rsp with sentinels, takes one write fault on a read-only page, and lets a VEH fix the page and resume. It then reports the highest qword that changed. Ryzen AI 9 365 (Zen 5, AVX-512), Windows 11 26200:

XState features 0x8e7, AVX 1, AVX-512F 1
no vector state live   frame 0xd60 bytes; highest overwritten qword: rsp-0x748 .. rsp-0x780
ymm0/ymm5 live         frame 0xd60 bytes; highest overwritten qword: rsp-0x648 .. rsp-0x680
zmm0/zmm5 live         frame 0xd60 bytes; highest overwritten qword: rsp-0x408 .. rsp-0x440
zmm16/zmm31 live       frame 0xd60 bytes; highest overwritten qword: rsp-0x8 .. rsp-0x40  <- inside the 128-byte SysV red zone

(The range covers the eight rsp alignments the repro tries.) On this machine Hi16_ZMM is the last component and sits directly under rsp, so the red zone is only hit while zmm16-31 hold data. None of the code our guest threads run through uses zmm16-31: ntdll, KernelBase, ucrtbase and vcruntime140 don't (checked by disassembly), our libs don't either, and the PS5's Zen 2 guest code has no AVX-512. So here the resumed shared-view faults never reach a guest red zone. That matches Sword of the Sea: with no view arming at all (0 resumed faults) our in-game Wwise heap crash still happens, so that crash is a separate bug.

A CPU without AVX-512 (e.g. an i5-14400F) has no opmask/ZMM components above AVX. So the AVX upper halves (or whatever component comes last there) probably sit right under rsp, and any live ymm register would land in the red zone. That would explain why the boot race reproduces on such machines and not on ours. I haven't been able to test a non-AVX-512 CPU. @parl0v, could you run it? If the ymm0/ymm5 live line says "inside the red zone", that confirms it.

Either way, a clean run on one CPU says nothing about another, so leaf framing (--frame-red-zone) or not resuming faults on guest threads are the host-independent fixes.

rzxsave.c (gcc/clang, x64: gcc -O1 rzxsave.c -o rzxsave.exe)

// Which bytes below rsp does Windows overwrite when a VEH resumes an access violation?
// Build: gcc -O1 rzxsave.c -o rzxsave.exe   (x64 mingw-w64 or clang)
#include <windows.h>
#include <intrin.h>
#include <stdint.h>
#include <stdio.h>

#define SLOTS 512 /* qwords probed below rsp: 4 KiB */
void probe(volatile uint64_t* target, uint64_t* out, uint64_t skew, uint64_t mode);
__asm__(
    ".text\n.globl probe\nprobe:\n"
    "  push %rbx\n  push %rsi\n  mov %rsp, %rbx\n  sub %r8, %rsp\n  mov %r9, %rsi\n"
    "  mov $1, %rax\n"                                   /* sentinel i at rsp - 8*i */
    "1: movabs $0x5a5a5a5a00000000, %r9\n  or %rax, %r9\n  mov %rax, %r10\n  neg %r10\n"
    "  mov %r9, (%rsp,%r10,8)\n  inc %rax\n  cmp $513, %rax\n  jne 1b\n"
    "  cmp $1, %rsi\n  jne 2f\n  vbroadcastss val(%rip), %ymm0\n  vbroadcastss val(%rip), %ymm5\n"
    "2: cmp $2, %rsi\n  jne 3f\n  vbroadcastss val(%rip), %zmm0\n  vbroadcastss val(%rip), %zmm5\n"
    "3: cmp $3, %rsi\n  jne 4f\n  vbroadcastss val(%rip), %zmm16\n  vbroadcastss val(%rip), %zmm31\n"
    "4: movq $1, (%rcx)\n"                               /* the faulting store */
    "  mov $1, %rax\n"
    "5: mov %rax, %r10\n  neg %r10\n  mov (%rsp,%r10,8), %r9\n  mov %r9, -8(%rdx,%rax,8)\n"
    "  inc %rax\n  cmp $513, %rax\n  jne 5b\n"
    "  mov %rbx, %rsp\n  cmp $3, %rsi\n  jne 6f\n  vpxord %zmm16, %zmm16, %zmm16\n  vpxord %zmm31, %zmm31, %zmm31\n"
    "6: vzeroupper\n  pop %rsi\n  pop %rbx\n  ret\n"
    ".data\nval: .float -0.0909\n.text\n");

static char* page;
static uint64_t frameDepth;
static LONG CALLBACK Resume(EXCEPTION_POINTERS* info) {
    if (info->ExceptionRecord->ExceptionCode != EXCEPTION_ACCESS_VIOLATION) return EXCEPTION_CONTINUE_SEARCH;
    DWORD old;
    VirtualProtect(page, 4096, PAGE_READWRITE, &old);
    frameDepth = info->ContextRecord->Rsp - (uint64_t)(uintptr_t)info->ContextRecord;
    return EXCEPTION_CONTINUE_EXECUTION;
}
static __attribute__((noinline)) void commitStack(void) { volatile char big[65536]; for (int i = 0; i < 65536; i += 4096) big[i] = 0; }

int main(void) {
    int r[4];
    __cpuid(r, 1);
    const int avx = (r[2] >> 28) & 1;
    __cpuidex(r, 7, 0);
    const int avx512 = (r[1] >> 16) & 1;
    printf("XState features 0x%llx, AVX %d, AVX-512F %d\n", (unsigned long long)GetEnabledXStateFeatures(), avx, avx512);
    page = VirtualAlloc(NULL, 4096, MEM_COMMIT | MEM_RESERVE, PAGE_READWRITE);
    AddVectoredExceptionHandler(1, Resume);
    const char* names[4] = {"no vector state live", "ymm0/ymm5 live", "zmm0/zmm5 live", "zmm16/zmm31 live"};
    for (uint64_t mode = 0; mode < 4; ++mode) {
        if ((mode == 1 && !avx) || (mode >= 2 && !avx512)) continue;
        int nearestMin = SLOTS * 8, nearestMax = 0;
        for (uint64_t skew = 0; skew < 64; skew += 8) {
            static uint64_t out[SLOTS];
            DWORD old;
            commitStack();
            VirtualProtect(page, 4096, PAGE_READONLY, &old);
            probe((volatile uint64_t*)page, out, skew, mode);
            int nearest = SLOTS * 8;
            for (int i = 0; i < SLOTS; ++i)
                if (out[i] != (0x5a5a5a5a00000000ull | (uint64_t)(i + 1))) { nearest = (i + 1) * 8; break; }
            if (nearest < nearestMin) nearestMin = nearest;
            if (nearest > nearestMax) nearestMax = nearest;
        }
        printf("%-22s frame %#llx bytes; highest overwritten qword: rsp-%#x .. rsp-%#x%s\n", names[mode], (unsigned long long)frameDepth,
            nearestMin, nearestMax, nearestMin <= 128 ? "  <- inside the 128-byte SysV red zone" : "");
    }
    return 0;
}

Another title and host: Sword of the Sea (PPSA25344, UE5). Windows 10 19045, Intel i5-14400F (Raptor Lake, no AVX-512), RTX 4060, MinGW-w64 GCC. Base is the #44 working-group snapshot sots/next (6b3c17ca4), which maps direct memory as views (#2054's private pages switched off there to avoid a red menu).

Crash and proof. The boot crashes in 6 of 23 starts at app.exe+0x37f77b3, a read of 0x740 in 0x1437f74c0. That's a leaf called from a UE ParallelFor lambda that keeps an object pointer at -0x38(%rbp) with rsp = rbp-0x28, i.e. at rsp-0x10. I recorded every exception the vectored handler resumes (thread, rip, rsp, kind, fault address) and printed the crashing thread's newest on FATAL. 3 of 3 crashes look the same:

resumed faults so far: 49239 (retarget 0, guest-write 49239, watch 0, sse4a 0); this thread's newest:
  guest-write at app.exe+0x37f7706, rsp 000000000924f308 (crash rsp +0), address 0x11400da0c0, 0 ms ago

+0x37f7706 is a store inside the same leaf, between its good read of the slot and the bad one.

Volume. About 41,000 resumed write faults in the first 40 s of boot, on about 41,000 distinct pages (only 18 pages faulted twice). So it's first-touch writes into ranges Collect armed, mostly texture-sized ranges written once by the streamer, not re-arm churn.

#2054 on this title. With #2054's private pages switched back on (and APS5_WRITE_WATCH_IMPORTS=watch, which the menu needs on that base), it's 0 crashes in 21 boots, against 6 in 23. View faults per boot fall under 8,192 (the counter's first step). The cost on this host is a 2.6× slower menu (3.0 vs 7.9 FPS, 2 runs each). The GPU driver reads about 96 GB of guest memory per 130 s instead of 9 GB, and copies about 9.7 GiB in and out of imports. So on this title #2054 trades the crash for speed until the import path is cheaper.

How many functions are exposed. In SotS's relinked app.exe, 604 of about 237,000 functions are leaves that keep values below rsp. That's small enough to treat statically. I have a relinker prototype, --frame-red-zone (branch sots/red-zone-framing in my fork): for a leaf with a push rbp; mov rbp, rsp; push … prologue, a trampoline stub lowers rsp by 128 after the pushes and raises it before each pop …; pop rbp; ret exit, so a fault's frame lands below the locals. It frames 80 of the 604 here, including 0x1437f74c0. Frameless leaves addressing -N(%rsp) directly are the larger part and not done.

Result with the prototype, shared views kept (no #2054): 25 of 25 boots clean, plus 2 smoke boots, against 6 of 23 without it. The fault volume is unchanged at 40,960 per boot; only the clobber is gone. Menu speed is unchanged too: 8.09 FPS vs 7.91/7.97. So for this crash, framing the exposed leaves fixes it without giving up the shared-view path, and it should be host-independent (no XSAVE layout dependence). It doesn't cover the frameless leaves, so other titles' crash functions (DREDGE's [rsp-0x30] routine, Astro Bot's +0xec6090) need that part first.

Another host and title variant: ASTRO BOT (PPSA21567). Host is Windows Server 2025 build 26100, AMD Ryzen 9 5900X (Zen 3) and RTX 3060, building main plus open PRs.

The crash. The audio thread SceSndzAudioOutMain crashes in the title's look-ahead limiter about 70-80 s after boot, in 5 of 5 runs. That leaf keeps its loop state in the red zone, from rsp-0x8 down to rsp-0x30. Just before the crash, the thread takes write faults on tracked shared views at the limiter's vmovups ymm store. Which slot is lost depends on rsp alignment, and it matches the layout measured above:

`rsp % 64`	First slot written by the exception frame	Result in the limiter
0x28	`rsp-0x30`	Block index lost, then a store through a pointer past its buffers
0x08	`rsp-0x10`	Ring base zeroed (read of 0x1000)

A standalone probe on the same host agrees. With live YMM upper halves, a vector-handler-resumed fault writes the area starting rsp % 64 bytes below rsp. With clean YMM state, nothing below rsp changes.

The trigger. The title's direct-memory pool import is refused. The host import budget was 5973 of 6144 MiB live, so buffers fall back to the heap mirror. The faults come from WindowsMappings::Collect arming those views. Raising APS5_HOST_IMPORT_MIB to 16384 removes the refusal, but the crash stays at the same site, still following rsp alignment.

Fault-free tracking such as #637 or #2054, the debugger-continue approach, or relinker-side red-zone protection should all fix this title.

Another title and host: Resident Evil Village (PPSA01556, 01.210.000, RE Engine), on Windows 11 Home 26300 with an AMD Ryzen 9 5900HS (Zen 3 laptop, no AVX-512) and an RTX 3070 Laptop GPU. The base is main 18c438b with the open pull requests the title needs to get this far (listed in #5169).

The crash. In 2 of 5 boots, GameThread reads 0x18 at app.exe+0x69cb959 (mov 0x18(%rax),%rax). rax is read as 0 from an array of 0x30-byte entries that is all zeros.

The array is filled by a frameless merge leaf (app.exe+0x1469cbb..). It keeps its loop state below rsp, at -0x8 to -0x28(%rsp) (a source pointer, counts, bounds), and copies the entries with vmovups ymm0/ymm1.
A diagnostic that records the faults the vectored handler resumes shows that leaf's store at app.exe+0x69cbc6b faulting on a write-tracked view, 56 bytes below the caller's rsp.
Afterwards the slots below rsp hold zeros and 3f800000 pairs, like YMM upper halves. The loop reloaded its state from them, stopped short and left zeros, which the caller dereferenced.
That boot had 154,437 resumed write faults in its first 52 s.

#5052 doesn't cover this title. A relink with --frame-red-zone reports 297248 functions, 3036 leaf functions keep values below RSP, 0 framed; skipped 3036: no push rbp; mov rbp, rsp prologue. All of RE Engine's red-zone leaves are frameless.

#2054 removes the faults, at a cost on Windows. It conflicts with main, and on the same base it needed one more change. Since 9873fae the direct memory section is SEC_RESERVE, and #2054's copy window wrote into uncommitted section pages. That crashed 25 s into boot in sceKernelMunmap → EraseMappings → SavePrivate, and committing the window's pages fixed it. With #2054, boot had 0 resumed write faults, but the title screen dropped from about 0.5 to 0.17 frames per second. Imports of the private direct memory are compared on every use: 121,462 compares, 1.6 TB and 170 s of CPU by the title screen, against 252 compares without #2054. That's the cost Sword of the Sea reported. I can post the conflict resolutions if they help.

The title's audio thread (AK::EventManager, Wwise) also crashes at a later scene change, on a list record that reads as if shifted by 4 bytes. Its nodes lie in a write-tracked range, so it may be the same mechanism, but I haven't shown that yet.

Still on current main with Disney Epic Mickey: Rebrushed (PPSA15657, UE4). Windows 11, RTX 4080, main ce5c839 unmodified, relinked with its relinker. About 3 minutes into the first level:

FATAL: unhandled exception 0xc0000005
  rip app.exe+0x55fac65
  write of 0x0000000000000060

This is the same leaf as before: eboot 0x55eaad0, a bounding-box tree refit that keeps a pointer below rsp. Its null base comes from a red-zone slot that was overwritten. I haven't tried #637, #5596 or #5052 with this title yet.

AI-assisted: yes (Claude Code)

#5596 (the port of #637 onto current main, DotDebian's design) is merged. On Windows, direct memory mapped at a single address is now private write-watched memory, so CPU writes there no longer take the faults that overwrote the red zone. GTA V, which crashed this way with write watch on, now loads Story on main with it.

AI-assisted (Claude Code).

@johnmaia

#5596 closed this on merge: singly mapped direct memory is now tracked without write faults on Windows, which removes the red-zone overwrite for those writes (GTA V, PPSA04264, no longer crashes loading Story). Faults on other paths are what #5052 covers.

@johnmaia your Epic Mickey run was on ce5c83953, before #5596. Could you retry PPSA15657 on current main? If the leaf at eboot 0x55eaad0 still faults, post it here and this gets reopened.

@oneandonlydean

@oneandonlydean retried PPSA15657 (Epic Mickey: Rebrushed) on Windows 11, RTX 4080, with main a310431 unmodified and the game relinked with its relinker. Since #5596 the title no longer reaches the point where the red-zone fault happened. It stops earlier, during the level load (one run) or before the main menu's Continue (the other), on the debt throw from #5024:

[videoout] presentation failed: AGC graphics: storing storage image results pending over memory outside the write watch without the guest bytes saved before them is not implemented

The same build with only #5596 reverted (git revert -m 1 3e4fe4ce3), with the same input sequence, loads the level and stays in it for 10 minutes without that throw or any fault (8.5 FPS).

Build	Runs	Result
`main` `a310431`	2	throw above, before or during the level load
`main` `a310431` with #5596 reverted	1	level loaded, 10 min, no throw, no fault

So on this title #5596 moves storage images that have results pending onto memory that leaves the write watch without saved guest bytes, the case #5024 left as a throw. Whether the 0x55eaad0 red-zone fault is gone can't be checked until this is resolved. In the reverted run Mickey stood still in the level, while the earlier fault came after about 3 minutes of play. Logs are available if useful.

AI-assisted: yes (Claude Code)

@johnmaia

One more data point for the throw @johnmaia hit. Sword of the Sea (PPSA25344) on Windows 11, RTX 5070 Laptop, NVIDIA 617.42, on main 52d134c with #5596 plus our lab perf commits; none of those touch Texture.cpp or the write watch. One fresh-dir run got through the post-New-Game window, where this title used to fault on FAsyncLoadingThread (#361: wild pointers, a 12 GB garbage allocation). 281 s later, still on the loading screen, it stopped on the same debt throw from #5024:

AGC graphics: storing storage image results pending over memory outside the write watch without the guest bytes saved before them is not implemented

So it isn't specific to Epic Mickey. With #5596, storage images with pending results land on memory that has left the write watch without saved guest bytes. On our current-14-based runs without #5596, the #361 faults hit about 30 % of New Game starts, so this is the one case that blocks both titles now.

@johnmaia

@johnmaia thanks for the bisect. On Windows, #5596 makes singly mapped direct memory private, so its host import is now a direct import that leaves the write watch (before, it went through a shared alias that stayed watched). A storage image whose results were still pending when its memory was first imported then had no saved guest bytes, and its next store threw.

#5657 saves those bytes, and the blocks the CPU wrote while the memory was watched, at the moment the range leaves the watch. Could you run PPSA15657 on it with the same steps?

fix(agc): keep an import's resident memory compared once it is shared

fix(libc): refuse to collect a shared page marked to be compared

fix(libc): stop write-protecting resident runs on Windows

local(agc): owned proofs by write watch, no guest-thread write faults

fix(libc): stop write-protecting resident runs on Windows

Navigation Menu

Repository navigation

Windows: resuming a write fault overwrites the guest red zone when AVX state is live #361

Mechanism

Evidence

Standalone reproduction

Workaround and its cost

Possible directions

Related

1. AVX-512 hosts are affected too, once zmm16-31 are live

2. How often a title maps the same physical range twice

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Windows: resuming a write fault overwrites the guest red zone when AVX state is live #361

Description

Mechanism

Evidence

Standalone reproduction

Workaround and its cost

Possible directions

Related

Activity

boykopovar commented on Oct 3, 2026

adriadam10 commented on Oct 3, 2026

QDarkRex commented on Oct 4, 2026

SP4C3B4R-8 commented on Oct 4, 2026

frankea commented on Oct 10, 2026

TheAggie commented on Oct 10, 2026

1. AVX-512 hosts are affected too, once zmm16-31 are live

2. How often a title maps the same physical range twice

oneandonlydean commented on Oct 10, 2026

rm-NoobInCoding commented on Oct 10, 2026

johnmaia commented on Oct 10, 2026

xXPBSXx commented on Oct 11, 2026

queeeef commented on Oct 12, 2026

AlxcBTW commented on Oct 12, 2026

parl0v commented on Oct 12, 2026

fdfawef324 commented on Oct 13, 2026

EagleBearerYS commented on Oct 13, 2026

johnmaia commented on Oct 14, 2026

tibzejoker commented on Oct 14, 2026

oneandonlydean commented on Oct 14, 2026

johnmaia commented on Oct 14, 2026

AlxcBTW commented on Oct 14, 2026

oneandonlydean commented on Oct 14, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions