Skip to content

fix(libkernel): map Windows shared memory with the regular file checks - #12296

Merged
adriadam10 merged 1 commit into
boykopovar:mainfrom
JJenkx:windows/kernel-shm-mmap-checks
Oct 30, 2026
Merged

adriadam10 merged 1 commit into
boykopovar:mainfrom
JJenkx:windows/kernel-shm-mmap-checks

Conversation

@JJenkx

@JJenkx JJenkx commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

What

On Windows, mmap sent a shared memory descriptor straight to the shared file mapper (DoMapFileBacked). It skipped the access checks and the private path that a regular file descriptor takes. Shared memory objects are files there: anonymous objects are delete-on-close temporary files, and named objects are files under AnyPS5/shm. Skipping those steps produced two errors:

  • MAP_PRIVATE of a named or anonymous object failed with EOPNOTSUPP (45), because DoMapFileBacked refuses private mappings on Windows. FreeBSD maps them, and a private mapping of a regular file on Windows already works as a copy of the file read at mmap (MapPrivateFileCopy).
  • A writable MAP_SHARED mapping of a read-only descriptor failed with EIO (5): the section was created read-only and the writable view then failed. FreeBSD and the regular-file path return EACCES.

Shared memory descriptors now take the same handle lookup, access checks and private copy as a regular file. Only the refusal of directories, the random device and non-disk handles stays limited to other descriptors. Shared mappings still go through the same section code as before.

A new Windows test, guest_named_shm_windows_mapping (GuestNamedShmWindowsMapping.cpp), maps a named object:

  • shared, through two descriptors;
  • against pread and pwrite;
  • at an offset;
  • private, with its stores staying out of the object;
  • read-only, plus a refused writable shared mapping of a read-only descriptor;
  • after shm_unlink.

Until now no Windows test mapped a named object; the named-object mapping tests are Linux-only. The libkernel debt entry that said so is updated, and is now shorter than on main.

Tested

Windows 11 Enterprise 10.0.22621, Ryzen 9 5950X, WinLibs GCC 15.2.0 UCRT r7, Ninja, Release, BUILD_TESTING=ON, Vulkan drivers hidden. Base main 660a0b306d.

  • Without the fix (the new test on main's PosixMemory.cpp): it aborts at "a private mapping of the named object failed (guest errno 45)". With the private case moved last, it aborts at "a writable shared mapping of a read-only descriptor was not refused with EACCES (guest errno 5)". The shared, offset and pread/pwrite checks already pass on main.
  • With the fix: these 17 tests pass in each of 3 runs. They are every Windows test whose source calls mmap, plus the named shared-memory Windows tests: guest_named_shm_windows_mapping, guest_named_shm_windows (with its short_temp and long_short_temp variants), guest_shm_open, guest_jit_shared_memory, guest_memory, guest_mmap_fixed_anonymous, guest_low_address_memory, guest_mprotect_unmapped, guest_msync, guest_kernel_stack_protect, guest_posix_filesystem, guest_system_configuration and the guest_kernel_app_archive* tests.

Not tested on Linux or macOS. The change is inside the _WIN32 branch only.

Overlap. tools/pr_overlap.py reports two textual conflicts with no shared behaviour:

Proposed merge order: this PR first, then #11625 and #10645 take the small textual merge; asked on #11625 and #10645. If either author prefers to go first, this PR will declare Depends on that PR and be rebased after it lands. #12339's author agreed that this PR lands before #12339, whose copy of the same check then becomes unreachable.

Current Linux qualification — 2026-10-29

Rebased onto main 5454cd2e455 as dc1d801c4ce47fe1083ac473619f60400ad560f6. Production, regression and CMake topic additions/removals are preserved exactly. The debt resolution keeps main’s #12339 read/execute permission details alongside this PR’s regular-file mapping statement; all recursive submodule pointers match main.

Linux: libc/libkernel build, and five memory/shared-memory groups pass three repetitions (15 passes, zero skips). The actual Windows production translation unit also passes MinGW GCC16.2.0 cross syntax checking on Linux. That is not Windows execution: the existing Windows pause remains in force, so earlier actual Windows tests retain their historical source pins. Conventions and whitespace pass.

The #11625/#10645 author-order questions remain unanswered, so no agreement is claimed. #12339 has now merged; its production check and debt details are preserved in this refresh. AI-assisted: yes.

Evidence: immutable Linux qualification bundle. Its SHA-256 is ca6b35926bcc6d3f21a9650517894c86b65594e2e2b653c5c32cd1c1de89c835. No fresh Windows runtime execution is claimed; older measurements retain their recorded source pins.

AI-assisted: yes (Codex).

Checklist

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 4563 core/libs/prx/libkernel/CMakeLists.txt
PR 4630 docs/dev/debt/libkernel.md
PR 5021 core/libs/prx/libkernel/CMakeLists.txt
docs/dev/debt/libkernel.md
PR 6936 core/libs/prx/libkernel/CMakeLists.txt
docs/dev/debt/libkernel.md
PR 7088 docs/dev/debt/libkernel.md
PR 7105 docs/dev/debt/libkernel.md
PR 7121 docs/dev/debt/libkernel.md
PR 7135 docs/dev/debt/libkernel.md
PR 7149 docs/dev/debt/libkernel.md
PR 7192 core/libs/prx/libkernel/CMakeLists.txt
docs/dev/debt/libkernel.md
PR 7447 docs/dev/debt/libkernel.md
PR 7464 docs/dev/debt/libkernel.md
PR 7509 docs/dev/debt/libkernel.md
PR 7804 docs/dev/debt/libkernel.md
PR 8605 docs/dev/debt/libkernel.md
PR 9156 core/libs/prx/libkernel/CMakeLists.txt
docs/dev/debt/libkernel.md
PR 9242 docs/dev/debt/libkernel.md
PR 9293 core/libs/prx/libkernel/CMakeLists.txt
docs/dev/debt/libkernel.md
PR 9297 core/libs/prx/libkernel/CMakeLists.txt
docs/dev/debt/libkernel.md
PR 9302 core/libs/prx/libkernel/CMakeLists.txt
docs/dev/debt/libkernel.md
PR 9326 core/libs/prx/libkernel/CMakeLists.txt
docs/dev/debt/libkernel.md
PR 9383 core/libs/prx/libkernel/CMakeLists.txt
PR 9389 core/libs/prx/libkernel/CMakeLists.txt
docs/dev/debt/libkernel.md
PR 9557 core/libs/prx/libkernel/CMakeLists.txt
docs/dev/debt/libkernel.md
PR 9566 core/libs/prx/libkernel/CMakeLists.txt
docs/dev/debt/libkernel.md
PR 9843 docs/dev/debt/libkernel.md
PR 10442 docs/dev/debt/libkernel.md
PR 10552 docs/dev/debt/libkernel.md
PR 10645 core/libs/prx/libkernel/CMakeLists.txt
docs/dev/debt/libkernel.md
PR 11002 core/libs/prx/libkernel/CMakeLists.txt
docs/dev/debt/libkernel.md
PR 11215 docs/dev/debt/libkernel.md
PR 11625 core/libs/prx/libkernel/CMakeLists.txt
core/libs/prx/libkernel/DirectMemory/PosixMemory.cpp
docs/dev/debt/libkernel.md
PR 11827 docs/dev/debt/libkernel.md
PR 11939 core/libs/prx/libkernel/CMakeLists.txt
docs/dev/debt/libkernel.md
PR 12104 docs/dev/debt/libkernel.md
PR 12560 docs/dev/debt/libkernel.md
PR 12688 core/libs/prx/libkernel/CMakeLists.txt
docs/dev/debt/libkernel.md
PR 12700 core/libs/prx/libkernel/CMakeLists.txt
docs/dev/debt/libkernel.md
PR 12741 docs/dev/debt/libkernel.md
PR 12868 core/libs/prx/libkernel/CMakeLists.txt
core/libs/prx/libkernel/DirectMemory/PosixMemory.cpp
PR 12965 core/libs/prx/libkernel/CMakeLists.txt
docs/dev/debt/libkernel.md

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the Windows-only mapping change against the regular-file path and the new regression test. Descriptor lookup and access-mode checks now run for shared-memory handles as well, while disk-type refusal stays scoped to other descriptors; shared mappings still use the existing section mapper and private mappings use the copy path. The test covers shared visibility through two descriptors and pread/pwrite, nonzero offsets, private copy isolation, read-only access enforcement, and post-unlink lifetime. Current Windows CI and conventions pass; the author reports the relevant Windows mmap suite passing three times. Linux/macOS were not tested, but the code change is confined to _WIN32. No blocking issue found.

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Windows shared-memory path now takes the same descriptor lookup, access checks, and private-copy path as regular files, and the added test covers shared mappings across descriptors, file I/O coherence, offsets, read-only access, EACCES, unlink lifetime, and private-copy isolation. Hosted conventions and Windows checks pass; the author reports the Windows behavior tests, while the other platform jobs are skipped. One repository-rule item remains: the overlap check explicitly reports a conflict with open PR #11625 in PosixMemory.cpp, and #11625 is still open. CONTRIBUTING.md asks authors to agree which overlapping PR lands first and record the merge order with Depends on; the current description says no dependency. Please coordinate with #11625’s author and update the merge-order line, then rebase after the selected PR lands so the combined change preserves both behaviors.

@MarcZX10

Copy link
Copy Markdown
Contributor

The Windows mapping implementation and its hosted test run look good. One integration-rule item remains: the PR body identifies a same-line rename in #11625 and an adjacent debt-entry overlap with #10645, then says either can land first. CONTRIBUTING asks the PR authors to agree on the order and record it with Depends on. I do not see agreement from those PR authors in this thread. Please coordinate a landing order with both authors and update the checklist so the overlap can be rebased predictably; this is the only item I am holding review for.

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The implementation and Windows test coverage look sound, and the Windows hosted build passes. I am requesting one contribution-rule correction before approval: this PR itself identifies overlaps with #11625 (the mapping-descriptor line) and #10645 (the libkernel debt entry), but the description says “Either can land first” without showing agreement from those authors or declaring an order under Depends on. CONTRIBUTING requires the overlapping PR authors to agree on order and record it. Please coordinate and update the checklist; I found no new code defect in this pass.

@babyhuey

Copy link
Copy Markdown
Contributor

Windows run of this head ba5f6c7a4b1, merged onto main c02d287bd0e (clean merge, local only), with the selection CI's skipped Windows product job uses: Windows 11, Ryzen 7 5800X, RTX 3080, WinLibs GCC 15.2, Release, BUILD_TESTING=ON. The tree was configured fresh after the merge, then select_tests.py select / ninja -f select.ninja selected / --target libs / select_tests.py ctest --parallel 4 --timeout 120.

AI-assisted: yes (Claude Code)

@adriadam10 adriadam10 self-assigned this Oct 28, 2026

@adriadam10 adriadam10 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked the PosixMemory.cpp change and guest_named_shm_windows_mapping: shared-memory descriptors now take the same handle lookup, access checks and private copy as regular files, the directory/random-device/non-disk refusal stays limited to other descriptors, and the test pins EACCES (13) for the writable shared mapping of a read-only descriptor and private-store isolation. I found no code change to ask for, and the Windows run from @babyhuey shows no failures from this PR.

  • The only open item is the landing order with #11625 and #10645 that @MarcZX10's change request is holding. Neither author has answered the questions yet. Once one of them confirms an order (or says it does not matter), record it under Depends on if it puts another PR first, and ask @MarcZX10 to re-review; no other change is needed.

@medazizktata

Copy link
Copy Markdown
Contributor

Overlap with #12339 (mine): I agree this PR lands first. Both PRs refuse a writable shared mapping of a read-only shared-memory descriptor with EACCES. Once this one is in, #12339's copy in DoMapFileBacked_nid_no_patch can no longer be reached. So I'll rebase #12339 onto this PR, drop that check, and reconcile the shared Windows debt entries there. Nothing is needed on this side beyond listing #12339 in the overlap section; #12339 records Depends on: #12296. Details: #12339 (comment)

On Windows, mmap sent a shared memory descriptor straight to the shared
file mapper, skipping the checks and the private path that a regular
file descriptor takes. A private mapping of a named or anonymous object
then failed with EOPNOTSUPP, and a writable shared mapping of a
read-only descriptor failed with EIO instead of EACCES. Shared memory
objects are files on Windows, so they now take the same access checks
and the same private copy as a regular file; only the directory, device
and file-type refusals stay limited to other descriptors.

guest_named_shm_windows_mapping covers shared mappings through two
descriptors, pread and pwrite against them, an offset mapping, a private
mapping, read-only mappings and the mappings after shm_unlink.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@JJenkx
JJenkx force-pushed the windows/kernel-shm-mmap-checks branch from ba5f6c7 to dc1d801 Compare October 29, 2026 12:52
@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 29, 2026
@JJenkx

JJenkx commented Oct 29, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto main 5454cd2e455 as dc1d801c4ce47fe1083ac473619f60400ad560f6. Production, regression and CMake topic additions/removals are preserved exactly. The debt resolution keeps main’s #12339 read/execute permission details alongside this PR’s regular-file mapping statement; all recursive submodule pointers match main.

Linux: libc/libkernel build, and five memory/shared-memory groups pass three repetitions (15 passes, zero skips). The actual Windows production translation unit also passes MinGW GCC16.2.0 cross syntax checking on Linux. That is not Windows execution: the existing Windows pause remains in force, so earlier actual Windows tests retain their historical source pins. Conventions and whitespace pass.

The #11625/#10645 author-order questions remain unanswered, so no agreement is claimed. #12339 has now merged; its production check and debt details are preserved in this refresh. AI-assisted: yes.

Evidence: immutable Linux qualification bundle. Its SHA-256 is ca6b35926bcc6d3f21a9650517894c86b65594e2e2b653c5c32cd1c1de89c835. No fresh Windows runtime execution is claimed; older measurements retain their recorded source pins.

@adriadam10
adriadam10 dismissed MarcZX10’s stale review October 29, 2026 14:28

Dismissing a stale review: the requested changes were addressed after it (new head) and the merge queue blocks on it. Please request changes again if anything is still open.

@adriadam10
adriadam10 merged commit 28b490e into boykopovar:main Oct 30, 2026
12 checks passed
@adriadam10

Copy link
Copy Markdown
Collaborator

Re-checked after the rebase: PosixMemory.cpp and guest_named_shm_windows_mapping are unchanged, and the debt entry keeps #12339's read/execute details next to this PR's mapping statement. #12339's author confirmed this lands first and will rebase onto it, and no answer came on #11625 and #10645, so the order is settled; merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants