Skip to content

test(libkernel): skip executable named shm mappings where /dev/shm is noexec - #15776

Open
root-over wants to merge 3 commits into
boykopovar:mainfrom
root-over:test/libkernel-named-shm-noexec-skip
Open

root-over wants to merge 3 commits into
boykopovar:mainfrom
root-over:test/libkernel-named-shm-noexec-skip

Conversation

@root-over

Copy link
Copy Markdown
Contributor

What

guest_mmap_executable fails on main in containers. The NVIDIA and AMD GPU CI runs on #10069 list it under "Failing on main too" (Subprocess aborted).

CheckExecutableSharedMappings maps named shared memory with PROT_EXEC. On Linux, named objects are files under /dev/shm, and Docker mounts /dev/shm noexec by default. Reproduced with docker run --tmpfs /dev/shm:rw,noexec,nosuid,nodev (ubuntu:24.04, host binaries through chroot):

mmap(0x200000000, 32768, PROT_READ|PROT_EXEC, MAP_SHARED|MAP_FIXED, 5, 0) = -1 EPERM (Operation not permitted)
Guest memory check failed at core/libs/tests/GuestMemory.cpp:3576

The same container with /dev/shm mounted exec passes, so the mount option is the only cause.

  • The test reads the flags of /dev/shm with statvfs. When it is noexec, the test:
    • still runs every regular-file, anonymous (memfd), JIT and non-executable named mapping;
    • skips only the executable named ones;
    • prints skipped, the host mounts /dev/shm noexec, so named shared memory is not mapped executable and returns 77.
  • guest_mmap_executable gets SKIP_RETURN_CODE 77. Windows is unchanged.
  • New debt entry: on such a host a PROT_EXEC mapping of a named object fails, where FreeBSD 11.4 maps POSIX shared memory executable. The library change this would need (named objects outside /dev/shm) is left out of this PR.

Tested

Ubuntu, kernel 7.0, GCC 15.2.0, Release.

/dev/shm main this PR
host (rw,nosuid,nodev) passes passes
Docker, noexec aborts at GuestMemory.cpp:3576 skipped (77), the other mappings checked

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: none
  • Unblocks: none
  • GPUs tested: n/a
  • AI-assisted: yes

@root-over
root-over force-pushed the test/libkernel-named-shm-noexec-skip branch from 0474e40 to 89b812e Compare November 4, 2026 15:29
CheckExecutableSharedMappings maps each descriptor with every protection
from 0 to 7. Its check now takes whether the executable protections are
mapped, so a caller can run the readable and writable cases alone. Every
caller still passes true, so the test runs as before.
… noexec

guest_mmap_executable maps named shared memory with PROT_EXEC, and named
objects are files under /dev/shm. Docker mounts /dev/shm noexec by default,
so the host refuses the mapping with EPERM and the test aborted in the GPU
CI containers, which list it as failing on main.

The test now reads the mount flags of /dev/shm with statvfs. When it is
noexec it still runs every regular file, anonymous, JIT and non-executable
named mapping, skips the executable named ones and returns 77, which the
test registers as a skip.
…noexec /dev/shm

Named shared memory objects are files under /dev/shm on Linux, so on a host
that mounts it noexec, as Docker does by default, a PROT_EXEC mapping of a
named object fails where FreeBSD 11.4 maps POSIX shared memory executable.
Anonymous objects are a memfd and are not affected.
@root-over
root-over force-pushed the test/libkernel-named-shm-noexec-skip branch from 89b812e to bbf62b9 Compare November 4, 2026 15:30
@adam25136

Copy link
Copy Markdown

Windows run, for head bbf62b9350 merged into main 9ab6a281ea (git merge --no-ff, clean merge). On Windows the change sets namedExecutable to true, so this checks that the named mapping is still checked as executable there.

Host Windows 10 Pro 22H2 (19045), i9-12900KF, WinLibs GCC 15.2.0 r7, Release, BUILD_TESTING=ON
GPU NVIDIA GeForce RTX 5080 (10de:2c02), driver 621.14
Selection 3,940 tests, chosen by tools/select_tests.py select for the 3 changed files
Run select_tests.py ctest build --parallel 4 --timeout 120 with ANYPS5_REQUIRE_VULKAN=1 ANYPS5_NO_SHADER_CACHE=1; main 9ab6a281ea ran its full suite (4,030 tests) the same way on this machine
Result 3,926 passed, 12 skipped, 2 failed, 0 new

No title was run.

AI-assisted: yes. Claude Code ran the builds and tests and drafted this comment; the numbers are from the runs on this machine.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants