Skip to content

fix(relinker): scope host dependencies to required guest modules - #6106

Merged
adriadam10 merged 2 commits into
boykopovar:mainfrom
Zinebyte:fix/deferred-host-dependencies
Oct 18, 2026
Merged

adriadam10 merged 2 commits into
boykopovar:mainfrom
Zinebyte:fix/deferred-host-dependencies

Conversation

@Zinebyte

@Zinebyte Zinebyte commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

What

A Linux executable with no startup module imports can run normally, but scanning an unused plugin under --module-dir Media/Plugins makes it fail at startup if that plugin declares an unavailable host library. The plugin is already marked deferred; the builder nevertheless adds every discovered module's host dependencies to the executable and to every other guest module.

Build each guest's host dependency list from its own declarations, preserving the existing libc alias and the native kernel dependency required by Linux writer-generated initializer helpers. Include a guest's host requirements in the executable only when it is required at startup; Windows keeps its eager host list. Deferred plugins still require their libraries when explicitly loaded, and invalid deferred host names are still rejected. Startup guest selection is unchanged.

The synthetic Linux host fixture explicitly declares an RW, non-executable PT_GNU_STACK, permitting native loading on glibc 2.41 and later without executable-stack tunables.

Tested

Linux x86-64, GCC 13.3, Release relinker-only build on current main af7db81e6ef71282fdec4f10b089d1da6c75555e, refreshed head ed1884134fcde70c9bb8314379115e2f7de391e7: all 185 configured CTests pass in 8.74 seconds, including the merged finalizer regression. Conventions and whitespace checks pass. All four files changed by the original dependency-scoping commit are byte-identical to the earlier b50ab4eb head; this refresh adds the required finalizer kernel fixture copy on current main.

Native Linux loader controls cover unused plugins (startup exits 42), required guest hosts, explicit deferred loading, missing required hosts, and startup/deferred initializers. Metadata controls cover Linux, macOS and Windows. The new controls fail against the unmodified builder; the fixed builder passes. Independent adversarial review checked libc aliases, shared-host deduplication, invalid names and a bundled guest named libkernel.prx.

The earlier hosted Build 38776790189 passed at previous head b50ab4eb: Linux 181/181 tests, Windows 176/176 and macOS 172/172. Hosted Build 38972981441 passed on refreshed head ed1884134fcde70c9bb8314379115e2f7de391e7: the Linux, Windows and macOS relinker jobs succeeded, and the latest Conventions check passed. The full Linux and Windows runtime jobs were skipped under the contribution policy, so this run does not establish a successful full runtime build. No retail-title execution or new local native macOS/Windows loader run is claimed.

The reported executable-stack fixture issue was independently reproduced using Debian glibc 2.41 extracted inside the workspace: the original host library fails dlopen with “cannot enable executable stack”; the corrected library loads. A corrected library declaring a missing dependency still fails normally. No GLIBC_TUNABLES, LD_PRELOAD or host-library installation was used. The directory suite also passes on host glibc 2.39.

Integration

#5614 has merged into main, so its finalizer regression is now present on this branch. This refresh includes the reported one-line fixture correction: copy the existing host fixture to libs/libkernel.prx alongside libs/libc.prx. All finalizer argument and unload assertions remain unchanged.

The optional HasInit guard adjustment for still-open #6684 remains integration-only: that peer adds zero-address initializers and the HasInit member. A checkout combining both PRs must use that member for the generated helper dependency. The standalone branch follows the current main writer's Init != 0 condition and does not import the unmerged peer.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw; no silent stubs added
  • No notes, investigation .md files or images added to the repository
  • New third-party code is a submodule built from source (none added)
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Unblocks: none
  • GPUs tested: n/a
  • AI-assisted: yes

@Dextroid17

Copy link
Copy Markdown
Contributor

I ran the guest module tests on Linux (glibc 2.44). guest_module_dir fails on this machine, and I think the cause is the test's hand-built host library, not the change.

Setup: head 78b906f merged into main at ff23657 (no conflicts; the PR's base is 315 commits behind), default CMake build (CMAKE_BUILD_TYPE empty, unoptimized), clean environment (env -i). ctest -R '^guest_module_(args|dir|identity)$', 3 runs each.

  • main alone: all 3 tests pass in 3 of 3 runs (main's test_guest_module_dir.py does not have check_plugin_host_dependencies).
  • with this PR: guest_module_args and guest_module_identity pass; guest_module_dir fails 3 of 3 in check_plugin_host_dependencies (linux target, runtime case), at the ctypes.CDLL load after libOptional.prx is written: "libOptional.prx: cannot enable executable stack as shared object requires: Invalid argument".

host_module() writes 3 program headers (LOAD, LOAD, DYNAMIC) and no PT_GNU_STACK (checked with readelf -lW). Without that header glibc treats the library as needing an executable stack, and since glibc 2.41 dlopen fails for such a library unless the stack is already executable (https://sourceware.org/git/?p=glibc.git;a=blob_plain;f=NEWS;hb=glibc-2.41).

With GLIBC_TUNABLES=glibc.rtld.execstack=2 added to the same environment (glibc 2.42+; it makes the stack executable at program start), the same ctest command passes 3 of 3 runs with this PR. With GuestModuleBuilder.cpp put back to main's version (the PR's test changes kept), all three tests fail in 3 of 3 runs, with or without the tunable; guest_module_dir fails at ('linux', 'runtime'), before any dlopen. So the tests do catch main's behaviour.

The relinker (ubuntu-latest, g++) check passes. That job ran on Ubuntu 24.04, which has glibc 2.39, so it likely never hits this.

Adding a PT_GNU_STACK header with RW flags to host_module() should make the test pass on newer glibc. I have not tried that change.

I did not check Windows or macOS.

AI-assisted: yes

@Zinebyte
Zinebyte force-pushed the fix/deferred-host-dependencies branch from 78b906f to b50ab4e Compare October 15, 2026 17:22
@nicolasfalesy

Copy link
Copy Markdown
Contributor

Tested on Linux (Arch), c++ (GCC) 16.2.1 20260810, Release, BUILD_TESTING=ON, main at 5d9dcc42d2ec5401aebd089028b8d4d7410a77b4.

This PR and #5614 break guest_module_fini together. main plus #5614 alone passes, and main plus #5614 plus this PR fails (2 of 2):

OSError: libkernel.prx: cannot open shared object file: No such file or directory

Cause: for Linux modules with an init function, this PR now adds libkernel.prx to the module's needed libraries (GuestModuleBuilder.cpp:310). The test of #5614 loads the converted module with only libc.prx as a provider (test_guest_module_fini.py:67-70), so ctypes.CDLL cannot resolve libkernel.prx.

Whichever PR lands second should copy the host module as libkernel.prx into the provider directory in that test.

AI-assisted: yes

@Zinebyte
Zinebyte force-pushed the fix/deferred-host-dependencies branch from b50ab4e to ed18841 Compare October 16, 2026 21:59
@adriadam10 adriadam10 self-assigned this Oct 16, 2026
@adriadam10

Copy link
Copy Markdown
Collaborator

Checked the per-module host dependency scoping against the diff: relinker build and tests in scope 137/137 pass, and reverting GuestModuleBuilder.cpp makes guest_module_dir, guest_module_args and guest_module_identity fail. guest_module_fini only gains the libkernel.prx fixture. Both earlier reports (executable-stack fixture, libkernel.prx in the finalizer test) are addressed on this head; merging once the full Build is green.

@github-actions

github-actions Bot commented Oct 17, 2026 •

Copy link
Copy Markdown

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 2780 core/relinker/relinker/src/guest/GuestModuleBuilder.cpp
PR 6314 core/relinker/relinker/src/guest/GuestModuleBuilder.cpp
PR 6977 core/relinker/relinker/src/guest/GuestModuleBuilder.cpp

@adriadam10
adriadam10 merged commit de2f884 into boykopovar:main Oct 18, 2026
10 of 11 checks passed
SandstromPL added a commit to SandstromPL/AnyPS5 that referenced this pull request Oct 19, 2026
boykopovar#6106 made a Linux guest module with an initializer depend on
libkernel.prx, which exports the module-argument getter and setter that
its DT_INIT stub imports. That check still tested Init != 0, so a module
whose DT_INIT is 0 got the stub but not the dependency. With no other
libkernel.prx loaded, the executable then stopped with a symbol lookup
error, and a --module-dir module failed to load with "undefined symbol:
__aps5_get_pending_module_args_nid_no_patch". The check now uses HasInit,
like the writers.

guest_zero_initializer now checks that the Linux module lists
libkernel.prx, and its native load provides one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
JJenkx pushed a commit to JJenkx/AnyPS5 that referenced this pull request Oct 19, 2026
boykopovar#6106 made a Linux guest module with an initializer depend on
libkernel.prx, which exports the module-argument getter and setter that
its DT_INIT stub imports. That check still tested Init != 0, so a module
whose DT_INIT is 0 got the stub but not the dependency. With no other
libkernel.prx loaded, the executable then stopped with a symbol lookup
error, and a --module-dir module failed to load with "undefined symbol:
__aps5_get_pending_module_args_nid_no_patch". The check now uses HasInit,
like the writers.

guest_zero_initializer now checks that the Linux module lists
libkernel.prx, and its native load provides one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
JJenkx pushed a commit to JJenkx/AnyPS5 that referenced this pull request Oct 20, 2026
boykopovar#6106 made a Linux guest module with an initializer depend on
libkernel.prx, which exports the module-argument getter and setter that
its DT_INIT stub imports. That check still tested Init != 0, so a module
whose DT_INIT is 0 got the stub but not the dependency. With no other
libkernel.prx loaded, the executable then stopped with a symbol lookup
error, and a --module-dir module failed to load with "undefined symbol:
__aps5_get_pending_module_args_nid_no_patch". The check now uses HasInit,
like the writers.

guest_zero_initializer now checks that the Linux module lists
libkernel.prx, and its native load provides one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants