Skip to content

fix(relinker): select guest startup dependencies from required roots - #5559

Merged
adriadam10 merged 3 commits into
boykopovar:mainfrom
JJenkx:fix/guest-executable-import-dependencies
Oct 15, 2026
Merged

adriadam10 merged 3 commits into
boykopovar:mainfrom
JJenkx:fix/guest-executable-import-dependencies

Conversation

@JJenkx

@JJenkx JJenkx commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

What

Choose guest module startup dependencies from the roots that actually need them. Dependencies of runtime-only modules discovered with --module-dir stay deferred until a required root uses them. A relocated executable import resolved to a unique guest provider requires that module and its dependency chain even without a DT_NEEDED entry; previously the import was renamed for guest binding but its provider could remain unloaded. Unreferenced dynamic symbols and R_X86_64_NONE relocations do not require an otherwise deferred module.

The branch includes the earlier unpublished runtime-only dependency fix (original commit b32fd11, preserved as a separate commit) and the executable-import correction. Windows retains its current eager dependency handling. Linux and macOS select startup roots from required modules and executable bindings. Logical export-library namespaces are unchanged.

Tested

Rebased as three linear commits on main d92a8dad390af1807cc91622b924252141816a5a, published head a38314933da4209c8ee36faafa3e7c6a8f7976ea. The third commit copies the existing synthetic host module under libkernel.prx as well as libc.prx, following Zinebyte's reproduction of the required initializer dependency when this topic is composed with #6106. It changes only the test fixture. The branch merges cleanly into current main.

Fresh Linux Release relinker-only build passed; 173/173 CTests passed, including guest_deferred_dependencies, with zero skips. Local conventions and whitespace checks passed. The current PR's relinker CI matrix also passed on Linux, macOS and Windows; full project build jobs were skipped.

Composition with current #6106 head 78b906fb67a8edff3969f60d26ced129f981139e cherry-picked cleanly and passed guest_deferred_dependencies; 172/173 relinker CTests passed initially. Its new guest_module_dir test failed to load synthetic libOptional.prx on this host because the fixture requested an executable stack. I reproduced the identical deterministic failure with #6106 alone on current main. A local two-line non-executable-stack ELF header correction in that test made 172/172 #6106-alone CTests and 173/173 composed CTests pass. That correction is separate and is not part of this PR.

Earlier validation

  • The final synthetic regression against a fresh build with runtime-only deferral but without the executable-import correction fails because the imported provider is absent from DT_NEEDED: ('imported', 'provider', ['libc.prx']).
  • Fresh independent Linux Release relinker-only build of this topic: all 141 checks pass without skips. The dependency test runs six scenarios, including an executable that calls the imported function through its relocated slot and exits with its return value, 42. It verifies provider initialization, deferred modules, unreferenced symbols and no-op relocations. Linux native execution is checked; macOS output dependency lists are inspected for the three new scenarios. No native Windows or macOS run of this topic is claimed.
  • Full bundled Linux Release development build passes with 32 build threads; all 122 relinker checks pass without skips. Integration commit 43a50a4, based on upstream 90c6839 plus other development changes. Local topic and integration conventions pass.
  • Astro Bot PPSA21564 v01.018.000: in the development integration, conversion now includes the bundled libScePsml module as a startup dependency. All 13 previously unresolved #guest imports from that module resolve in ldd -r. Loading the provider exposes four additional imports, leaving 22 distinct unresolved names and four missing libraries. Native startup and gameplay remain unvalidated. The integration also includes the separate alias fix in test(agc): fix and re-enable 3 of 8 bitrotted AGC driver tests #23, which this topic does not need for its generic tests.

The initial fork review used base 1acfa7f78; those integration results are historical.

Full CI run 38323687739 passed at the earlier topic head 25c1cc8ac. Linux: 937 passed and 43 skipped (980 total); Windows: 639 passed and 357 skipped (996 total). All Linux, Windows and macOS relinker jobs, register-context jobs and write-watch jobs passed. These full runs validate the independent topic, beyond the earlier local integration. GitHub conventions passed.

Checklist

  • One topic per PR: guest startup dependency selection
  • No comments added to code
  • Generic module and import dependency handling
  • Unimplemented paths still fail
  • No game assets, binary excerpts, screenshots or investigation logs added
  • No new third-party code
  • Depends on: none
  • Unblocks: none
  • AI-assisted: yes

Historical upstream validation

Former head c37a4b8b6b85d94cb9bcc6c75b8b86905ebf3c29 contains main d1670009fbfdfe57f8f26ee09ced77fe6b18c2ca. The topic's production change is unchanged by the latest rebase. Previous validated build passes at 4b26d31b1db2317ca538a3e7fd0e614257db1520 against main 7a0c838ca48; those results apply to their explicitly recorded earlier head/base. Earlier full Linux and Windows validation passes for c37a4b8b6; expected skips remain unvalidated.

Independent macOS validation

jotitadev reports testing former head c37a4b8b6 on main 84b01f9cd, Apple M2 Max / macOS 27.2 / AppleClang. The arm64 relinker-only build passed 157/157 cases. Under Rosetta 2, 151/157 passed; the same six failures reproduced on unmodified main, and guest_deferred_dependencies passed. Dependency inspection confirms that the imported executable loads its required provider while leaving the unused consumer deferred.

Their runtime check used the synthetic imported-module fixture, a stand-in startup shim and the separate #5362 fixture correction. With those stated conditions, the relocated call returned 42 and the provider initialized once. The uncorrected fixture fails dyld on both main and this topic. This is contributor-reported fixture evidence, not a native game run or a claim that this branch includes #5362.

@jotitadev

Copy link
Copy Markdown
Contributor

macOS data point, since the description only inspects the macOS dependency lists.

Apple M2 Max, macOS 27.2 (26B5091g), AppleClang, Ninja Release, relinker-only (-DANYPS5_RELINKER_ONLY=ON -DBUILD_TESTING=ON). Head c37a4b8b6 merged without conflicts into main 84b01f9cd.

  • arm64 build: ctest --parallel 8 --timeout 30 157/157 pass, including guest_deferred_dependencies.
  • x86_64 build under Rosetta 2 (-DCMAKE_OSX_ARCHITECTURES=x86_64, x86_64 Python): 151/157. The 6 failures (pr_overlap_encoding, progress_history, stubgen_stdout_newlines, release_package, linux_fixed_base, guest_module_args) fail the same way on unmodified main 84b01f9cd on this machine (five are Python 3.9 incompatibilities of the only x86_64 Python here; guest_module_args is the fixture test(relinker): keep the guest_module_args fixture's GOT out of executable memory #5362 fixes); guest_deferred_dependencies passes.
  • The new test run against a main relinker fails at the first scenario: ('runtime', 'consumer', [...provider..., ...consumer..., 'libc.prx']).
  • Mach-O imported scenario, otool -L: main loads provider.prx.guest.prx and consumer.prx.guest.prx; this PR loads only provider.prx.guest.prx.

Runtime check of the imported Mach-O output under Rosetta 2, with libs/libc.prx = x86_64 guest_module_args_host and libs/libkernel.prx = a stand-in whose Aps5StartGuest_nid_no_patch switches to the prepared stack and jumps to the entry (the fixture's Linux exit syscall is caught as SIGSYS and turned into _exit(rdi)):

Written with AI assistance (Claude Code). I ran the builds and tests above on my machine.

@Zinebyte

Copy link
Copy Markdown
Contributor

Small Linux fixture correction for composition with #6106 (d3e2f294ddc6d9252ab6dfd1904a234b622153c5), checked against this PR's current head c37a4b8b6b85d94cb9bcc6c75b8b86905ebf3c29.

test_guest_deferred_dependencies.py supplies the host fixture as libs/libc.prx, but #6106 makes the generated initializer helpers' libkernel.prx dependency explicit. The deferred load at line 127 consequently fails with:

OSError: libkernel.prx: cannot open shared object file: No such file or directory

Copying the same existing host fixture under the required kernel name fixes it:

diff --git a/core/relinker/relinker/tests/test_guest_deferred_dependencies.py b/core/relinker/relinker/tests/test_guest_deferred_dependencies.py
--- a/core/relinker/relinker/tests/test_guest_deferred_dependencies.py
+++ b/core/relinker/relinker/tests/test_guest_deferred_dependencies.py
@@ -117,6 +117,7 @@ def main():
                 libraries = case / "libs"
                 libraries.mkdir()
                 shutil.copyfile(host, libraries / "libc.prx")
+                shutil.copyfile(host, libraries / "libkernel.prx")
                 events = case / "startup-events.txt"
                 output.chmod(0o755)
                 run = subprocess.run([str(output)], capture_output=True, text=True, timeout=30,

Fresh local verification: Linux x86-64, GCC 13.3, Release relinker-only build, main 6239ceffc84e0c104904e5454f8f3c1eb22a4049 composed with #6100, #6103, #6105, #6106, #6080 and this PR. Without this line, guest_deferred_dependencies fails in 0.18 s. With only this fixture correction, 174/174 CTests pass in 22.82 s, including that test in 0.23 s. Conventions and whitespace checks pass. Earlier isolation also confirmed that this PR alone passes and adding #6106 triggers the failure.

This follows the two-copy fixture pattern already used in test_guest_module_args.py and preserves required-library enforcement. Could you include this one-line fixture update?

Prepared and verified with OpenAI Codex assistance.

@adriadam10

Copy link
Copy Markdown
Collaborator

Checked the dependency selection on the merged tree: relinker build and ctest 133/133 with no skips, and guest_deferred_dependencies fails when GuestModuleBuilder.cpp is reverted. It covers runtime-only deferral, executable/bundled roots, imported provider (native run returns 42), unreferenced symbols and no-op relocations, plus macOS dependency lists. Merging once the full build is green.

@adriadam10
adriadam10 merged commit 38d2454 into boykopovar:main Oct 15, 2026
10 of 11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants