Repository navigation
fix(relinker): bind imports declared from a guest module only to that module's exports on Linux and macOS - #11880
Conversation
… module's exports on Linux and macOS An import that declares a bundled guest module which does not export its NID was still renamed to the guest suffix on Linux and macOS when another bundled module exports the NID, so the dynamic linker bound it to that module. For guest-module imports the builder already chooses the provider only among the declared module's exports and records such an import as a system import of that module; the guest-module rename now follows that decision, and the executable rename applies the same rule, as Windows does. Such an import keeps its plain NID, and an executable import no longer makes an unrelated module from --module-dir start with the executable; undeclared and host-declared imports and Windows outputs are unchanged.
…declared-module-binding
rooklz
left a comment
rooklz
left a comment
There was a problem hiding this comment.
Build-verified on macOS (the host lane left untested in the description).
Code parity. Every rule in the description maps to the code at a5e1bc5: systemImports receives exactly the undefined non-empty-name imports without a provider in their declared module (empty names continue before any insert, host-declared imports take the early arm, and the provider loop has restricted candidates to the declared module since #2212), the registry takes its imports only from systemImports (line 479), so the old rename indeed contradicted the registry and the Windows output. For the executable arm, the rebuilt dynamic section carries import names already stripped of #lib#mod with ImportModules keyed by relocation address (SysVDynamicSectionBuilder), so executableModules is the declared module and the old condition fired for any bundled exporter, dragging the unrelated module in through executableProviders. The windows || guard on the new findGuest lookup is needed: on Windows findGuest emplaces case-folded aliases into guestNames, which the earlier executable pass must not trigger; on Linux/macOS it is a pure lookup. Host-declared, undeclared and declared-and-exported imports keep their old paths, and the exported arm of the guest rename is untouched.
Red/green. With this PR's test file over the base GuestModuleBuilder.cpp (801d5b9), guest_symbol_names fails with AssertionError: ('other', ['ZZZZZZZZZZZ#guest', 'entry#guest']) — identical to the counter-check in the description; at a5e1bc5 it passes.
Tests (macOS, Release, ANYPS5_RELINKER_ONLY). ctest -R 'guest_|module|import|registry': 44/44, same set as the Windows run. Full suite: 361/361, matching the local main baseline (the two failures on your machine are Windows-specific).
Overlap. pr_overlap: no shared lines with any open PR that merges with main; #10861 is the only uncomparable. Hand-checked at 8a4fa9a: its GuestModuleBuilder.cpp hunks sit in the top-of-file helpers and the syscall scan (406-435), disjoint from both change sites here, and a trial merge conflicts only in CodeInstructionCollector.cpp with GuestModuleBuilder.cpp auto-merging — same conflict main itself has with #10861. Trial merge with #11479 (needed-name alias loop + a hunk below 489): clean.
|
Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.
|
|
Checked the declared-module condition against the builder's own provider choice (the executable rename at line 248 and the guest-module rename at line 336), the new test's control and missing-export cases for both the guest module and the executable, and the relinker jobs on Linux, macOS and Windows; merged. |
What
On Linux and macOS,
GuestModuleBuilder::Buildrenames a guest-module or executable import that declares a bundled guest module (NID#lib#mod, mapped throughDT_SCE_NEEDED_MODULEandDT_NEEDED) to<NID>#guestwhenever any bundled module exports the NID, also when the declared module does not export it, so the dynamic linker (global-scope lookup on Linux, flat lookup on macOS) binds it to another bundled module that does. The builder's own provider choice (lines 278 to 296 onmain, restricted to the declared module by #2212) finds no provider there and records the import as a system import of the declared module, so the outputs contradict the registry and the Windows output. On untouched 48c160c, withsce_module/provider.prxexporting onlyother,sce_module/third.prxexportingentryandconsumer.prximportingentry#A#Bdeclared fromprovider, the Linux consumer keptentry#guestwithDT_NEEDEDonly$ORIGIN/provider.prx.guest.prxwhile its--registryfile listed{"nid": "entry", "library": "provider.prx"}, the macOS consumer containedentry#guest, and the Windows program exited0xC0000139; an executable importingentry#A#Bdeclared fromprovidergotentry#gueston Linux and macOS, and withthird.prxin--module-dir Mediathe Linux executable'sDT_NEEDEDalso listed$ORIGIN/app0/Media/third.prx.guest.prx, so the unrelated module started with the executable.#guestname on Linux and macOS only when that module exports the NID, the provider decision the builder already makes and Windows already applies. The guest-module rename (line 334 onmain, 336 here) also requires that the import is not insystemImports, which holds the imports without a provider in their declared module;declaresHoststays, because imports with an empty name never entersystemImports. The executable rename and itsexecutableProvidersinsert (line 247 onmain, 248 here) require, for an import that declares a guest, that this guest is among the NID's providers;declaredis looked up only for non-Windows targets, becausefindGuestadds case-folded aliases toguestNameson Windows. This completes fix(relinker): bind Linux guest imports by declared module identity #2212 (144b1ea), which restricted the provider choice but left both renames checking only whether the declared module is a host module (fix(relinker): keep Linux imports declared from host modules off guest exports #1529, 5f4a28c) and whether any bundled module exports the NID. Undeclared imports (empty library, including SDK named imports and the case of relinker: RetroArch (PPSA99169) can't convert its libretro cores: two are refused for DF_SYMBOLIC, and cores exporting the same name make another core's import ambiguous #5927), host-declared imports and imports whose declared module exports the NID are unchanged. Such an import keeps its plain NID, as host-declared imports do; no message is added. Windows outputs do not change: both renames only run for Linux and macOS.test_guest_symbol_names.pygets one loop afterguest-unmatched-ps4:provider.prxexportsentry(control) orother,third.prxexportsentry. Aconsumer.prximportingentry#A#Bdeclared fromprovidermust keepentry#guestin the control and plainentryotherwise, and an executable importingentry#B#Bdeclared fromprovidermust import{entry#guest}and{entry}respectively.pr_overlap.pyreports no shared lines with open pull requests that merge withmain. It cannot compare this branch with fix(relinker): analyze referenced code beyond SCE scan boundaries #10861 (GuestModuleBuilder.cpp), because that pull request does not merge withmain. Checked by hand at fix(relinker): analyze referenced code beyond SCE scan boundaries #10861's head 8a4fa9a: it changes the includes and helpers at the top ofGuestModuleBuilder.cppand the syscall scan of the code segments inBuild(lines 406 to 435 onmain), not the executable import filter (line 247) or the guest rename (lines 330 to 334) that this PR changes. A trial merge of this branch with fix(relinker): analyze referenced code beyond SCE scan boundaries #10861 conflicts only inCodeInstructionCollector.cpp, asmainwith fix(relinker): analyze referenced code beyond SCE scan boundaries #10861 does. fix(relinker): reject a needed module name that several bundled modules match only by extension #11479 also editsGuestModuleBuilder.cpp, in the needed-name alias loop (lines 110 to 174 onmain); the two merge without conflict.Tested
Windows 11, GCC 15.2.0 (WinLibs UCRT), relinker-only Release build (
-DANYPS5_RELINKER_ONLY=ON -DBUILD_TESTING=ON); synthetic fixtures only, no title or homebrew.guest_symbol_nameswith the new cases on untouched 48c160c fails at the guest-module case withAssertionError: ('other', ['ZZZZZZZZZZZ#guest', 'entry#guest']), after the control passed. Run directly with the test's inputs, untouched code gives the executable{'entry#guest'}on Linux for both providers. With the change the consumer has['ZZZZZZZZZZZ#guest', 'entry'], the executable imports{'entry'}, neither macOS output containsentry#guest, and the--module-dir Mediaexecutable'sDT_NEEDEDlists onlyprovider.prx.guest.prx; the controls are unchanged. All files the six Windows conversions (both guest-module inputs and the four executable inputs) write are byte-identical before and after; the Windows programs exit0xC0000139for the missing export, and the controls exit0x2a(guest-module input) and0x1(executable input,guest_import_fixtures.executable), before and after.ctest -R 'guest_|module|import|registry' -j 5on 65dba8a: 44 tests passed; run alone,guest_symbol_names(1.1 s),guest_module_identity(7.4 s),windows_import_modules(10.1 s),call_registry(3.3 s),guest_registry(0.6 s),linux_guest_named_imports(0.6 s),weak_imports(1.1 s) andmacos_guest_weak_imports(0.4 s) passed.main(801d5b9) into the branch without conflicts (a5e1bc5); since 48c160c,mainchangedGuestModuleBuilder.cpponly through fix(relinker): skip backup copies of guest modules in the online converter too #10997, which appends lines after line 489, away from the lines of this PR. On a5e1bc5,ctest -R 'guest_|module|import' -j 6: 44 tests passed, among themguest_symbol_names(1.3 s),guest_module_identity(9.6 s),guest_needed_modules(4.8 s),import_symbol_types(4.1 s) andwindows_import_modules(13.1 s).ctest --test-dir build-relinker -j 6: 369 tests, 366 passed,long_pathsskipped; failures known on this machine onmainas well:select_tests(Windows symlink fixture) andpatch_advisor_unit(path with spaces, which open test(tools): compare quoted and resolved paths in the patch advisor test #10667 fixes); both failed again alone.python tools/check_conventions.py --base upstream/mainwith this title and description passed on a5e1bc5.Checklist
main; no other open PR implements the same functionsNotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file.mdfiles or images added to the repository (attach them to this PR)