Repository navigation
feat(build): add ANYPS5_GUEST_LIFTED to build the libraries for ps5lift - #15799
oberonlift wants to merge 17 commits into
Conversation
✅ NVIDIA GPU test: passedPR head
Failing on main too (3, not counted against this PR)Validation IDs already present on main (13 IDs in 809 tests)Skipped (53)ReproduceAutomated build and test on a contributor's NVIDIA machine; it does not replace upstream CI. A new comment is posted for each new push. Full logs. |
39a6b84 to
19e1243
Compare
✅ AMD GPU test: passedPR head
Unstable on main (5, these also failed in recent main runs on this machine; not counted against this PR)Failing on main too (13, not counted against this PR)Passed on retry (1, failed in the parallel run, passed when re-run with a 360 s timeout; not counted against this PR)Validation IDs already present on main (8 IDs in 809 tests)Skipped (62)ReproduceAutomated build and test on a contributor's AMD machine; it does not replace upstream CI. A new comment is posted for each new push. Full logs. |
19e1243 to
a14472a
Compare
✅ NVIDIA GPU test: passedPR head
Unstable on main (3, these also failed in recent main runs on this machine; not counted against this PR)Failing on main too (3, not counted against this PR)Passed on retry (1, failed in the parallel run, passed when re-run with a 360 s timeout; not counted against this PR)Validation IDs already present on main (9 IDs in 812 tests)Skipped (59)ReproduceAutomated build and test on a contributor's NVIDIA machine; it does not replace upstream CI. A new comment is posted for each new push. Full logs. |
a14472a to
28f9dbb
Compare
28f9dbb to
9634bbe
Compare
✅ NVIDIA GPU test: passedPR head
Unstable on main (3, these also failed in recent main runs on this machine; not counted against this PR)Failing on main too (3, not counted against this PR)Passed on retry (2, failed in the parallel run, passed when re-run with a 360 s timeout; not counted against this PR)Validation IDs already present on main (9 IDs in 812 tests)Skipped (61)ReproduceAutomated build and test on a contributor's NVIDIA machine; it does not replace upstream CI. A new comment is posted for each new push. Full logs. |
9634bbe to
1988618
Compare
✅ NVIDIA GPU test: passedPR head
Unstable on main (1, these also failed in recent main runs on this machine; not counted against this PR)Failing on main too (3, not counted against this PR)Validation IDs already present on main (9 IDs in 819 tests)Skipped (61)ReproduceAutomated build and test on a contributor's NVIDIA machine; it does not replace upstream CI. A new comment is posted for each new push. Full logs. |
1988618 to
c7d8a9b
Compare
The fenv functions read and write the x87 control, status and environment and MXCSR through GuestCpu's inline helpers, which issue the same x87 and SSE instructions, so a build for another host replaces them in one header. feholdexcept keeps the exceptions masked as the fnstenv it replaces did. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
sceKernelReadTsc and the TSC frequency measurement read the counter through GuestCpu::ReadTsc, which is __rdtsc. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Spin loops in the driver yield through PreciseSpinPause, which is now defined on every platform and uses the aarch64 yield hint, and WriteChanged falls back to its byte loop on targets other than x86-64. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…able helpers The worker's spin timing reads the counter through GuestCpu::ReadTsc and pauses through PreciseSpinPause. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
webrtc's CPU feature query runs cpuid and xgetbv through new GuestCpu helpers, which issue the same instructions as <cpuid.h>. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ps5lift's runtime is the library a lifted program links. It maps the guest images, dispatches indirect calls, lays out TLS as x86-64 does, runs x87 code on Berkeley SoftFloat, answers cpuid as the PS5's Zen 2, calls native functions through guest function pointers and back, and provides the GuestCpu functions, the guest FPU state and timestamp counter that AnyPS5's libraries read when they are built for a lifted guest. `tools/ps5lift.py runtime` builds it with SoftFloat, a new submodule, into a shared library for a target, and the program's main into an archive. Its known gaps are a ps5lift section of the technical debt. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A workflow builds the runtime with LLVM 18 on GitHub's ubuntu-24.04-arm runner every night, on request, and for pull requests that change ps5lift or the workflow. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GuestCpuCpuid and GuestCpuXgetbv give a library built for a lifted guest the PS5's answers that lifted code gets from cpuid and xgetbv, so it can pick the code paths the guest CPU supports. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Clang rejects a reinterpret_cast from const void* to a function pointer as casting away qualifiers, so only Apple builds cast the vtable entry to void* first, and a Clang build on Linux failed to compile libc. Every build now takes the const_cast form, which GCC compiles the same. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The libraries' lifted-guest code paths were only reachable by compiling their sources outside this build with the macro defined by hand, which cannot produce the generated and configured headers that libSceVideoOut, libSceSsl, libSceVorbisDec, libSceUsbd and libSceVideodec2 need. The option defines ANYPS5_GUEST_LIFTED for every target under core/libs, so the libs target builds all 368 libraries for a lifted guest on an aarch64 host. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The full build always asked ffmpeg-core for the x86-64 package, so in an aarch64 build every library that links FFmpeg (libSceAjm, libSceAudiodec, libSceAvPlayer, libSceVideodec2) failed to link. The architecture now follows CMAKE_SYSTEM_PROCESSOR. A processor with no package, and MinGW on arm64, which has none either, stop the configuration. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A lifted build's libraries call the GuestCpu functions that ps5lift's runtime provides. On Linux they resolved against the executable at load time; macOS binds every undefined symbol to a named library, so the libraries have to link the runtime. ANYPS5_LIFTED_RUNTIME names it, and libc links it publicly, so every library that links libc gets it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Under ANYPS5_GUEST_LIFTED GuestCpu's helpers call GuestCpu*_nid_no_patch functions, which ps5lift's runtime implements on the guest's emulated CPU state: the fenv functions, the timestamp counter, cpuid and xgetbv then see the guest's FPU and the PS5's CPU. The soft-float builtins compute in the default floating-point environment through <cfenv> and pick NaN results as x86 does, the 16-byte atomics use __atomic_compare_exchange_n, Bessel uses std::sqrt and std::fma, and the x86 call-trace thunks and the setjmp/longjmp assembly are left to the host runtime, which owns the guest registers. Direct builds are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The libraries link ps5lift's runtime, but their RPATH named only their own directory, so a library that a process loads with dlopen, as some tests load libc, could not find the runtime. In a lifted build their RPATH also names the runtime's directory. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The runtime refuses to describe the guest images until the executable hands it its lifted program, and a test program under core/libs is a native program with none, so libraries that look up guest images stopped. In a lifted build the test programs there link an object that hands the runtime an empty program, and link the runtime. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…arm64 The workflow builds the runtime, configures AnyPS5 with ANYPS5_GUEST_LIFTED on GitHub's ubuntu-24.04-arm runner and runs the tests that ps5lift/tests/lifted_tests.txt names on lavapipe: tools/lifted_tests.py builds the files their commands name and runs them with ctest. The 101 tests that pass with this change are the ones whose programs do not link libc, such as the relinker's and the shader recompiler's. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ForwardExports.cpp defines each forwarded export as an x86 jmp to the export it aliases, which does not assemble for aarch64. Lifted builds leave the file out: a lifted guest reaches an export only through an import bridge generated from its C declaration, and these aliases have none. The debt file lists the gap. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
c7d8a9b to
da948de
Compare
Merge after #15798. Until the PRs before it merge, this PR also shows their changes; oberonlift#9 shows only its own.
ps5lift is a static recompiler for PS5 executables: it lifts a game's x86-64 code to LLVM IR and compiles it ahead of time to aarch64, which runs on Linux against AnyPS5's own libraries with no interpreter or JIT. It runs AnyPS5's PS5-style test programs and a homebrew media player to its home screen; commercial games are not playable yet. Branch
ps5lift-previewmerges all of its PRs (how to try it); native Apple Silicon support is coming along fast. It is a separate approach from the ARM64 compiler in #5021; #1835 has an introduction.This is 3 of 8 PRs that let AnyPS5's libraries run a lifted program; the lifter itself follows them.
x86-64: unchanged. The option is off by default, and with it off nothing is compiled differently, apart from a
const_castthat libc'sstd::error_categorycode now takes in every build and that GCC compiles to the same object.What
A build option that compiles the libraries for a guest that ps5lift has recompiled to aarch64.
ANYPS5_GUEST_LIFTEDdefines the macro for every target undercore/libs, so thelibstarget builds all 368 libraries for a lifted guest on an aarch64 host, including those that need generated and configured headers. It needs Linux and Clang; configuring it otherwise stops with an error.GuestCpu.hppgets its lifted half: the helpers callGuestCpu*_nid_no_patchfunctions, which ps5lift's runtime implements on the guest's emulated CPU state, so the fenv functions, the timestamp counter,cpuidandxgetbvsee the guest's FPU and the PS5's CPU.ANYPS5_LIFTED_RUNTIMEnames ps5lift's runtime (feat(ps5lift): add the runtime for lifted programs #15798), which provides theGuestCpufunctions. libc links it publicly, so every library that links libc gets it.<cfenv>and pick NaN results as x86 does,lrintandllrintraise invalid out of range and inexact when they round, ascvtsd2sidoes, the 16-byte atomics use__atomic_compare_exchange_n, Bessel usesstd::sqrtandstd::fma, and the call-trace thunks and thesetjmp/longjmpassembly are left to the host runtime, which owns the guest registers. Direct builds are unchanged.std::error_categorycode casts a vtable entry away fromconstbefore it reinterprets it in every build, as Apple builds already did; Clang rejects the other form.CMAKE_SYSTEM_PROCESSORinstead of always being the x86-64 one. A processor with no package, and MinGW on arm64, which has none either, stop the configuration.core/libslink the runtime andcore/libs/tests/LiftedEmptyProgram.cpp, which hands it an empty program.ForwardExports.cppout of lifted builds. Its exports are x86jmps to the exports they alias, and a lifted guest reaches an export only through an import bridge generated from its C declaration, which these aliases don't have; the debt file lists the gap.ANYPS5_GUEST_LIFTEDon GitHub's Linux arm64 runner and runs the tests thatps5lift/tests/lifted_tests.txtnames on lavapipe;ps5lift/tools/lifted_tests.pybuilds the files their commands name and runs them with ctest. The list holds the 101 tests whose programs do not link libc.#5021 also changes some of the same files:
JumpBuffer.cpp.Tested
Compiled for x86-64 Linux before and after this PR (GCC 13, same flags): all 154 objects it reaches are byte-identical.
On a DGX Spark (aarch64 Linux, NVIDIA GB10, LLVM 18), with all eight PRs and ps5lift on top: every library builds for a lifted program, 2,285 of AnyPS5's own tests pass in that build on lavapipe, and ps5lift's 69 tests pass, among them its libc, exceptions, kernel, fiber and module-loading fixtures and upstream's libc test programs.
The conventions check passes on this branch.
Checklist
main; no other open PR implements the same functionsNotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file.mdfiles or images added to the repository (attach them to this PR)🤖 Generated with Claude Code