Repository navigation
sync(upstream): integrate Batch 42 relinker, AGC driver, libc/libkernel and PRX exports - #583
Open
RINNECODER wants to merge 181 commits into
Open
RINNECODER wants to merge 181 commits into
RINNECODER wants to merge 181 commits into
Conversation
(cherry picked from commit 13b6650)
…sing the module On x86-64 macOS, including under Rosetta, the host owns GS (it holds the pthread TSD base, which the macOS TLS stubs read), so a guest instruction that reads or writes through GS cannot be translated, and the TLS scan refused the whole module. Prospero Win's (PPSA99995) Wine ntdll has six. Overwrite each in place with ud2 and NOP padding of its own length, as the Windows writer does in boykopovar#6102, and refuse one that is also an AMD-only trampoline site. (cherry picked from commit 55c6254)
(cherry picked from commit 4187f0b)
(cherry picked from commit 5f56417)
__std_find_trivial(_unsized), __std_count_trivial, __std_min/max/minmax_element, __std_reverse_trivially_swappable, __std_reverse_copy_trivially_copyable and __std_swap_ranges_trivially_swappable_noalias for 1-, 2-, 4- and 8-byte elements, matching the libc.prx bundled with PPSA08427 including the rescan of the first aligned block in the unsized find. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Q8tinbP1MUpUaDsbNWnGnu (cherry picked from commit 6103a3e)
copysignl, fdiml, fmodl, remainderl, remquol, logbl, ilogbl, rintl, roundl, lroundl, lrintl, llrintl, sqrtl, modfl, nanl, scalblnl, fmal, rint(f), lrint(f), llrint(f), drem(f), scalbln(f), scalb(f), finite(f), __fpclassifyl, __isinfl, __isnanl, __isnormall, _LDtest, _Dclass, _FDclass, _LDclass and _LDsign, matching the libc.prx bundled with PPSA08427 bit for bit, including NaN propagation, errno and FreeBSD fmal special cases. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Q8tinbP1MUpUaDsbNWnGnu (cherry picked from commit 4f608a8)
Morrowind imports fchown and stops at it once rtprio_thread and the pthread concurrency controls are in. Accept the guest's own owner and group (0) or unchanged (-1), fail an invalid descriptor with EBADF, and throw for another owner or group, since ownership is not modelled. (cherry picked from commit 443df05)
(cherry picked from commit 2be7b07)
(cherry picked from commit 540c3a1)
MarkWritten and StoreOwnBytes return 0 for a range that is not wholly watched, and they stamped none of its blocks. The watched blocks such a store wrote kept their old stamps, so UnchangedSince and StoredOver over them still read as unchanged after a GPU write that crossed into an unwatched import. A vouched shader table column or a cached range then kept bytes that had been replaced. Both now stamp each block whose part of the store starts or ends in watched memory, as the wholly watched path stamps every block. They still return 0 and note the store as unstamped. This costs one or two coverage lookups per block, and only on this path. (cherry picked from commit c886c28)
… path The runtime layout printed ../app0/sce_module/... under app0/ when the output path contained . or .. components (./out.elf), because the module paths are normal and the output directory was not. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit f40acdb)
A driver store and an own store across a watched block and an unwatched one must stamp the watched block and no other. On Linux, a store across the halves of a split block must stamp it, and a store into its unwatched half alone must not. (cherry picked from commit a0dc9d3)
(cherry picked from commit f40d220)
(cherry picked from commit 73ab57d)
(cherry picked from commit 5356078)
…AY face A DIM 2D image_load through a cube T# already reads texel (x, y) of the T#'s BASE_ARRAY face (boykopovar#4139, measured on RDNA1). A DIM 2D image_store to a cube T# had no matching runtime mode and threw "image descriptor is incompatible with the static runtime image interface". Crimson Desert writes a 16_16_16_16_FLOAT cube this way from a compute shader. A mip-less image_store to an image the program only stores to now takes the same face mode, writing texel (x, y) of the BASE_ARRAY face, the face the measured load reads. Stores were not measured on hardware; the debt entry says so. Formats that store through a conversion (SINT) keep throwing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit dbf176f)
(cherry picked from commit 1fee29b)
Co-authored-by: SuperCatElvis <39642318+SuperCatElvis@users.noreply.github.com> (cherry picked from commit f25f226)
(cherry picked from commit 8363c01)
(cherry picked from commit af691b7)
(cherry picked from commit 51e9e5e)
… output macos_gs_traps converts the forms Wine's ntdll in Prospero Win (PPSA99995) uses, a store, an ALU read and a register-address form shorter than a jump, and checks that each becomes ud2 and NOP padding of its own length before the next instruction, with none of the original bytes left. (cherry picked from commit 8da2483)
The macOS writer sets up guest TLS only for a module with a non-empty PT_TLS. A GS access in a module without one must still be trapped. (cherry picked from commit 6f89fd7)
… offset overflow (cherry picked from commit 477f8a9)
A flush stores a storage image's pending results to guest memory and keeps the 64 KiB blocks the write tracker shows written since the image was last in sync. An address-based dispatch marks the pages it stored to only when its batch's completion runs. A flush between the GPU's store and that completion found the blocks unwritten and stored the image's older texels over the newer bytes; a CPU-resolved indirect dispatch then read those texels as its workgroup counts. FlushPending now looks for an unfinished batch that writes the range or holds it in an address-based lease (Recorder::PendingWriterSerial). Batches that already finished are completed first. If the writer is still unfinished, the flush finishes the in-flight batches up to it, or syncs when the writer is in the open batch, so its marks land and its blocks are kept. Inside a completion action it does not wait. If the wait throws, the images it took stay registered as pending. The test stores an image, then a global dword into the image's memory, and flushes the image twice: once with the writer's batch finished on the GPU but its completion not run, and once with the writer still in the open batch. Each time the dword is kept and a texel in another block is stored. Without write watching the in-flight case is skipped; the open-batch case still runs, as the write-back then compares guest memory, which the GPU has not written yet at the flush. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 9fffcea)
(cherry picked from commit b27e4c9)
…acOS sceNetCtlGetInfo found the host's default route in /proc/net/route, which macOS does not have, so every IPv4 query threw "the host routing table is unavailable" there. On macOS it now reads the IPv4 routes that have a gateway with sysctl(NET_RT_FLAGS, RTF_GATEWAY) and keeps those to 0.0.0.0 that are up, neither rejecting nor host routes, on an interface that still exists. An interface-scoped default route ranks after the unscoped one, the route macOS sends traffic by. The parsing is exported as NetCtlDefaultRoutes_nid_no_patch so it can be tested on a routing table built by hand. (cherry picked from commit 8af0802)
… route A routing table built by hand with an unscoped default route, a host route, a host route to 0.0.0.0, a rejecting default route, an interface-scoped default route, one that is down, one on an interface that does not exist and a truncated last message must give the unscoped route first and the scoped one after it, and must count every match while writing only as many as fit. (cherry picked from commit 6d4cde8)
…s first use sceAgcDriverResolveShaderAbi and sceAgcDriverResolveGraphicsStagesAbi prepare every stage when the title creates its pipeline, and threw when the recompiler could not: Resident Evil Village stops there on a pixel shader whose T# pointer is a phi of user data and a ballot, and a compute shader that calls user-data pointers stopped the same way although its dispatch links the callees. ResolveShaderAbi and PrepareGraphicsStages now catch the failure inside their own preparation transaction, so the ShaderPreparationScope that libSceAgc opens around the call still commits. The shader is logged once, marked deferred like a shader deferred at registration, and the ABI key is recorded, so a repeated call prepares nothing. The draw or dispatch that uses the stage prepares it as before and fails there if it cannot. ResolvePreparedGraphics skips the rectangles of a deferred stage that has no artifact yet, which also lets a stage deferred at registration through sceAgcDriverResolveGraphicsAbi. Driver failures, ProcessShutdown and a failure stored by the shader's registration preparation still propagate. (cherry picked from commit 8c55d15)
Every OpGroupNonUniformBallot now goes through SpirvModule::AddSubgroupBallot. A repeated ballot of the same predicate in the same block becomes an OpCopyObject of the first ballot. Within one block the active invocations do not change, so the result is identical. The cache is cleared at every label, function boundary, function call and demote, and helper functions keep their own cache. When SPIRV-Tools is enabled, block merging and specialization can still put two ballots of one predicate into the same block after emission, so ValidateAndOptimizeSpirv collapses repeated block ballots the same way on every module it returns, optimized or not. NVIDIA 580's SPIR-V compiler segfaulted (libnvidia-gpucomp+0xe3facc) on GTA V compute shaders that ballot one predicate many times in a block. One such shader had 163 repeated ballots across three blocks. With them collapsed, both crashing captures (compute-fe506552cec99bfc and compute-c02e5e3f35010fa5) compile. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit d83f541)
# Conflicts: # core/libs/prx/libSceAgcDriver/CMakeLists.txt
Drop the second ParseInteger copy (main already carries it) and list the sceRtcInit/sceRtcEnd silent stubs as upstream's debt doc does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Reverts 3fc8b2b: all 149 stubs it appends already exist in the fork's Export.cpp (redefinition errors). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Upstream moved it to MathExact.cpp (raw x87 bits, pseudo-denormal aware); drop the older Math.cpp copy. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two Batch 42 picks each added it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Reverts 264467d: all 20 stubs it appends already exist in the fork's Export.cpp (redefinition errors). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Reverts 96e373f: it calls WindowsLoadImage::GetImageBase, which the fork's load image does not have. Pure refactor, no behaviour lost. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Owner
Author
|
@codex review |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configuration
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fd3402589c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| "PATH=path_list_prepend:$<TARGET_FILE_DIR:libSceGameLiveStreaming>") | ||
| endif() | ||
|
|
||
| add_test_executable(guest_game_live_streaming_tests tests/GuestGameLiveStreaming.cpp) |
There was a problem hiding this comment.
When BUILD_TESTING=ON, this repeats the identical add_test_executable(guest_game_live_streaming_tests ...) block already present at lines 298–306. CMake requires target names to be unique, so configuration stops at this second declaration before any tests can build; remove the newly duplicated block.
Useful? React with 👍 / 👎.
| @@ -79,7 +82,6 @@ target_sources(libc PRIVATE | |||
| ${CMAKE_CURRENT_SOURCE_DIR}/src/RealPath.cpp | |||
| ${CMAKE_CURRENT_SOURCE_DIR}/src/CxxAbiSupport.cpp | |||
| ${CMAKE_CURRENT_SOURCE_DIR}/src/Threads.cpp | |||
| ${CMAKE_CURRENT_SOURCE_DIR}/src/Backtrace.cpp | |||
| ${CMAKE_CURRENT_SOURCE_DIR}/src/exception/Exports.cpp | |||
There was a problem hiding this comment.
Removing src/Backtrace.cpp from target_sources(libc ...) drops the only definitions of sceLibcBacktraceGetBufferSize_nid_postfix and sceLibcBacktraceSelf_nid_postfix. Guest modules importing those supported functions can no longer resolve them, and the existing Unix guest_backtrace_tests executable directly references both symbols and therefore cannot link.
Useful? React with 👍 / 👎.
|
|
||
| set_target_properties(agc_driver_graphics_tests PROPERTIES CXX_EXTENSIONS OFF) | ||
| configure_windows_unwind(agc_driver_graphics_tests) | ||
| if(BUILD_TESTING) |
There was a problem hiding this comment.
On MinGW builds, this replacement also removes configure_windows_unwind(agc_driver_graphics_tests), even though the target contains many throw/catch paths. That hook renames the executable's .eh_frame section to .ehfram, which the custom libc unwinder scans on Windows; without it, exceptions crossing graphics-test frames cannot be unwound reliably and the newly registered tests may terminate instead of reaching their handlers.
Useful? React with 👍 / 👎.
…ushes The deferred-registration pushes used upstream's five-field initializer; the fork's PreparedGraphicsStage has only snapshot and entry. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… check Upstream's section-relative fixture uses a section symbol (Info 3, value 0) with a non-zero addend, which the fork's symbol==addend check rejected. Drop the now-dead info rewrite for symbol-less RELATIVE entries. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Pm4::Validate now accepts misaligned wait addresses, but the Metal pending label check, DispatchAhead and the draw-pipeline keep check still used the raw address. Add Pm4::WaitAddress and use it at all WAIT_REG_MEM readers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
DecodeDrawDispatch is the fork-only decode Metal uses; upstream's NormalizeUnusedViewportDepth was only added to the Vulkan DrawDecode path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Reverts 57466a0 (upstream e8544ef). Its fix, upstream caae92c, does not apply to the fork (ShaderMemory.cpp, ResourceMaterializer.cpp, SpirvImageEmitter.cpp, SpirvDescriptors.cpp conflict), so the globbed test would fail. Re-add it together with a port of caae92c. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tests/Graphics.cpp defined struct LiveCounter twice (also on main), so the graphics test target could not compile. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
test_guest_relative_section_symbols and test_guest_none_relocations import it, but the fork never took the module (upstream 723d02c), so both failed with ModuleNotFoundError. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…atch arguments Hand port of upstream f6c8e84. CPU reads of DISPATCH_INDIRECT arguments now finish leased batches (Graphics::FinishLeaseWork) so an address-based producer has run; a read-ahead whose counts differ from the re-read is dropped. Conflicts kept the fork's re-decode after adopting a parked packet and its NVIDIA split message (now with the CPU-read note). The new test drops upstream's code-address PrepareDispatch argument, which the fork's VulkanDevice does not take. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
guest_game_live_streaming_tests was added twice by this sync; the agc_constant_select and guest_binary_word duplicates are already on main. Any configure that reaches these directories with BUILD_TESTING failed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Dinkumware atomics pick (5e45c8d) dropped it, removing the only definitions of sceLibcBacktraceSelf and sceLibcBacktraceGetBufferSize. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The fence-failure test pick replaced the target's tail and dropped CXX_EXTENSIONS OFF and configure_windows_unwind, which MinGW needs for the libc unwinder to see the test's throw/catch frames. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Owner
Author
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 51a5d97eae
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| @@ -754,7 +773,7 @@ if(BUILD_TESTING) | |||
| add_test(NAME application_heap_lazy COMMAND application_heap_tests lazy) | |||
| endif() | |||
| if(WIN32) | |||
| set_tests_properties(application_heap_replacement application_heap_invalid_metadata application_heap_default application_heap_exit_order application_heap_lazy | |||
| set_tests_properties(application_heap_replacement application_heap_invalid_metadata application_heap_default application_heap_exit_order application_heap_scripting application_heap_lazy | |||
There was a problem hiding this comment.
On every Windows configuration with BUILD_TESTING=ON, this includes application_heap_scripting, but no test with that name is registered anywhere in the repository; the surrounding block only creates the replacement, invalid-metadata, default, exit-order, and lazy tests. As confirmed by cmake --help-command set_tests_properties (“If the test is not found, CMake will report an error”), this makes Windows configuration fail before anything can build.
Useful? React with 👍 / 👎.
| if os.name == 'nt': | ||
| compiler.write_text('@echo off\nif "%~1"=="-print-sysroot" if "%~2"=="" exit /b 0\nif "%~1"=="--version" if "%~2"=="" exit /b 0\n>>"%~dp0compiler-invoked" echo %*\nexit /b 99\n', encoding='utf-8') | ||
| else: | ||
| compiler.write_text('#!/bin/sh\nif [ "$#" -eq 1 ]; then case "$1" in -print-sysroot|--version) exit 0;; esac; fi\nprintf "%s\\n" "$*" >> "$(dirname "$0")/compiler-invoked"\nexit 99\n', encoding='utf-8') |
There was a problem hiding this comment.
With the repository's CMake 3.28.3, running this newly registered test invokes the fake compiler as -Wl,--help; the script records that as an unexpected invocation, so every one of the eight test methods fails before exercising the shader-cache assertions. This test runs in the full Linux ctest job, so the fake compiler must also handle CMake's linker probe (or the fixture must avoid treating that probe as a failure).
Useful? React with 👍 / 👎.
8dd0b5d added the name to the Windows set_tests_properties list, but the add_test from upstream was never picked, so Windows configure failed on the unknown test. Register it as upstream does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…probes CMake 3.28 calls the compiler with -Wl,--help, -Wl,--version and -v --help; the fake compiler logged them as unexpected and every case failed. Take upstream's probe handling. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Integrates upstream Sync Batch 42 (five slices
team/u42-0..4, each cherry-picked with-xontoorigin/main, merged--no-ff).150 upstream commits applied (153 picked, 3 reverted after build), plus fork ports
e094fb91e(PositionExportLayout fix on the Metal copies) and954c4cb4f(follow-up to 538b63e).Applied, by area (upstream PRs)
Reverted during integration
feat(libScePad): export missing firmware pad interfaces(upstream feat(libScePad): export missing firmware pad interfaces boykopovar/AnyPS5#6600): all 149 stubs already exist in the fork (redefinition errors).feat(libSceAudioPropagation): export missing firmware functions from database(feat(libSceAudioPropagation): export missing firmware functions from database boykopovar/AnyPS5#6849): all 20 stubs already exist in the fork.refactor(relinker): take the macOS image base from the load image(feat(relinker): place macOS executables at a fixed base with --macos-fixed-base boykopovar/AnyPS5#6802, 96e373f): needsWindowsLoadImage::GetImageBase, absent in the fork. Pure refactor.Integration fixes: union of the AGC test lists in
libSceAgcDriver/CMakeLists.txt; oneParseIntegercopy inSonyCrt.hpp(main already had it); one__fpclassifyl(upstream's raw-bits version inMathExact.cpp); oneGuestEbadf; thesceRtcInit/sceRtcEndsilent-stub debt entry.Skipped (summary)
Gates
check_conventions.py --base origin/mainrosetta_image_cachefails identically onorigin/main(2 overlap checks), sources untouched hereagc-map --expect-hash 51a268f2e73efb9cagc-live-mapvsreplay-ref-v3.pngagc-menuvsreplay-ref-v3.pngRisky resolutions to review
RelinkerPipeline.cppscanSegments copy with NOP-filled jump tables (f25f226);SyscallScanner.cppnow throws on size over buffer (80a987f);main.cppOutputIdentityDirectoryno longer Windows-only (502b557);GuestImageReader.cppkeeps the fork's RELATIVE-with-symbol block (5f56417).GuestMemory.cppstampWatchedPart passescomplete=falsefor partly watched stores (c886c28);ShaderRegistry.cppResolveShaderAbi deferral ported onto the fork's transaction version;TessellationLowering.cppdynamic-address branch only (9f429e6);BufferPool.cppsource fix only (5001f87).Review fixes (2026-10-23)
Pm4::WaitAddressstruct LiveCounterin tests/Graphics.cpp (also on main)guest_import_fixturesapplication_heap_scriptingin properties but not registered🤖 Generated with Claude Code