Skip to content

feat(libSceAjm): run PS4 batch buffer jobs in the console's chunk layout - #17

Closed
ntamas94 wants to merge 7710 commits into
Get-fix:ps4/limbo-gpufrom
ntamas94:feat/ps4-ajm-batch-buffer
Closed

ntamas94 wants to merge 7710 commits into
Get-fix:ps4/limbo-gpufrom
ntamas94:feat/ps4-ajm-batch-buffer

Conversation

@ntamas94

@ntamas94 ntamas94 commented Oct 21, 2026 •

Copy link
Copy Markdown

What

Knowledge is Power: Decades (CUSA12435, Wwise), the one PS4 title observed, builds its AJM batches with the PS4 SDK's inline batch helpers, which end in four libSceAjm exports: sceAjmBatchJobControlBufferRa, sceAjmBatchJobRunBufferRa, sceAjmBatchJobRunSplitBufferRa and sceAjmBatchStartBuffer. Unlike the PS5 builders, whose batch records are private to the library, here the batch buffer belongs to the title: the SDK code reads the position each builder returns, sizes the buffer for the console's jobs and may write chunks itself. So the buffer has to use the console's layout. On main, boykopovar#9433 added the PS4 libSceAjm with three of the four as throwing stubs, and sceAjmBatchJobRunBufferRa (ElslOCpOIns) is only a throwing stub in libSceAjmi.

This implements the layout in libSceAjm.native/src/Ajm.cpp, which libSceAjm is built from:

  • The builders write one job each: a job chunk (instance as payload, byte count of the job's chunks), an optional return-address chunk, the input buffers, a flags chunk (control or run, flags bits 32-51 as payload), the output buffers and the output control (sideband) buffer. They return the position after the job.
  • sceAjmBatchStartBuffer walks the chunks, skips inline-data chunks, turns each job into the library's existing control, run or statistics job and runs it with the existing decoders (ATRAC9, MP3, M4AAC, Opus), so the sideband results are the ones the PS5 path writes. Like sceAjmBatchStart, it rejects a context that initialization did not allocate, and the batch belongs to its context.
  • A chunk header is read as the console's builders write it: bits 0-3 the kind, bit 4 the output direction of a buffer chunk (0x11 output run, 0x12 output control), bits 6-25 the payload. The builders leave bits 5, 26, 27 and 31 as they were, and a title that reuses its batch memory carries the previous batch's bytes there, so those bits are ignored.
  • An ATRAC9 run with RUN_MULTIPLE_FRAMES reports which side ran out, as the console does: PARTIAL_INPUT (0x8) when the input ran out with room left, on a superframe boundary or not; NOT_ENOUGH_ROOM (0x10) when the room ran out with input left, after any number of frames, and with nothing consumed for a zero room; both (0x18) when the input ended just as the output filled, which is every Wwise run job. The ATRAC9 format reports the frame's bitrate (frame bytes x 8 x sample rate / 256: 168000, 84000 and 384000 measured). Resampled runs keep the earlier rule, as they were not measured.
  • A run job read from a PS4 batch buffer without RUN_MULTIPLE_FRAMES decodes exactly one ATRAC9 frame and reports 0, as the probe measured on the PS4. A job from the PS5 builders still decodes its whole input, which FMOD in PPSA29141 relies on; the PS5 console's behaviour there is not measured.
  • Buffer chunks follow what probe batches on the PS4 showed: a chunk of size zero is dropped whatever its address, so the job runs as without it, and a null address with a nonzero size refuses the whole batch before any of its jobs runs: sceAjmBatchStartBuffer returns 0x80930011 (MALFORMED_BATCH) and fills the error record with 0x80930016 (INVALID_ADDRESS), the job chunk's address, the chunk's offset in the job and the job's return address. A size that is not a multiple of 8, a chunk that overruns its job or the batch, and a job without a flags chunk are refused the same way, not measured; the record then holds 0x80930011. Unknown chunk kinds and control flags other than RESET, INITIALIZE and SIDEBAND_GAPLESS_DECODE throw.
  • The three stubs in libSceAjm/Export.cpp are dropped. The throwing sceAjmBatchJobRunBufferRa of libSceAjmi stays, declared as sceAjmBatchJobRunBufferRa_nid_disambig1: the NID patcher exports it under the name's NID, so a title importing it from libSceAjmi still gets the not-implemented error at call time, and the two libraries no longer export one name.

The layout (chunk kinds 0, 1, 2, 3, 4, 6, 7, 17, 18, the builders' chunk order, statistics instance 0x80000) follows shadPS4's PS4 library reimplementation. A GoldHEN user-mode plugin dumped 32 545 batches that Knowledge is Power: Decades (CUSA12435, Wwise) started on a PS4. They confirmed the 64-byte control and 80-byte run jobs byte for byte, the header bits above, and the 0x18 result and bitrate of all 14 680 traced run jobs, which each decode 4 frames into exactly 4 frames of room. Probe runs of one of the title's ATRAC9 streams through the same builders on the PS4 (fresh instance per case: input ending with room left, room ending with input left, half a superframe of room, zero room, exact fit) gave the result rules above, showed that a run without RUN_MULTIPLE_FRAMES decodes exactly one frame on the console (kept as the whole input here, which a PS5 title needs; noted in the debt), that the console rejects the tests' synthetic silent superframe after three frames where LibAtrac9 decodes four, and that LibAtrac9 matches the console's PCM of that stream within 4 LSB (83.5% of samples exactly). The statistics instance and the inline chunk come from shadPS4 only; the debt entries say what is measured and what is not. The PS5 sceAjmBatchStart path and its records are unchanged, apart from the ATRAC9 exact-fit result and bitrate, which both paths share.

Tested

Windows 11 Pro 26200, WinLibs GCC 15.2.0 r7, NVIDIA RTX 2080, on ps4/limbo-gpu at 45625370e4, merged into this branch. The same change is open upstream as boykopovar#9637; this branch carries the identical Ajm content for the integration branch.

  • guest_ajm passes, with seven new cases:
    • TestConsoleBufferJobs builds a control job (with a return address), a run job and a split run job with the three builders, checks the returned positions and the chunk words, starts the batch with sceAjmBatchStartBuffer and checks the control result and both run sidebands (result, consumed, written, total samples, frames) and the decoded PCM.
    • TestConsoleHandWrittenJobs writes a batch word by word: an inline chunk holding ATRAC9 initialize parameters, a control job whose input points into it, a run job and a statistics job. It checks their results, then that a size not a multiple of 8, a job without flags and an overrunning job return 0x80930011, and that an unknown chunk kind throws.
    • TestConsoleStaleHeaderBits runs a hand-written batch with every bit the console's builders never write set, and checks it runs exactly as the clean batch.
    • TestAt9FlagRules runs the five fits above and checks the result, the bytes consumed, the output written and the frame count of each.
    • TestConsoleBufferEdges runs a job with zero-size input and output chunks beside real ones (decodes as without them), a job whose only output has size zero (0x10, nothing consumed), and two batches with a null input or output address, which are refused with the measured error record while the control job before them stays unrun.
    • TestConsoleEmptyBatch checks the empty-batch contract: a null batch address is 0x80930005 with size 0 or 16, a null batch id is 0x80930005, and a batch of zero bytes starts, returns an id and completes at once.
    • TestConsoleSingleFrameRun runs a console job without RUN_MULTIPLE_FRAMES on a whole superframe with room for four frames and checks that one frame is decoded with result 0, the PCM after it untouched.
    • The existing ATRAC9 cases whose input exactly fills the output now expect 0x18; the helpers that stream a decode accept 0x8 and 0x10 where one side runs out.
  • guest_ajm_vibration passes; its exact-fit run now expects 0x18 (it expected 0 before). Vibration streams were not in the trace.
  • check_conventions passes; the patched libSceAjmi.prx still exports ElslOCpOIns (objdump).
  • PS4 probe batches (FW 10.71, ps4probe, the console's builders, a control job before each run job): zero-size input and output chunks alone, beside a real chunk and with a null address; null addresses with a nonzero size as the first or second input chunk and as the output chunk. They gave the rules above; the refused batches ran nothing.
  • Knowledge is Power: Decades, locally on my bring-up branch, with the first version of this change (before the header-bit and result rules above): Wwise builds every batch with these builders and a return address (80-byte run jobs, 64-byte control jobs, up to 6 jobs per batch). In four 150 s runs it ran about 30,800 jobs per run on five stereo ATRAC9 instances (control flags 0x6001, run flags 0xc00000001001, 56-byte sidebands), every one with result 0 under that commit's rules, about 650 s of decoded PCM per run, and the title reached its animated title screen. One of the four runs stopped before that on a UnityGfxDeviceWorker fault in the guest heap that runs with every AJM job skipped also hit; it is not in AJM. Without this PR the title aborts at its first sceAjmBatchJobControlBufferRa (not implemented).

boykopovar#9483 changes the last lines of sceAjmBatchStart's job loop, next to where this moves the batch registration into RegisterBatch, so the two conflict in three lines; boykopovar#9481 and boykopovar#9034 also add debt entries to libSceAjm.native.md. I can rebase onto whichever lands first.

Checklist

🤖 Generated with Claude Code

@Get-fix

Get-fix commented Oct 21, 2026

Copy link
Copy Markdown
Owner

Thanks @ntamas94. boykopovar#6977 is now a draft integration branch, and its own work is split into boykopovar#9429 to boykopovar#9433 (details). The maintainers asked that the branch not land anything without its own review, so could you open this upstream against main, on top of boykopovar#9433, with Depends on? Once it's open, I'll merge it into ps4/limbo-gpu, list it as a member and run Limbo with it. ps4/limbo-gpu now moves to the split branches, so this pull request may show conflicts here.

AI-assisted: yes (Claude Code)

🤖 Generated with Claude Code

@ntamas94

Copy link
Copy Markdown
Author

Thanks @Get-fix. Opened upstream against main as boykopovar#9637: one commit with this PR's net change, on top of boykopovar#9433, which is already merged, so Depends on: none. guest_ajm, guest_ajm_vibration and check_conventions pass on main at da54c652e. I'll leave this one open until you've merged boykopovar#9637 into ps4/limbo-gpu.

oneandonlydean and others added 27 commits October 29, 2026 12:16
…ffer

feat(libSceAjm): implement sceAjmBatchJobInlineBuffer
…inux-guest-ident-padding

fix(relinker): zero the e_ident padding of Linux guest modules
…JavaSpaceChar and u_isUWhiteSpace

Replace the five Unknown stubs whose NIDs hash these ICU names with the
uchar.cpp definitions of ICU 78.1 (049e0d6a), over the Zs, Zl, Zp and
White_Space code points that ICU 65.1 to 78.1 ship. Values outside
U+0000..U+10FFFF return 0, as ICU's trie error value gives.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ply_Scaled, FT_Vector_Transform_Scaled and ft_highpow2

Name four Unknown Ot exports by their NIDs and implement them after
FreeType 2.4.12 ftcalc.c and ftutil.c and bundled 2.13.3: truncating
MulDiv, the scaled matrix product and vector transform, and the highest
power of two. Inputs on which the FT_LONG64 and 32-bit builds of
FreeType 2.4.11-2.5.3 and bundled 2.13.3 can differ throw before any
output is written.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tionEnableMirroring, GetPredictedDisplayTime and SetRenderConfig

Without a reprojection session the three calls return 0x8a72000b
(reprojection not initialized) for every argument and write nothing.
A retail PS5 answered so from a payload before sceHmd2Initialize,
with every argument null and with every argument pointing at a zeroed
16 KiB buffer (firmware 13.60, T3cm0's hmd2s log on boykopovar#7798), and in a
PPSA23479 game process (firmware 13.40, frankea on boykopovar#8645).
sceHmd2ReprojectionInitialize throws on main, so no session can
exist; the in-session answers stay unimplemented and are recorded in
docs/dev/debt/libSceHmd2.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kldstat was a NotImplemented stub. It now follows FreeBSD 11.4's
sys_kldstat (sys/kern/kern_linker.c): only the version field is read
(EFAULT when that fails), and a version other than 1056
(struct kld_file_stat_1) or 2080 (struct kld_file_stat) is EINVAL.
No linker file is listed here, as for kldnext, so every file id then
gets ENOENT and the structure is never written.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
sceBluetoothHidThreadParamInitialize threw NotImplemented. It now fills
the thread parameter block that sceBluetoothHidRegisterCallback reads:
the 32-bit priority at offset 0, the 64-bit affinity mask at offset 8
and 0 in the 32 bits at offset 16, leaving bytes 4 to 7 and 20 to 23
as they were, and returns 0. A null block returns 0x81540001.

This follows BunnyFl's native x86-64 run of firmware 13.42's
libSceBluetoothHid, not a console:
boykopovar#11124 (comment)
The test checks the run's bytes for (700, 0x3f) and the null answer.
The run does not show the success value, range checks, bytes 20 to 23
or calls before sceBluetoothHidInit; the debt entry records them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… scePthreadMutexattrInitForInternalLibc

Both call libkernel's pthread_mutex_init and pthread_mutexattr_init and
return 0x80020000 | errno on error, as shadPS4 (ce74386) exports them as
ORBIS(posix_pthread_mutex_init) and ORBIS(posix_pthread_mutexattr_init)
and ChonkyStation4 (c101a23) maps them to its POSIX init functions. A
mutex made without attributes is error-checking. FreeBSD 11.4's
malloc-internal _pthread_mutex_init_calloc_cb makes a normal mutex
instead; the gap is recorded in the libkernel_sys debt file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…-concurrency

fix(libc): return the CPU count of the default thread mask from std::thread::hardware_concurrency
…-sensor-channels

fix(libkernel): return -49 from sceKernelGetSocSensorTemperature channel 64 as a PS5 payload process does
agc_compute_pipeline_capture_optional_byte_cap encoded, hashed and wrote eight
32 MiB records to the temp directory to fill the capture directory. Write only
reads the sizes of existing records, so the fixture now creates the eight
capture files at their sizes without writing data: SetEndOfFile on Windows,
resize_file elsewhere. On Windows CI the test timed out 9 times from 10-20 to
10-28 when the runner's writes to the temp disk slowed down.
The MinGW std::filesystem::last_write_time reads and sets file times
through the UCRT stat and utime calls, which convert through local time
using TZ. With a TZ other than the Windows zone, including the one libc
applies from the guest environment, times shifted by the difference:
under TZ=IST-5:30 on an Eastern host a temporary written that second
was 9.5 hours old, so housekeeping deleted it, and a TZ west of the host
set the active cache stamp hours in the past. Housekeeping now reads
times with GetFileAttributesExW, refreshes the stamp with SetFileTime
and compares in system_clock. The housekeeping test runs its child
with a TZ 12 hours from the host zone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When a save has no stored mtime, its parameters fall back to the save
directory's modification time. On Windows std::filesystem reads that
through the UCRT stat call, which converts through local time using TZ,
so with a TZ other than the Windows zone, including one libc applies
from the guest environment, the reported time was off by the
difference between the zones. The fallback now reads the time with
GetFileAttributesExW. guest_savedata_search checks it under TZ values
12 hours either side of UTC.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…fix and _sceUlobjmgrUnregisterObject_nid_postfix
…he throwing calls

The LncUtil collapsed dispatch entry, written by d4f2911 and
50c8103 (boykopovar#9137), said its protected service implementations remain
unchanged. Of the functions whose signatures d4f2911 restored,
sceSystemServiceGetAppId answers for the running title since
7c83407 (boykopovar#8354) and sceLncUtilGetAppId since 6950410 (boykopovar#10801).
Only the status, GPU-crash, dialog-activation, shared-memory,
continue and VSH-ready calls still throw.
adriadam10 and others added 28 commits October 29, 2026 14:33
…ocale-coverage

test(http): ensure URI locale regression runs in Linux CI
…ker-macos-guest-writer-reader-checks

refactor(relinker): drop macOS guest writer checks that the guest image reader already makes
…port-types

docs(libScePad): note the port types ten titles pass to scePadOpen
…entory

feat(libSceMouse): declare 7 additional exports
…valid-port

fix(input-mapper): handle invalid configured port without leaking stack
…entaccess-request-validation

test(libSceNpEntitlementAccess): cover request bounds and parameter checks in GuestEntitlements
…array-operators

feat(libc): export the remaining aligned and array operator new and delete overloads
…dns-info-to-kernel

feat(libSceNet): track sceNetSetDnsInfoToKernel gap and test throw
fix(libkernel): return EFAULT from poll for an unreadable or unwritable descriptor array
feat(libSceFreeTypeOt): implement the FreeType stream calls
…-skip-cpu-publish-of-compared-import

test(agc): skip the sparse import's CPU publish where host-imported pages read as written after a GPU read
…tack-above-arena

fix(libkernel): move the macOS main stack out of the guest arena
…d-label

fix(agc): publish small tiled writes before labels
…tation-dialog-uninitialized-coverage

test(libSceInvitationDialog): cover uninitialized and terminated argument validation
…hread-key-exhaustion

test(libkernel): check pthread_key_create fails with EAGAIN once keys run out
…cket-argument-errors

test(libSceNet): cover sceNetSocket family and type errors
…amic-scale

feat(tools): extend progress history chart scales dynamically
fix(libkernel): return EFAULT from pselect for an unreadable timeout or signal mask
fix(libc): open a directory with freopen in a read mode on Windows
…egister-past-last-buffer

fix(libSceVideoOut): return INVALID_VALUE for a registration past the last buffer as the console does
…t-write

fix(libc): keep setvbuf stream buffers in host memory
…r-constructor-array-overflow

fix(relinker): validate constructor array address range against 64-bit overflow
…nd null-batch result

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… the rate to the DCE driver

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…rning a stored port timestamp

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nd output-first order

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ffer

# Conflicts:
#	core/libs/prx/libSceAgcDriver/Graphics/include/GuestTextureResource.hpp
#	core/libs/prx/libSceAgcDriver/Graphics/include/State.hpp
#	core/libs/prx/libSceAgcDriver/Graphics/src/DepthSurface.cpp
#	core/libs/prx/libSceAgcDriver/Graphics/src/State.cpp
#	core/libs/prx/libSceAjm.native/src/Ajm.cpp
#	core/libs/prx/libSceVideoOut/tests/Flip.cpp
#	core/libs/tests/GuestAjm.cpp
#	docs/dev/debt/libSceAjm.native.md
#	docs/dev/debt/libSceAjmi.md
#	docs/dev/debt/libSceAudioOut.md
#	docs/dev/debt/libSceVideoOut.md
@ntamas94

Copy link
Copy Markdown
Author

Closing: the upstream twin boykopovar#9637 was merged into main on 2026-10-25 (ae7d59258e), so ps4/limbo-gpu can take the PS4 batch buffer change from there. This branch also picked up an accidental merge of main on 2026-10-29, which is why it now shows main's whole diff and fails conventions; nothing in it needs to land here.

AI-assisted: yes (Claude Code)

@ntamas94 ntamas94 closed this Oct 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.