Skip to content

fix(libScePlayGo): accept chunk ids when the dump has no chunk definitions - #3238

Merged
adriadam10 merged 2 commits into
boykopovar:mainfrom
Tamalero:fix/playgo-undeclared-chunks
Oct 11, 2026
Merged

adriadam10 merged 2 commits into
boykopovar:mainfrom
Tamalero:fix/playgo-undeclared-chunks

Conversation

@Tamalero

Copy link
Copy Markdown
Contributor

What

On PS5 dumps, scePlayGoGetLocus, scePlayGoGetEta, scePlayGoGetProgress, scePlayGoPrefetch and scePlayGoSetToDoList return SCE_PLAYGO_ERROR_BAD_CHUNK_ID (0x80B2000C) for every chunk except 0.

Cause: ValidChunks() reads only /app0/playgo-chunkdefs.xml. PS5 dumps don't have it: the package's chunk table isn't part of the dump, and sce_sys/playgo-scenario.json lists no chunks. So the set falls back to {0}.

Marvel's Guardians of the Galaxy (PPSA01748 01.006.000) asks for all of its chunks in one scePlayGoGetLocus call (ids 1–10 alongside chunk 10). If that call fails, it never marks the chunks as installed. It then puts them on the to-do list and waits forever on the main thread, with no window ever created. With this change it creates its window, mounts its save data and keeps loading.

Fix: a chunk table now records whether definitions were found.

  • No definitions: the table is unknown and a dump is fully installed, so any chunk id is accepted and reported as LOCAL_FAST.
  • playgo-chunkdefs.xml present: validation is unchanged.
  • scePlayGoGetChunkId still reports the known set ({0} without definitions).

Tested

Linux (CachyOS), GCC 16, Release, on current main.

  • guest_playgo passes.
    • The 0xFFFF to-do case now expects success (no definitions in the test).
    • New case: scePlayGoGetLocus({1, 10}) returns 0 with locus 3, and scePlayGoGetChunkId still counts 1 chunk.
  • Negative check: the updated test aborts on the current ValidChunks().
  • PPSA01748 under gdb, breakpoints around its scePlayGoGetLocus call:
    • before: count=2 ids: 1 10 returns 0x80b2000c, and the main thread waits on the chunk state;
    • after: it gets past the stall and runs until a later, unrelated stop.
  • tools/check_conventions.py passes.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed in TechnicalDebt
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • AI-assisted: yes

🤖 Generated with Claude Code

@github-actions github-actions Bot added the intake paused Closed while PR intake is limited to authors with at least 10 PRs merged into main label Oct 11, 2026
@github-actions

Copy link
Copy Markdown

Thank you for your interest in the project! We are sorry, but we cannot accept more PRs while we work through the existing backlog. This PR has been automatically closed and labeled "intake paused". This is not a judgment of the quality of your changes.

@github-actions github-actions Bot closed this Oct 11, 2026
@oneandonlydean oneandonlydean removed the intake paused Closed while PR intake is limited to authors with at least 10 PRs merged into main label Oct 11, 2026
@oneandonlydean

Copy link
Copy Markdown
Collaborator

Reopened.

Two things before merging:

  • core/libs/prx/libScePlayGo/Export.cpp:27: the comment still says unknown IDs must be rejected. Update it to the new rule.
  • Add a TechnicalDebt entry: without playgo-chunkdefs.xml every chunk id is reported as installed while scePlayGoGetChunkId still lists only chunk 0, and the console's answer for an id outside the package's chunk table is not checked.

@SleglJan

Copy link
Copy Markdown
Contributor

Linux check with 007 First Light (PPSA11386 v01.018.000), which stops on the same scePlayGoGetLocus BAD_CHUNK_ID (row 5 of #2523). Setup: RTX 3080 (NVIDIA 615.71.09), Release with SPIRV-Tools off, relinked and run in a bubblewrap sandbox, one run each with a 120 s limit:

For the TechnicalDebt entry requested above: three retail PS5 packages I looked at carry the chunk table as the package entry playgo-chunk.dat, which extracts into sce_sys/. They are 007, Until Dawn and The Last of Us Part II Remastered. The file has a plgx header and 100, 64 and 100 chunks respectively. If a stricter rule is wanted later, SleglJan#19 reads the count from that file. I won't open it upstream, since this PR covers the stop.

AI-assisted: yes (Claude Code)

@Mouhand-Kaddo

Copy link
Copy Markdown
Contributor

Additional platform validation of standalone PR head 7b50fa975e0ef4df77259fa9eddd530e258701bf (2026-10-11):

Windows: passed — Windows 11, WinLibs GCC 15.2.0 UCRT/POSIX/SEH r7; CMake 4.3.1 / Ninja; Release, BUILD_TESTING=ON.

1 checks, 0 failures, 0 CTest skips. Built the selected targets and dependencies; this is focused validation, not a full build or full-suite run.

Checks: guest_playgo.

Command after building: ctest --test-dir <build> --verbose --no-tests=error --timeout 30 -R "^guest_playgo$".

No title/gameplay validation. These results cover the standalone head, not a merge with current main.

@adriadam10

Copy link
Copy Markdown
Collaborator

Thanks. Added the two items @oneandonlydean asked for as a separate commit:

  • Export.cpp: the chunk-set comment now says IDs are rejected only when playgo-chunkdefs.xml exists, and accepted otherwise.
  • TechnicalDebt.md (Unknown function info): without the chunk definitions every chunk id is reported as installed while scePlayGoGetChunkId lists only chunk 0; the console's answer for ids outside the package's chunk table is unchecked.

Checked: guest_playgo passes and fails without the Export.cpp change; 95 tests of libs/PlayGo/libkernel pass. Merging once CI is green.

@adriadam10
adriadam10 force-pushed the fix/playgo-undeclared-chunks branch from 7b50fa9 to 7bc7619 Compare October 11, 2026 07:25
@github-actions github-actions Bot added needs rebase Conflicts with main, rebase needed docs labels Oct 11, 2026
Tamalero and others added 2 commits October 11, 2026 09:44
…tions

PS5 dumps carry no playgo-chunkdefs.xml (only sce_sys/playgo-scenario.json, which
lists no chunks), so the valid set fell back to chunk 0 and every query for another
chunk returned SCE_PLAYGO_ERROR_BAD_CHUNK_ID. A title that asks for all of its chunks
in one scePlayGoGetLocus call then never sees them as installed and waits forever.

Without definitions the chunk table is unknown and a dump is fully installed, so any
chunk id is accepted and reported as local. With playgo-chunkdefs.xml present,
validation is unchanged. scePlayGoGetChunkId still reports the known set.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… in TechnicalDebt

Co-Authored-By: Claude <noreply@anthropic.com>
@adriadam10

Copy link
Copy Markdown
Collaborator

Rebased onto current main: only conflict was the adjacent TechnicalDebt line (sceKernelGetAppInfo), kept both. guest_playgo and 15 tests of libs/PlayGo pass on the rebased tree; merging once CI is green.

@adriadam10
adriadam10 force-pushed the fix/playgo-undeclared-chunks branch from 7bc7619 to db3c429 Compare October 11, 2026 07:50
@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 11, 2026
@adriadam10
adriadam10 merged commit 98da5a9 into boykopovar:main Oct 11, 2026
12 checks passed
@xXPBSXx

xXPBSXx commented Oct 12, 2026

Copy link
Copy Markdown
Contributor

Data point on this change, from Sifu (PPSA03001, details on #3209): with this merge Sifu stops at start-up (guest protection or unmap range has a hole, later Unreal's Ran out of memory allocating 5565054976 bytes), and with the old chunk check restored locally (unknown chunk IDs rejected unless playgo-chunkdefs.xml lists them) it runs. Which behaviour is right for a dump without chunk definitions is not known to me; this only records that accepting every ID changes Sifu's start-up path.

AI-assisted (Claude Code).

@xXPBSXx

xXPBSXx commented Oct 12, 2026

Copy link
Copy Markdown
Contributor

Mechanism for the Sifu stop (details on #3209), measured with a local trace on main a11c8f7 (Windows, Sifu PPSA03001, 60 s runs, the trace prints every ValidateChunks call with its ids and result):

  • Old check (ids other than 0 rejected when there is no playgo-chunkdefs.xml): the title asks about chunk 0 (n=1, result 0), then chunk 1 (n=1, result 0x80b2000c) and stops asking. Two calls in all; Sifu then runs on (about 250 stdout lines, textures and shaders).
  • New check (every id accepted): the same title asks one id at a time in ascending order (n=1, ids 0, 1, 2, ...), every call succeeds, and it kept going: 4,023 consecutive ids in a 60 s run with the trace slowing it. So it enumerates chunk ids until a call fails, and accepting every id means it never gets the failure. (Which of scePlayGoGetLocus, GetEta, GetProgress or Prefetch it calls is not distinguished by my trace; all go through this check.)
  • Where the 5,565,054,976 bytes come from, an inference: 5565054976 = 65536 * 84916 exactly, i.e. one record of 84,916 bytes for each of the 65,536 possible 16-bit chunk ids, which is what an enumeration that never hits an error would size its table for. The Ran out of memory allocating 5565054976 bytes message of Sifu and Demon Slayer (Demon Slayer -Kimetsu no Yaiba- The Hinokami Chronicles (PPSA03490 v01.000.000) on Windows: start-up needs seven sceRudp* imports that no library exports; with them it runs until the packed pixel ancillary error (#2548) #3073) has that exact size; I did not trace the allocation itself.

A narrower condition that I think satisfies both titles, as a suggestion for discussion, not a tested change: the title of this PR (Guardians of the Galaxy) has a PlayGo package without a visible chunk table (its sce_sys/playgo-scenario.json exists but lists no chunks, per the description). My Sifu and Demon Slayer dumps have no playgo-* file at all in sce_sys (Sifu: about, trophy2, param.json, and so on; no scenario, no chunk, no hash-table file), which suggests packages without PlayGo data. Accepting unknown ids only when sce_sys/playgo-scenario.json (or playgo-chunk.dat) exists, and keeping {0} otherwise, would keep Guardians working and Sifu's enumeration ending at chunk 1. I have not checked that Guardians' dump has that file (I do not have the title), and I did not test the condition with Sifu.

@Tamalero, @adriadam10: would that file test fit your findings for Guardians? I am not opening a PR for this unless you want one.

AI-assisted (Claude Code).

@xXPBSXx

xXPBSXx commented Oct 12, 2026

Copy link
Copy Markdown
Contributor

Follow-up trace for Sifu (PPSA03001) after this change: with an undefined chunk table the title enumerates PlayGo ids one per call and never stops while every id is accepted (about 36-39 million calls in 100 s, ids wrapping at 65536). Details and limits (the logging build did not reproduce the OOM, so no count limit was tested): #3209 (comment)

AI-assisted (Claude Code).

@Tamalero

Copy link
Copy Markdown
Contributor Author

@xXPBSXx Checked against our dumps:

  • Guardians of the Galaxy (PPSA01748): sce_sys/playgo-scenario.json is present (1 scenario, id 0, type playmode, 10 languages, no chunk list). There is no playgo-chunk.dat and no playgo-chunkdefs.xml. So your file test would keep Guardians on the accept-unknown path it needs.
  • High on Life (PPSA15388, Unreal Engine 4): no playgo-* file at all in sce_sys. It reaches gameplay with the old rule (ids other than 0 rejected) on an older base, and with the current accept-all rule on main, so it works either way.

So the scenario-file condition fits both of our titles. I have not tested it as a change.

Tested and written by Claude (AI assistant) on a contributor's machine.

@xXPBSXx

xXPBSXx commented Oct 12, 2026

Copy link
Copy Markdown
Contributor

A finding about this change from Sifu (PPSA03001), whose dump has no playgo-chunk.dat and no playgo-chunkdefs.xml: with every id accepted, the title spins on scePlayGoGetLocus with a single id per call and an ever increasing id (it wraps at 65536) and never gets an error. A temporary counter in ValidateChunks shows at least 250 million calls in 100 s, after which the title runs out of memory (its own allocator: a 16-byte element array growing to about 348 million entries, Ran out of memory allocating 5565054976 bytes) and traps in libc.prx. A local test that accepts only ids below 256 when no table exists ran 100 s without that failure in 8 of 8 runs (library hash checked); without it the failure occurred in 9 of 9. Details and limits: #3209 (comment)

Question to those who know the Guardians of the Galaxy case (ids 1 to 10 in one scePlayGoGetLocus call): which bound for the no-table case would you prefer? Options I see: a fixed upper limit (256 is my assumption, not a console value; the retail packages mentioned above have 64 to 100 chunks); reading the count from playgo-chunk.dat when it exists (does not help dumps without it, like Sifu); or accepting ids only up to the highest id requested so far plus a margin. I have not opened a PR; the local branch changes one line plus the guest_playgo case that now expects 0xFFFF to be accepted.

AI-assisted (Claude Code).

@xXPBSXx

xXPBSXx commented Oct 12, 2026

Copy link
Copy Markdown
Contributor

The change is now open as #4491 (ids below 256 accepted without a chunk table; Sifu, Demon Slayer and Returnal numbers in its description). The chunk count read from playgo-chunk.dat (#3435) and the Guardians case of #3238 (ids 1 to 10) are unchanged.

AI-assisted (Claude Code).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants