Skip to content

fix(shader): report a truncated memory, image or export instruction as out of range - #2034

Merged
adriadam10 merged 1 commit into
boykopovar:mainfrom
TheAggie:fix/shader-truncated-decode-out-of-range
Oct 10, 2026
Merged

adriadam10 merged 1 commit into
boykopovar:mainfrom
TheAggie:fix/shader-truncated-decode-out-of-range

Conversation

@TheAggie

@TheAggie TheAggie commented Oct 9, 2026

Copy link
Copy Markdown
Contributor

What

ReadRawComputeShader (the fallback for compute programs that were never registered) reads a growing prefix of guest memory, 64, 128, 256 words and so on, and decodes it. When the decoder throws std::out_of_range, it reads more. The instruction-length check, the scalar decoder and the vector decoders throw std::out_of_range for an instruction cut off by the end of the span.

The memory decoders (SMEM, DS, MUBUF, MTBUF, FLAT, through requireTwoWords and the generic memory check), the image decoder (MIMG and its NSA payload) and the export decoder threw std::runtime_error instead. So when a two-word instruction of those families lay across a window boundary, the read failed instead of growing. Whether it happened depended on where the program's instructions fell. vkQuake (PPSA99010) terminated with truncated FLAT instruction in some runs and not others.

Those five throw sites now use std::out_of_range, like the rest of the decoder. The messages are unchanged. Nothing in the tree catches std::runtime_error around decoding; the one test that matches a truncation message (LdsCondxchg, truncated DS instruction) catches std::exception.

Test (agc_driver): a raw program of 63 s_nop, a two-word s_load_dword in words 63 and 64, and s_endpgm reads back as 66 words. It fails (truncated SMEM instruction) without the change.

Tested

Windows 11 Pro (26200), MinGW-w64 GCC 15.2.0 (WinLibs r7), Release, NVIDIA GeForce RTX 5070 Ti.

  • The full suite (488 tests) passes on this branch alone. It was run from the generated CTestTestfile.cmake commands because antivirus blocks ctest.exe on this machine.
  • tools/check_conventions.py --base origin/main passes.

Not tested on Linux or macOS.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed in TechnicalDebt
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • AI-assisted: yes

🤖 Generated with Claude Code

…s out of range

ReadRawComputeShader decodes a growing prefix of guest memory and reads
more when the decoder throws std::out_of_range, which the vector, scalar
and instruction-length checks throw for an instruction cut off by the end
of the span. The memory (SMEM, DS, MUBUF, MTBUF, FLAT), image and export
decoders threw std::runtime_error instead, so a two-word instruction
across a window boundary failed the read: vkQuake terminated on "truncated
FLAT instruction". These now throw std::out_of_range too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot added shader core/shader agc libSceAgc* labels Oct 9, 2026

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the complete current diff. Only truncated-span checks in the memory, image/NSA, and export decoders change exception type; the diagnostic strings and invalid-PC checks remain unchanged. The regression crosses the initial 64-word read window, and current Linux, Windows, conventions, and report checks are green. Approved.

@adriadam10

Copy link
Copy Markdown
Collaborator

Checked: build + ctest (libSceAgc, libSceAgcDriver) green locally; agc_driver fails with the three decoder files reverted and passes with them. Every remaining "truncated" throw in the decoders is now std::out_of_range, so ReadRawComputeShader grows its window for all of them. Merging.

@adriadam10
adriadam10 merged commit 56bfc4d into boykopovar:main Oct 10, 2026
6 checks passed
SleglJan pushed a commit to SleglJan/AnyPS5 that referenced this pull request Oct 10, 2026
…-decode-out-of-range

fix(shader): report a truncated memory, image or export instruction as out of range
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agc libSceAgc* shader core/shader

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants