Skip to content

feat(libSceIme): open a panel-less IME fed by the host keyboard - #4756

Merged
adriadam10 merged 1 commit into
boykopovar:mainfrom
p3n1cillin:claude/ime-host-keyboard
Oct 14, 2026
Merged

adriadam10 merged 1 commit into
boykopovar:mainfrom
p3n1cillin:claude/ime-host-keyboard

Conversation

@p3n1cillin

Copy link
Copy Markdown

What

sceImeOpen threw, so Minecraft Preview (PPSA19639) stopped when entering a world (#1277; the bring-up post worked around it locally by returning an error). This PR implements the IME as a session without a panel, with the text typed on the host keyboard:

  • libSceIme: sceImeOpen validates the parameters, takes the initial text from inputTextBuffer, places the caret after it and queues the Open event (position and sceImeGetPanelSize size). Edits keep inputTextBuffer and a copy in work in sync and queue UpdateText (one edit area: position and change in UTF-16 units) or UpdateCaret (one per UTF-16 unit). Enter sends PressEnter (or inserts a line break with the multiline option); Escape sends PressClose. sceImeUpdate(handler) delivers the queue with the arg from sceImeOpen when given the handler passed to it, outside the lock, so a handler may call sceImeClose. sceImeSetText, sceImeSetCaret, sceImeSetTextGeometry, sceImeGetPanelPositionAndForm and sceImeClose work while a panel is open. Typed text is cut to maxTextLength without splitting a surrogate pair.
  • libSceVideoOut: new ImeTextInput forwards SDL text input (already enabled by SDL on desktop, so host keyboard layouts and host IMEs work) and Backspace, Delete, Left, Right, Home, End, Enter and Escape from the game window. While a panel is open, key presses no longer reach the keyboard-to-pad bindings, as the console's panel takes the controller; releases still do, so no button sticks.
  • SceTypes: EventHandler gets APS5_VABI, since it is a guest function now called from host code.

Event ids, caret directions, payload layout and error codes follow the PS4 reimplementation in shadPS4 ime.cpp; its SDK-version-dependent masks are not applied. A text filter or external keyboard filter throws (contract unknown). The panel-less behaviour is listed under Silent stubs and every assumption is recorded in docs/dev/debt/libSceIme.md.

New guest_ime_open covers every sceImeOpen error, the Open event and handler arg, delivery only to the opening handler, insertion with UTF-8 to UTF-16 conversion and surrogate pairs, the length limit, Backspace/Delete/caret moves at the edges, SetText/SetCaret, Enter and Escape, the multiline option, a close from inside the handler dropping the remaining events, and the SDL event translation (window filter, key repeats, key-up). Removing the surrogate guard, offsetting the Backspace edit index or dropping the handler check each make it fail.

Tested

Windows 11, MinGW-w64 GCC 15.2.0 (WinLibs r7), BUILD_TESTING=ON: guest_ime_open, guest_ime_parameters and guest_ime_keyboard pass; libSceVideoOut builds. python tools/check_conventions.py --base origin/main and tools/check_test_registration.py pass. Not tested in Minecraft (no copy here) and the pad suppression is not covered by a test, since PadInput needs an SDL window.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Unblocks: PPSA19639 Minecraft Preview (PPSA19639) #1277
  • AI-assisted: yes

🤖 Generated with Claude Code

@p3n1cillin
p3n1cillin force-pushed the claude/ime-host-keyboard branch from 3055e55 to f6f4914 Compare October 12, 2026 21:37
@xXPBSXx

xXPBSXx commented Oct 13, 2026

Copy link
Copy Markdown
Contributor

Windows test: head 3055e553b merged onto current main 90c683946 (clean merge, no conflicts). Windows 10, MinGW GCC (WinLibs), Ninja Release, -j2: full build 0 errors; ctest with the filter guest_ime_open: guest_ime_open passed (1 test(s), 0 failed); python tools/check_conventions.py --base upstream/main passed. I did not run the new tests against main without the change.

AI-assisted (Claude Code).

@oneandonlydean

Copy link
Copy Markdown
Collaborator

This needs a run of PPSA19639 on this head: does entering a world get past sceImeOpen, and does the title take the typed text? The event ids and payloads come from a PS4 reimplementation, and the description says it wasn't tried in the title.

The Windows test above ran 3055e553b, not the current head, and nobody has built it on Linux yet.

@p3n1cillin

Copy link
Copy Markdown
Author

I don't have PPSA19639 to run, so I can't show it here. @ddrsilva, you reported the sceImeOpen stop when entering a world in #1277: could you try this head with your Minecraft Preview build and say whether it gets past sceImeOpen and takes typed text? Until someone can, it can stay open or be closed, whichever you prefer.

@simonbbby

Copy link
Copy Markdown
Contributor

Linux verification of exact PR head f6f49148f2bb32438a79ee3b0e9eccc4c53142b5, Ubuntu 24.04 / GCC 13.3 / Ninja Release, BUILD_TESTING=ON, ANYPS5_ENABLE_SPIRV_TOOLS=ON:

  • Built guest_ime_open_tests, guest_ime_parameters_tests, guest_ime_keyboard_tests, and libSceVideoOut successfully.
  • ctest -R '^guest_ime_(open|parameters|keyboard)$' --repeat until-fail:3 --timeout 30 --no-tests=error: all three tests passed three consecutive runs each, no skips or timeouts.
  • An independent temporary harness also checked the actual PadInput implementation with default bindings: Space drives Cross before opening IME; a held Space releases while IME is open; new Space/A presses are suppressed during IME; Cross and left-stick input resume after closing it. All checks passed. The harness sends constructed SDL events directly to PadInput::HandleEvent and captures PadPublishInput_nid_postfix; this is a component test, without an interactive window.
  • Negative control: rebuilt the same harness against the parent revision's PadInput.cpp, retaining the PR's IME library. It fails at the assertion that a new Space press is suppressed during IME. Rebuilding against the PR's PadInput.cpp passes.

This covers the missing Linux build/test evidence and the keyboard-to-pad capture logic. I have not run PPSA19639 or verified its acceptance of the event IDs/payloads; that title check remains outstanding.

@adriadam10

Copy link
Copy Markdown
Collaborator

Checked the build and ctest for libSceIme/libSceVideoOut on the merge with main (guest_ime_open passes, 46/46 in scope), the error order and event layout against the cited shadPS4 revision, and the third-party Linux/Windows runs above (including the PadInput suppression harness with its negative control). sceImeOpen threw before, so only titles that call it change behaviour. Not run on PPSA19639; merging once the Windows build is green.

@ddrsilva

Copy link
Copy Markdown

Tested the IME implementation from head f6f49148f2bb32438a79ee3b0e9eccc4c53142b5 with Minecraft Preview 1.21.40.22 (PPSA19639) on Windows 11, MinGW-w64 GCC 15.2, RTX 2080 Ti.

The functional checks passed in my isolated integrated Minecraft build:

  • Opened the copied world's name field, typed IME4756 with the physical keyboard, and pressed Enter. The title displayed and accepted the text. Both the world's levelname.txt and its level-info cache persisted IME4756.
  • Loaded that renamed world into actual gameplay. Terrain and HUD rendered, and no sceImeOpen stop or fatal exception occurred during this run.
  • GuestImeOpen, GuestImeParameters, and GuestImeKeyboard from this PR all passed.

Scope: this was not a build/run of the entire unmodified PR head. I used the PR's libSceIme/Export.cpp, ImeInput.hpp, and both ImeTextInput files byte-for-byte, and adapted its VideoOut event-loop and pad-suppression additions to my existing working Minecraft integration. That integration includes separate boot/rendering/performance changes; its base is 2a6910ac (upstream b5f5362d plus local work). The EventHandler guest ABI was already equivalent there. The existing backend/other modules were retained, and native-input mode was disabled for this test. This is title-specific functional evidence for the IME implementation, not independent validation of every change on current upstream, all text fields, other titles, or an FPS improvement.

@p3n1cillin

Copy link
Copy Markdown
Author

Since the Windows CI job only runs on push or manual dispatch, here is a local Windows run: Windows 11, MinGW-w64 GCC 15 (winlibs), Ninja, Release. I built head f6f49148f merged into current main (a0f5916a2); it merges cleanly and the full tree builds. ctest -R "ime_|video|pad_input|mouse": 28/28 pass, with 6 video_out_flip_*/wqhd_no_vulkan tests skipping themselves because ctest runs SDL offscreen (no Vulkan surface). guest_ime_open, guest_ime_parameters, guest_ime_keyboard and pad_input pass.

@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 14, 2026
@oneandonlydean

Copy link
Copy Markdown
Collaborator

#4704 moved the keyboard and mouse handling in PadInput::HandleEvent into the new keyboard/mouse handler, so this conflicts with main now. Could you rebase and move the ImeCapturesKeyboard keydown check into that handler? It's ready to merge after that.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@p3n1cillin
p3n1cillin force-pushed the claude/ime-host-keyboard branch from f6f4914 to 435be99 Compare October 14, 2026 07:56
@p3n1cillin

Copy link
Copy Markdown
Author

Rebased onto main 1209d3707 (head 435be9905). Since 38bc0fd, key and mouse handling lives in Pad::KeyboardMouse::HandleEvent. So this PR's one PadInput.cpp change, not mapping a key-down to the pad while the IME takes the keyboard, now sits just before that call: if (event.type == SDL_KEYDOWN && ImeCapturesKeyboard_nid_postfix()) return;. Key-ups still reach it, so held keys are released. Nothing else changed. On Windows 11 (MinGW GCC 15, Release) the full tree builds, ctest -R "ime_|video|pad_input|mouse|keyboard" passes 30/30 (including guest_ime_open, guest_ime_parameters, guest_ime_keyboard and pad_input), and the conventions check passes. Now shows as mergeable.

@github-actions github-actions Bot removed the needs rebase Conflicts with main, rebase needed label Oct 14, 2026
@adriadam10

Copy link
Copy Markdown
Collaborator

Checked the build and ctest for libSceIme/libSceVideoOut on the merge with main (guest_ime_open passes, 52/52 in scope), the PadInput change after the rebase onto #4704, the error order and event layout against the cited shadPS4 revision, and the debt entries. Title run by @ddrsilva on PPSA19639 and the Linux/Windows runs above cover the rest; merging once the Windows build is green.

@adriadam10
adriadam10 merged commit 1594e5a into boykopovar:main Oct 14, 2026
8 checks passed
adriadam10 pushed a commit that referenced this pull request Oct 19, 2026
sceImeDialogInit wrote "Slayer" into an empty text buffer and the
dialog finished on the second status poll, so a title never got text
from the user: ProsperoRadio's search typed "Slayer" by itself and
found no station.

The dialog now runs until the user answers on the host keyboard, the
way #4756 made sceImeOpen's panel-less IME take it. While no IME panel
is open, libSceIme hands the game window's text and editing keys to
a sink that the dialog registers. Typing edits the text up to
maxTextLength without splitting a surrogate pair, Enter writes it to
the title's buffer and finishes with end status 0, and Escape finishes
with 1. Abort still finishes with 2. A second Init reports busy, and a
null parameter or buffer or a maximum length outside 1 to 2048 is
refused, as in shadPS4.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants