Repository navigation
feat(libSceIcu): implement u_isspace, u_isblank, u_isWhitespace, u_isJavaSpaceChar and u_isUWhiteSpace - #12630
Conversation
…JavaSpaceChar and u_isUWhiteSpace Replace the five Unknown stubs whose NIDs hash these ICU names with the uchar.cpp definitions of ICU 78.1 (049e0d6a), over the Zs, Zl, Zp and White_Space code points that ICU 65.1 to 78.1 ship. Values outside U+0000..U+10FFFF return 0, as ICU's trie error value gives. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Progress report (3a2b919 → f70f9e3)📈 System libraries: 45.94% (+0.03%, +5 implemented, -5 removed functions) ✅ 5 implemented
🗑️ 5 removed
Changed libraries (implemented exports)
|
|
Other open pull requests touch the same code. Merging one will break the other, or both implement the same thing. If this one builds on them, list them in Depends on.
Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.
|
|
Windows run of this head (
AI-assisted: yes (Claude Code). |
|
@Lindamea1 reviewed at f70f9e3: PASS. The five ICU space predicates are written as Unicode character-set tables and match ICU's uchar.cpp definitions (current Zs set, U+180E excluded); five stubs are removed and the debt count goes 107 -> 102. The new @adriadam10 exact-head CI is green (run 40790128160); we'll merge this in the next train unless you'd rather merge it yourself. |
What
On
main, libSceIcu exports these five ICU character property functions as Unknown stubs that throw. Their NIDs arecompute_nid(tools/nid_names.py) of the plain ICU names, so each_nid_postfixdefinition replaces its stub (Export.cppL319-L322, L344-L352, L369-L372, L409-L412) under the same export NID:u_isspacetcSjvPP2-4su_isblanktwrAIl4N+EM(uint32_t)c <= 0x9F, otherwise Zs (18)u_isWhitespaceO3zT1gO1ZNMu_isJavaSpaceCharmY5RbYVLzr4u_isUWhiteSpaceU089pStD8BwZs is U+0020, 00A0, 1680, 2000..200A, 202F, 205F and 3000; Zl is U+2028 and Zp is U+2029. White_Space is U+0009..000D, 0020, 0085, 00A0, 1680, 2000..200A, 2028, 2029, 202F, 205F and 3000. Each function takes a
UChar32(std::int32_t) and returns aUBool(std::int8_t) of 1 or 0. Every otherint32_treturns 0, including negative values, surrogates and values above U+10FFFF. The functions keep no state, have no error paths and use no host ICU; Linux and Windows run the same code.Reference, ICU 78.1 at 049e0d6a (firmware 13.42 reports ICU 78.1, #10961):
uchar.cppL196-L244: the five definitions withIS_THAT_CONTROL_SPACEandIS_THAT_ASCII_CONTROL_SPACE.u_isUWhiteSpacetests bitUPROPS_WHITE_SPACE(uprops.hL185-L186) ofu_getUnicodeProperties(c, 1).uprops.hL381-L393:TAB,CR,NBSP,FIGURESPandNNBSP.umachine.hL269:typedef int8_t UBool.utrie2.hL844-L855 reads the trie's error slot when(uint32_t)c > 0x10FFFF.propsTrie(uchar_props_data.hL1524-L1537) andpropsVectorsTrie(L4238-L4251) have initial and error value 0: category Cn and no White_Space bit.UnicodeData.txt(Zs, Zl and Zp rows 33, 161, 5193, 7392-7402, 7432, 7433, 7439, 7487 and 11279) andppucd.txt(WSpacerows 1364-1368, 1388, 1504, 1532, 7363, 9911-9921, 9957, 9958, 9966, 10026 and 14348) that ICU 78.1 ships.uchar.cppL196-L245 atrelease-65-1, fd123bf0) and 68.2 (uchar.cppL196-L245 atrelease-68-2, 84e1f26e) differ from 78.1 only by(UBool)casts and by writing 0x85 asNL, andUBoolisint8_tthere too. The releases between them and up to 77.1 differ from 78.1 at most in the same way: the casts remain until 75.1 andNLuntil 68.2.Space.cppkeeps Zs and White_Space as fixed range tables, and Zl and Zp as the code points U+2028 and U+2029, instead of ICU's trie data. To check this against every ICU release from the 65 to 68 range that #11115 gives for the 3.20 module up to the 78.1 that 13.42 reports (17 releases, listed below), I emulated ICU's code paths for the five functions over the generated tries in each release'suchar_props_data.h. The emulation covers every code point, -0x10000..-1 and 0x110000..0x11FFFF. Every release gives exactly the sets above (29, 18, 25, 19 and 25 code points) and 0 for every out-of-range value: both tries' error slots hold 0, which means category Cn andpropsVectorsrow 0, whose White_Space bit is clear. The emulation follows the 78.1 code, and each release matches it: apart from the casts andNL, the five definitions are the same, and so areGET_PROPS,u_getUnicodeProperties, the UTrie2 lookup macros and constants inutrie2.h, andUPROPS_WHITE_SPACE(bit 0).UnicodeData.txtandppucd.txtlist the same Zs, Zl, Zp andWSpacecode points in every release. The ICU version of the 3.20 module itself is not measured.ICU releases checked
Firmware availability: BunnyFl's full 13.42 export list has
u_isUWhiteSpaceamong the 104 exports it shares with the 3.20 table.u_isspace,u_isblank,u_isWhitespaceandu_isJavaSpaceCharare not among its 299 exports, by name or NID, so these four are 3.20-inventory exports kept for older firmware, like those of #10961 and #11013.Changes:
core/libs/prx/libSceIcu/Space.cppandSpace.hpp(included fromlibSceIcu.hpp) add the five exports;Export.cppdrops their five stubs.core/libs/prx/libSceIcu/CMakeLists.txtbuildsSpace.cppand registers the newguest_icu_spacetest (tests/Space.cpp).docs/dev/debt/libSceIcu.md: a new Functional entry (unverified on a console, ICU 78.1 with fixed sets, 13.42 availability) and the Unknown count from 107 to 102.tools/progress.py --module libSceIcugoes fromlibraries 11/120 (9.17%)tolibraries 16/120 (13.33%).Overlap: no open PR mentions any of the five names or NIDs (
gh pr list --searchfor each, and agit log -Gsearch of the 5569 commits that the 1366 open PR heads fetched at about 20:30Z add tomain, which does find #12320'su_isISOControl). Agit merge-treeof this branch with each of the 14 open PR heads that change libSceIcu files merges cleanly with #10962 to #10967 and #12451. With #12320 only the Unknown count sentence indocs/dev/debt/libSceIcu.mdconflicts, because #12320 moves it and sets it to 92; whichever lands second recounts (with both,Export.cppkeeps 82 Unknown stubs). #11088, #11115, #11118, #11122, #11132 and #11136 conflict inCMakeLists.txtand the debt file in the same hunks as withmain; this branch's side of the debt hunk differs frommain's only in the Unknown count. At 20:49Z those 14 heads were still current, and none of the 9 PRs opened after the fetch (#12562 to #12570) changes a libSceIcu file.Tested
Linux (Gentoo, kernel 7.2.8, AMD Ryzen 7 9800X3D), GCC 16.2.1, Ninja, Release build with
BUILD_TESTING=ONof this branch onmain319621b.ctest --test-dir build -R '^(guest_icu_space|guest_icu_cstring|guest_icu_version|guest_icu_unescape|public_headers)$' --output-on-failure -j8 --timeout 300: 5/5 passed. A direct run ofguest_icu_space_teststakes 0.019 s.python3 tools/check_test_registration.py build: passed (1516 test executables, none built and not run).tools/select_tests.py dump,selectwith the 7 changed files,ninja -f select.ninja selected,select_tests.py ctest): 490 of 2746 tests selected, 490/490 passed.python3 tools/check_conventions.py --base 319621ba2, also with this title and description: passed.guest_icu_spacetest (core/libs/prx/libSceIcu/tests/Space.cpp) calls each function for every value from -0x1000 to 0x110FFF, forINT32_MIN, -1, 0x110000 andINT32_MAX, and for each true code point plus 0x110000, 0x200000, 0x1000000, 0x80000000 and 0xFFFF0000 (modulo 2^32). It requires exactly 1 for the code points in a literal list written per function from the ICU 78.1 data, and exactly 0 for every other value. A script checked the five lists againstUnicodeData.txtandppucd.txtwith theuchar.cppformulas.Space.cpp, which passes, and with 11 mutated copies, each of which aborted:u_isWhitespacekeeping U+202F,u_isspacewithout U+0085 or returning 2,u_isblankwithout TAB or without its<= 0x9Fbranch,u_isUWhiteSpaceequal tou_isspace, with U+180E in place of U+1680 or with a 21-bit mask,u_isJavaSpaceChartruncating to 16 bits, Zs without U+205F, and the separators without U+2029.build/core/libs/libs/libSceIcu.prx:dlsymresolves the five NIDs, and their results for U+0009, 001C, 0020, 0085, 00A0, 2007, 2028, 0041, -1 and 0x110000 match the table above. NosceIcuUnknown_symbol for these NIDs is left in the patched or unpatched library.x86_64-w64-mingw32-g++16.2.0,-std=gnu++20 -fsyntax-only -Wall -Wextra -D_WIN32_WINNT=0x0602) onSpace.cpp,Export.cppandtests/Space.cpp. This is not a Windows build or run.Checklist
main; no other open PR implements the same functionsNotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file.mdfiles or images added to the repository (attach them to this PR)🤖 Generated with Claude Code