Repository navigation
Conversation
Progress report (b475792 → b394b9b)📈 System libraries: 46.8% (+0.14%, +20 implemented, -20 removed functions) ✅ 20 implemented
🗑️ 20 removed
Changed libraries (implemented exports)
|
|
Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.
|
12 tasks done
This was referenced Oct 28, 2026
Convert a UTF-32 string to UTF-16 following ICU 78.1 ustrtrns.cpp. Code points outside U+0000..U+10FFFF and surrogate code points are replaced by the substitution character, or fail with U_INVALID_CHAR_FOUND when it is U_SENTINEL. The destination length is the full preflight length, the terminator or the U_STRING_NOT_TERMINATED_WARNING and U_BUFFER_OVERFLOW_ERROR codes follow u_terminateUChars, and a supplementary code point that does not fit is counted but not split. The two sceIcuUnknown stubs with these NIDs are removed. A null UErrorCode throws where ICU dereferences it.
Host vectors with the UErrorCode values derived from ICU 78.1 ustrtrns.cpp: supplementary and boundary code points, NUL-terminated and counted sources, exact fit (U_STRING_NOT_TERMINATED_WARNING), overflow preflight lengths including the case where a supplementary code point does not fit but a later BMP code point does, surrogate and out-of-range replacement with BMP and supplementary substitution characters, U_INVALID_CHAR_FOUND with the sentinel, argument validation (U_ILLEGAL_ARGUMENT_ERROR), incoming failure and warning codes, and the throw on a null status.
Add the ICU 78.1 ustrtrns.cpp source, the error codes followed and the unverified firmware presence of u_strFromUTF32 and u_strFromUTF32WithSub, and update the remaining stub count.
Convert a UTF-16 string to UTF-32 following ICU 78.1 ustrtrns.cpp. A lead surrogate followed by a trail surrogate becomes one supplementary code point; any other surrogate is replaced by the substitution character, or fails with U_INVALID_CHAR_FOUND when it is U_SENTINEL. A supplementary substitution character is written as one code point. The preflight length, terminator and warning or overflow codes follow u_terminateUChar32s. The two sceIcuUnknown stubs with these NIDs are removed.
Host vectors derived from ICU 78.1 ustrtrns.cpp: surrogate pairs and the BMP boundaries, NUL-terminated and counted sources with an embedded NUL, unpaired lead and trail surrogates and reversed pairs replaced by BMP and supplementary substitution characters, U_INVALID_CHAR_FOUND with the sentinel, exact fit and overflow preflight lengths, argument validation, incoming failure and warning codes, and the throw on a null status.
Add the ICU 78.1 ustrtrns.cpp source and the unverified firmware presence of u_strToUTF32 and u_strToUTF32WithSub, and update the remaining stub count.
Convert Java modified UTF-8 to UTF-16 following ICU 78.1 ustrtrns.cpp. One to three byte sequences are decoded without the overlong and surrogate checks of standard UTF-8, so C0 80 is U+0000 and encoded surrogates pass through; four byte sequences and truncated or malformed ones are replaced by the substitution character, or fail with U_INVALID_CHAR_FOUND when it is U_SENTINEL. The bytes a malformed sequence swallows follow ICU's utf8_nextCharSafeBody. As in ICU, once the destination is full each substituted sequence adds one unit to the preflight length even when the substitution character is supplementary. The sceIcuUnknown stub with this NID is removed.
Host vectors derived from ICU 78.1 ustrtrns.cpp and utf_impl.cpp: one to three byte sequences including C0 80, overlong forms and encoded surrogates, four byte sequences replaced as one unit, the bytes consumed by truncated or ill-formed sequences, the sentinel failing with U_INVALID_CHAR_FOUND before and after the destination fills, exact fit and overflow preflight lengths including the supplementary substitution count quirk, argument validation, incoming failure and warning codes, and the throw on a null status.
Add the ICU 78.1 ustrtrns.cpp and utf_impl.cpp sources, the preflight quirk kept from ICU and the unverified firmware presence of u_strFromJavaModifiedUTF8WithSub.
Convert UTF-16 to Java modified UTF-8 following ICU 78.1 ustrtrns.cpp. Every code unit is encoded on its own, so surrogates become three bytes each and U+0000 becomes C0 80. Output stops at the first unit that does not fit and the remaining units are only counted for the preflight length; the terminator and warning or overflow codes follow u_terminateChars. The sceIcuUnknown stub with this NID is removed.
Host vectors derived from ICU 78.1 ustrtrns.cpp: one, two and three byte boundaries, U+0000 as C0 80, surrogates encoded individually, output that stops at the first unit that does not fit while later smaller units are only counted, exact fit and overflow preflight lengths, argument validation, incoming failure and warning codes, the throw on a null status and a round trip of all 65536 code units through u_strFromJavaModifiedUTF8WithSub.
Add the ICU 78.1 ustrtrns.cpp source and the unverified firmware presence of u_strToJavaModifiedUTF8.
Compare two UTF-16 strings in code unit or code point order following ICU 78.1 ustring.cpp. Two negative lengths compare as NUL-terminated strings and return the difference of the first differing units; otherwise the strings are compared over the shorter length and equal prefixes order by length as -1, 0 or 1. In code point order a surrogate that is part of a pair keeps its value and any other unit at or above U+D800 is lowered by 0x2800 so supplementary characters sort after the BMP. A null pointer or a length below -1 returns 0. The sceIcuUnknown stub with this NID is removed.
Host vectors derived from ICU 78.1 ustring.cpp for NUL-terminated and counted comparisons: the difference returned at the first mismatch, the -1, 0, 1 length ordering of equal prefixes, mixed length modes, embedded NUL units, code point order against code unit order for BMP units above the surrogate range, paired, lone lead and lone trail surrogates (with a pair split by the length limit), and null or negative-length arguments returning 0.
Add the ICU 78.1 ustring.cpp source and the unverified firmware presence of u_strCompare, and note that u_strCompareIter stays unimplemented.
Find the first or last occurrence of a NUL-terminated UTF-16 substring in a NUL-terminated string following ICU 78.1 ustring.cpp (u_strFindFirst and u_strFindLast). A match is rejected when it starts on a trail surrogate that follows a lead surrogate or ends on a lead surrogate that is followed by a trail surrogate, so it never splits a pair. A null substring or an empty one returns the string, and a null string with a substring returns null. The two sceIcuUnknown stubs with these NIDs are removed.
Host vectors derived from ICU 78.1 ustring.cpp: first and last matches, overlapping candidates, substrings longer than the string, null and empty arguments, an embedded NUL ending the search, and matches rejected or accepted at surrogate pair boundaries (a lone trail after a lead, a lead before a trail, matches that start or end inside a pair and later matches that do not).
Add the ICU 78.1 ustring.cpp source and the unverified firmware presence of u_strstr and u_strrstr.
u_isISOControl returns true for U+0000..U+001F and U+007F..U+009F and false for every other value including negative ones, as in ICU 78.1 uchar.cpp. u_forDigit maps a digit below the radix to a lowercase ASCII digit or letter and returns 0 for a radix outside 2..36 or a digit that is negative or not below the radix. The two sceIcuUnknown stubs with these NIDs are removed.
Check u_isISOControl over -4096..0x110010 and the int32 limits against the two control ranges, and u_forDigit over every int8 radix and digits -3..40 against the digit and lowercase letter mapping, plus the boundary radices 2 and 36, the out-of-range radices 0, 1 and 37, and the int32 limits.
Add the ICU 78.1 uchar.cpp source and the unverified firmware presence of u_isISOControl and u_forDigit.
…unctions Both functions are deprecated in ICU 78.1 and only report that user supplied locking is unsupported: when the status is not a failure it becomes U_UNSUPPORTED_ERROR and the callbacks are ignored (umutex.cpp). A status that already is a failure is left unchanged. A null UErrorCode throws where ICU dereferences it. The two sceIcuUnknown stubs with these NIDs are removed.
…ions Host checks derived from ICU 78.1 umutex.cpp: every non-failure status (zero and the warning range) becomes U_UNSUPPORTED_ERROR regardless of the callbacks, every failure status is left unchanged, and a null status throws.
Add the ICU 78.1 unames.cpp source and the unverified firmware presence of u_getISOComment.
Keep the data directory as a lock-protected copy, following ICU 78.1 putil.cpp: a null or empty path stores the empty string, any other path is copied so later changes to the caller's buffer do not show, and the getter returns the stored string, which is empty until a directory is set. The ICU_DATA environment variable and compile-time defaults are not consulted. The returned pointer stays valid until the next set, as in ICU. The two sceIcuUnknown stubs with these NIDs are removed.
Host checks derived from ICU 78.1 putil.cpp: the empty default, set and get, null and empty paths resetting the directory, the path being copied, a stable pointer between calls, backslashes kept as they are, a long path, and concurrent setters from several threads ending on one of their values.
Add the ICU 78.1 putil.cpp source, the unmeasured default directory and the unverified firmware presence of u_setDataDirectory and u_getDataDirectory.
…oneFilesDirectory Keep the time zone files directory as a lock-protected string following ICU 78.1 putil.cpp. Setting it with a failure status changes nothing; otherwise the previous value is cleared first, so a null path leaves the empty string and sets U_ILLEGAL_ARGUMENT_ERROR, and any other path is copied. The getter returns the empty string for a failure status and the stored value otherwise; it is empty until set because the ICU_TIMEZONE_FILES_DIR variable and build-time default are not consulted. A null UErrorCode throws where ICU dereferences it. The two sceIcuUnknown stubs with these NIDs are removed.
…ilesDirectory Host checks derived from ICU 78.1 putil.cpp: the empty default, set and get, failure statuses returning the empty string and leaving the stored value alone, warning statuses treated as success, a null path clearing the value with U_ILLEGAL_ARGUMENT_ERROR, empty paths, the path being copied, independence from the data directory, concurrent setters, and the throw on a null status.
Add the ICU 78.1 putil.cpp source, the unmeasured default and the unverified firmware presence of u_setTimeZoneFilesDirectory and u_getTimeZoneFilesDirectory.
Release the process-wide state this module keeps, following ICU 78.1 ucln_cmn.cpp: the functions stored by u_setMemoryFunctions are cleared and the data and time zone files directories return to their empty defaults. Calling it repeatedly or with nothing set is harmless, and the module can be configured again afterwards. The sceIcuUnknown stub with this NID is removed.
Host checks derived from ICU 78.1 ucln_cmn.cpp: the data and time zone files directories return to empty after a cleanup, repeated cleanups are harmless, the module can be configured again afterwards, and cleanups racing with setters from other threads end in the empty state.
Add the ICU 78.1 ucln_cmn.cpp source and the unverified firmware presence of u_cleanup.
MarcZX10
force-pushed
the
feat/libsceicu-ustring-exports
branch
from
October 28, 2026 14:35
0314ad9 to
b394b9b
Compare
github-actions
Bot
added
build
docs
prx
and removed
needs rebase
Contributor
|
Windows result, since the Tested section covers Linux only. Head
AI-assisted: yes (Claude Code) |
12 tasks done
12 tasks done
12 tasks done
12 tasks done
12 tasks done
This was referenced Oct 30, 2026
12 tasks done
12 tasks done
12 tasks done
12 tasks done
12 tasks done
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Implements 20 of the
sceIcuUnknown_*_nid_no_patchstubs of libSceIcu that the PS5 3.20 export inventory (PS5-3.20_Libs) names, using the names that the NIDs hash to (checked withtools/nid_names.py) and the prototypes and behavior of ICU 78.1, unicode-org/icu@049e0d6a420629ac7db77256987d083a563287b5, the commit the escape decoder already cites. The code is written independently; ICU is only the reference for prototypes, error codes and edge cases. The stubs are replaced by named exports, so the remaining stub count drops from 112 to 92.Utf32.cpp, ustrtrns.cpp):u_strFromUTF32,u_strFromUTF32WithSub,u_strToUTF32,u_strToUTF32WithSub.JavaUtf8.cpp, ustrtrns.cpp, utf_impl.cpp):u_strFromJavaModifiedUTF8WithSub,u_strToJavaModifiedUTF8.Compare.cpp,Search.cpp, ustring.cpp):u_strCompare,u_strstr,u_strrstr.Character.cpp, uchar.cpp, unames.cpp):u_isISOControl,u_forDigit,u_getISOComment.Configuration.cpp, umutex.cpp, cmemory.cpp, putil.cpp, ucln_cmn.cpp):u_setMutexFunctions,u_setAtomicIncDecFunctions(both reportU_UNSUPPORTED_ERRORas ICU does),u_setMemoryFunctions,u_setDataDirectory,u_getDataDirectory,u_setTimeZoneFilesDirectory,u_getTimeZoneFilesDirectory,u_cleanup. Stored state is guarded by mutexes.Not implemented:
u_strCompareIterstays a throwing stub because it needs theUCharIteratorlayout, which only the open iterator PR #11136 introduces. Nothing else from the Unicode property tables, converters or name data is touched.None of the 20 names is implemented or removed by the 13 open libSceIcu PRs (#10962-#10967, #11088, #11115, #11118, #11122, #11132, #11136, #12294); this was checked by name against each diff. Those PRs also edit
CMakeLists.txt,libSceIcu.hpp,Export.cppand the debt file, so a textual conflict on those lines is possible whichever lands second. This PR keeps its own files and does not copy code from them.Commits are cut per export group: implementation with its header and stub removal, then its tests, then its debt entry (39 commits). Only the final tree was built; the intermediate commits were not built one by one.
Availability caveat, recorded in the debt file: only 104 of the 661 inventory NIDs appear in the native libSceIcu export table, so whether each of these NIDs exists on current firmware is unproven. Firmware 13.42 is also reported (in #11132) to export no
u_setMemoryFunctions. The functions stay as compatibility exports like the existing escape decoder. A nullUErrorCodethrows where ICU would dereference it, and a result longer thanINT32_MAXthrows. No console behavior of these exports was measured.Tested
Linux x86-64 (host fails the CPU gate, so the archive build with the patched gate was used), GCC. Built
libSceIcuand ran withrun_tests.sh:guest_icu_utf32,guest_icu_java_utf8,guest_icu_compare,guest_icu_search,guest_icu_character,guest_icu_configuration, plus the existingguest_icu_unescapeandguest_icu_version: 8/8 passed.tools/check_conventions.py --base upstream/main,test_public_headers.py --audit-onlyandgit diff --check upstream/mainare clean. The six new test programs also ran standalone under AddressSanitizer and UBSan with no findings.The expected values in the tests were derived by reading the ICU source (UErrorCode values, preflight lengths, surrogate handling, the bytes swallowed by malformed UTF-8 sequences, the 0x2800 code point order adjustment). I checked by mutating the implementation that the UTF-32, comparison and search vectors fail when the pair or boundary logic is broken; I did not do that for every test. Not run: Windows, macOS, a console, or any title; the guest calling convention of the stored memory callbacks is never exercised because nothing calls them.
Checklist
main; no other open PR implements the same functionsNotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file.mdfiles or images added to the repository (attach them to this PR)