Skip to content

test(tools): decode check_test_registration CLI output as UTF-8 - #13839

Merged
oneandonlydean merged 1 commit into
boykopovar:mainfrom
CodingBreadd:test/test-registration-decode-utf8
Nov 1, 2026
Merged

oneandonlydean merged 1 commit into
boykopovar:mainfrom
CodingBreadd:test/test-registration-decode-utf8

Conversation

@CodingBreadd

Copy link
Copy Markdown
Contributor

What

tools/check_test_registration.py reconfigures stdout and stderr to UTF-8, but RegistrationTests.run_tool in tools/tests/test_test_registration.py reads its output with text=True, which decodes with the locale encoding. On Windows without Python's UTF-8 mode, the locale encoding is the ANSI code page. When the temporary directory holds a character whose UTF-8 bytes the code page cannot decode, such as с (d1 81) in a TEMP like C:\Users\Иван\AppData\Local\Temp under code page 1252, the error: line naming the missing CMakeCache.txt fails to decode in subprocess's reader thread (UnicodeDecodeError: 'charmap' codec can't decode byte 0x81), stderr is None, and test_missing_build_directory_is_reported_without_a_traceback raises TypeError.

run_tool now passes encoding="utf-8". The tool is unchanged. Same kind of fix as #13833 and #13837 for the select_tests, stubgen and check_skips tests.

Tested

Windows 11, Python 3.14 (ANSI code page 1252), WinLibs GCC 15.2.0 UCRT POSIX SEH toolchain on PATH.

python tools/tests/test_test_registration.py RegistrationTests.test_missing_build_directory_is_reported_without_a_traceback RegistrationTests.test_unsupported_generator_is_reported_without_a_traceback (the two run_tool users):

TEMP main this PR
C:\b\l3 тест 2 run, 1 error 2 run, OK
default 2 run, OK 2 run, OK
  • python tools/tests/test_test_registration.py with the default TEMP: 63 run, OK. With C:\b\l3 тест the CMake and Ninja cases still fail because the MinGW toolchain here cannot configure a build in a directory outside the ANSI code page; this change does not touch them.
  • python tools/check_conventions.py --base origin/main: all checks passed.
  • Not run here: Linux and macOS, where the locale is normally UTF-8 already.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: none
  • Unblocks: none
  • GPUs tested: n/a
  • AI-assisted: yes

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@MarcZX10 MarcZX10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed — looks good

The explicit UTF-8 decoding matches the tool's UTF-8 output and fixes the test harness's locale-dependent read of that output.

What I ran

Reviewed head fe18b92006ec70131fbfe832828959f2c628559b against base 809c7fa2dfb1b21e2b31b3114b7d51bc14ee88ab; the merge preview was clean. Environment: Debian/Linux x86_64, Python 3.11.2. No build or compilation was needed.

python3 tools/pr_precheck.py --repo boykopovar/AnyPS5 --pr 13839 --output-dir /tmp/anyps5-pr-precheck/boykopovar-AnyPS5-pr-13839
python3 -m unittest tools.tests.test_test_registration
python3 tools/check_test_registration.py --scripts
python3 /tmp/anyps5-negative-control-13839.py

Results

Check Result
Diff check Passed; no whitespace errors.
Conventions and PR template Passed.
Python unit suite 63 tests passed.
Test-script registration audit 230 scripts accounted for.
Merge preview Clean against 809c7fa2dfb1b21e2b31b3114b7d51bc14ee88ab.
Hosted eligibility, intake, labels Passed.
Hosted conventions Queued at the final snapshot.
Hosted Windows relinker and ASan In progress at the final snapshot.
Hosted Ubuntu/macOS relinker jobs Queued.
Hosted product Linux/Windows, register-context, write-watch Skipped.
Test-quality helper Could not run because the helper script was absent from the isolated precheck checkout.
Unedited check output
All conventions checks passed.

...............................................................
----------------------------------------------------------------------
Ran 63 tests in 0.640s

OK

230 test script(s), each run by an add_test or a workflow

Negative control

Assertion Change tested Did the old behavior fail?
The tool's UTF-8 error output can be read when the parent uses a Windows-style ANSI decoder. Ran the unchanged CLI from the reviewed head with a Cyrillic temporary path; forced cp1252 decoding for the base behavior and UTF-8 decoding for the fix. Yes for the base behavior: UnicodeDecodeError on byte 0x81. The UTF-8 path returned the expected error (exit=2) without a traceback.

This reproduces the decoding boundary on Linux; I did not run Windows locally. The precheck's automatic base-suite control passed under this host's UTF-8 locale, so it did not independently discriminate the platform-specific case. The author reports that the targeted base test fails and the PR test passes on Windows 11 with code page 1252.

Unedited negative-control output
base decode control: expected UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 247: character maps to <undefined>
UTF-8 control: exit=2; error line decoded; traceback absent

Sources

Not verified

I did not run on Windows or macOS. The Windows 11/code-page 1252 test results in the PR description are author-reported. The queued/in-progress hosted relinker jobs had not finished at the snapshot; they are separate from the locally run Python suite.

@oneandonlydean oneandonlydean self-assigned this Nov 1, 2026
@oneandonlydean
oneandonlydean merged commit 8daa361 into boykopovar:main Nov 1, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants