Skip to content

test(tools): decode select_tests and stubgen CLI output as UTF-8 - #13833

Merged
oneandonlydean merged 1 commit into
boykopovar:mainfrom
CodingBreadd:test/decode-tool-output-utf8
Nov 1, 2026
Merged

oneandonlydean merged 1 commit into
boykopovar:mainfrom
CodingBreadd:test/decode-tool-output-utf8

Conversation

@CodingBreadd

@CodingBreadd CodingBreadd commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

What

tools/select_tests.py and tools/stubgen.py reconfigure stdout and stderr to UTF-8, but three test call sites read their output with text=True, which decodes with the locale encoding:

  • BuildErrorTests.run_tool in tools/tests/test_select_tests.py (3 tests, 10 cases);
  • test_dangling_targets_fail_instead_of_selecting_no_tests in the same file;
  • test_readonly_second_output_preserves_both_outputs in tools/tests/test_stubgen.py, which runs only on Windows.

On Windows without Python's UTF-8 mode, the locale encoding is the ANSI code page. When the temporary directory holds a character whose UTF-8 bytes the code page cannot decode, such as с (d1 81) in a TEMP like C:\Users\Иван\AppData\Local\Temp under code page 1252, the error message that names the path fails to decode in subprocess's reader thread (UnicodeDecodeError: 'charmap' codec can't decode byte 0x81), stdout/stderr are None, and the assertions raise TypeError.

These call sites now pass encoding="utf-8", as tools/tests/test_audit_dump.py, tools/tests/test_configure_seed.py and other tool tests already do. The tools are unchanged. Same kind of fix as #13269 for test_pr_overlap_export_cache.py; no overlap with it.

Tested

Windows 11, Python 3.14 (ANSI code page 1252), WinLibs GCC 15.2.0 UCRT POSIX SEH toolchain on PATH.

Command TEMP main this PR
python tools/tests/test_stubgen.py C:\b\l3 тест 28 run, 1 error 28 run, OK (2 skipped)
python tools/tests/test_select_tests.py BuildErrorTests GraphClosureTests C:\b\l3 тест 7 run, 11 errors (subtests) 7 run, OK
python tools/tests/test_stubgen.py default OK 28 run, OK (2 skipped)
python tools/tests/test_select_tests.py default OK 42 run, OK

The other test_select_tests.py classes configure a CMake/Ninja build in the temporary directory, which the MinGW toolchain here cannot do in a directory outside the ANSI code page; they are not affected by this change.

  • python tools/check_conventions.py --base origin/main: all checks passed.
  • Not run here: Linux and macOS, where the locale is normally UTF-8 already.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: none
  • Unblocks: none
  • GPUs tested: n/a
  • AI-assisted: yes

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@oneandonlydean oneandonlydean self-assigned this Nov 1, 2026
@oneandonlydean
oneandonlydean merged commit e3a6267 into boykopovar:main Nov 1, 2026
14 of 15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants