Skip to content

fix(ci): fail the relinker test step when no test is registered - #14187

Merged
oneandonlydean merged 2 commits into
boykopovar:mainfrom
MarcZX10:ci/relinker-test-step-needs-tests
Nov 2, 2026
Merged

oneandonlydean merged 2 commits into
boykopovar:mainfrom
MarcZX10:ci/relinker-test-step-needs-tests

Conversation

@MarcZX10

Copy link
Copy Markdown
Contributor

What

The relinker-only test step in build.yml runs ctest --test-dir build-relinker without --no-tests=error. A configuration that registers no test therefore passes with "No tests were found". The other test steps in the same file (register-context, write-watch, relinker-asan) already pass that flag. With ctest 3.25.1, a directory with no tests exits 0 without the flag and 8 with it.

No configuration registers zero tests today: the relinker-only configuration has 233 add_test lines. The change closes the gap for a future CMake change that moves the registrations behind a condition that is false in that configuration.

Fix, in two commits:

  • b0dbe27b48: --no-tests=error on that step.
  • be8ea1e172: tools/tests/test_relinker_test_step.py reads the relinker ctest line of build.yml and requires the flag. It is registered in CMakeLists.txt after select_tests.

Overlap: the overlap check fails closed on ten open pull requests that touch CMakeLists.txt and conflict with main (#1835, #5021, #7642, #9539, #10193, #10897, #11104, #13578, #13928, #13929), so it cannot compare them. I searched their CMakeLists.txt patches for select_tests, tools_self_inspect and relinker_test_step: none mentions them. This PR's registration sits after select_tests.

Tested

Linux, Python 3.11.2, on upstream/main 1bc97b9527 plus these commits:

  • python3 tools/tests/test_relinker_test_step.py: OK with the fix.
  • Negative control: on the unfixed workflow the test fails with '--no-tests=error' not found in ' - run: ctest --test-dir build-relinker --output-on-failure --timeout 120 --parallel 4'.
  • build.yml parses as YAML (PyYAML).
  • python3 tools/check_conventions.py --base upstream/main and git diff --check upstream/main: clean.

Not run: the relinker job itself. The flag only changes behaviour when no test is found, which the current configuration does not produce.

Checklist

  • Based on current main
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none
  • Fixes: none
  • Unblocks: none
  • GPUs tested: n/a
  • AI-assisted: yes

The relinker-only test step runs ctest without --no-tests=error, so a configuration that registers no test passes with 'No tests were found'. The sibling steps for the register-context, write-watch and relinker-asan tests already pass that flag. ctest 3.25.1 exits 0 without it and 8 with it when no test is found.
The test reads the relinker ctest line of build.yml and checks that it passes --no-tests=error. Against the previous line it fails.
@github-actions

github-actions Bot commented Oct 31, 2026 •

Copy link
Copy Markdown

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 1835 CMakeLists.txt
PR 5021 CMakeLists.txt
PR 7642 CMakeLists.txt
PR 9539 CMakeLists.txt
PR 10193 CMakeLists.txt
PR 13560 .github/workflows/build.yml
PR 13578 CMakeLists.txt
PR 13928 CMakeLists.txt
PR 13929 CMakeLists.txt
PR 14212 CMakeLists.txt
PR 14264 CMakeLists.txt

@maskym

maskym commented Nov 1, 2026

Copy link
Copy Markdown
Contributor

Windows result for head be8ea1e172, merged alone into main f2d874c1fd (the PR's Tested section names no Windows run).

Host: Windows 11 26300, Python 3.13.5.

python -m unittest tools.tests.test_relinker_test_step: OK, three runs. Not run on a PS5.

AI-assisted: yes (Claude Code), the runs and the numbers above are from this machine.

@oneandonlydean
oneandonlydean merged commit 573edc5 into boykopovar:main Nov 2, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants