Skip to content

CI: the VideoOut display tests never run on Linux or Windows; under Xvfb they run in 10 s and caught the EOP flip regression #6899

Open

Description

@hashaam1217

What

17 VideoOut tests skip on the Linux CI job, and Windows CI skips them as well: video_out_flip_{controls,eop,eop_aftergpu,eop_main,labels,labels_main,lifetime,overlay,reopen,unopened}, video_out_frame_dump_queue, video_out_hdr_present and guest_video_out_{hdr,high_frame_rate,output_mode,resolution_status,wqhd_detection}. They use SDL_VIDEODRIVER=offscreen (the ANYPS5_VIDEO_OUT_TEST_DRIVER default from #3810), and SDL2's offscreen driver has no Vulkan surface, so each one reports no display or Vulkan device. tools/check_skips.py accepts that reason, so the job stays green.

This hides real regressions. #1202 (pipelined draws on by default) broke presentation of EOP flips from #5539: video_out_flip_eop and _eop_main fail and _eop_aftergpu times out. CI could not see it; it was found by running the tests by hand and bisecting. The fix is #6616.

Running them under Xvfb

On Ubuntu 24.04 with lavapipe (Mesa 25.2.8):

Xvfb :99 -screen 0 1920x1080x24 -nolisten tcp &
cmake -S . -B build -DANYPS5_VIDEO_OUT_TEST_DRIVER=x11
DISPLAY=:99 ANYPS5_GPU=llvmpipe ctest --test-dir build -R video_out

On main plus #6616, 16 of the 17 run and pass, in 9.9 s for all 21 video_out tests. guest_video_out_high_frame_rate still skips, because Xvfb offers no 120 Hz mode. check_skips.py already accepts that reason. On main without #6616, the three EOP tests fail as described above.

A hardware Vulkan driver cannot present on Xvfb, which has no DRI3: Intel ANV fails with No DRI3 support detected. So this works with lavapipe, which is what the Linux job already uses.

Suggestion

On the self-hosted Linux runner: install xvfb, configure with -DANYPS5_VIDEO_OUT_TEST_DRIVER=x11, and run the ctest step under xvfb-run -a (or with a started Xvfb and DISPLAY set). Once that is in place, check_skips.py could stop accepting no display on the Linux job, so a lost display shows up as a failure instead of a skip. The runner is yours to configure, so I have not opened a workflow PR. I'm happy to send one if this direction is wanted.

AI-assisted: yes.

Activity

  1. MarcZX10 commented on Oct 16, 2026

    @MarcZX10
    Contributor

    I’m prototyping the Linux CI change from your suggestion: configure ANYPS5_VIDEO_OUT_TEST_DRIVER=x11 and run CTest under xvfb-run, with ANYPS5_GPU=llvmpipe. The repo’s build instructions cover X11/XRandR development packages but not the self-hosted runner’s Xvfb package. Is xvfb-run already installed on the [self-hosted, linux, x64, anyps5] runner, or should the workflow install/provision it? I’m keeping this separate from the EOP production fix in #6616.

  2. MarcZX10 commented on Oct 16, 2026

    @MarcZX10
    Contributor

    I opened #6932 with the Linux X11/Xvfb CI change. The workflow checks for both xvfb-run and xauth, and installs xvfb/xauth with apt if either is missing. The focused VideoOut target builds locally, but runtime assertions on my host stop at the emulator CPU feature guard; I have not claimed a local CTest pass. The PR checks are still running, with the Windows relinker CTest step pending.

  3. MarcZX10 commented on Oct 16, 2026

    @MarcZX10
    Contributor

    Update: #6932 has now passed its PR checks, including the Linux, macOS, and Windows relinker test jobs. The full Linux/Windows build suites are skipped on pull requests by the workflow conditions; the Xvfb VideoOut configuration will be exercised by the full Linux build on main after merge. My local VideoOut test attempt remains blocked before assertions by this host’s CPU feature guard.

  4. MayankManas commented on Oct 20, 2026

    @MayankManas
    Contributor

    Windows results with a real display, for comparison with the Xvfb run above. ANYPS5_VIDEO_OUT_TEST_DRIVER=windows, ctest -R video_out -V with ANYPS5_GPU per device, ANYPS5_REQUIRE_VULKAN=1, ANYPS5_NO_SHADER_CACHE=1, VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation. Windows 11 Pro 26200, WinLibs GCC 15.2.0 r7, Release, main e9cc4a8, display in HDR.

    With offscreen, 13 of the 14 video_out_flip_* tests skip (decode needs no window). With windows:

    GPU Passed Skipped Failed
    NVIDIA GeForce RTX 5070 Ti, driver 617.42 24 1 1: video_out_hdr_probe
    Intel UHD Graphics 770, driver 32.0.101.7088 23 1 2: video_out_hdr_probe, video_out_flip_eop_aftergpu

    All 14 flip tests, guest_video_out_* and video_out_hdr_present pass on the NVIDIA GPU. The skip is guest_video_out_wqhd_detection_no_vulkan. No VUID- and no was not found but was requested by env var VK_INSTANCE_LAYERS in either log.

    Both failures are in the tests, not in VideoOut, and CI cannot reach either of them:

    AI-assisted: yes

  5. LeandroLP23 commented on Oct 26, 2026

    @LeandroLP23
    Contributor

    AMD and a second NVIDIA generation for the Windows half, following MayankManas's run above. Same setup: -DANYPS5_VIDEO_OUT_TEST_DRIVER=windows, ctest -R video_out -V with ANYPS5_REQUIRE_VULKAN=1, ANYPS5_NO_SHADER_CACHE=1 and VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation (Vulkan SDK 1.4.363.0), WinLibs GCC 15.2.0 r7, Release, main 035ac96edd, Windows 11 10.0.26300, displays in SDR. Both runs drew on the machine's real display in the logged-in session (on the desktop through an interactive-logon scheduled task, since an SSH session has no display).

    GPU Display Passed Skipped Failed
    AMD Radeon 860M (integrated, RDNA 3.5), AMD 24.30.58.08 2880x1800 at 120 Hz 29 1: guest_video_out_wqhd_detection_no_vulkan 0
    NVIDIA GeForce RTX 5060 Ti (Blackwell), NVIDIA 621.01 2560x1440 at 143 Hz, plus a 1920x1080 30 0 0

    With the Linux half done in #6932, what remains here is Windows CI, which has no display; on real Windows displays the suite now passes on AMD, NVIDIA (two generations) and Intel apart from the two test issues already filed.

    AI-assisted: yes (Claude Code)

  6. Abhi2april commented on Nov 5, 2026

    @Abhi2april

    #6932 (merged) closed the Linux half: the Linux job runs ctest -R video_out under xvfb-run with -DANYPS5_VIDEO_OUT_TEST_DRIVER=x11 and ANYPS5_GPU=llvmpipe, and it passes 16 of 17 in 9.9 s. The Windows job still configures with offscreen (the default from #3810), so 13 of the 14 video_out_flip_* tests skip with no display or Vulkan device — only video_out_flip_decode runs (it needs no window) — and tools/check_skips.py accepts that reason, which keeps the job green. The guest_video_out_* tests skip the same way. That is the state the two manual runs by @MayankManas (Windows 11 26200, WinLibs GCC 15.2.0 r7, header main e9cc4a8, HDR display) and @LeandroLP23 (Windows 11 10.0.26300, Vulkan SDK 1.4.363.0, main 035ac96, SDR) already measured on AMD and two NVIDIA generations.

    for the Windows job:

    • In the Windows build job's ctest step, set ANYPS5_VIDEO_OUT_TEST_DRIVER=windows, which the two runs above already validate on real displays. Required focused-test env per the CONTRIBUTING rule: ANYPS5_REQUIRE_VULKAN=1, ANYPS5_NO_SHADER_CACHE=1, VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation. Unlike the Linux side, no new display stack is needed: the Windows job uses the runner's real display, as the manual runs did — those runs are the baseline the CI job should reproduce.
    • tighten tools/check_skips.py for the Windows job: stop accepting no display or Vulkan device once the driver is windows. Keep accepting the two skips that are host-display-dependent — video_out_hdr_present (runner display may not be HDR) and guest_video_out_high_frame_rate (runner display may lack a 120 Hz mode; on the Xvfb Linux job it skips for the same reason).
    • then the 13 skipped video_out_flip_*, plus video_out_frame_dump_queue and guest_video_out_{hdr,output_mode,resolution_status,wqhd_detection}, must run and pass. A presentation regression like the EOP one that fix(agc): present EOP flips when draws are pipelined #6616 fixed would then fail CI instead of hiding behind a skip.

    What I could not check from my side (I am a first-time contributor without runner access): whether the self-hosted Windows runner keeps a logged-in session with a live display, and whether the ctest step belongs in the Windows full-build job in build.yml or somewhere else. If the direction is fine, I open the PR and a collaborator confirms it with gh workflow run build.yml --repo boykopovar/AnyPS5 --ref main --field full=false-style run as in CONTRIBUTING.

    On the PR checklist I would put Depends on: #6932 only if the shared check_skips.py changes conflict with their merged surface, Fixes: #6899
    AI-assisted: yes.

  7. LeandroLP23 commented on Nov 5, 2026

    @LeandroLP23
    Contributor

    One thing to check before the Windows step: the two manual runs you cite, mine and MayankManas's, used machines with a real GPU and Vulkan driver. windows-latest has neither. tools/check_skips.py --job windows already says so ("the runner has no Vulkan driver"), and the Windows ctest step in build.yml doesn't set ANYPS5_REQUIRE_VULKAN.

    On that runner, ANYPS5_VIDEO_OUT_TEST_DRIVER=windows would most likely move the flip tests from "no display" to "no usable Vulkan device", and they would still skip. Adding ANYPS5_REQUIRE_VULKAN=1 to that step would turn the device skips into failures: the video_out ones, the agc* tests the job skips today, and any other test that reports no Vulkan device.

    So the Windows half also needs a Vulkan device on the runner. The counterpart of the Linux job's llvmpipe would be Mesa's lavapipe built for Windows, for example from mesa-dist-win, registered through VK_DRIVER_FILES. I haven't tried that on a hosted runner, so I can't say whether the video_out tests pass on it or whether the runner's session gives them a display. One trial run of the job with that change would answer both.

    AI-assisted: yes (Claude Code)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions