Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions docs/board-enablement.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,38 @@ pack below and confirm `dropped: 0` at the source framerate.
> path is identical; their slower ARM cores only affect demux/parse + the fb
> memcpy, not the offloaded decode/scale/convert.

#### Screen rotation and video on Pi 1 / 2 / 3

`screen_rotation` interacts badly with video on these boards, and the outcome
depends on the angle — because VideoCore IV has **no rotate control at all**.
`v4l2-ctl -l` on the ISP (`/dev/video12`) exposes only `horizontal_flip` and
`vertical_flip`; no bcm2835 video device exposes `V4L2_CID_ROTATE`. (The same
limit shows up on the pi3-64 vc4 DRM plane, which offers 0°/180° + reflect
only — forum 6730.)

| `screen_rotation` | path | Pi 2, 1080p30 |
|---|---|---|
| 0 | all hardware | ~27 fps |
| 180 | ISP `horizontal_flip` + `vertical_flip` | ~27 fps |
| 90 / 270 | software `videoflip` (no HW path) | **~2 fps** |

* **180 is free.** hflip + vflip compose to a 180 rotation and ride along on
the `v4l2convert` pass that already does the aspect-fit scale, so the
pipeline stays fully hardware and the result is pixel-identical to the
software rotation.
* **90 / 270 are effectively unsupported for video on Pi 1 / 2 / 3** (issue
#3198). The quarter turns need a transpose, which no VideoCore IV block can
do, so `videoflip` rotates each full-resolution frame on a single CPU core
and 1080p collapses to ~2 fps. The picture is *correct*, just far too slow to
use. Rotating still images and web pages costs nothing per frame and is
unaffected — this limit is video-only. **Use a Pi 4 / Pi 5 for
portrait-mounted video**, where the platform layer rotates instead.

Note that reordering to scale-before-flip does not rescue 90/270: the case that
actually matters — portrait content on a portrait-mounted panel — rotates to
exactly fill the framebuffer, so there is no downscale to exploit ahead of the
flip (measured slightly *worse*, 1.9 vs 2.1 fps).

## Goal: hardware-accelerated playback on every board

Every clip Anthias displays should decode in hardware on the target board.
Expand Down
49 changes: 39 additions & 10 deletions src/anthias_viewer/gst_fbdev_player.py
Original file line number Diff line number Diff line change
Expand Up @@ -68,15 +68,38 @@
# even half-cadence instead of juddering on irregular sync drops.
MAX_OUTPUT_FPS = 30

# Operator screen_rotation → GStreamer ``videoflip`` method. None for
# an unrotated panel (the common case) so the videoflip element is
# omitted entirely and the pipeline stays fully hardware.
# Operator screen_rotation → GStreamer ``videoflip`` method. 180 is
# absent deliberately: the bcm2835 ISP does that one in hardware (see
# ISP_FLIP_CONTROLS). None for an unrotated panel (the common case) so
# the videoflip element is omitted entirely and the pipeline stays
# fully hardware.
#
# 90/270 have no hardware path on VideoCore IV (Pi 1/2/3) and cost
# roughly an order of magnitude in frame rate — measured on a Pi 2 at
# 1080p30: 27 fps unrotated vs 2 fps rotated, with videoflip pegging a
# core (issue #3198). The rotation is visually correct, so the element
# stays as the honest best effort, but rotated 1080p video is
# documented as unsupported on these boards (docs/board-enablement.md).
# Reordering to scale-before-flip does not rescue it: the real case —
# portrait content on a portrait panel — rotates to exactly fill the
# framebuffer, so there is no downscale to exploit (measured 1.9 fps
# vs 2.1 fps, i.e. slightly worse).
GST_VIDEOFLIP_METHODS = {
90: 'clockwise',
180: 'rotate-180',
270: 'counterclockwise',
}

# 180 costs nothing on the bcm2835 ISP: a horizontal + vertical flip
# composes to a 180 rotation, and both are v4l2 controls on the same
# ``v4l2convert`` pass that already does the aspect-fit scale. Measured
# on a Pi 2 at 1080p30: 27 fps (vs 2 fps through the software
# videoflip) and pixel-identical to the software rotation. These are
# the *only* rotate-ish controls the hardware exposes — there is no
# V4L2_CID_ROTATE on any bcm2835 video device, which is what rules out
# 90/270 above.
ISP_FLIP_CONTROLS = 'c,horizontal_flip=1,vertical_flip=1'
ISP_FLIP_ROTATION = 180


def compute_fit_dims(
src_width: int,
Expand Down Expand Up @@ -178,7 +201,9 @@ def parse_args(argv: list[str]) -> argparse.Namespace:
'--rotation',
type=int,
default=0,
choices=sorted({0, *GST_VIDEOFLIP_METHODS}),
# 180 is handled by the ISP rather than videoflip, so it isn't
# in GST_VIDEOFLIP_METHODS — it's still an accepted rotation.
choices=sorted({0, ISP_FLIP_ROTATION, *GST_VIDEOFLIP_METHODS}),
)
parser.add_argument('--audio-device', default='')
parser.add_argument('--fb-device', default=FB_DEVICE)
Expand All @@ -189,16 +214,20 @@ def build_sink_description(args: argparse.Namespace) -> str:
"""gst-parse description for playbin's ``video-sink`` bin.

``videorate`` caps the frame rate (drop-only — never duplicates),
``videoflip`` is inserted only when the operator rotated the
screen, ``v4l2convert`` (bcm2835 ISP) does the scale + colour
convert in hardware, and the named capsfilter is the handle the
CAPS probe re-pins with the aspect-fit dimensions.
``videoflip`` is inserted only for the 90/270 rotations the
hardware can't do, ``v4l2convert`` (bcm2835 ISP) does the scale +
colour convert — and, at 180, the rotation — in hardware, and the
named capsfilter is the handle the CAPS probe re-pins with the
aspect-fit dimensions.
"""
parts = [f'videorate drop-only=true max-rate={MAX_OUTPUT_FPS}']
flip = GST_VIDEOFLIP_METHODS.get(args.rotation)
if flip:
parts.append(f'videoflip method={flip}')
parts.append('v4l2convert name=fit_convert')
convert = 'v4l2convert name=fit_convert'
if args.rotation == ISP_FLIP_ROTATION:
convert += f' extra-controls={ISP_FLIP_CONTROLS}'
parts.append(convert)
parts.append(
f'capsfilter name=fit_caps '
f'caps={build_fit_caps_string(args.fb_format)}'
Expand Down
31 changes: 28 additions & 3 deletions tests/test_gst_fbdev_player.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
import pytest

from anthias_viewer.gst_fbdev_player import (
ISP_FLIP_CONTROLS,
MAX_OUTPUT_FPS,
build_fit_caps_string,
build_sink_description,
Expand Down Expand Up @@ -112,12 +113,36 @@ def test_sink_description_is_hw_pipeline_with_rate_cap() -> None:
assert 'fbdevsink device=/dev/fb0' in desc
# Unrotated panel → no videoflip, pipeline stays fully hardware.
assert 'videoflip' not in desc
assert 'extra-controls' not in desc


def test_sink_description_rotation_adds_videoflip() -> None:
desc = build_sink_description(_args(rotation=90))
assert 'videoflip method=clockwise' in desc
@pytest.mark.parametrize(
('rotation', 'method'),
[(90, 'clockwise'), (270, 'counterclockwise')],
)
def test_sink_description_90_270_use_videoflip(
rotation: int, method: str
) -> None:
# No bcm2835 device exposes a rotate control, so the quarter turns
# have no hardware path and fall back to the software videoflip
# ahead of the ISP (issue #3198).
desc = build_sink_description(_args(rotation=rotation))
assert f'videoflip method={method}' in desc
assert desc.index('videoflip') < desc.index('v4l2convert')
assert 'extra-controls' not in desc


def test_sink_description_180_rotates_in_the_isp() -> None:
# hflip + vflip compose to a 180 rotation, and the ISP does both as
# v4l2 controls on the aspect-fit pass it already runs — so 180
# keeps the full-speed hardware pipeline instead of dropping to
# ~2 fps through videoflip (issue #3198).
desc = build_sink_description(_args(rotation=180))
assert 'videoflip' not in desc
assert (
f'v4l2convert name=fit_convert extra-controls={ISP_FLIP_CONTROLS}'
in desc
)


def test_parse_args_rejects_non_cardinal_rotation() -> None:
Expand Down