Skip to content

feat(utils): add prefetch to get_video_frames_generator#2273

Open
Ace3Z wants to merge 2 commits into
roboflow:developfrom
Ace3Z:feat/get-video-frames-generator-prefetch
Open

feat(utils): add prefetch to get_video_frames_generator#2273
Ace3Z wants to merge 2 commits into
roboflow:developfrom
Ace3Z:feat/get-video-frames-generator-prefetch

Conversation

@Ace3Z
Copy link
Copy Markdown
Contributor

@Ace3Z Ace3Z commented May 25, 2026

Closes #1411.

@LinasKo asked for a worked threading example that produces a real FPS improvement for the get_video_frames_generator path. This PR adds an opt-in prefetch: int = 0 argument: when > 0, frames are decoded in a background thread and buffered in a bounded queue, so a CPU-bound consumer can overlap with decode I/O.

for frame in sv.get_video_frames_generator("video.mp4", prefetch=8):
    run_inference(frame)

Default stays 0 (synchronous, behaviour unchanged). The threaded path is a thin wrapper that drives the existing sync generator on one daemon=True thread and pushes frames through a Queue(maxsize=prefetch). No new dependencies, ~30 added lines in video.py. Pattern matches the reader-thread already in process_video further down the same file.

Benchmark

150-frame 1080p h.264 video, fixed CPU consumer simulated with time.sleep:

consumer work per frame sync prefetched=8
10 ms 63.8 fps 78.3 fps (+23%)
20 ms 38.2 fps 39.6 fps (+4%)
40 ms 21.4 fps 22.1 fps (+3%)

Decode alone on this video is ~10 ms/frame, so the speed-up is largest when the consumer cost is roughly decode-bound. Heavier consumers asymptote to no benefit, which is the right behaviour.

Test

test_get_video_frames_generator_prefetch_matches_sync runs the generator twice on the same dummy video with prefetch=0 and prefetch=4 and asserts the two outputs are frame-for-frame identical. Full pytest src/ tests/ is green (1859 passed). Pre-commit clean.

@Ace3Z Ace3Z requested a review from SkalskiP as a code owner May 25, 2026 11:35
@Borda Borda requested a review from Copilot May 26, 2026 16:40
@codecov
Copy link
Copy Markdown

codecov Bot commented May 26, 2026

Codecov Report

❌ Patch coverage is 84.61538% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 78%. Comparing base (2fdb970) to head (397c5c5).

Additional details and impacted files
@@           Coverage Diff           @@
##           develop   #2273   +/-   ##
=======================================
  Coverage       78%     78%           
=======================================
  Files           66      66           
  Lines         8412    8451   +39     
=======================================
+ Hits          6580    6614   +34     
- Misses        1832    1837    +5     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an opt-in prefetch parameter to get_video_frames_generator to overlap frame decoding with CPU-bound consumers by decoding frames on a background thread and buffering them in a bounded Queue. This targets the FPS bottleneck raised in #1411 while keeping the default (prefetch=0) behavior unchanged.

Changes:

  • Extend get_video_frames_generator(..., prefetch: int = 0) with a threaded prefetch path when prefetch > 0.
  • Add internal _prefetched_frames_generator that drives the existing synchronous generator from a daemon thread and yields frames from a queue.
  • Add a regression test ensuring prefetched output matches the synchronous frame sequence exactly.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
src/supervision/utils/video.py Adds prefetch option and a new internal queued background-reader generator.
tests/utils/test_video.py Adds a test asserting prefetched iteration matches the synchronous generator frame-for-frame.

Comment on lines +309 to +322
def reader() -> None:
try:
for frame in get_video_frames_generator(
source_path=source_path,
stride=stride,
start=start,
end=end,
iterative_seek=iterative_seek,
prefetch=0,
):
frame_queue.put(frame)
finally:
frame_queue.put(None)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. The reader now captures Exception, parks it as a sentinel on the queue, and the outer generator re-raises it before yielding the next frame. Added a regression test that points the generator at a missing file and asserts the error reaches the consumer.

Comment on lines +323 to +330
thread = threading.Thread(target=reader, daemon=True)
thread.start()
while True:
frame = frame_queue.get()
if frame is None:
break
yield frame

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. Added a threading.Event that the outer generator sets in a finally block when the consumer stops or raises. The reader checks it both before each frame and inside the put() loop, and uses a 0.1s put timeout so a full queue does not leak the thread. Added a regression test that breaks out after three frames and then iterates the same file again to confirm nothing blocks.

Adds an opt-in prefetch: int = 0 parameter. When > 0, frames are decoded
in a background thread and buffered in a bounded queue, letting a
CPU-bound consumer overlap with decode I/O.

Default 0 keeps the original synchronous behaviour unchanged. The threaded
path drives the existing sync generator on a daemon thread and pumps
frames through a Queue(maxsize=prefetch). No new dependencies.

Closes roboflow#1411.
@Ace3Z Ace3Z force-pushed the feat/get-video-frames-generator-prefetch branch from bd3f788 to ba1f44f Compare May 26, 2026 17:12
@Ace3Z
Copy link
Copy Markdown
Contributor Author

Ace3Z commented May 29, 2026

Friendly ping. @Borda, would you have a moment to take a look? Happy to address any feedback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Increasing Video FPS running on CPU Using Threading

3 participants