Skip to content

fix: keep macOS webcam recordings continuous with VP8 - #1297

Open
tonisives wants to merge 1 commit into
getopenscreen:mainfrom
tonisives:fix/macos-webcam-recording
Open

tonisives wants to merge 1 commit into
getopenscreen:mainfrom
tonisives:fix/macos-webcam-recording

Conversation

@tonisives

@tonisives tonisives commented Oct 17, 2026 •

Copy link
Copy Markdown

Follow-up, 2026-10-27

The blanket VP8 preference needs revision. The review measured dropped frames at 4K, and later local traces found separate camera-buffer and hardware H.264 encoding stalls.

The local workaround now copies camera pixels into owned frames and uses software H.264 through Chromium's disable-accelerated-video-encode switch on macOS. It passed simultaneous screen, microphone and 1080p30 camera recording, pause/resume, playback and full MP4 export. The final camera take ran for 203.829 s at 29.976 fps, with a maximum frame gap of 69 ms. Software H.264 at 4K still needs measurement.

The branch currently contains the original VP8 proposal below. The frame-copy and software-H.264 changes are local; the follow-up comment records the diagnosis, exact workaround and test scope. The compositor and WebM metadata fixes remain applicable independently.

Original proposal

On macOS, MediaRecorder's H.264 webcam encoder can stop producing frames while the camera stream stays live. A local reproduction with a Logitech C922 produced only 13 frames over 56 seconds; VP8 produced 3,828 frames over 128 seconds with a maximum frame gap of 68 ms. A second installed-build take produced 1,932 frames over 64 seconds, maximum gap 66 ms.

Prefer VP8 for macOS webcam recording, independently of the native H.264 screen recorder. Keep the existing H.264 preference on Windows/Linux and negotiate supported fallbacks. Check the codec's VideoToolbox configurations before selecting hardware presentation: VP8 CPU frames need the existing NV12/CVPixelBuffer conversion to appear in the Metal compositor. Preserve unknown-sized WebM Cluster bytes when patching duration, including recordings under 2 MB.

The machine-specific build/install scripts used for local testing are excluded. Autofocus and camera constraints are unchanged. The held-screen preview clock fixes already on main address a separate editor-only freeze.

Related issue

No issue linked; reproduced locally on macOS.

Type of change

  • Bug fix

Release impact

  • Patch

Desktop impact

  • macOS

Screenshots / video

Private camera recordings were checked locally and are not attached. The testing checklist records the measured continuity and the partial desktop smoke-test scope.

Testing of the original PR patch

  • Node 22: full unit suite, 351 files; 4,909 passed, 2 skipped.
  • Focused webcam recorder options, webcam capture target and WebM duration tests: 25 passed.
  • Both app and test TypeScript checks pass.
  • Biome check of the five changed TypeScript files passes.
  • Native codec regression test passes: VP8 uses CPU presentation; H.264 advertises VideoToolbox.
  • Release compositor addon builds on Apple Silicon with the vendored LGPL FFmpeg SDK.
  • Capture-to-editor and MP4 camera presentation tested on the v2.0.0 local build carrying this patch. The port to current main has unit/type/build checks; a new desktop capture smoke test on that exact main build and physical Windows/Linux capture were not run.

Summary by CodeRabbit

  • Bug Fixes
    • Improved webcam recording compatibility across macOS, Windows, and Linux by selecting an available video format for each platform.
    • Webcam recording bitrate now reflects the camera’s delivered dimensions.
    • Fixed duration updates for WebM recordings of all sizes while preserving their video data.
    • Unsupported hardware-decoding formats on macOS now use the CPU presentation path, supporting preview and export.

@coderabbitai

coderabbitai Bot commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review.

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 89d559b7-b24f-4ed3-8667-b18a9e509107
📥 Commits

Reviewing files that changed from the base of the PR and between 763cfed and 7ce05cc.

📒 Files selected for processing: 7
  • crates/compositor/src/pipeline_macos.rs
  • electron/recording/webm-duration.test.ts
  • electron/recording/webm-duration.ts
  • src/hooks/useScreenRecorder.ts
  • src/hooks/webcamRecorderOptions.test.ts
  • src/hooks/webcamRecorderOptions.ts
  • technical-documentation/testing/manual-e2e-checklist.md

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.

Walkthrough

The changes update platform-specific webcam codec selection, route all on-disk WebM duration patches through a header-only splice path, and gate macOS VideoToolbox setup on codec support. Tests cover codec selection, WebM byte preservation, and decoder support.

Changes: Camera recording and playback

Layer / File(s) Summary
Webcam codec selection
src/hooks/webcamRecorderOptions.ts, src/hooks/useScreenRecorder.ts, src/hooks/webcamRecorderOptions.test.ts
The new helper selects codecs in platform-specific order, omits mimeType when no candidate is supported, and sets the bitrate from delivered camera dimensions. The screen recorder uses this helper. Tests cover codec preferences, fallback, and bitrate.
On-disk WebM duration patching
electron/recording/webm-duration.ts, electron/recording/webm-duration.test.ts
Files of all sizes now use the header-only splice path. Tests check patching a file under 2 MB and preservation of Cluster bytes after duration patching.
macOS decoder setup
crates/compositor/src/pipeline_macos.rs, technical-documentation/testing/manual-e2e-checklist.md
VideoToolbox setup is attempted only when the codec supports the device type and pixel format. Tests check VP8 and H.264 support. The manual test log records macOS capture results and remaining test gaps.

Priority: High

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix

Merge Risk: Minimal · up to 7ce05

The camera-recording changes are ready to merge after normal checks; no concrete blocking issue remains identified.

Pre-merge checks: Passed 4
Check name Status Explanation
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Title check Passed The title clearly summarizes the primary change: maintaining continuous macOS webcam recordings by preferring VP8.
Description check Passed The description covers the change, related issue status, bug-fix and patch classification, macOS impact, testing scope, results, and known limitations. It does not use the template's exact Summary hea…
Full details: Description check

Explanation

The description covers the change, related issue status, bug-fix and patch classification, macOS impact, testing scope, results, and known limitations. It does not use the template's exact Summary heading, but the required summary information is present.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@EtienneLescot EtienneLescot left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the detailed report. We measured this on an M1 Mac mini (macOS 27) inside the app's own Chromium, with the recorder options from main and from this PR, 60 s per take:

Take Frames fps Max gap Gaps > 100 ms CPU
H.264, 1080p 1799 30.0 76 ms 0 30 %
VP8, 1080p 1799 30.0 65 ms 0 156 %
H.264, 2160p 1799 30.0 66 ms 0 72 %
VP8, 2160p 1082 18.2 1299 ms 54 348 %

The 2160p source was the camera upscaled frame by frame to 3840×2160, since our only camera there tops out at 1080p. The source delivered ~1800 frames; the VP8 encoder dropped the rest.

So VP8 holds 1080p (at about five times the CPU), but not the default 2160p preset: there it produces the same kind of frozen webcam the PR sets out to fix. We couldn't reproduce the H.264 stall with our virtual camera. That doesn't rule it out on a C922, but it means we can't trade it for a certain stall at 4K. Could you limit VP8 to takes the camera delivers at 1080p or below, and keep H.264 above that? Any other approach that holds 30 fps at 2160p on Apple Silicon works too.

The compositor change is good and needed on its own. With main's addon a VP8 webcam is missing from the export entirely; with yours it exports smoothly, and H.264 exports are bit-identical. The new Rust test passes on Apple Silicon. Feel free to split it into its own PR if you'd like it merged first.

Please also rebase: the checklist row conflicts with main.

@tonisives

tonisives commented Oct 27, 2026 •

Copy link
Copy Markdown
Author

Follow-up after longer tests: the blanket VP8 preference in this PR needs to change. Your 4K measurements exposed a CPU cost that the initial short 1080p takes missed.

On the affected M3 Pro, camera capture kept delivering about 30 fps while hardware H.264 encoding stalled during simultaneous native screen and webcam recording. One 60-second trace had 1,800 renderer inputs, eight GPU encode calls and one encoded output. Process sampling found VTCompressionSessionEncodeFrame blocked, with the VideoToolbox service waiting inside AppleVideoEncoder. Camera-only tests passed. The condition inside VideoToolbox that causes the wait is still unresolved.

I also reproduced the frozen tail with the built-in FaceTime HD camera on macOS 27.2: H.264 stopped after 52 frames at 1.701 s while the screen recorded 125.900 s. Checking only camera track FPS, container duration or gaps between existing packets missed this failure.

Two local changes now work together:

  • Copy camera pixels into owned VideoFrames and release the borrowed capture frames before encoding. This improved the earlier VP8 stalls: worst measured encode time fell from 930 ms to 70 ms. Copying alone did not reliably fix hardware H.264.
  • Keep H.264 preferred and disable Chromium's hardware video encoding on macOS before startup:
app.commandLine.appendSwitch("disable-accelerated-video-encode");

A trace confirmed OpenH264::EncodeFrame was used. GPU compositing and video decoding remained enabled; the separate native screen recorder and compositor kept their existing encoder paths. Chromium's MediaRecorder hardware fallback runs on an encoder error, which the blocked call did not produce.

Results on the local v2.0.0-based build, Electron 41.2.1 / Chromium 146.0.7680.188, FaceTime HD at 1080p30, with screen and microphone recording:

Take Camera frames Last camera PTS Max gap Gaps >100 ms
Software H.264 comparison 12,326 411.185 s 198 ms 3
Installed build, normal launch, pause/resume 6,111 203.829 s 69 ms 0

The final take decoded completely with 6,111 distinct camera frames. Its full UI MP4 export had 6,096 frames, clean video/audio decoding and 6,096 distinct camera-region images. The comparison's recording renderer averaged 51.9% CPU over 45 seconds, where 100% is one core; that excludes the native recorder and Apple's encoder service.

This validates a local 1080p workaround. The C922 was disconnected for the latest tests, and software H.264 at 4K has not been measured. These changes are not on this PR branch yet. I recommend keeping H.264 as the upstream default while working out a targeted software fallback for affected recordings. The codec-aware compositor path and WebM metadata fixes in this PR remain useful independently.

Further hardware isolation on the same Mac: a minimal MediaRecorder process using the actual frame-copy code, FaceTime HD at 1080p30, and native screen/microphone capture passed with 1,798 frames through 59.909 s (max gap 53 ms). A 90-second synthetic 1080p30 run alongside a second native 4K encoder saved 2,697 frames through 89.868 s (max gap 44 ms). Traces confirmed hardware encoding.

A separate synthetic stress case is reproducible: with two 4K streams, MediaRecorder saved only 186 frames through 6.17 s from 30 s of input, twice. Output kept arriving but fell far behind wall time. The native foreground control completed all 900 frames, with submission stretched to 32.51 s. A single MediaRecorder 4K stream passed. Reducing the second stream to 30 fps still left a large backlog.

This supports keeping hardware H.264 as the upstream default. The original 1080p app freeze remains unexplained by the minimal tests; the blocked VideoToolbox stack alone does not identify a Chromium defect. The software switch remains a local workaround, while the synthetic backlog gives us a smaller case to investigate.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants