Skip to content

Playback: audio callback load and dropout meter (System Usage, ui.audio_load) - #1067

Open
sanderdewijs wants to merge 1 commit into
storytold:mainfrom
sanderdewijs:feat/dsp-load-meter
Open

sanderdewijs wants to merge 1 commit into
storytold:mainfrom
sanderdewijs:feat/dsp-load-meter

Conversation

@sanderdewijs

@sanderdewijs sanderdewijs commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

What

A load and dropout meter for the audio callback.

  • Each device callback is timed against how long its buffer lasts. Window ▸ System Usage shows:
    • the average and peak load over the last half second, with a bar;
    • the heaviest callback;
    • the number of late callbacks: callbacks whose render took longer than the buffer lasts, so the device ran dry (a dropout).
    • A Reset button starts the worst load and the count again.
  • ui.audio_load {reset?} returns the same for agents: measured, average, peak, worst, late, callbacks, buffer_ms. The MCP reaches it through execute with --connect.
  • System Usage now counts instruments as plugin instances. It used to count inserts only, so a session with 28 inserts and 14 instruments showed 28.

How

crates/playback/src/load.rs: the audio thread is the only writer. It keeps a local half-second window and publishes through atomics, so there are no locks and no allocation in the callback. The UI asks for a reset with a flag, and the audio thread clears the counters at its next callback. cpal doesn't pass on the device's own overload notices, so late callbacks are counted here instead. On the web (wasm32), Instant isn't available and the meter reports measured: false.

Why

Glitches during realtime playback were hard to tell apart: plug-in behaviour, scheduling jitter, or plain CPU shortage. Without a number, every fix was a guess. On a real 19-track session the meter immediately separated two causes:

  1. A plug-in in trial mode: 9 late callbacks, up to 276%. After activation: 0.
  2. A remaining isolated 97% spike on a ~13% average, which is the jitter [Audit] The audio callback blocks on rayon's global pool #886 describes.

It also gives #886, #1006 and #804 a before/after measure on the same session.

Tests

  • load.rs unit tests: per-window averaging and peaks, late counting, worst, reset, and hostile timings (NaN, infinite, zero and negative durations are ignored).
  • ui-egui integration test: ui.audio_load without an audio engine reports measured: false and every field, and the command is listed for agents.
  • Checked in the running app on macOS (built-in output, 512 and 1024 frames): Window ▸ System Usage and ui.audio_load show the live values.

cargo xtask ci passes locally (macOS). Docs: gaps.md (G-01), hardware-parity.md (new row: not yet verified on Windows, Linux or FreeBSD; not measured on the web).

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features
    • The System Usage window now shows audio callback load, including average, peak, and worst load, plus late callback counts. A load indicator updates while measurements are available.
    • Use Reset to clear audio-load statistics. When measurements aren’t available, the window indicates that audio load hasn’t been measured.
    • The ui.audio_load command provides audio-load metrics and supports resetting them.
    • Active plugin counts now include track instruments.

@coderabbitai

coderabbitai Bot commented Oct 25, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
Commits

Reviewing files that changed from the base of the PR and between 691dd3b and 58c54c0.

Files selected for processing: 3
  • crates/playback/src/lib.rs
  • crates/playback/src/stream.rs
  • crates/ui-egui/src/panels/windows.rs

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.

Walkthrough

The playback crate now measures audio callback load and exposes the measurements through Player. The System Usage window displays available measurements, and the ui.audio_load command returns them and can request a reset.

Changes: Audio Load Metering and Reporting

Layer / File(s) Summary
Callback metering and Player access
crates/playback/src/load.rs, crates/playback/src/stream.rs, crates/playback/src/lib.rs
The callback records render time and buffer duration. The load meter publishes average and peak load over 0.5 seconds and tracks worst load, late callbacks, callback count, and buffer duration. Player provides snapshot and reset methods. Silent-clock players initialize an unmeasured meter. Tests cover publication, reset, and invalid timing values.
System Usage and command reporting
crates/ui-egui/src/panels/windows.rs, crates/ui-egui/src/menus.rs, crates/ui-egui/tests/it/*, docs/gaps.md, docs/hardware-parity.md
The System Usage window displays load metrics and provides a reset control. The ui.audio_load command returns measurements and can request a reset. Integration tests check the command without an audio engine. Documentation records audio-load reporting and platform status. Plugin-instance totals now include track instruments.

Priority: Normal

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant AudioCallback
  participant LoadMeter
  participant Player
  participant UICommand
  participant SystemUsage
  AudioCallback->>LoadMeter: record render time and buffer duration
  SystemUsage->>Player: request audio_load snapshot
  Player->>LoadMeter: read snapshot
  LoadMeter-->>Player: return published measurements
  UICommand->>Player: retrieve or reset audio_load
  Player->>LoadMeter: read or request reset
  Player-->>UICommand: return serialized measurements
Loading

Suggested reviewers: dylanmtaylor, echelon

Merge Risk: Minimal · up to 58c54

No actionable merge-blocking risk was identified; the change is ready for normal validation.

Security architecture risk: Low · up to 58c54

The new command reports aggregate callback statistics and can request a statistics reset; it does not change playback or device settings. The inspected ownership and recovery paths keep this control separate from audio processing. Remaining uncertainty concerns end-to-end command access and platform behavior, rather than an identified security defect.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • inferred — For a caller already able to invoke the UI command interface, the new operation is bounded to reading the active Player's aggregate metrics and requesting that meter's counter reset. Its parameters provide no selector for another session, device, file, or service.

Security Findings and Attack Paths

  • inferred — The inspected new parameter-to-state path terminates at a snapshot read and an atomic reset request. It does not introduce a path from command parameters to device reconfiguration, plugin execution, or transport mutation. No security defect was established on this path.

Trust Boundaries and Controls

  • observed — The new command is registered in the existing UI command dispatcher, which SoundApp::run invokes before engine commands. The dispatcher infrastructure is unchanged relative to the PR base; this establishes reuse of that path, not independent verification of external transport authentication.

Resilience and Maintainability Implications

  • inferred — Failure containment remains separate from telemetry: guarded rendering handles panic and silencing, while the meter records after the rendering block. Reset does not clear fault flags or resume playback, and replacement prevents an old stream from publishing into the new Player's meter.
Pre-merge checks: Passed 4
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title names the two main parts of the change: an audio callback load and dropout meter in Playback, and its exposure in System Usage and through the ui.audio_load command. The changeset adds LoadM…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title names the two main parts of the change: an audio callback load and dropout meter in Playback, and its exposure in System Usage and through the ui.audio_load command. The changeset adds LoadMeter and AudioLoad in load.rs, wires the meter into the output stream callback, adds the System Usage display and reset button, and adds the audio_load_command with its integration test, so the title matches the main change.

Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

…io_load)

Each device callback is timed against how long its buffer lasts. Window >
System Usage shows the average and peak load over the last half second,
the worst load and the callbacks that ran past their buffer (dropouts);
ui.audio_load returns the same for agents. The audio thread is the only
writer (atomics, no locks or allocation). System Usage also counts
instruments as plugin instances.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sanderdewijs

Copy link
Copy Markdown
Contributor Author

Merge-order note: this PR and #1076 each pass on their own, but they interact. #1076 makes SoundApp::player private behind app.player(), and this PR still reads app.player directly (System Usage and ui.audio_load in panels/windows.rs). Git merges them without a conflict there, so the combination only fails at compile time.

@eliets2 eliets2 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First-pass review notes (automated; combined from specialist angles: tests, UI code, user experience, docs and text, API and interfaces). Please treat as suggestions and verify before acting.

Summary: This adds an audio callback load and dropout meter: the playback thread times each callback against its buffer duration and publishes average, peak, worst, late-callback count, and buffer size through atomics, shown in Window ▸ System Usage with a Reset button and exposed to agents via ui.audio_load {reset?}; it also changes System Usage to count instruments as plugin instances.

Logical units: Load measurement in crates/playback/src/load.rs; System Usage UI in crates/ui-egui/src/panels/windows.rs; ui.audio_load command; plugin-instance counting fix; unit/integration tests and docs updates.

Findings:

  • [tests] crates/playback/src/load.rs, LoadMeter::record: the guard accepts audio = INFINITY and a finite busy / audio that overflows to infinity, so buffer_ms, worst, peak, and average can be published as non-finite, and hostile_timings_are_ignored does not cover these cases. Fix: require audio.is_finite() and a finite computed load, and add those cases to the hostile-timings test.
  • [API and interfaces] crates/playback/src/load.rs, LoadMeter::reset / record: reset clears only worst/late/callbacks on the next callback, not average, peak, buffer_ms, or the in-progress LoadWindow, so a snapshot after reset:true mixes pre-reset average/peak with post-reset counters, and a reset with no following callback leaves the old snapshot visible. Fix: clear the published counters in reset() itself (or show a pending state), and pin the intended semantics with tests for average/peak and the partial window across a reset and for reset-then-snapshot before any callback.
  • [UI code] crates/ui-egui/src/panels/windows.rs, system_usage: the load bar uses load.peak.clamp(0.0, 1.0) with no text, so any overload above 100% — the dropout case this meter exists to show — renders identically to exactly 100%, and the bar shows peak while the label leads with average. Fix: label the bar as peak, add the value as the bar's text, and add an overload indication (e.g. red fill/text) when peak > 1.0 or late > 0.
  • [UI code] crates/ui-egui/src/panels/windows.rs, system_usage: ctx.request_repaint_after(...) is only called in the load.measured branch, so if the window is opened while unmeasured, "Audio load: not measured" will not refresh on its own when measurement starts. Fix: request the repaint in both branches.
  • [user experience] crates/ui-egui/src/panels/windows.rs, system_usage: the "Late callbacks (dropouts): … · worst …%" label and the Reset button are in one ui.horizontal inside a fixed 320px window, so the long label can push the button out or clip it at large counts. Fix: put the late/worst text on its own line above the button, or shorten the label.
  • [user experience] crates/ui-egui/src/panels/windows.rs, system_usage: a non-zero dropout count is rendered as plain text with no visual distinction from the normal case. Fix: highlight late > 0 with a warning color/icon so dropouts are scannable.
  • [user experience] crates/ui-egui/src/panels/windows.rs, system_usage: {:.0}% for average/peak/worst rounds low loads (e.g. 0.4%) to 0%, hiding a healthy-but-nonzero reading. Fix: use one decimal place, or adaptive precision below 10%.
  • [user experience] crates/ui-egui/src/panels/windows.rs, system_usage: the unmeasured branch only says "Audio load: not measured", with no reason. Fix: state why measurement is unavailable.
  • [tests] crates/ui-egui/tests/it/audio_load.rs, the_audio_load_is_unmeasured_without_an_audio_engine: only the no-player path is tested, with no test that reset: true returns the pre-reset values and then clears them. Fix: add that reset test.

Overlaps: None listed.

Verdict: Needs changes. The 3 most important points are: (1) non-finite load values can be published from infinite/overflowing timings; (2) reset leaves stale/mixed average, peak, and window data, including when playback is stopped; (3) the peak bar is clamped to 100% with no overload indication, hiding the dropouts the meter is meant to show.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants