Repository navigation
Description
Context
PR #113 ripped bridge.py's voice/perception/vision/audio code but kept empty-dict stubs for the data caches the dashboard reads via getter callables:
- `_vision_cache` (perception card "vision" tile, /ui/vision/large modal)
- `_audio_cache` (perception card "audio caption" tile)
- `_scene_synthesis_cache` (perception card "scene narrative" tile)
- `_perception_state` (perception card device-state + sound-direction sparkline)
- `_identity_display_name` ("Speaking with…" hint)
- `_dashboard_last_user_line_getter` (last user utterance display)
- `_dashboard_sound_balance_series` (sound direction history)
- `_dashboard_vision_failures_last_hour` (Dashboard: Vision-failure aggregator #74 content-filter / vision failure aggregator)
The stubs are wired into the existing getter callables so the dashboard renders without crashing — but every card shows empty / "no recent data" / 0 counts.
What this issue tracks
Rewire each stubbed getter to fetch live data from dotty-behaviour (port 8090) instead of returning empty.
dotty-behaviour endpoints that already exist
- `GET /api/perception/state` — per-device state dict (drives perception card)
- `GET /api/perception/feed` — SSE stream of events (drives the live feed sidebar — already wired separately)
- `GET /api/vision/latest/{device_id}` — most recent VLM description
dotty-behaviour endpoints that DON'T yet exist (need adding)
- `GET /api/vision/cache` or `GET /api/vision/cache/{device_id}` — full vision_cache dump (description + jpeg_bytes + timestamp + source + room_match_person_id) — needed by /ui/vision/large modal
- `GET /api/audio/cache` — audio caption cache
- `GET /api/scene-synthesis/recent` — recent narrative entries
- `GET /api/perception/last-user-line` (or merge into /state) — last user utterance
- `GET /api/perception/sound-direction-series` — historical sound direction bucket counts (currently lives on bridge as in-memory; needs porting to dotty-behaviour)
- `GET /api/safety/vision-failures-recent-hour` (or similar) — for Dashboard: Vision-failure aggregator #74 content-filter aggregator
Implementation shape
Two options:
(a) HTTP fetch in bridge.py getters (Recommended for speed)
Replace each getter's empty-dict return with a synchronous `requests.get("http://localhost:8090/...\")\` call. Cache the result for a few seconds to avoid hammering dotty-behaviour on every dashboard refresh. Add timeout + circuit-breaker so a slow dotty-behaviour doesn't take down the dashboard.
(b) WebSocket / SSE push from dotty-behaviour
Have dotty-behaviour push perception updates to bridge via a long-lived connection. bridge maintains its own local cache from those pushes. More efficient but adds bidirectional state.
Path (a) matches the dashboard's existing HTMX polling model — Tier 1 of polling on each card already drives the read frequency naturally. Path (b) is over-engineered for production-of-one.
Sequencing
Best done in tiles, smallest-first, so each adds an incremental dashboard surface:
- Perception card device-state — easiest, /api/perception/state already exists
- Vision tile + /ui/vision/large modal — needs new /api/vision/cache endpoint
- Scene-synthesis tile — needs new /api/scene-synthesis/recent endpoint
- Audio caption tile — needs new /api/audio/cache endpoint
- Sound direction sparkline — needs porting in-memory series to dotty-behaviour first
- Content-filter / vision-failure aggregator (Dashboard: Vision-failure aggregator #74) — needs new /api/safety/* endpoint
Discovery
Surfaced during PR #109 bring-up: container deployed clean, dashboard reachable at /ui/, but cards render empty because the rip (PR #113) faithfully stubbed the data sources rather than re-pointing them. The stub-vs-rewire decision was the right one for #113 scope — this is the follow-up.
Activity
#115 Tile 1: rewire perception card to dotty-behaviour (#117)
#115 Tile 2: rewire vision card + /ui/vision/large modal (#118)
#115 Tiles 3 + 4: rewire scene_synthesis + audio cards (#119)
#115 Tile 5: sound-balance sparkline from dotty-behaviour (#120)
BrettKinny commentedon May 23, 2026
Closed 2026-05-24 — 5 of 6 tiles shipped via PRs #117/#118/#119/#120:
Tile 6 (#74's vision-failure aggregator) needs new failure-tracking infrastructure in dotty-behaviour — not just an exposure tile like 1-5. The dispatch/vlm.py module has clear error sentinels (
VLM_OFFLINE_SENTINEL,VLM_NETWORK_ERROR_SENTINEL) but no rolling failure log. Genuine new feature work, ~1-2 days. File a fresh issue if/when Brett wants the dashboard surface._dashboard_vision_failures_last_hour()left as the empty-dict stub in bridge.py — degrades cleanly to showing 0 in the dashboard, no crash.Vision-alignment pass: pay down post-#36 / #115 drift (#129)