Skip to content

Perception card audio + synthesis bench validation #33

Closed

Description

@BrettKinny

Phase A is merged on dotty-stackchan main (63d0cc2 + follow-ups 1fe4b52, 2464ff9, 63359e3) and deployed to the RPi 2026-04-28. Bench-verify on hardware (not yet run):

  • (a) curl -F file=@clip.wav -F question="Describe what you hear." -H "device-id: <mac>" http://<ZEROCLAW_HOST>:8080/api/audio/explain returns a sensible description and the dashboard Audio section updates within 2 s.
  • (b) The chosen AUDIO_CAPTION_MODEL (default google/gemini-2.5-flash) actually accepts the OpenAI-style input_audio content-block on OpenRouter — first call will reveal a 4xx if the model id needs to change, override via env if so.
  • (c) Cover camera 65 s, confirm the Perception card's image-age badge flips to text-warning.
  • (d) Wait 5 min with a face present, confirm tail /root/zeroclaw-bridge/logs/scene-synthesis-$(date +%Y-%m-%d).ndjson shows a line and the dashboard Environment section appears.
  • (e) Walk in front of camera, confirm face_recognized triggers a synthesis emit within ~120 s (min-interval guard).
  • (f) find /root/zeroclaw-bridge -name "*.jpg" -o -name "*.wav" -o -name "*.mp3" returns nothing (no media on disk).
  • (g) Confirm the robot-modal scene-context panel no longer duplicates the photo / Sees / Hears (per the dedupe in 63359e3) — only the recent-events ring + security cycles should show.

Migrated from local tasks.md on 2026-05-17.

Activity

  1. added
    status:activeReady to start, not blocked
    area:bridgebridge.py + bridge/ dashboard service (Unraid-resident post-#36)
    area:dashboardBridge dashboard UI / HTMX templates
    on May 16, 2026
  2. BrettKinny commented on May 23, 2026

    @BrettKinny
    OwnerAuthor

    Closed 2026-05-24 — most checks now passive-verified by the post-#36 architecture + Wave 8 rewire (#117-#120):

    • (a) /api/audio/explain works (verified during today's session against synthetic clips)
    • (c) Perception card image-age badge: card now reads live from dotty-behaviour's vision_cache (PR #115 Tile 2: rewire vision card + /ui/vision/large modal #118), age badge wired via /ui/perception
    • (d) scene-synthesis is alive in dotty-behaviour (perception scene-synthesis loop started (interval=300s min_gap=120s) in container logs)
    • (e) face_recognized triggers scene_synthesis: verified by scene_synthesis device=… reason=face_recognized log line during walk-in test (2026-05-23 20:33:02)
    • (g) Scene-context panel dedupe per 63359e3 shipped pre-Pi-runtime voice path v1 + full RPi decommission #36 and is preserved post-cutover

    The two unverified checks ((b) AUDIO_CAPTION_MODEL real OpenRouter call, (f) media-on-disk check) need a bench session — fold into #122 if they matter. Closing as substantially-verified.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:bridgebridge.py + bridge/ dashboard service (Unraid-resident post-#36)area:dashboardBridge dashboard UI / HTMX templatesstatus:activeReady to start, not blocked

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions