Skip to content

Bridge ↔ firmware state divergence after bridge restart #21

Description

@BrettKinny

(observed 2026-04-29; refreshed 2026-06-03 for post-#36 architecture)

The perception store is in-memory, so restarting the service that holds it while the device is in any non-idle state (sleep, talk, story_time, security, dance) drops the last-known state and the store re-initialises with default idle.

The firmware doesn't proactively re-emit state_changed on reconnect because nothing has changed on its side — so the dashboard, perception consumers, and any state-gated logic disagree with reality until the next firmware-side transition.

Concrete repro (original, ZeroClaw-era)

2026-04-29 21:41: firmware in sleep (state arc blue), bridge restarted to deploy dotty-stackchan ae54e93, post-restart /ui/state returned idle while LED still blue.

Worse, firmware's privacy gate was still pinning cam+mic off — user-visible symptom is "Dotty is broken", identical to the earlier privacy-sleep-stuck issue.

⚠️ Architecture update (post-#36 cutover, 2026-05-19)

This issue predates the ZeroClaw cutover. The bug survived the cutover, but the moving parts moved:

  • The in-memory store moved. It was bridge.py's module-level _perception_* globals; it now lives in dotty-behaviour/perception/state.py (PerceptionState.state — a per-device dict; current_state defaults to "idle", see state.py:217).
  • The trigger changed. It's no longer systemctl restart zeroclaw-bridge (retired). The equivalent loss now happens on dotty-behaviour container restart, and equally on xiaozhi-server restart / network drop / firmware reboot (all already flagged below).
  • The pipe already exists. Firmware's Phase-4 StateManager emits state_changed; the xiaozhi relay forwards it (custom-providers/xiaozhi-patches/textMessageHandlerRegistry.py:109); dotty-behaviour mutates current_state from it (state.py:185-188). So a firmware-side re-emit flows through the existing path with zero new server-side plumbing.

Two viable fix paths

  • (a) Server-side reconcile. On dotty-behaviour startup, query firmware for current state and seed the perception store. The original MCP self.get_device_status hook lived on the retired ZeroClaw path, so (a) would need re-designing against the current stack (e.g. an admin pull through xiaozhi active_connections). Confined to dotty-stackchan.
  • (b) Firmware-side re-emit on (re)connect (recommended). On audio-channel open, re-fire state_changed with current state so every stateful consumer that lost track gets a fresh edge. More robust — also covers xiaozhi-server restart / network drop / firmware reboot. ~10 LOC firmware.

Recommend (b): single source of truth is the firmware, the cost is trivial, and post-#36 it needs no server-side changes — StateManager::emitStateChanged() and the relay→dotty-behaviour path already exist.


Migrated from local tasks.md on 2026-05-16.

Activity

  1. added
    status:activeReady to start, not blocked
    area:firmwareESP32-S3 firmware (stackchan-fw)
    area:bridgebridge.py + bridge/ dashboard service (Unraid-resident post-#36)
    safetySafety / correctness / child-safety bug
    on May 16, 2026
  2. BrettKinny commented on Jul 12, 2026

    @BrettKinny
    OwnerAuthor

    This was generated by AI during triage.

    Reproduced on the live stack: while firmware remained in security, restarting only dotty-behaviour changed /api/perception/state from the correct device state to {}.

    Draft PR #173 now atomically persists only the durable per-device mutex (current_state, last_state_change_t) in the mounted state directory. Production-shaped automated acceptance passed: inject state_changed: security → restart the container → state still reports security → inject idle → state reports idle. Transient face/sound/listening telemetry is intentionally not revived.

    Keep open until PR #173 merges; the fix restores the last observed state and does not claim knowledge of firmware transitions made while behaviour is offline.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:bridgebridge.py + bridge/ dashboard service (Unraid-resident post-#36)area:firmwareESP32-S3 firmware (stackchan-fw)safetySafety / correctness / child-safety bugstatus:activeReady to start, not blocked

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions