MoonI80 streaming ring: 48×256 (12,288 lights) via '595 expander, dual-core prime, WDT self-heal, expert mode

.gitignore

@@ -125,3 +125,8 @@ __pycache__/
 # the site by moondeck/docs/mkdocs_hooks.py (via gen_api.py). Regenerated every build.
 /docs/moonmodules/core/moxygen/
 /docs/moonmodules/light/moxygen/
+
+# Read-only hardware snapshots (flash/NVS dumps pulled off reverse-engineered boards).
+# Root-anchored + dir-scoped, like /build/ above: a bare `.snapshots` would also swallow any
+# nested path of that name anywhere in the tree.
+/.snapshots/

CLAUDE.md

@@ -79,6 +79,8 @@ The **one exception** is `esp32/main/CMakeLists.txt`: ESP-IDF builds use IDF's b
 
 **The one nuance: a *rigorous* change gets a heads-up first.** "Flash freely" is the default for ordinary iteration (a code fix, a UI tweak, a normal reflash to see it run). But when the change is *rigorous* — it could leave a board in a bad state or is disruptive to recover from: erasing/repartitioning flash, a bootloader / partition-table / sdkconfig change, changing the flash-size variant, a first-flash of an untested board, a long full-erase, or anything that risks bricking or a bootloop — **say what you're about to do and why, and wait for the go-ahead.** The test is *reversibility*: a routine reflash is a keystroke to redo, so just do it; a change that could cost real recovery effort (or hardware) is worth one sentence of confirmation first. When unsure which side a change falls on, ask.
 
+**Invite the product owner to test — then STOP and wait.** Whenever a change produces something the product owner can *see or judge* (LEDs on a bench board, a UI screen, a rendered effect, a live device behaviour), the agent's job ends at "it's running on <board/URL>, here's what to look at" — **not** at the agent's own verdict. Say what changed, where to look, and what would count as good or bad, then **stop and wait for their observation before drawing conclusions, writing them into docs, or moving to the next step.** The product owner's eyes are the measurement; serial logs and API reads are supporting evidence, not a substitute. This is the *[Agent Roles](#agent-roles)* division made concrete: the product owner tests on hardware before approving, and an agent that races ahead — reaching a conclusion, updating a doc, starting the next task — has quietly taken that decision away from them and is often *wrong* (this rule exists because it happened repeatedly: conclusions written up from a bench the product owner never got to look at). The trigger is simply *"could the product owner see this?"* — if yes, hand it over and wait. Doubly so before anything irreversible-ish (a revert, a reconfigure, a reflash) that would destroy the very state they were about to look at: leave it running.
+
 The full gate lists per lifecycle event (commit, push, PR merge, release) live in **Lifecycle Events** below.
 
 **Mandatory subtraction.** Periodically review and remove code and docs that no longer earn their place. If nothing can be removed, justify why. This applies to `docs/backlog/` and `docs/history/` too: both grow *and shrink*. A backlog item that ships is deleted (it lives in the code now); a history entry whose lesson has been absorbed — folded into a principle, a doc, or simply internalised — gets pruned. The permanent record is the git commits; these two folders are a working narrative layered on top, kept only as long as they still earn it. Don't treat either as append-only.

docs/adr/0014-own-i80-dma-driver-below-esp-lcd.md

@@ -0,0 +1,72 @@
+# 14. Our own i80 DMA driver, one level below esp_lcd
+
+Date: 2026-07-14
+
+## Status
+
+Accepted
+
+## Context
+
+`I80LedDriver` drives parallel WS2812 output through ESP-IDF's `esp_lcd` i80 bus. It pre-encodes a whole frame and sends it as **one** `esp_lcd_panel_io_tx_color` transaction, which is gapless and correct — the peripheral streams the buffer and stops.
+
+That single-transaction design is also the driver's ceiling. The DMA must read the whole frame in one unbroken stream, so the frame has to be somewhere the DMA can sustain, in one contiguous allocation. The measured consequences:
+
+- The **74HCT595 expander** renders correctly only while its ×8 frame fits internal DMA RAM — about **96 lights per strand** on the ESP32-S3. Above that the frame lands in PSRAM and the strands garble.
+- **Parlio** caps at ~**4,096 lights**: a hardware 65,535-byte single-transfer limit, and a contiguous-block limit below that.
+- The **classic ESP32** caps at ~**2,048 lights**: its i80 backend is the I2S peripheral, whose DMA cannot address PSRAM at all.
+
+The obvious fix is to split the frame into several transactions. **We built that, and it does not work.** On a dense frame the strands flash full-brightness white; on a sparse frame the fault hides. The cause is in IDF, not in our code — every transaction starts with `lcd_start_transaction` (`esp_lcd/i80/esp_lcd_panel_io_i80.c`):
+
+```c
+lcd_ll_reset(bus->hal.dev);        // reset the LCD peripheral
+lcd_ll_fifo_reset(bus->hal.dev);   // flush the FIFO
+gdma_start(...);
+esp_rom_delay_us(4);               // hard-coded busy-wait
+lcd_ll_start(bus->hal.dev);
+```
+
+**`esp_lcd` resets the peripheral between transactions.** For an LCD panel that is harmless — a panel is addressed, not clocked continuously. For WS2812 it is fatal: the protocol is one unbroken self-clocked bit stream, and a mid-frame reset corrupts everything after it. IDF's Parlio driver resets its FIFO in the same place (`esp_driver_parlio/src/parlio_tx.c`). So **no chunking strategy inside IDF's LED-adjacent drivers can be gapless**, at any chunk size or boundary. This is why hpwit's driver — the reference implementation for this class of LED output — hand-rolls its DMA rather than using IDF's.
+
+There is, however, an opening. The LCD peripheral has **no data-length register**: `lcd_ll_set_phase_cycles()` sets `lcd_dout` as a *boolean enable*, and IDF's own comment reads *"Number of data phase cycles are controlled by DMA buffer length"*. The peripheral clocks out exactly what the DMA feeds it and stops when the chain ends. Therefore **one `gdma_start()` over an arbitrarily long descriptor chain, plus one `lcd_ll_start()`, is a single continuous gapless stream spanning as many buffers as we like.** `esp_lcd` discards that capability by re-arming per transaction; the hardware never required it.
+
+The descriptors are almost free: a 144 KB frame (16 lanes × 1,024 lights) needs 37 descriptors — 444 bytes.
+
+## Decision
+
+**Build a second i80 implementation, `MoonI80`, on IDF's HAL and GDMA link-list APIs (`lcd_ll_*`, `gdma_link_*`) — one level below `esp_lcd` — and ship it alongside the existing driver rather than replacing it.**
+
+Three parts to the decision:
+
+1. **One level below `esp_lcd`, not down to the registers.** `gdma_link_*` and the LCD HAL are the APIs IDF's own drivers are built on. We are declining `esp_lcd`'s transaction *policy*, not its abstractions. No raw register pokes.
+
+2. **The whole frame in one descriptor chain (phase 1).** We already pre-encode the frame, so the DMA can simply read it — no ISR refill, no ring, no real-time deadline, and therefore no underrun for WiFi to cause. This is strictly simpler than hpwit's design, which needs a CPU refill only because it transposes per-LED.
+
+3. **Both drivers ship.** `I80LedDriver` remains the default and the **reference implementation**: correct, memory-capped, and the thing MoonI80 is measured against. MoonI80 is the challenger. It replaces the reference only when it demonstrably beats it on the same bench. Both are registered module types, so the A/B is a swap in the UI with no reflash.
+
+An **internal-RAM ring with CPU refill** (hpwit's shape, and the only thing that can ever work on the classic ESP32) is deferred to a phase 2, and **gated on phase 1 measuring that the silicon — not `esp_lcd` — is the wall.** The ring is a superset of the same descriptor machinery (`GDMA_FINAL_LINK_TO_HEAD` closes the chain), so it is an extension, not a rewrite.
+
+## Consequences
+
+**What we gain in phase 1, precisely.** We own the descriptor chain and fire it with a single `gdma_start` + `lcd_ll_start`, so `esp_lcd`'s per-transaction re-arming (the peripheral reset that corrupts a WS2812 stream) is gone, and with it the whole `lli full` mount-failure class — the chain is mounted once, owner-checking off. **That is all phase 1 gains.** It does *not* yet lift a memory ceiling: it still mounts one contiguous frame buffer per transfer, exactly as `esp_lcd` did. And it does not touch Parlio, whose 65,535-byte and contiguous-block caps are its own peripheral's, not `esp_lcd`'s.
+
+What phase 1 *buys* is the **capability** the ceilings need: a chain we control, which phase 2 closes into a ring over small internal buffers. Only then does the frame stop needing to be one contiguous DMA-reachable block.
+
+**What we give up, and it is real.** This diverges from *[Industry standards, our own code](../../CLAUDE.md#principles)* — we are leaving a maintained IDF driver for code we own. The justification is that the maintained driver **cannot express the behaviour the hardware supports and WS2812 requires**, and that is demonstrated from IDF's source, not assumed. But we now carry: the GPIO/clock/bus setup `esp_lcd` was doing for us, the interrupt plumbing, and the risk of drifting against future IDF versions. Keeping `I80LedDriver` as the reference is the mitigation — if MoonI80 rots, the working path is still there and still default.
+
+**What phase 1 measured (2026-07-14, board B — the question this ADR was written to settle).**
+
+MoonI80 renders correctly on real hardware: the SE16 at 4,096 lights (direct, 16 lanes) and board B through the 74HCT595 expander, both confirmed by eye, with **zero GDMA mount failures** and a wire time matching the `esp_lcd` reference to within 0.15% (19,646 µs vs 19,674 µs — so our own peripheral configuration produces the same waveform).
+
+And it answered the open question, by removing the suspect rather than reasoning about it:
+
+| pixel clock | frame in PSRAM | result |
+|---|---|---|
+| **2.67 MHz** (direct) | 2,048 lights | **drives** — 7,712 µs on the wire |
+| **26.67 MHz** (expander) | *any* size — 54 KB or 144 KB | **never completes** |
+
+Same board, same PSRAM, same descriptor chain, same driver. The only variable is the clock. **The S3's GDMA cannot sustain a PSRAM read at the expander's 10× rate** — a '595 is serial-in, so each WS2812 slot is shifted out over 8 bus words, and the bus must run ten times faster to keep the slot's duration.
+
+This **kills the hypothesis that motivated the build**: the `esp_lcd` path failed with thousands of `lli full` descriptor-mount errors, which pointed hard at its descriptor handling. MoonI80 removes that mechanism entirely (own chain, mounted once, owner-checking off) — the mount errors are gone, and the transfer still never completes. The `lli full` storm was a **symptom, not the cause**. Six earlier hypotheses about this bug were proposed and refuted (see [lessons.md](../history/lessons.md)); this is the first one killed by a controlled experiment with a working control condition rather than by a story that stopped fitting.
+
+**Consequence: phase 2 is now justified by measurement, and this driver is its foundation.** The fix is the internal-RAM ring — close the chain into a ring (`GDMA_FINAL_LINK_TO_HEAD`) over small *internal* buffers and refill them from the PSRAM frame in our own EOF callback, a bulk sequential CPU read, so the DMA never reads PSRAM at the expander's clock at all. Every piece of that (our own link list, our own EOF hook, one continuous `lcd_ll_start` that is never re-armed) exists *only* because we own the DMA; `esp_lcd` can express none of it. The ring extends this machinery rather than replacing it, exactly as the decision above anticipated.

docs/adr/README.md

@@ -23,3 +23,4 @@ Agents do not read this directory automatically, only when a decision's rational
 | [0011](0011-data-exchange-pull-and-prepare-pass-not-pubsub.md) | Inter-module data/events: pull + prepare-pass, not pub/sub | Accepted |
 | [0012](0012-ha-discovery-wled-default-mqtt-opt-in.md) | HA discovery: WLED by default, MQTT discovery opt-in | Accepted |
 | [0013](0013-no-migration-code-robust-persistence-plus-documented-breaks.md) | No migration code — robust persistence + documented breaks | Accepted |
+| [0014](0014-own-i80-dma-driver-below-esp-lcd.md) | Our own i80 DMA driver, one level below esp_lcd | Accepted |

docs/backlog/backlog-core.md

@@ -481,3 +481,11 @@ Legend: **Adopt-1.0** (small, high value) · **Defer-1.x** (needs engine work or
 They are **not** CI failures (CI is Debug) and each one inspected so far is a false positive of the same shape: a bounds check exists, but GCC cannot prove the degenerate case (e.g. `HttpServerModule.cpp:2400` warns "writing 1 byte into a region of size 0" on a line *guarded* by `if (wsLen + headerLen > sizeof(hdr)) return false;`). Still worth clearing: each is either a genuinely unprovable bound (fix the code) or a bound the code knows but does not state (make it explicit). Doing it in one pass beats letting them mask a real one later.
 
 Not done with the multi-destination/tab-UI merge because 17 warnings across four core files is its own change, not a tail on someone else's.
+
+## Preview stream: tail byte-phase desync under backpressure
+
+**Symptom (board B, 2026-07-14):** on a large grid the preview's bottom region (the frame tail) shows per-pixel noise; with a SOLID effect it flickers pure R / pure G / pure B at max brightness. Pure primaries from solid content = the frame bytes read at an offset that is not a multiple of 3 — tearing alone cannot discolor identical frames, so the resumable sender's offset accounting slips. Survives a browser refresh; occurs "occasionally" (per-frame), worst when the device is network-starved.
+
+**Suspect:** `HttpServerModule::sendBufferedFrame` (the zero-copy resumable send whose body is the live producer buffer, draining across transport ticks) — the resume-after-partial-socket-write path. Verify by logging the resume offset vs bytes actually written on EWOULDBLOCK; the fix is offset accounting, plus a resync rule (a client that missed bytes gets a fresh frame header, not a phase-shifted tail — the *robust to any input* bar).
+
+**Not** the shift-register transport bug and **not** buffer corruption: the composite buffer is proven correct (Solid writes every light; the preview alone garbles).