Repository navigation
Commit 24071dd
authored
feat(nrf): hardware watchdog with reset-reason and phase breadcrumbs (#139)
* feat(nrf): hardware watchdog with reset-reason and phase breadcrumbs
nRF had no watchdog, so any unbounded wait was permanent: nrfx_spim.c's
`while (!nrf_spim_event_check(END)){}` and the six Wire_nRF52 TWIM spins
have no timeout and no yield, and checkTransferTimeouts() cannot help
because it runs FROM loop(), which is what is stuck. This makes that
class recoverable -- the residual PLAN_PHASE2_BOUND_WAITS D-L accepted
and D-K assumed unrecoverable.
Portable module, not an nRF-only one: every feed site and breadcrumb
stamp lives in code that compiles for both targets, so an nRF-only API
would mean #ifdef TARGET_NRF around ~20 call sites in shared files.
Follows the ble_transport pattern -- one header with no vendor includes,
two whole-file-gated implementations, ESP32 stubbed. Its reset-reason
decode moves out of main.cpp, a net #ifdef reduction there.
Timeout is 300 s, and the number alone is not what makes it safe. The
longest span the firmware cannot instrument is a REFRESH_FULL on a
7-colour split-buffer panel: 4 BUSY_WAIT entries x 30 s, sent to BOTH
controllers, ~240 s inside one bbepRefresh(). What keeps that from
resetting a healthy device is the feed immediately before all 15
bb_epaper entry points, so the dog faces one call rather than that call
plus everything preceding it. Margin is ~1.25x -- re-check it when
adding a panel.
Three details that are easy to get wrong:
- RESETREAS must come from readResetReason(). The core reads AND
clears the register in init() before setup(), so reading the
peripheral (or sd_power_reset_reason_get) returns zero forever and
reports every watchdog reset as a power-on.
- GPREGRET2 needs two access paths. sd_power_gpregret_* are numbered
from SOC_SVC_BASE_NOT_AVAILABLE and cannot be used before
ble.begin(); direct register access is correct while the SoftDevice
is disabled, SVCs once it is enabled.
- A running WDT cannot be stopped or reconfigured, and which resets
clear it is NOT established by anything in-tree. So inherit-detection
via RUNSTATUS runs on every build INCLUDING the disabled one, and
feeds every enabled RREN channel of whatever it finds. A disabled
build that inherited a live dog it never fed would be a brick.
Boot-loop containment: armed before the boot panel path so boot wedges
are covered, with a 3-strike counter in GPREGRET2 bits 5:4 entering a
safe mode that refuses panel work at both epdSessionAcquire and pwrmgm.
Strikes clear after 10 min of uptime rather than on a successful
refresh -- refresh-based clearing would never accumulate (every boot
refreshes) and would make safe mode permanent (safe mode never
refreshes).
Not verified on hardware. The 240 s figure is read from bb_ep.inl, not
measured; T2 in the plan is the gate before this reaches devices.
Safe mode rejects transfers late and generically -- a clean NACK needs a
"device in safe mode" code, which must originate in opendisplay-protocol.
Plan, decisions and four rounds of review findings:
docs/PLAN_NRF_HARDWARE_WATCHDOG_2026-08-01.md
* fix(nrf): watchdog debug instrumentation and reset-reason logging fix
Timeout dropped from 300s to 120s, which is now below the ~240s worst-case
REFRESH_FULL span on a 7-colour split-buffer panel -- a healthy refresh on
that panel class will trip the watchdog mid-refresh. Known gap, documented
in platformio.ini; re-check before shipping to such a panel.
Adds two new breadcrumb phases (IDLE_OFF/IDLE_WARM replacing the shared
IDLE) so a freeze while the panel session is idle can be told apart from
one during keep-alive, plus four more (PWRMGM_AXP2101/RAIL/PINS/WIRE)
instrumenting pwrmgm()'s previously-blind power-up path, added after a
watchdog reset landed there with no breadcrumb to explain why. A
phaseName() lookup makes the retained phase human-readable in the boot
log instead of a bare integer.
WDT-DEBUG-tagged od_log_debug lines pair with each EPD-session and
pwrmgm() breadcrumb (stamped first, since the log call itself can hang on
the same USB CDC mutex delay() depends on) -- grep "WDT-DEBUG" to remove
the whole set later.
Also adds a bounded (2s cap), debug-build-only wait for the USB CDC host
to reconnect before the first log line: without it, the reset-reason and
retained-breadcrumb lines -- the whole point of this feature -- reliably
lose the race against USB re-enumeration after a reset and are silently
discarded by od_log's dark-port check, which doesn't count them as drops.1 parent 4c0de5d commit 24071dd
8 files changed
Lines changed: 1237 additions & 30 deletions
Large diffs are not rendered by default.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
84 | 84 | | |
85 | 85 | | |
86 | 86 | | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
87 | 103 | | |
88 | 104 | | |
89 | 105 | | |
| |||
154 | 170 | | |
155 | 171 | | |
156 | 172 | | |
157 | | - | |
| 173 | + | |
158 | 174 | | |
159 | 175 | | |
160 | 176 | | |
| |||
184 | 200 | | |
185 | 201 | | |
186 | 202 | | |
187 | | - | |
| 203 | + | |
188 | 204 | | |
189 | 205 | | |
190 | 206 | | |
| |||
213 | 229 | | |
214 | 230 | | |
215 | 231 | | |
216 | | - | |
| 232 | + | |
217 | 233 | | |
218 | 234 | | |
219 | 235 | | |
| |||
242 | 258 | | |
243 | 259 | | |
244 | 260 | | |
245 | | - | |
| 261 | + | |
246 | 262 | | |
247 | 263 | | |
248 | 264 | | |
| |||
291 | 307 | | |
292 | 308 | | |
293 | 309 | | |
294 | | - | |
| 310 | + | |
| 311 | + | |
295 | 312 | | |
296 | 313 | | |
297 | 314 | | |
| |||
311 | 328 | | |
312 | 329 | | |
313 | 330 | | |
314 | | - | |
| 331 | + | |
| 332 | + | |
315 | 333 | | |
316 | 334 | | |
317 | 335 | | |
| |||
331 | 349 | | |
332 | 350 | | |
333 | 351 | | |
334 | | - | |
| 352 | + | |
| 353 | + | |
335 | 354 | | |
336 | 355 | | |
337 | 356 | | |
| |||
355 | 374 | | |
356 | 375 | | |
357 | 376 | | |
358 | | - | |
| 377 | + | |
359 | 378 | | |
360 | 379 | | |
361 | 380 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
| 2 | + | |
2 | 3 | | |
3 | 4 | | |
4 | 5 | | |
| |||
510 | 511 | | |
511 | 512 | | |
512 | 513 | | |
| 514 | + | |
513 | 515 | | |
514 | 516 | | |
515 | 517 | | |
| |||
1031 | 1033 | | |
1032 | 1034 | | |
1033 | 1035 | | |
| 1036 | + | |
1034 | 1037 | | |
1035 | 1038 | | |
1036 | 1039 | | |
| |||
1039 | 1042 | | |
1040 | 1043 | | |
1041 | 1044 | | |
| 1045 | + | |
1042 | 1046 | | |
1043 | 1047 | | |
1044 | 1048 | | |
| |||
1058 | 1062 | | |
1059 | 1063 | | |
1060 | 1064 | | |
| 1065 | + | |
1061 | 1066 | | |
1062 | 1067 | | |
1063 | 1068 | | |
| |||
0 commit comments