Skip to content

feat: ReplyFile response Resources for NomadNet file downloads - #26

Open
rinchen wants to merge 5 commits into
ratspeak:mainfrom
rinchen:feat/reply-file-response
Open

rinchen wants to merge 5 commits into
ratspeak:mainfrom
rinchen:feat/reply-file-response

Conversation

@rinchen

@rinchen rinchen commented Sep 9, 2026

Copy link
Copy Markdown

Summary

  • Add RequestOutcome::ReplyFile so hosts can return raw file bytes as a response Resource with msgpack {"name"} metadata (Python NomadNet serve_file parity).
  • Expose pack_file_name_metadata and return Resource metadata from LinkClient::query so clients can save under the correct filename.

Test plan

  • cargo test -p rns-runtime reply_file_starts
  • cargo test -p rns-runtime pack_file_name_metadata
  • cargo check -p rns-tools

Serve raw file bytes as a response Resource with optional msgpack
filename metadata, matching Python NomadNet serve_file behavior.
File response Resources (NomadNet serve_file) carry raw bytes plus
filename metadata; expose both so clients can save under the right name.
rinchen added a commit to Colorado-Mesh/rsNomad that referenced this pull request Sep 9, 2026
Cargo.lock was regenerated against ratspeak/rsReticulum#26; the prior main
SHA lacks ReplyFile/zeroize so CI cannot resolve the lock with --locked.
rinchen added a commit to Colorado-Mesh/mesh-client that referenced this pull request Sep 9, 2026
Watch ratspeak/rsReticulum#26 and ratspeak/rsLXMF#7 via RATSPEAK_STACK_PR_ENTRIES
so update warns when CI pins can be cleared. Drop the rsNomad pin now that #7
merged, and document the stacked-PR pin workflow.
rinchen added a commit to Colorado-Mesh/mesh-client that referenced this pull request Sep 9, 2026
Follow rustfmt push on ratspeak/rsReticulum#26 so CI clones the current
PR head.
rinchen added a commit to Colorado-Mesh/mesh-client that referenced this pull request Sep 9, 2026
#966)

* feat(reticulum): Nomad file metadata, image preview, LXMF media ingest

Prefer Resource filename metadata on Nomad /file downloads, preview common
rasters in the Nomad panel, ingest FIELD_IMAGE and multi-file attachments,
and add Save/Reveal controls on chat attachment lines.

Depends on ratspeak/rsReticulum#26, Colorado-Mesh/rsNomad#7, ratspeak/rsLXMF#7.

* ci(reticulum): pin stacked Ratspeak PRs; keep rmpv for stub

Sidecar CI floated siblings to main without ReplyFile / multi-file LXMF
APIs. Pin rsReticulum#26, rsLXMF#7, and rsNomad#7 heads, and make rmpv
non-optional so stub builds can parse Nomad Resource filename metadata.

* ci(reticulum): CI-only stack pins without forcing full Vitest

Editing tests.yaml forced vitest_mode=full and failed per-shard coverage.
Load temporary ratspeak-stack-ci-pins.env from clone-ratspeak-stack.sh when
CI=true instead, and restore tests.yaml to match main.

* fix(reticulum): only reuse attachment preview bytes for matching fetchKey

Avoid saving a previous image's cached data URL under a new attachment
filename if the selection changes mid-read.

* chore(reticulum): track stacked rsReticulum/rsLXMF PRs in pnpm update

Watch ratspeak/rsReticulum#26 and ratspeak/rsLXMF#7 via RATSPEAK_STACK_PR_ENTRIES
so update warns when CI pins can be cleared. Drop the rsNomad pin now that #7
merged, and document the stacked-PR pin workflow.

* ci(reticulum): bump rsReticulum pin to ReplyFile PR tip

Follow rustfmt push on ratspeak/rsReticulum#26 so CI clones the current
PR head.

* fix(deps): floor smol-toml for audit; add Nomad/chat attachment i18n

pnpm audit was failing Code quality on GHSA-7w5x-hrqm-74c2 via markdownlint-cli2; add the missing download/save/reveal locale keys so check:i18n stays green.
@rinchen
rinchen marked this pull request as draft September 9, 2026 21:37
@rinchen
rinchen marked this pull request as ready for review September 9, 2026 21:56
NomadNet .allowed ACLs need the peer identity on catch-all request
handlers; DestinationRequest already exposed it for per-path handlers.
zevaryx pushed a commit to zevaryx/rettui that referenced this pull request Oct 7, 2026
rsReticulum moves to 1.3.0 (upstream main) with ratspeak/rsReticulum#26
merged in, which node hosting needs (file replies, their metadata and
the requester's identity). Until that pull request is merged upstream,
the submodule follows the rettui branch of zevaryx/rsReticulum
(6825661). From 1.3: path requests a busy or slow interface couldn't
take are kept and sent once it can, a failed Link's route is dropped
before recovery, Resource transfers have bounded waits, a shared
instance client attaches without a startup race, and on Windows a
process rettui starts no longer keeps its listeners open. The one
breaking change (recursive discovery internals) isn't used here.

rsLXMF moves to main, which needs rsReticulum 1.3: delivery and lxmd
changes rettui doesn't use, and a propagation node fix for messages
stored without a stamp (rettui stores them stamped).

No code changes. Checked against Python RNS 1.5.5 and LXMF 1.2.0: path
retries (52 s with the first request dropped, 12 s with the first Link
dropped), page fetch, ping, probe, direct and opportunistic messages
both ways (with ratchets), a propagated message and sync, the CLI as a
shared instance's client, and hosting a node (a page, a file with its
name, and an executable page given the visitor's identity). rsNomad's
own 104 tests pass against rsReticulum 1.3.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tv5VJUy237EdS4U5P6JwjR
@zevaryx zevaryx mentioned this pull request Oct 8, 2026
rinchen added a commit to Colorado-Mesh/mesh-client that referenced this pull request Oct 9, 2026
…m propagation fixes, translation/RRC polish, Ratspeak overlay rebase (#1198)

## Summary

This branch bundles 9 commits across five areas:

1. **Delivery reliability** — MECP emergency reports now retry until the
*network* acknowledges them (not just until the radio accepts them), and
regular chat messages auto-resend up to 3 times after a failure.
2. **MeshCore MQTT / Reticulum propagation fixes** — a reconnect loop
stuck on "attempt 1/3", and propagation sync failing forever on large
`/get` requests.
3. **Translation** — pack downloads actually work (Mozilla CDN returned
HTTP 406), and the settings UI is reorganized and clearer.
4. **RRC chat layout** — modern IRC-style columns, compact Translate
action, and no more overlapping rows / scroll jumps after translating a
line.
5. **Ratspeak upstream breakage** — rebased four sidecar overlays onto
the Rust 1.89 republish of rsReticulum / rsLXMF and forwarded rsLXST's
new Codec2 voice-audio event.

82 files changed, +4288 / −729.

## Commits

| Commit | Title |
| --- | --- |
| `0ab9fef9c` | fix: MeshCore MQTT retry escalation and oversized
Reticulum propagation /get |
| `522124acd` | feat: retry MECP until network-acknowledged; auto-resend
failed chat messages |
| `372156fcf` | fix(rrc): modern IRC column layout and compact translate
action |
| `dfbb454ad` | fix(translation): working pack downloads and clearer
translation settings |
| `cfea96aa8` | fix(rrc): re-measure only the translated row to stop
overlap and scroll jumps |
| `5c7a712fc` | fix(translation): label the toggle Hide translation
instead of Show original |
| `8fece1a86` | feat(app): collapse Translation settings and move them
below MECP |
| `a3482477c` | feat(app): collapse MECP settings and place Translation
above it |
| `aca665b73` | fix(reticulum): rebase Ratspeak overlays on new upstream
and forward Codec2 voice audio |

---

## 1. MeshCore MQTT retry escalation + oversized Reticulum propagation
`/get` (`0ab9fef9c`)

### MeshCore MQTT reconnect loop

**Problem:** On `close`, `MeshcoreMqttAdapter` measured session duration
from `lastConnected`, which was never cleared. A reconnect that dropped
*before* its own CONNACK measured from the **previous** session's
CONNACK, looked like a long stable session, and reset the retry budget —
so the adapter looped on "attempt 1/3" forever instead of escalating.

**Fix (`src/main/meshcore-mqtt-adapter.ts`):** consume the CONNACK
timestamp on close (`connectedAt = lastConnected; lastConnected =
null`). A session without its own CONNACK now measures 0s and counts
toward escalation. The debug log now reports `disconnect #N` plus `, no
CONNACK` when applicable.

**Tests:** `src/main/meshcore-mqtt-adapter.test.ts` (+80 lines) covers a
pre-CONNACK drop after a previously stable session.

### Reticulum propagation `/get` larger than the Link MDU

**Problem:** rsLXMF's `PropagationClient` sent `/get` bodies as a single
packet. With a long "haves" list the body exceeded the Link MDU,
transport rejected it as `InvalidPacket`, and every sync failed at
`PurgeRequested` — the node was never purged and sync never completed.

**Fix:**

- New Ratspeak overlay
`reticulum-sidecar/patches/rsLXMF-propagation-client-request-resource.patch`:
request bodies larger than the Link MDU are sent as a **request
Resource** (Python `Link.request` parity). Upstream:
[ratspeak/rsLXMF#8](ratspeak/rsLXMF#8).
- Apply script
`scripts/apply-rsLXMF-propagation-client-request-resource.sh` + test
`scripts/apply-rsLXMF-propagation-client-request-resource.test.mjs`;
registered in `scripts/lib/ratspeak-overlay-apply-list.sh` and
`RATSPEAK_PATCH_ENTRIES` in `scripts/update.sh` so `pnpm run update`
warns when it can be dropped. Documented in
`reticulum-sidecar/patches/README.md`.
- Sidecar (`src/stack/live.rs`, `src/stack/propagation_bridge.rs`):
client `/get` failures **after the link is up** are reported as
`propagation request failed: RequestFailed` instead of `NoLinkProof`.
- Renderer (`reticulumPropagationSync.ts`): maps that to the new
`reticulumPropagation.syncRequestFailed` message rather than the
misleading "announce recovery" hint.

---

## 2. MECP retry until network-acknowledged; auto-resend failed chat
messages (`522124acd`)

### MECP emergency reports

**Before:** `sendEmergencyText` did a live send and only fell back to
the outbox when the radio was unavailable / send threw / (Reticulum) no
receipt. Once the radio accepted the packet, the report was considered
done even if nobody on the mesh heard it.

**After:**

- `sendEmergencyText` (`src/renderer/lib/emergencySend.ts`) **always**
enqueues an `priority: 'emergency'` outbox row. The drain sends it
immediately when the radio is up.
- The drain removes the row **only after a network ACK**, awaited via
the new `src/renderer/lib/networkAckAwait.ts`:
  - **Meshtastic:** routing ACK, implicit ACK, or MQTT ack
  - **MeshCore:** DM ACK, or channel repeater "heard"
  - **Reticulum:** LXMF delivery receipt
  - **Any protocol:** a peer MECP `R01`/`B02` ACK or relay
- No attempt cap; emergency backoff is 30s → 1m → 2m → 5m.
- Per-row abort registry: **Stop retrying** cancels an in-flight ACK
wait and deletes the row. The outbox bubble shows "awaiting ACK" and a
retry countdown (`useOutboxRowAwaitingAck.ts`,
`formatRetryCountdown.ts`, `chatOutboxDrain.ts`).
- `sendTextWithOutboxFallback` keeps the old
live-send/enqueue-on-failure behaviour for incident ACKs (`'normal'`
priority so they don't compete with emergency reports). Shared enqueue
code factored into `enqueueOutboxText`.

### Auto-resend for regular messages

- New `src/renderer/lib/autoResend/` (`autoResendController.ts`,
`autoResendPolicy.ts`) + `useAutoResend` hook mounted from `App.tsx`: a
failed regular message is automatically resent **3 times** with 15s / 1m
/ 3m backoff.
- `ChatPanel` shows the retry status with a **Cancel retry** control.
- Superseded failed bubbles are removed so the chat doesn't fill with
duplicates — new IPC `db:deleteFailedOutboundMessage(protocol,
packetId)`:
- Types in `src/shared/electron-api.types.ts`, handler in
`src/main/index.ts`, preload in `src/preload/index.ts`, mock in
`vitest.electronApiMock.ts`, IPC security test updated.
- The SQL only matches `status = 'failed'` rows, so a packet-id
collision cannot delete delivered or inbound history. Sender is
validated; `packetId` goes through `safeNonNegativeInt`; protocol is
whitelisted.
- Outbox `MAX_ATTEMPTS` aligned to 1 + 3.
- New capability `hasChannelDeliveryAck` on `ProtocolCapabilities`
(Meshtastic `true`, Reticulum `true`, MeshCore `false` — MeshCore
channel floods only report companion accept, so auto-resend skips them).
- `messageStore` emits added/renamed events so the controller can track
a message across the temporary-id → packet-id rename.

### Docs

`docs/development/mecp.md`, `docs/development/emcomm.md`,
`docs/development/chat.md`, `docs/emcomm.md`, `docs/troubleshooting.md`,
`README.md`.

### Tests

`networkAckAwait.test.ts`, `autoResendController.test.ts`,
`useChatOutbox.test.ts`, `useEmergencyOutboxDrain.test.ts`,
`emergencySend.test.ts`, `ChatPanel.retryStatus.test.tsx` (incl. axe),
`messageStore.test.ts`, `protocol-capabilities.test.ts`,
`emcommSafety.contract.test.ts`.

---

## 3. RRC chat layout (`372156fcf`, `cfea96aa8`)

- **IRC column layout** (`RrcChatView.tsx`): timestamp / right-aligned
nick / message columns, so wrapped and multi-line messages hang under
the message text instead of under the nick. Nicks drop the `<angle
brackets>` and truncate with a `title` tooltip.
- **Compact Translate action** (`MessageTranslateButton.tsx`): same
compact size as Copy and Reply, so it no longer inflates row height.
- **Overlap / scroll-jump fix:** translating a line called
`virtualizer.measure()`, which reset *every* cached row height to its
estimate. Unchanged rows were never re-measured, so lines overlapped and
the scroll position jumped. Now only the affected row is re-measured,
matching Chat and MeshCore Rooms.

**Tests:** `RrcChatView.test.tsx`, `MessageTranslateButton.test.tsx`.

---

## 4. Translation (`dfbb454ad`, `5c7a712fc`, `8fece1a86`, `a3482477c`)

### Working pack downloads

Mozilla's attachment CDN answers Electron's `net.fetch` (Chromium stack)
with **HTTP 406**, so every pack download failed. Packs now download
with Node `fetch` and a `mesh-client/<version>` User-Agent
(`createTranslationPackFetch` in
`src/main/ipc/translation-handlers.ts`). LibreTranslate keeps using
`net.fetch`. Download errors now name the asset that failed
(`packManager.ts`).

### Clearer settings (`AppTranslationSection.tsx`)

- "Languages I read" (checkbox grid; the target language is always read)
and automatic translation are separate sections with their own guidance.
- The long pack list is replaced with a **"Needed for your settings"**
summary (one-click download) and a searchable per-language table with
directional buttons, progress, cancel, delete and retry.
- New helper `neededTranslationPacks(target, readLanguages)` in
`lib/translation/helpers.ts` computes the packs required to bring every
unread language into the target.
- The toggle under a translated message is now labeled **Hide
translation** instead of "Show original" — the original always stays
visible; the toggle only hides/shows the translation beneath it
(`TranslatedMessageBlock.tsx`).

### App panel layout

- Translation and MECP are now **collapsed `<details>` sections** in the
App panel (MECP keeps its red styling). Final order: Translation, then
MECP.
- Settings search entries (`lib/settingsSearchEntries/app.ts`) follow
the panel order.

**Tests:** `translation-handlers.test.ts`,
`AppTranslationSection.test.tsx`, `helpers.test.ts`,
`TranslatedMessageBlock.test.tsx`, `AppPanel.test.tsx`.

---

## 5. Ratspeak upstream breakage: overlay rebase + Codec2 voice audio
(`aca665b73`)

**Why CI failed:** On the evening of Oct 8, Ratspeak republished
rsReticulum (`0f8731b`), rsLXMF (`8bad5e2`) and rsLXST (`8acab49`) for
Rust 1.89, which collapsed many nested `if`s into let-chains. Sidecar CI
floats `origin/main`, so `clone-ratspeak-stack.sh` failed on every
sidecar job (`reply-file-query-metadata patch did not apply on
rsReticulum @ 0f8731b`), and that cascaded into "Reticulum sidecar
coverage" and "Merge coverage". This was unrelated to the feature
changes above; Vitest was green.

Overlays regenerated against the new upstream (base SHAs recorded in
`reticulum-sidecar/patches/README.md`):

- **`rsReticulum-reply-file-query-metadata.patch`** (carry of
[ratspeak/rsReticulum#26](ratspeak/rsReticulum#26),
unchanged upstream since Sept): kept the `file_spec` response-Resource
block ahead of upstream's let-chain `fetch_spec` block in
`link_manager.rs`.
- **`rsReticulum-ble-rnode-host-rssi-cache.patch`**: re-inserted the two
`remember_peripheral_host_rssi` calls into the let-chain
`resolve_ble_target` paths.
- **`rsLXMF-propagation-client-lrproof-diagnostics.patch`**: re-applied
on the let-chain LRPROOF / `handle_link_proof` code. **Behavior
change:** an invalid LRPROOF now only records `LrproofInvalid` and stays
`LinkEstablishing` instead of marking the client `Failed`. That makes
upstream's
`invalid_client_lrproof_allows_later_valid_binding_before_lrrtt` test
pass (the old overlay broke it, but CI never runs rsLXMF's tests), so a
garbage or spoofed proof can no longer kill a sync. The establish
timeout still fails it, and the sticky error is still reported to the
renderer.
- **`rsLXMF-propagation-client-request-resource.patch`** (added in this
PR): rebased on the regenerated lrproof overlay and upstream's let-chain
`cleanup()`; nested `if` in `fail_outbound_request` collapsed to satisfy
Clippy.

**rsLXST voice (`reticulum-sidecar/src/stack/voice_session.rs`):**
rsLXST now reports non-Opus profiles (native Codec2, e.g.
`BandwidthUltraLow`) as `AudioFramesReceived` and `Audio*Stream*` events
instead of the `Opus*` variants, so the `bridge_service_event` match no
longer compiled. `AudioFramesReceived` now forwards to `voice.audio`
exactly like `OpusFramesReceived`, so received audio on Codec2 calls
isn't dropped. The `Audio*` stream lifecycle events are ignored, like
their `Opus*` equivalents. The bridge test now also asserts that a
Codec2 frame reaches the dedicated audio bus.

**Verified locally** against a fresh `clone-ratspeak-stack.sh` at the
new upstream tips: all overlays apply; sidecar `cargo fmt --check`,
`cargo clippy --all-targets --features rns-stack,rns-ble,rns-rnode-tcp
-- -D warnings` and `cargo test` (797 passed); rsLXMF `lxmf-core` Clippy
and tests (579 passed); rsReticulum `rns-runtime`/`rns-tools` Clippy;
overlay apply-script Vitest tests.

---

## i18n

New/changed keys in `en/translation.json`; all 15 other locales updated
via `i18n:auto-translate`.

## Platform parity

All changes are shared across linux, darwin and win32; no OS-specific
branches were added.

## Test plan

- [x] Pre-commit (staged Vitest, ESLint, typecheck, `check:*`) passed on
each commit
- [ ] CI green
- [ ] MECP: send a report with no peer in range → row stays queued with
"awaiting ACK" and countdown; bring a peer in range → row clears on ACK;
**Stop retrying** removes the row
- [ ] Regular DM on Meshtastic with destination offline → 3 auto-resends
at ~15s/1m/3m, failed duplicates removed, **Cancel retry** stops it
- [ ] MeshCore channel message → no auto-resend (no channel delivery
ACK)
- [ ] MeshCore MQTT: broker that drops before CONNACK → attempt counter
escalates past 1/3
- [ ] Reticulum: propagation sync against a node with a large haves list
completes and purges
- [ ] Translation: download a pack from "Needed for your settings";
cancel/delete/retry in the per-language table
- [ ] RRC: translate a line in a long channel → no overlap, scroll
position stays put; wrapped lines hang under message text


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Failed chat messages now retry automatically up to three times, with a
countdown and the option to cancel; manual resend remains available.
* Emergency reports stay in the outbox and retry until network
acknowledgment. Outbox status shows acknowledgment or retry countdown,
and **Stop retrying** cancels a pending send.
* Translation settings now include searchable language packs, multi-pack
downloads, progress and retry controls, and language selection.
* Chat messages display in clearer columns, with improved row sizing
when translations expand.
* Translation and MECP settings now appear in collapsible sections,
initially closed.
  * Received voice audio is now forwarded to the voice interface.
* **Bug Fixes**
  * Propagation sync failures are now reported more clearly.
* **Documentation**
* Updated chat, emergency messaging, and troubleshooting guidance to
reflect retry behavior and controls.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant