feat(opp-solana): drive resumable inbound dispatch; delete the depot-side terminal budget

[valthon]

Important

Must merge together with https://github.com/Wire-Network/wire-solana/pull/419.
SOLANA_MAX_CHUNK_BYTES here and MAX_CHUNK_BYTES there are one coupled protocol
constant: epoch_in requires every non-final chunk to be exactly that size, so
merging either side alone breaks multi-chunk envelopes with ChunkSizeMismatch.
The relay also cannot send dispatch_limit until the program accepts it.

Relay side of resumable OPP dispatch. Pairs with Wire-Network/wire-solana#419 —
they must land together.

What changes

• The depot-side Solana terminal-account budget is deleted. buildenv's packing gate
  and queueout's admission check were capping what could ever be sent to Solana, based
  on a depot-side estimate of the outpost's account budget. Chunking plus a per-call
  dispatch limit replaces it, so the depot no longer needs to model the outpost's
  transaction shape.
• epoch_in carries dispatch_limit, and the terminal call loops until the on-chain
  dispatched_count cursor drains. Each round greedily packs attestations while their
  account union still fits, always taking at least one so an oversized effect makes
  progress rather than wedging the epoch. Three exits: cursor drained, consensus not yet
  reached, or the cursor failed to advance (another operator is draining it).
• extract_inbound_effects() is now the authoritative indexed walk of the envelope;
  the five per-effect extractors became dedup views over it, so per-attestation account
  manifests stay aligned with attestation indices.

Packet budget

SOLANA_MAX_CHUNK_BYTES drops 672 -> 668. dispatch_limit adds 4 bytes to every
epoch_in call including data chunks, where it is ignored — that pushed a full chunk from
1230 to 1234 bytes against Solana's 1232-byte limit, so solana_types.cpp::serialize
threw host-side before any RPC. Coupled by protocol to MAX_CHUNK_BYTES in wire-solana:
non-final chunks must be exactly this size so offsets line up at
chunk_index * MAX_CHUNK_BYTES. Overhead is 562 bytes, ceiling 670; 668 restores the
2-byte margin that existed before dispatch_limit.

Adds epoch_in_full_data_chunk_fits_packet_limit, which serializes a full data chunk and
asserts it fits. This replaces the SEC-94 fixture parity test and covers changes to the
instruction's argument tuple, not just its account list — the old fixture's contract
was account-list-only, so it would not have caught this. Verified to fail at 672 and pass
at 668.

Verification

• test_outpost_solana_client_plugin — No errors detected
• e2e flow tests 13/13: https://github.com/Wire-Network/wire-platform-build-system/actions/runs/31217669964
  (built against wire-solana fc061a46 + wire-sysio 00e456ee; every other repo's
  revision identical to the prior run, so the result is attributable to this change)

[heifner]

Review of the resumable-dispatch change: 10 findings, ranked by severity. Nine are anchored inline on the diff; one sits in a file this PR does not touch and is included here.

---

Retry idempotence / fee burn — plugins/batch_operator_plugin/src/outpost_opp_job.cpp:120 (file not in this diff)

Nothing records that chunks were already staged, and _last_outbound_epoch only advances on a clean return (outpost_opp_job.cpp:120-134). Every throw path in the new, much longer delivery (deadline, program abort, decode failure, stale cursor read) leaves it unset, so the cron re-enters deliver_outbound_envelope every 15 s, which unconditionally re-sends all data chunks from offset 0 (outpost_solana_client.cpp:908-923) before reaching the terminal loop. For a wedged epoch that is an unbounded fee burn and RPC load — 4 tx/tick for the dev-026 envelope, 99 tx/tick at the 64 KiB cap — with no backoff, and the re-upload itself eats into the same 15 s budget, making the deadline-throw wedge self-reinforcing. Suggest probing the chunk_buffer PDA (or a local memo) before re-uploading.

[heifner]

Lockstep break: the program on the manifest-pinned wire-solana next has none of this contract.

Checked wire-solana next (local == origin/next, and wire-platform-manifest/default.xml pins next):

• programs/liqsol-core/src/states/opp_states.rs:54 — MAX_CHUNK_BYTES = 672, and epoch_in.rs:122-134 requires non-final chunks to be exactly 672 → every envelope > 668 bytes from this relay is rejected with ChunkSizeMismatch (6027); the epoch never assembles and SOL OPP stalls in both directions. Envelopes ≤ 668 bytes are single-chunk and still pass, so this breaks intermittently under load.
• epoch_in.rs:17 / lib.rs:1031-1047 — no dispatch_limit arg. anchor 0.31.1 uses BorshDeserialize::deserialize (trailing bytes tolerated), so on a single-chunk terminal call the 4 extra bytes are silently dropped and dispatch runs unbounded — no error, no signal, while the relay believes it throttled.
• opp_states.rs:704-710 — EpochDeliveries has no dispatched_count; no resumability exists.
• errors/opp_errors.rs tops out at 6063 — no InvalidDispatchLimit / EffectAccountMissing; missing effect accounts still log-and-skip (inbound.rs:78-90, :710-717, :803-825), so on the pinned program a missing account is a silently lost remit with the epoch then closed — the exact failure this PR exists to close.

No commit on any wire-solana ref implements the new contract (git log --all -S"dispatch_limit" is empty). This PR must land/deploy in lock-step with the companion program change, and the two new error codes should be appended at 6064/6065 (the numeric table in opp-outpost-technical-spec.md:72-105 and scripts/opp/patch-idl-errors.js both key on stable numbering).

[heifner]

settled is never seeded from the on-chain cursor — any re-drive or late delivery against a partially-drained cursor wedges the epoch permanently.

read_epoch_dispatch_progress is first called only at line 963, after the first terminal epoch_in has been submitted. So the first call of every invocation packs accounts for [0, batch_end) and claims dispatch_limit = batch_end, while the program settles [dispatched_count, dispatched_count + dispatch_limit).

Concrete wedge: an epoch needs 2 terminal rounds; round 1 settles [0,5), then the 15 s delivery deadline trips at line 932. outpost_opp_job.cpp:120-144 catches, leaves _last_outbound_epoch unset, and retries next tick — which re-enters with settled = 0, packs accounts for [0,5), and sends dispatch_limit = 5; the program dispatches [5,10) from its cursor and aborts with EffectAccountMissing per this PR's own contract. execute_tx_and_confirm throws on a failed tx (libfc solana_client.cpp:1737-1741), so the self-heal read at 963 is never reached, and every retry repeats identically: attestations 5-9 (remits/slashes) never dispatch. Same wedge via a late operator delivering after another operator partially drained (its first call sees cursor K>0), MAX_TERMINAL_ROUNDS exhaustion + the retry_pending redelivery, or a nodeop restart. Note the single-round case is benign ([K,total) ⊂ accounts for [0,total)), so this bites exactly the multi-round case the feature exists for.

Fix: seed settled = read_epoch_dispatch_progress(epoch_index).dispatched_count before the loop, and skip the loop when settled >= total_attestations (still sending one terminal call to record delivery if needed).

[heifner]

Zero-attestation path: on envelope decode failure this asserts before any terminal call — the delivery is never recorded. Regression vs. the base.

count_inbound_attestations returns 0 on ParseFromArray failure (lines 411-416); the packing while then never runs, dispatch_limit = 0, and this FC_ASSERT throws. All data chunks were already staged (line 917), but no terminal epoch_in is ever sent → EpochDeliveries never records this operator → consensus can never tip → the depot row stays PENDING_DELIVERY and the job re-uploads the whole envelope every 15 s forever. The merge base (2b74bed5a0) sent the terminal finalize unconditionally with zero extras and let effects log-and-skip on-chain.

The PR contradicts itself on this path: the extract_inbound_effects wlog (lines 429-435) still says "submitting epoch_in with no extras", and the new test's comment (test_outpost_solana_client_plugin.cpp:880-882) states "the relay submits a terminal call claiming nothing" — behavior the loop no longer performs (the test only asserts on the extractors, so nothing catches it).

A genuinely zero-attestation envelope is not producible by the depot (sysio.msgch.cpp check(included_count > 0, ...)), so decode failure is the live trigger — guard total_attestations == 0 and still send the delivery-recording terminal call.

[heifner]

Commitment mismatch: the write confirms at processed, this read defaults to confirmed, no re-poll — a stale cursor read produces false breaks.

execute_tx_and_confirm returns once the tx is processed (libfc solana_client.hpp:45, solana_client.cpp:1707-1718); get_account_info defaults to confirmed (solana_client.hpp:586) and read_epoch_dispatch_progress calls it immediately with no commitment argument and no retry (line 687). When the confirmed bank lags the processed slot, the read returns the pre-tx EpochDeliveries — or nullopt on the epoch's first delivery, i.e. zero-valued progress — so the loop breaks at 969 ("consensus not yet reached" when this very tx reached it) or 979 ("another operator is draining" when this relay advanced the cursor). Result: only the first batch dispatches per invocation and the remainder waits for a re-drive, which then hits the settled = 0 wedge.

read_inbound_envelope already pins its read commitment deliberately (lines 1006-1010); this read should pin processed (or poll until the read catches up to the write) to match.

[heifner]

Loop budgets don't fit the feature: the 15 s delivery deadline was sized for one terminal call, and MAX_TERMINAL_ROUNDS = 64 can be exceeded by a legal envelope — with exhaustion returning silently as success.

• batch_operator_plugin.cpp:38-39 gives the entire delivery (N confirmed chunk txs + 2 get_account_info per unique reserve + M rounds of confirmed tx + read) 15 s — equal to the cron period. Any envelope needing ≥ 3 rounds makes the deadline throw at line 932 the normal exit path, feeding the unseeded-cursor wedge. The deadline was not raised in this PR.
• With the depot's 16-account SVM cap deleted, MAX_ENVELOPE_BYTES = 65'536 permits ~500 small SPL swap-remits across distinct reserves (~5 attestations per 16-account round → 100+ rounds). After 64 rounds the loop falls through with no log and no error, returns last_sig as success, and strands the cursor mid-envelope with nothing scheduled to resume it (_last_outbound_epoch advances, so this operator never re-drives).

At minimum: log/throw on round exhaustion, and size MAX_TERMINAL_ROUNDS + the delivery deadline against the depot-side worst case now that the depot no longer bounds it.

[heifner]

The always-take-one rule's justification is wrong, and the 16-account budget lost its measurement test.

The comment above ("its surplus accounts log-and-skip on-chain") is false twice: (a) an oversized single attestation's accounts are all included — nothing is surplus/omitted; the real failure is transaction::serialize's 1232-byte packet assert (libfc solana_types.cpp:309-313), which throws locally on every attempt — the deterministic packer rebuilds the identical oversized tx each 15 s retry, i.e. exactly the "wedging the epoch" the comment claims is avoided; (b) per this PR's own contract, a genuinely missing account now aborts the terminal call, it does not log-and-skip.

Compounding: this PR deletes the SEC-94 fixture test that measured a terminal epoch_in with 16 dynamic accounts against the packet limit, so MAX_TERMINAL_DYNAMIC_ACCOUNTS = 16 (line 48) is now an unmeasured hand estimate — the budget counts only the extras, never the ~13 IDL-declared accounts or the compute-budget pre-ix. The new epoch_in_full_data_chunk_fits_packet_limit test covers only the data-chunk shape; please add the terminal-shape equivalent (static accounts + 16 extras ≤ 1232), which is the same failure class that test was added to prevent.

[heifner]

Design conflicts with the standing outpost-remit-is-immediate rule — the rule/spec must change in the same coordinated landing.

wire-platform-manifest/.claude/rules/outpost-remit-is-immediate.md: "every remit-class attestation in that envelope MUST trigger its on-chain transfer inline, in the same tx that processes the envelope. There is no concept of 'pending remit,' 'queued release,' 'delayed payout,' or any kind of waiting state on the outpost... the dispatch loop IS the release."

The resumable cursor is precisely a queued release: after the consensus-tipping tx, undispatched remits sit in a pending state until later terminal calls that may never come (see the unseeded-cursor and round-exhaustion findings on this file). If the cursor design is the intended direction — and the depot-side rationale in buildenv reads that way — the rule doc and wire-solana's normative opp-outpost-technical-spec.md (lines 150, 838-842, 882, and 915, which still codifies log-and-skip, 672-byte chunks, and the 5-arg epoch_in) need updating in the same landing, or every diagnosis playbook built on them will misread production.

[heifner]

queueout lost its only chain_code registration check — an authorized caller's typo now creates a permanent, unreachable READY row.

All 8 production call sites derive their code from sysio.chains (epoch:686,729; opreg:634,673,745; uwrit:349,561; reserv:153), so this is not an attack vector — but it is an ops footgun: a governance msig holding one of the authorized authorities pushing queueout directly with an unregistered code used to abort with "chain_code is not registered"; now it succeeds silently. epoch::advance only fans buildenv to registered active outposts, the PROCESSED-drain (lines 1853-1860) skips mismatched chain codes, and no prune action exists — the row is permanent sysio-pool RAM, re-walked by the Phase-1 bystatus scan on every buildenv for every outpost, every epoch.

Related cleanup if the removal stands: sysio.reserv.cpp:93-95 ("sysio.msgch::queueout fails loudly for direct callers"), sysio.msgch_tests.cpp:164-170, and sysio.reserv_tests.cpp:411-414 all still describe the deleted guard — a future reader will conclude msgch is the backstop and relax the reserv-side guards. Also buildenv's chains fetch (new lines 1648-1651) now asserts with the bare default "key not found" — worth passing the message overload.

[heifner]

The resumable-dispatch state machine itself ships untested, and the five whole-envelope extractors are now dead public surface.

The new tests cover extract_inbound_effects / count_inbound_attestations only. Nothing drives the packing loop, dispatch_limit sizing, cursor resume, the consensus_reached/stall breaks, or the 16-account budget — the two confirmed bugs in that loop (the zero-attestation FC_ASSERT and the unseeded cursor) both live in the untested region and both contradict comments in the same files, which is why they survived.

Separately: extract_inbound_recipient_pubkeys, extract_inbound_swap_remit_spl_targets, extract_inbound_swap_revert_spl_targets, extract_inbound_swap_remit_reserve_seeds, and extract_inbound_reserve_create_cancelled_seeds now have zero production callers (deliver_outbound_envelope uses extract_inbound_effects directly) but remain exported from the public header purely for tests — and the comments at outpost_solana_client.cpp:525/539 mislabel the two SPL-target views as "deduplicating", which they are not. Either test the loop through a seam and delete/relocate the views, or the dead surface plus wrong docs will drift.