diff --git a/.github/workflows/rust.yml b/.github/workflows/rust.yml index 70f900138..d8ec73d4b 100644 --- a/.github/workflows/rust.yml +++ b/.github/workflows/rust.yml @@ -6,6 +6,8 @@ on: - "**/*.rs" - "**/Cargo.toml" - "Cargo.lock" + # README code blocks are compiled as doctests (hardy-btpu). + - "**/README.md" - ".github/workflows/rust.yml" - "proto/**" - "tests/**" @@ -143,10 +145,10 @@ jobs: steps: - uses: actions/checkout@v7 - - name: Setup Rust toolchain (32-bit bare-metal target) + - name: Setup Rust toolchain (32-bit bare-metal targets) uses: actions-rust-lang/setup-rust-toolchain@v1 with: - target: thumbv7em-none-eabihf + target: thumbv7em-none-eabihf, thumbv6m-none-eabi - name: Use cached dependencies and artifacts uses: Swatinem/rust-cache@v2 @@ -167,10 +169,11 @@ jobs: # critical-section impl — only final binaries need one. # TODO: extend to hardy-bpa once its --no-default-features build is # confirmed clean on this target. - - name: Build cbor (no_std, 32-bit) + - name: Build cbor + btpu (no_std, 32-bit) run: > cargo build --locked --no-default-features -p hardy-cbor + -p hardy-btpu --target thumbv7em-none-eabihf - name: Build bpv7 + eid-patterns (no_std, 32-bit) @@ -181,6 +184,16 @@ jobs: -p hardy-eid-patterns --target thumbv7em-none-eabihf + # thumbv6m-none-eabi (Cortex-M0) has no atomic compare-and-swap at all, + # so alloc::sync::Arc does not exist there and `bytes` needs + # portable-atomic, which btpu's `critical-section` feature enables. + - name: Build btpu (no_std, no atomic CAS) + run: > + cargo build --locked --no-default-features + --features hardy-btpu/critical-section + -p hardy-btpu + --target thumbv6m-none-eabi + msrv: runs-on: ubuntu-latest steps: diff --git a/Cargo.lock b/Cargo.lock index 1880b0e69..e961a43cd 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -856,6 +856,9 @@ name = "bytes" version = "1.12.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "fc652a48c352aef3ea3aed32080501cf3ef6ed5da78602a020c991775b0aff04" +dependencies = [ + "portable-atomic", +] [[package]] name = "bytes-utils" @@ -2025,6 +2028,32 @@ dependencies = [ "serde_json", ] +[[package]] +name = "hardy-btpu" +version = "0.1.0" +dependencies = [ + "bytes", + "futures", + "futures-core", + "portable-atomic", + "rand 0.10.2", + "rand_core 0.10.1", + "serde", + "serde_json", + "smallvec", + "thiserror", + "tower", +] + +[[package]] +name = "hardy-btpu-fuzz" +version = "0.0.0" +dependencies = [ + "bytes", + "hardy-btpu", + "libfuzzer-sys", +] + [[package]] name = "hardy-cbor" version = "2.0.0" diff --git a/Cargo.toml b/Cargo.toml index 70f5bb5ac..0bbb42223 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -12,6 +12,8 @@ members = [ "bpv7", "bpv7/tools", "bpv7/fuzz", + "btpu", + "btpu/fuzz", "cbor", "cbor/tools", "cbor/fuzz", diff --git a/btpu/CHANGELOG.md b/btpu/CHANGELOG.md new file mode 100644 index 000000000..ab40d0af2 --- /dev/null +++ b/btpu/CHANGELOG.md @@ -0,0 +1,17 @@ +# Changelog + +All notable changes to `hardy-btpu` are documented in this file. + +The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this crate adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). + +## [Unreleased] + +### Added +- Initial release: a `#![no_std]` + `alloc` implementation of the Bundle Transfer Protocol - Unidirectional (draft-ietf-dtn-btpu) with message framing for the FEC extension (draft-ietf-dtn-btpu-fec). The crate is a pure protocol library with no dependency on `hardy-bpa`, `hardy-bpv7`, or an async runtime; a convergence-layer crate composes it with `hardy-bpa`. +- `codec`: the wire format. `decode_pdu` is a lazy, zero-copy iterator with two-tier fault containment: a malformed message interior is skipped via its header length and iteration continues, while a framing fault stops the walk and keeps everything already parsed; an encapsulated bundle of undeterminable extent is reported with its first byte and its offset from the start of the PDU. Unknown message types and their flag bits relay byte-exact. `decode_pdu_with` takes `DecodeOptions`: FEC decoding (off by default, since the provisional FEC type values are Private Use) and a `BundleExtent` hook through which a caller that parses bundles lets the decoder delimit bare bundle frames and encapsulated bundles, trimming link padding and continuing past them per Section 7.3. Hint chains fold to one item per type while decoding, and a malformed Bundle Length hint is carried as an unknown hint rather than failing its message. `Hints` is the one-item-per-type set, latest wins, ordered by type, that the sender and receiver APIs share. +- `transfer`: `WindowSize` and `TransferId`. The wraparound-safe receive window and the sender's transfer-number allocator, which enforces the Section 5 rule by gating on the span of outstanding numbers rather than their count, are internal to the crate. +- `sender`: segmentation, PDU packing, and cancellation. A segmented transfer is one queue entry whose segments are cut from its buffered chunks as PDUs are packed, each copied straight into the PDU, so a large bundle costs a handle rather than one message per segment. `begin`, `push`, and `finish` take a bundle in chunks through a `SendHandle`, so a CLA need not buffer it whole: a segment goes out once its bytes have been pushed, or under `SegmentCutStrategy::Half` once at least half a segment's have, an overrun is refused, `is_push_ready` paces pushes against the queue bound without letting a producer wait on bytes only it would supply, an underrun at `finish` cancels the bundle, and dropping a handle does not cancel it. A transfer's window slot is released by the sender itself when its Transfer End is packed into a PDU, since a unidirectional link offers nothing to anchor an explicit completion call to; `cancel` takes any `SendId`, or a `SendHandle`, and reports whether it cancelled anything: for a transfer it frees the slot early and only queues a Transfer Cancel if part of the transfer was emitted, placing it at the front of the queue; a Bundle Message or bare frame still queued is simply removed. The queue is packed in order, except that a transfer whose next segment is waiting on its producer or does not fit the room left in the PDU is passed over, so transfers interleave as Section 4.1 permits. `SenderConfig` fixes the PDU size, window, send queue bound (`SendQueueBytes`, in bytes queued and not yet packed), segment cut strategy (`SegmentCutStrategy`), and `LinkFraming` at construction: fixed-size frames (every PDU padded) or variable-length PDUs (unpadded), the latter optionally emitting a fitting, hint-free bundle as a bare bundle frame through the same queue as everything else. `enqueue` returns a `SendId`, an opaque ID whose `kind()` says whether the bundle travels as a transfer, a Bundle Message, or a bare frame, and which `cancel` and `is_outstanding` take in place of any wire transfer number, and `next_pdu` returns a `Pdu`, or `next_pdu_with` one packed as its `NextPduOptions` say (with `flush`, waiting transfers also send what they have buffered, for a CLA whose timer says the producer has gone quiet): the packed `Bytes`, sized to its content and sharing a bare frame's buffer rather than copying it, with a `Carried` entry for every bundle it holds bytes of, flagged when it holds the last, so a CLA can report per-bundle outcomes. The list is a `CarriedList`, a `SmallVec` of four inline entries; `next_pdu_into` refills a caller-owned one, which keeps its heap buffer once it has one, and `CarriedList::with_capacity` sizes it to the documented bound for valid bundles. `SendOptions` takes caller hints as `Hints`, so a repeated type never reaches the wire and a relay can pass on a received set. `Sender`'s `Debug` summarises its configuration and queue rather than printing queued bundle bytes. +- `receiver`: reassembly, window expiry, duplicate/conflict rejection, and `reset`. `Received` carries the transfer's hints as `Hints`. `receive_pdu` is infallible: every fault and disposition is a `ReceiverEvent`, so a fault late in a PDU never discards the events before it. A transfer the receiver rejects is reported once with its `RejectReason`, and its later messages are dropped as `DropReason::Rejected` with the same reason. `receive_pdu_into` fills a caller-owned list instead, so a CLA can reuse one allocation for the event list, whose length grows with the number of messages in the PDU. Memory is bounded by construction: the mandatory `MaxTransferSize` polices each transfer's bundle bytes exactly (`TooLarge`) and budgets its per-segment and hint bookkeeping separately (`TooFragmented`); an optional `MaxSegments` replaces the per-segment charge with a direct segment-count limit, and `MaxSegments::for_link_pdu_size` derives a recommended value from the link's PDU size, so honest senders on links with PDUs under 64 bytes are not refused. `MaxRetainedBytes` bounds the state held across all in-progress transfers, rejecting the transfer that would exceed it as `ReceiverFull`; it is enforced as configured, defaults to one transfer's full allowance (2 GiB with the default 1 GiB cap), and `MaxRetainedBytes::for_transfers` derives it from a count of transfers to hold. The `ReceiverConfig` rustdoc covers sizing, and `Receiver::retained_bytes` reports the charged total for export as a metric. Segments shorter than half their PDU and retained hint values are copied out so no PDU is pinned by a fragment, and empty segments (including an empty Transfer End) are stored so a spec-conforming sender always completes. Window expiry costs only the transfers it removes, and `TransferExpired` events are reported oldest first, across the 2³² roll-over included. Delivered, cancelled, and rejected transfer numbers are remembered for the life of the window, so repeats never re-deliver or re-open a transfer; a repeat of a segment still held is reported as `DropReason::Duplicate`, so every repeat produces an event, and a Cancel for any in-window number is honoured even before its segments arrive. With FEC decoding on, a transfer that mixes core and FEC messages, or changes its FEC Instance ID, FEC Encoding ID, or FEC message form, is rejected and stays closed, as the FEC draft requires; a transfer that completes with no data is rejected like an empty Bundle Message. `ReceiverConfig` carries the window, the cap, the optional segment limit, the optional retention limit, and the FEC switch. +- Validated configuration newtypes (`PduSize`, `WindowSize`, `MaxTransferSize`, `MaxSegments`, `MaxRetainedBytes`, `SendQueueBytes`) in the `core::num::NonZero` shape (`const fn new`, `const` `get`/`MIN`/`MAX`/`DEFAULT`, `Display` and the binary, octal, and hex formats, `FromStr`, `Hash`, `Ord`, `From` to and from `NonZero*` where only non-zero is required), so invalid sizes are rejected at the edge and no constructor panics. Every `TryFrom` returns the shared `OutOfRange` error, and every `FromStr` the shared `ParseError`. Error and event types are `Clone + PartialEq + Eq`; every module with an error type exposes a `Result` alias. +- Optional features: `serde` (the configuration structs and newtypes, with validation on deserialize), `rand` (`try_from_rng` and `from_rng` constructors seeding the initial transfer number, from a fallible RNG such as `SysRng` or an infallible one), `critical-section` (builds on targets without atomic compare-and-swap, such as Cortex-M0, by moving `bytes` onto `portable-atomic`), and `tower` (`Service` for `Sender`, an infallible `Service` for `Receiver`, and a `Stream` PDU drain with waker-based backpressure that wakes every parked producer). +- Fuzz targets under `fuzz/` for the decoder (re-encode, byte-exact relay, and decode round trip) and for the receiver (delivered bundles within the cap, a PDU fault only ever last), each across the FEC and extent-hook options. diff --git a/btpu/Cargo.toml b/btpu/Cargo.toml new file mode 100644 index 000000000..3e65c6d61 --- /dev/null +++ b/btpu/Cargo.toml @@ -0,0 +1,40 @@ +[package] +name = "hardy-btpu" +description = "Bundle Transfer Protocol - Unidirectional (BTP-U) codec and transfer logic" +version = "0.1.0" +edition.workspace = true +rust-version.workspace = true +license.workspace = true +repository.workspace = true +readme = "README.md" +keywords = ["dtn", "bpv7", "btpu", "convergence-layer", "unidirectional"] +categories = ["network-programming"] + +[lib] +path = "src/lib.rs" +crate-type = ["rlib"] + +[features] +default = [] +serde = ["dep:serde"] +rand = ["dep:rand_core"] +tower = ["dep:tower", "dep:futures-core"] +# Enable this feature on targets without native atomic CAS instructions (e.g., thumbv6m / Cortex-M0). +# Requires a critical-section implementation - see https://docs.rs/portable-atomic +critical-section = ["bytes/extra-platforms", "dep:portable-atomic", "portable-atomic/critical-section"] + +[dependencies] +bytes = { version = "1", default-features = false } +thiserror = { version = "2", default-features = false } +serde = { version = "1", default-features = false, features = ["derive"], optional = true } +rand_core = { version = "0.10", default-features = false, optional = true } +tower = { version = "0.5", default-features = false, optional = true } +futures-core = { version = "0.3", default-features = false, optional = true } +portable-atomic = { version = "1", default-features = false, optional = true } +smallvec = { version = "1", features = ["const_new"] } + +[dev-dependencies] +rand = "0.10" +tower = { version = "0.5", features = ["util", "limit"] } +futures = "0.3" +serde_json = "1" diff --git a/btpu/README.md b/btpu/README.md new file mode 100644 index 000000000..50c245d81 --- /dev/null +++ b/btpu/README.md @@ -0,0 +1,81 @@ +# hardy-btpu + +A `no_std` implementation of the Bundle Transfer Protocol - Unidirectional (BTP-U): the wire codec, the transfer window, and a sender and receiver pair for one-way, frame-based links. + +Part of the [Hardy](https://github.com/ricktaylor/hardy) DTN Bundle Protocol implementation. + +## Overview + +This crate implements [draft-ietf-dtn-btpu](https://datatracker.ietf.org/doc/draft-ietf-dtn-btpu/), which carries bundles over convergence layers that offer no IP services and no return channel (broadcast radio, satellite downlinks, CCSDS frames), and frames the messages of the FEC extension in [draft-ietf-dtn-btpu-fec](https://datatracker.ietf.org/doc/draft-ietf-dtn-btpu-fec/); no FEC scheme is implemented. It is a pure protocol library, `#![no_std]` with `alloc`, with no dependency on `hardy-bpa`, `hardy-bpv7`, or an async runtime, so a convergence-layer crate drives it against the link and composes it with `hardy-bpa`. There is no BTP-U convergence layer in Hardy yet; this crate is the protocol engine one would be built on. + +Two layers are exposed, the lower usable without the upper: the `codec` module for PDU encode and decode, and the `sender` and `receiver` modules for a ready-to-use engine configured by `SenderConfig` and `ReceiverConfig`. The `transfer` module holds the types they share, `WindowSize` and `TransferId`. + +## Features + +- **Segmentation and reassembly**: bundles that fit a PDU travel as a single message; larger ones are cut into segments as PDUs are packed, filling the tail of a PDU another message started, and reassembled at the receiver, with out-of-order and repeated messages handled. +- **Streamed sending**: a CLA can push a bundle into the sender in chunks as it arrives, through `Sender::begin`, `push`, and `finish`, rather than buffering it whole; each segment goes out once its bytes have arrived (or, under `SegmentCutStrategy::Half`, once half a segment's have), a transfer still waiting on its producer is passed over rather than holding up the queue, and `SendQueueBytes` bounds the bytes held, with `is_push_ready` telling a producer when to push rather than drain without ever waiting on itself. A CLA with a timer can ask `next_pdu` to flush, sending what waiting transfers have buffered. +- **Streamed delivery**: optionally, the receiver releases a segmented bundle's bytes in order as they arrive, as `TransferStarted`, `TransferData`, and `TransferFinished` events, rather than holding the bundle until it is complete; `Receiver::refuse` lets a CLA give up on a transfer its consumer will not take. +- **Transfer window**: the Section 5 sliding window, enforced on the span of outstanding transfer numbers at the sender and with roll-over at the receiver; the sender releases a transfer's slot itself when its last segment is packed. +- **Fault containment**: receiving is infallible, every fault and policy disposition is a `ReceiverEvent`, and a malformed message is skipped via its header length without discarding the rest of the PDU. +- **Bounded memory**: a mandatory per-transfer cap on reassembled bundle size, with per-segment and hint bookkeeping budgeted separately so a flood of tiny segments cannot exhaust the receiver, and an optional per-transfer segment limit, derivable from the link's PDU size, for links with small frames, and a receiver-wide limit on retained state across all in-progress transfers (on a lossy link, size it for the whole window; see the `ReceiverConfig` docs), and a `RetentionBudget` that the receivers of one link, one per peer, and the CLA itself can share. +- **Link framing**: fixed-size padded PDUs (CCSDS-style) or variable-length PDUs (datagrams), optionally padded up to a floor such as Ethernet's 46-octet minimum payload, optionally emitting fitting bundles as bare frames on links shared with raw-bundle peers, through the same queue as everything else. +- **Forward compatibility**: unknown message types, unknown hints, and reserved flag bits relay byte-exact; the provisional FEC message types decode only when switched on. +- Feature flag: `serde` -- `Serialize`/`Deserialize` for `SenderConfig`, `ReceiverConfig`, and the validated newtypes they hold, which re-validate on deserialize. +- Feature flag: `rand` -- `try_from_rng` and `from_rng` constructors that seed the initial transfer number from an RNG, such as the operating system's `rand::rngs::SysRng`. +- Feature flag: `tower` -- `tower::Service` implementations for `Sender` and `Receiver` and a `futures_core::Stream` PDU drain, with waker-based backpressure. Requires `std` at the consumer level. +- Feature flag: `critical-section` -- builds on targets without native atomic compare-and-swap (such as `thumbv6m`, Cortex-M0) by switching `bytes` to `portable-atomic`'s critical-section fallback. Requires a `critical-section` implementation from the HAL or runtime. + +## Usage + +A loopback, with the `rand` feature enabled: bundles enqueued on a `Sender` are drained as PDUs, fed to a `Receiver`, and come back out as events. + +```rust +use bytes::Bytes; +use hardy_btpu::receiver::{Receiver, ReceiverConfig, ReceiverEvent}; +use hardy_btpu::sender::{SendOptions, Sender, SenderConfig}; +use rand::rngs::SysRng; + +// The configuration types validate their spec-defined ranges on +// construction; the defaults (1500-byte PDUs, a window of 16, fixed-size +// framing, a 1 GiB bundle-size cap) are always valid. The spec recommends +// an unpredictable initial transfer number, so the sender draws it from the +// operating system RNG (the `rand` feature); without that feature, pass one +// to `Sender::new`. +let mut sender = Sender::try_from_rng(SenderConfig::default(), &mut SysRng)?; +let mut receiver = Receiver::new(ReceiverConfig::default()); + +let bundle = Bytes::from(vec![0x9F; 4000]); // stands in for an encoded bundle +let id = sender.enqueue(bundle.clone(), SendOptions::default())?; + +// Drain PDUs to the link; here the link is the receiver. A segmented +// transfer's window slot is released when its last PDU is packed, so the +// caller has nothing to acknowledge. Each PDU lists the bundles it +// carries; the one flagged `completes` holds the bundle's last bytes. +let mut delivered = Vec::new(); +let mut sent = false; +while let Some(pdu) = sender.next_pdu() { + for event in receiver.receive_pdu(pdu.data) { + match event { + ReceiverEvent::Received { data, .. } => delivered.push(data), + _ => {} // cancellations, expiries, drops, faults: see ReceiverEvent + } + } + sent |= pdu.carried.iter().any(|c| c.id == id && c.completes); +} +assert!(sent); +assert_eq!(delivered, vec![bundle]); +# Ok::<(), Box>(()) +``` + +Wire-format details, event semantics, memory bounds, and the padding pitfalls of bare bundle frames are documented in the rustdoc. + +## Documentation + +- [Design](docs/design.md) +- [Test Coverage](docs/test_coverage_report.md) +- [Changelog](CHANGELOG.md) +- [API Documentation](https://docs.rs/hardy-btpu) + +## Licence + +Apache 2.0 -- see [LICENSE](../LICENSE) diff --git a/btpu/docs/design.md b/btpu/docs/design.md index 5a89b51fb..c6754384d 100644 --- a/btpu/docs/design.md +++ b/btpu/docs/design.md @@ -1,89 +1,349 @@ -# hardy-btpu Sender/Receiver Design (tranche 2) +# hardy-btpu Design -Design for the second tranche of work on the high-level `Sender`/`Receiver` layer of `hardy-btpu`, turning the initial implementation into the protocol engine for real convergence layers. The codec and transfer-window layers from the initial contribution are unchanged by this design. +A `no_std` protocol library for the Bundle Transfer Protocol - Unidirectional (BTP-U): the wire codec, the transfer window, and a `Sender`/`Receiver` pair that a convergence-layer crate drives against a frame-based link. -## Design Goals - -The initial crate implements [draft-ietf-dtn-btpu](https://datatracker.ietf.org/doc/draft-ietf-dtn-btpu/) correctly at the wire level, but its `Sender`/`Receiver` layer assumes a single deployment shape: fixed-size link PDUs, lossless-enough links, whole bundles in memory. This design generalises that layer to serve three concrete consumers without per-link forks: +This document describes the crate as it is. Direction that has been sketched but not built is confined to [Next steps](#next-steps-not-yet-implemented-or-agreed-in-detail) at the end, and nothing above that heading depends on it. -- **Constant-bit-rate framed links** (CCSDS-style): fixed-length PDUs, padding mandatory, blind repetition for loss protection. -- **Ethernet**: variable-length frames with a 46-octet minimum payload, padding wasteful beyond the minimum, blind repetition for loss protection. -- **QUIC datagrams** ([QUBICLE](https://datatracker.ietf.org/doc/draft-ek-dtn-qubicle/) unreliable service, [RFC 9221](https://www.rfc-editor.org/rfc/rfc9221.html)): self-delimiting datagrams where padding actively wastes congestion-window budget, and where the usable PDU size changes during a connection. - -A fourth goal comes from inside Hardy rather than from a link type: the BPA is moving to a streaming bundle pipeline (`Sink::dispatch_streamed`, storage `stream_out()`), and this layer must bound its memory use the same way — no full-bundle buffering on either the send or receive path. +## Design Goals -Two non-goals, stated explicitly because both were considered and rejected: this layer does not provide reliability (see the repetition decision below), and it does not couple to `hardy-bpa`'s streaming traits (see the integration section). The crate remains `no_std` + alloc, sans-io, runtime-free. +- A pure protocol library. The crate is `#![no_std]` + `alloc`, sans-io, and depends on neither `hardy-bpa`, `hardy-bpv7`, nor an async runtime. BTP-U's target links include CCSDS frames and broadcast radio, where the surrounding software may be embedded or a different runtime entirely, so the protocol engine must be usable from any of them and testable without one. That includes targets without atomic compare-and-swap: the crate holds nothing in an `Arc` (the receiver owns its `BundleExtent` hook in a `Box`, beside rather than inside the state it mutates), and the `critical-section` feature moves the one atomic user, `bytes`, onto `portable-atomic`. The cost is that the crate never performs I/O and never drives itself: a convergence-layer crate pumps PDUs in and out. +- Faults split by trust boundary. Everything decoded from the wire is untrusted, so processing it surfaces events, never errors. `Result` is reserved for trusted local operations, meaning constructor validation and `enqueue`. A hostile or corrupt PDU therefore cannot make the receiver "fail", only report what it dropped. +- Memory bounded by default. A unidirectional receiver gets no say in what a peer sends, so every buffer a remote peer can grow has a cap that is on unless the operator explicitly chooses otherwise, and the cap budgets what is actually retained (per-segment bookkeeping and hint bytes) separately from the payload bytes it polices. The send side has a queue depth that its admission gate (`poll_ready`, or `is_send_queue_full` for a direct caller) applies before each bundle; `enqueue` itself never refuses on depth. +- Zero-copy on the receive path, within the memory bound. Received PDUs are `Bytes`, and every segment the codec yields is a slice of the PDU it arrived in. A segment is copied out of its PDU only when keeping the view would pin far more memory than the segment is worth; reassembly copies once where a consumer needs contiguous bytes. The send path slices the enqueued bundle into segments and encodes them into a PDU buffer, which is one copy per byte sent. +- Link-agnostic sending. The sender knows only the PDU size and the link's framing discipline (fixed-size frames or variable-length PDUs). It does not know whether the link is UDP, Ethernet, or a CCSDS virtual channel. ## Architecture Overview -The CLA is a demand-driven pump between two pull interfaces: +The crate has three layers plus an optional adapter. The codec and the engine are public, and the codec is usable without the engine, so a CLA with unusual needs can stop at the codec and build the rest itself. The window layer between them is crate-internal. ```text - ingress egress - link ──► CLA ──► dispatch_streamed storage stream ──► CLA ──► link - │ (BPA pulls segments) │ - ▼ ▼ - btpu Receiver btpu Sender - (emits in-order chunks (next_pdu(max_len), - as the prefix extends) segments at pack time) + CLA crate (async, owns the socket or frame device) + │ enqueue(bundle, options) ▲ ReceiverEvent::Received, ... + ▼ │ + sender::Sender receiver::Receiver ── high-level engine + │ next_pdu() ▲ receive_pdu(Bytes) + ▼ │ + transfer::TransferNumberAllocator / TransferWindow ── Section 5 window (internal) + │ ▲ + codec::{encode_message, pad_pdu} / codec::decode_pdu_with ── wire format + │ ▲ + └────────────── link PDUs ─────┘ ``` -On egress, the link pulls: the CLA calls `next_pdu(max_len)` when the link reports send capacity (a QUIC datagram slot, a frame interval, a socket becoming writable), passing the capacity actually available *now*. On ingress, the BPA pulls: the CLA feeds received PDUs to the `Receiver`, which emits reassembled bundle data as a stream of in-order chunks that the CLA forwards into `dispatch_streamed`, backpressured by the BPA's bounded channel. Neither direction holds a complete bundle in memory. +- `codec` encodes and decodes messages, hints, and headers. Its decode path is a lazy iterator over one PDU, parameterised by `DecodeOptions`: whether to interpret the FEC extension's message types, and an optional `BundleExtent` hook through which a caller that parses bundle formats tells the decoder how long an encapsulated bundle is. +- `transfer` holds the two Section 5 window primitives, the receiver's sliding window and the sender's transfer-number allocator, which are `pub(crate)`, and the public types the engine exposes, `WindowSize` and `TransferId`. The primitives stay internal because a caller holding them would hold raw wire numbers, which is what the opaque `SendId` and `TransferId` exist to avoid, and because a CLA that builds its own engine from the codec can implement Figure 2 in a few lines. +- `sender` and `receiver` are the engine a typical CLA uses: segmentation and PDU packing on one side, reassembly and window bookkeeping on the other, each built from a configuration struct of validated newtypes. `fec` carries the FEC extension's four message types; no scheme is implemented. +- The `tower` feature wraps `Sender` and `Receiver` as `tower::Service`s and exposes the sender's PDU drain as a `Stream`, with waker-based backpressure. It is a thin adapter over the same `&mut self` methods, not a second engine. + +The CLA drives both engines. On egress it calls `enqueue` as the BPA forwards bundles and `next_pdu` whenever the link can take a frame. On ingress it hands each received PDU to `receive_pdu` and acts on the returned events. Both engines are single-owner `&mut self` state machines with no interior locking. ## Key Design Decisions -### Pack-time segmentation with per-call PDU capacity +### Receive-side faults are events, not errors + +`Receiver::receive_pdu` is infallible. Every decode fault and every policy disposition is a `ReceiverEvent` appended to the same list as the bundles that were delivered, so a fault late in a PDU never discards the events produced by the well-formed prefix before it. The alternative, a `Result` that fails the whole PDU, was rejected because a PDU legitimately carries messages from several transfers, and one bad message must not cost the others their delivery. The list's length is the peer's choice, since most messages can produce an event: a PDU packed with 8-byte Cancels for unknown transfers yields one `MessageDropped` per 8 bytes, several times the PDU's size in memory. Coalescing repeated drops into a counted event would bound it, at the price of events that no longer map one-to-one onto messages; `receive_pdu_into` instead clears and refills a list the caller keeps, as `next_pdu_into` does on the sending side, so the allocation is paid once for the largest PDU seen rather than on every PDU. -The initial implementation fixes `pdu_size` at `Sender` construction and cuts a bundle into segments eagerly inside `enqueue()`. That is the wrong binding time for two of the three target links: the QUIC datagram limit derives from the negotiated `max_datagram_frame_size` *and* the live path MTU, and can shrink mid-connection (path migration, PMTU discovery), stranding already-cut segments that no longer fit. It also forces whole-bundle ingestion, which conflicts with the streaming pipeline. +Fault containment inside the codec has two tiers, following the self-framing property of the Section 7 header and the skip-and-continue rule of Section 7.3. When a message's extent is known from its header length but its interior is malformed, `decode_pdu` yields an error for that message and resumes at the next boundary, and the receiver reports `MalformedMessage`. When the framing itself fails, because the header is truncated, the length runs past the buffer, or an encapsulated bundle of unknown extent is reached, no further boundary can be found, so the iterator stops and the receiver reports `MalformedPdu` as the last event of that PDU. `MessageIter::is_exhausted` lets the receiver tell the two apart. -Instead, `enqueue` records the bundle (or accepts it as a chunk stream) and segmentation happens in `next_pdu(max_len)`, cutting segments against the capacity offered on each call. The configured `pdu_size` becomes an upper bound and the default for callers with genuinely fixed frames. +Hints are ignorable by definition (Section 7.3), so a hint that is framed but malformed, such as a Bundle Length hint with a value length Section 9.1 does not allow, is carried as an unknown hint rather than failing the message that carries it. Only a hint chain that runs past its message is a fault. -One constraint makes this subtler than it looks: [BTP-U §6](https://datatracker.ietf.org/doc/draft-ietf-dtn-btpu/) requires any repeated Message to be an exact copy of an already emitted Message. Re-cutting a segment at a different boundary on a repeat pass would violate this. The first emission pass therefore pins each segment's byte offsets, and later passes re-cut at exactly those boundaries even if the offered capacity has changed since. Offsets are cheap to retain (a `u64` pair per segment); the segment *bytes* are not retained — see the repetition decision. +Benign dispositions are data rather than faults. A message outside the window, a repeat of a segment the transfer already holds or of a delivered, cancelled, or rejected transfer, a Cancel for a transfer that is not in progress (Section 8.4), or a message that contradicts the transfer's established segment sequence each produce `MessageDropped` with a `DropReason`, and the caller decides whether to count, log, or ignore them. Every repeat is reported, before the transfer closes as `DropReason::Duplicate` and after it with the reason it closed, so a CLA can measure the sender's repetition (Section 6); the event list stays bounded by the PDU's message count. A transfer the receiver closes itself (over the cap or segment allowance, over the receiver-wide retention limit, mixing core and FEC messages, changing its FEC configuration, or completing with no data) produces one `TransferRejected` carrying a `RejectReason`, and its later messages are dropped as `DropReason::Rejected` with the same reason. Nesting the rejection reasons inside `DropReason`, rather than repeating them there, lets the event carry only the reasons a rejection can have while a consumer still sees why a later message was dropped, and can match every drop caused by an earlier rejection with one arm. -### Padding is a policy, not a behaviour +### Encapsulated bundles are delimited by the caller or not at all -The initial `next_pdu` unconditionally pads to the full PDU size, which is correct for exactly one of the three link types. Each consumer wants a different rule, so padding becomes configuration: pad to the full PDU size (constant-bit-rate links, today's behaviour), pad only up to a minimum length (Ethernet's 46-octet floor — although in practice the NIC's own zero-fill decodes as BTP-U indefinite padding, so even this is belt-and-braces), or no padding at all (QUIC datagrams, where every padding byte spends congestion-window budget that reliable streams on the same connection are competing for). +BTP-U reserves the message-type values that begin a bundle (Section 12.1) so that a bundle in its native format can stand where a message would (Section 7.3): a bare bundle frame on a shared link, or a bundle between BTP-U messages. Both bundle formats are self-delimiting, but only to a receiver that parses them, and this crate does not. The extent therefore comes from the caller: `DecodeOptions::bundle_extent` takes a `BundleExtent`, a one-method trait (any `Fn(&[u8]) -> Option` implements it) that a CLA holding `hardy-bpv7` satisfies in a few lines. With a hook, the decoder delivers exactly the bundle's bytes wherever it finds one and continues with whatever follows, as Section 7.3 asks of a receiver that implements the format. -### Priority interleaving replaces the single FIFO +Without a hook the decoder falls back to the only rule available: a bundle found before any message has been framed is taken to run to the end of the PDU, and one found after a message is a terminal fault. The fallback is correct only when the link delivers frames at exactly the length they were sent. On a link that pads frames, such as Ethernet's 46-octet minimum payload or fixed-length CCSDS frames, the padding is delivered as bundle bytes, and a peer that follows a bare bundle with anything else loses it. This is documented on `decode_pdu`, on `Receiver::with_bundle_extent`, and on the sender's `BundleFraming::Bare`, which is the one place this crate itself emits bare frames. A BTP-U PDU has neither problem, because link zero-fill after its last message decodes as Indefinite Padding; that is one reason the sender frames fitting bundles as Bundle Messages by default. -[BTP-U §4.1](https://datatracker.ietf.org/doc/draft-ietf-dtn-btpu/) permits interleaving Transfer Messages from different Transfers precisely so a large low-priority bundle cannot block a small urgent one. The initial implementation queues all of a bundle's segments contiguously in one FIFO, which makes head-of-line blocking structural. This also mismatches Hardy's forwarding model, where `Cla::forward(queue, ...)` already expresses per-bundle queue lanes that the CLA currently has nowhere to put. +### FEC decoding is opt-in because its type values are Private Use -The replacement is per-transfer queues with a scheduler: the packer fills each PDU by pulling from the highest-priority transfer with pending messages, round-robining within a priority class. Unsegmented Bundle messages and Transfer Cancels join the scheduler as single-message pseudo-transfers so that ordering and priority apply uniformly. A useful second-order effect: when repetition is configured, round-robin scheduling naturally spreads a message's repeats across different PDUs rather than emitting them back-to-back, which is strictly better protection against bursty frame loss at no extra cost. +The FEC draft's four message types have no IANA codes yet. The crate uses 0x70..=0x73 from the Private Use range of the message-type registry as provisional values, and a decoder interprets them only when `DecodeOptions::fec` (or `ReceiverConfig::fec`) is set. Otherwise they are unknown messages like any other Private Use value and relay byte-exact, so a deployment that assigns its own meaning to 0x70 is not misparsed, and an FEC message from a peer that was never agreed on cannot open a transfer and advance the receiver's window. When enabled, FEC transfers are tracked to detect mixing and configuration changes, and their Bundle Length hint is policed like a core transfer's, but no payload is stored and no reassembly is attempted. -### Repetition is the only loss mechanism — acknowledgement feedback was rejected +### Configuration is validated newtypes, grouped by engine -An earlier draft of this design proposed an emission ledger: retain emitted messages, consume per-datagram acknowledgement/loss reports from the QUIC stack (RFC 9221 datagram frames are ack-eliciting), and re-emit exactly the messages from lost PDUs — selective repeat instead of blind repetition. +`PduSize`, `WindowSize`, `MaxTransferSize`, `MaxSegments`, `MaxRetainedBytes`, and `SendQueueBytes` each enforce their range in a `const fn new` returning `Option` and in `TryFrom`, so an invalid value is a typed error at the edge and no constructor panics. They follow the `core::num::NonZero` shape: a transparent non-zero backing, so `Option` costs nothing; `const` `get`, `MIN`, `MAX`, and (where a default exists) `DEFAULT`; `Display` and the binary, octal, and hex formats, forwarded to the integer; `FromStr`; `Hash` and `Ord`; and, for the four whose only constraint is non-zero, `From` conversions to and from the matching `NonZero` type. Every `TryFrom` returns the one crate-level `OutOfRange` error naming the value and its range, rather than its module's error enum, so a configuration failure cannot be confused with a runtime one and the module enums hold only runtime errors. `FromStr` returns a crate-level `ParseError` that is either that `OutOfRange` or the integer parse failure tagged with the value's name, for a CLA that takes settings from command-line arguments or environment variables rather than through serde. Each range carries a guarantee: a `PduSize` spans one message header to the 20-bit content-length ceiling, which is exactly the guarantee `next_pdu` needs that anything `enqueue` accepted can be drained; a `WindowSize` is the Section 5 range; a `MaxTransferSize` has no "unlimited" value because an unbounded reassembly buffer is a memory-exhaustion lever handed to the peer, so an operator who wants no limit says `usize::MAX` explicitly; a `MaxSegments` is any non-zero `u32`, since the segment index space is 32 bits; a `MaxRetainedBytes` is any non-zero `usize`, enforced as given even below one transfer's full allowance, since that allowance depends on two other fields and an operator who sets less has chosen a smaller effective bundle size. -This was rejected as building QUIC inside QUIC. QUBICLE deliberately offers both services on one connection: a bundle whose delivery matters belongs on the reliable stream service, where QUIC's loss recovery is real and mature. Ack-driven repair in the CLA reconstructs a worse ARQ one layer up — heuristic loss declaration, roughly an RTT of repair latency (by which time data on an intentionally-unreliable flow is often stale), and retention buffers that conflict with the bounded-memory goal. There is also a semantic trap: an RFC 9221 acknowledgement confirms the *packet* carrying the datagram arrived, not that anything consumed it. +`SenderConfig` and `ReceiverConfig` group the newtypes with the non-numeric settings (`LinkFraming`, the FEC switch), all with defaults, and a `Sender` or `Receiver` is built from one in a single call. Everything that changes how queued data is framed is therefore fixed before anything can be queued; there is no builder method that could be applied to a sender with bare frames already in its queue. Under the `serde` feature the structs use kebab-case keys, missing fields take their defaults, and the newtypes serialize as plain integers and re-validate on deserialize, so a configuration file cannot smuggle in an out-of-range value. -What survives is deliberately smaller: the blind repetition count is the single loss knob, set per-enqueue, and the CLA may tune it from *aggregate* link statistics — QUIC lost-packet counters, Ethernet driver stats. BTP-U §6 explicitly anticipates link-layer signalling triggering increased repetition, so this is tuning a protocol-native parameter with a statistic, not acknowledgement-driven reliability. The protocol-native escalation beyond repetition, for deployments where the multi-segment completeness cliff bites, is the [FEC extension](https://datatracker.ietf.org/doc/draft-ietf-dtn-btpu-fec/) — not acknowledgements. +### The window is enforced on span, not count -A pleasant consequence of the rejection: messages need to be retained only until their last *scheduled* emission, after which nothing is kept. Combined with pack-time segmentation, a large transfer's unsent remainder lives in the storage stream rather than in the Sender, and repeat passes re-pull chunks (re-cutting at the pinned offsets) rather than holding segments across passes. +Section 5 forbids the sender from emitting any message whose transfer number is at or below the greatest emitted minus the window size. Because numbers are allocated sequentially, that is a bound on the span of outstanding numbers, not on how many there are: if transfer 3 were released before 0, admitting 4 would push 0 out of the receiver's window even though only one transfer is active, and a reordered End for 0 would then be dropped on arrival. The allocator therefore keeps outstanding numbers in allocation order and refuses to allocate while the oldest would fall out of range. Allocation order, not numeric order, survives the 2^32 roll-over. ### The window releases itself -The initial API requires the CLA to call `complete(transfer_number)` to free a window slot, but on a unidirectional link there is no acknowledgement to anchor that call to, and `next_pdu` returns an opaque buffer, so the CLA cannot even tell when a transfer's messages have finished leaving the queue. In practice the method's argument was ignored and any call released *some* slot — an API the caller cannot use correctly. +A unidirectional link offers no acknowledgement, so there is no event a CLA could anchor an explicit "transfer complete" call to. The sender therefore releases a transfer's slot at the one moment it can observe: when the transfer's End message is packed into a PDU, after which nothing further will be emitted for it. With a single FIFO queue and no repetition, that moment is also the last emission of the transfer, so the Section 5 rule holds without any bookkeeping beyond the allocator. The `tower` adapter's `poll_ready` gates every request on the window, unsegmented bundles included, because it cannot see the request; this never deadlocks, because a full window means End messages are queued and draining the `Stream` opens it. + +`cancel` is the only other way out of the window. It discards the transfer's queued messages and queues a Transfer Cancel only if some of the transfer had already been emitted, at the front of the queue so the receiver can drop the partial transfer without waiting out the backlog (a smaller number emitted early cannot raise the greatest emitted, so Section 5 is unaffected); segments leave the queue in index order, so the first segment still being queued means the receiver never learned of the transfer. The allocator is the single owner of the outstanding set, so a duplicate or bogus `cancel` frees nothing and queues nothing. + +`cancel` takes any `SendId`, whatever the bundle's kind, so a CLA can abandon whatever the BPA withdraws without first asking how it was framed. A Bundle Message or bare frame still in the queue is removed; it travels whole, so the receiver has seen none of it and nothing is sent in its place. `cancel` returns whether it changed anything, and returns `false` once the bundle's last bytes are packed, the point at which a PDU has already reported it `completes`, so the CLA never has two outcomes for one bundle. + +### Each PDU names the bundles it carries + +A CLA that reports per-bundle outcomes to the BPA (`ForwardBundleResult::Accepted` followed by one `transfer_outcome`) needs to know when a bundle has finished leaving, and only the sender knows which bytes went into which PDU. `enqueue` therefore returns a `SendId`, and `next_pdu` returns a `Pdu` holding the packed bytes and a list of `Carried` entries: every bundle with bytes in that PDU, once each, flagged `completes` when the PDU holds its last bytes. The CLA reports a bundle sent once the PDU flagging it is written, and failed if it cancels the bundle first or a write of any PDU carrying it fails; a cancelled transfer is never flagged. A PDU can complete several bundles (a Transfer End followed by Bundle Messages) and a bundle can span many PDUs, so the signal is attached to the PDU rather than offered as a query. A query would also be racy: the CLA may pack PDU k+1 before it has written PDU k, and once a scheduler repeats messages only the sender knows which emission is the last. Nothing assumes bundles complete in enqueue order, so the list survives interleaving. + +The ID is opaque. `SendId::kind` reports how the bundle travels as a `SendKind` (`Transfer`, `Message`, or `Bare`), and `Debug` shows the number for logs, but there is no public constructor and no accessor for the number. A caller therefore names a bundle only by an ID the sender issued, and every query and command (`cancel`, `is_outstanding`) takes that ID, never a transfer number: a wire value in caller hands would invite cancelling or querying by a number read off the link, or by one the window has since reissued, and gives a CLA nothing it needs. Internally a segmented bundle's ID holds its transfer number, which the window keeps unique while it is outstanding. Bundle Messages and bare frames share one wrapping `u32` counter, so their IDs are unique while queued unless 2³² of them are queued at once. The ID is eight bytes, the size of a `u64`, and a compile-time assertion keeps it there. Almost every PDU carries a bundle, so the list is a `CarriedList` that holds four entries inline and moves to the heap only beyond that, the way `SmallVec` does; a PDU of segments lists at most two transfers plus the Bundle Messages between them, so only a PDU packing several small bundles allocates, and `next_pdu` (and the tower `Stream`) otherwise costs one allocation per PDU, the `Bytes`. `next_pdu_into` clears and refills a list the caller keeps, and a list keeps its heap buffer once it has one, writing into it even when the entries would fit inline, so a drain loop stops allocating for the list once it has grown to its largest PDU. The bound that makes pre-sizing possible comes from the smallest valid BPv7 bundle, 28 bytes (a 20-byte primary block with its mandatory CRC and `dtn:none` EIDs, an empty payload block, and the array around them): each Bundle Message is at least 32 bytes, and transfers are packed in queue order, so at most two share a PDU, one ending and one starting in its tail with a segment of at least half the PDU; on a PDU of 52 bytes or more that segment is at least 32 bytes too, so a PDU carries at most `pdu_size / 32 + 1` entries, and `CarriedList::with_capacity` of that never grows. The sender does not parse bundles, so shorter enqueued data can exceed the bound; the list then grows rather than failing. A fixed per-PDU entry cap, closing a PDU early at N entries in the style of `IOV_MAX`, was rejected because it leaves link capacity unused, as padding under fixed-size framing, exactly when bundles are small. + +### Segments are cut at pack time against a fixed PDU size + +`Sender::enqueue` and `Sender::begin` decide only how a bundle travels. One that fits in a PDU, hints included, becomes a single Bundle Message and takes no window slot. Otherwise a transfer number is allocated and the transfer joins the one send queue as a single entry holding the bundle's chunks and a cursor. Its segments are cut in `next_pdu`, from the cursor, when the transfer supplies the next message: each segment's header is encoded into the PDU buffer and its data copied in after it from the chunks, so a segment spanning two chunks is not gathered first, and the send path copies each byte once. The PDU buffer is sized from the messages it is about to pack. So a queued gibibyte costs one entry and a `Bytes` handle, not 720 thousand messages, and the segmentation work is spread across the drain. Sizing is checked before the transfer number is taken, so a PDU too small to carry a segment never disturbs the number sequence. Because every full segment is sized against the PDU, and the PDU size never changes, the first entry able to supply a message always fits it in an empty PDU; `next_pdu` relies on this to make progress. + +A segment carries what remains of the bundle up to its capacity, the `PduSize` less its framing (and, for segment 0, less its hints). When the PDU already holds another message and the segment does not fit the room left, it is cut short to fill that tail, as the draft's Appendix A.1 shows, but only if the tail holds at least half the segment's capacity; otherwise the transfer is passed over for this PDU (see [A transfer that cannot supply is passed over](#a-transfer-that-cannot-supply-is-passed-over)). Every segment but the last therefore fills at least half a PDU, which is the size a receiver keeps as a view rather than copying (see [Receiver memory is bounded by what is retained](#receiver-memory-is-bounded-by-what-is-retained)), and the segment count is at most twice the full-size count, inside the fourfold allowance of `MaxSegments::for_link_pdu_size`. The Segment Index does not wrap: it runs from zero to the final index the End carries (Section 4), so a transfer has at most 2^32 segments. `enqueue` and `begin` refuse a bundle if its doubled count could reach a final index of `u32::MAX`, which the receiver treats as a count of segments it can never hold; that is about 2.9 TiB at the default PDU size, and a receiver's `MaxTransferSize` is far lower in practice. + +The first copy of a segment fixes its boundaries. A segment with copies still to emit keeps its length at the transfer's cursor and goes out at that length or not at all, so every copy is byte-identical as Section 6 requires. The count is one until repetition is implemented, and the cursor, the segment 0 hints, and, for an End, the window slot advance or are released only with the last copy (see [Repetition as the only loss mechanism](#repetition-as-the-only-loss-mechanism)). + +The PDU size being fixed for the life of the sender is deliberate. Varying it per `next_pdu` call was considered for QUIC datagrams and dropped: a QUIC convergence layer uses one PDU size below the 1200-byte payload every QUIC path must carry (RFC 9000 Section 14). The design has a known limit: a transfer that can supply its next segment goes out ahead of everything queued behind it, so a large bundle's segments hold up the bundles queued after it. That limit is the subject of [Priority interleaving](#priority-interleaving-in-place-of-the-single-fifo); nothing in the current API promises otherwise. + +Caller-supplied hints ride the Bundle Message or the first segment, since hints are transfer-scoped (Section 7.2). Both ends use one type for them, `codec::hint::Hints`: at most one item per hint type, the latest inserted winning, with a well-formed Bundle Length held inline and the other items sorted by type. `Received` delivers one and `SendOptions` takes one, so a relay passes a received set straight to its sender; a repeated type is collapsed before it reaches the wire rather than sent for the receiver to discard; and a delivered transfer that carried only the Bundle Length allocates nothing for its hints. The codec's `Message` types keep `Vec`, a plain view of the wire in its own order. The Bundle Length hint (Section 9.1) is always derived by the sender and a caller-supplied one is discarded, because the receiver uses it to reject oversized transfers early and the sender is the only party that knows the truthful value. An empty bundle is refused at `enqueue`: a Bundle Message's content must be a valid bundle (Section 8.1), and an empty one would also be the one message no minimum-size PDU could ever drain. + +### A bundle can be pushed in chunks through a send handle + +A CLA that receives a bundle as a stream, as `Cla::forward` supplies it, pushes the stream into the sender as it arrives instead of buffering the whole bundle first. + +```rust +impl Sender { + pub fn begin(&mut self, total_len: usize, options: SendOptions) -> Result; + pub fn push(&mut self, handle: &mut SendHandle, chunk: Bytes) -> Result<()>; + pub fn finish(&mut self, handle: SendHandle) -> Result; + pub fn cancel(&mut self, id: impl Into) -> bool; +} + +impl From for SendId { /* ... */ } +``` + +The handle is a token, not a borrow of the sender. A CLA drains `next_pdu` between pushes, usually from another task through a mutex, so it cannot hold `&mut Sender` for the life of a bundle. `SendHandle` is `#[must_use]` and not `Clone`. It counts the bytes pushed through it, which is why `push` takes it by `&mut`: an overrun is refused before the sender is searched, and `finish` detects an underrun without a lookup. + +Dropping a handle does not cancel its bundle. The token cannot reach the sender, and making it do so would need state shared between the two, an `Arc` with a lock or atomics, which the crate avoids for targets without atomic compare-and-swap. The CLA cancels an abandoned bundle explicitly with `cancel(handle)`; until then it supplies nothing and, if segmented, holds its window slot. `cancel` takes anything that converts into a `SendId`, and `From` consumes the handle, so `cancel(handle)` and `finish(handle)` end a handle the same way, while `cancel(id)` still serves a bundle from `enqueue`, or one whose handle has been finished but whose bytes are still queued. A CLA that keeps the sender in `Arc>` can wrap the handle in a guard that cancels on drop in a few lines of its own. The `SendHandle` docs say this first. + +`total_len` is required, cannot be zero (`Error::Empty`), and `Cla::forward` always supplies it. It decides at `begin` how the bundle travels, by the same rules and refusals as `enqueue`, and it supplies the Bundle Length hint for segment 0. A segmented bundle takes its transfer number and joins the queue at `begin`, so the window counts it from then and a slow producer holds its slot for as long as it pushes, which is the Section 5 span rule working as intended. A bundle that fits travels as a Bundle Message or bare frame: its chunks are held aside and gathered into one buffer, at most one PDU's worth, and it joins the queue when its last byte is pushed, so such bundles go out in the order they complete. Under `BundleFraming::Bare`, `begin` cannot see the bundle's first byte, so a fitting, hint-free bundle is begun as a bare frame, and a first chunk that does not start with a bundle-reserved byte is refused with `Error::NotABundle`; `enqueue` sees the byte and frames such data as a Bundle Message instead. + +The bundle is complete once `total_len` bytes have been pushed; the End is cut from them without waiting for `finish`. A push that would go past `total_len` fails with `Error::Overrun` and leaves the bundle as it was. `finish` consumes the handle and checks that every byte arrived; if some did not, it cancels the bundle as `cancel` would and returns `Error::Underrun`. A push for a bundle cancelled by its ID fails with `Error::NotInProgress`. An empty chunk is accepted and changes nothing. `enqueue(data, options)` is `begin`, one `push`, and `finish` in one call. + +A segment is cut only once the transfer holds the bytes it would carry, so a slow producer does not multiply the segment count. `SenderConfig::segment_cut_strategy` relaxes this: under `SegmentCutStrategy::Half` a segment is cut once at least half a full segment's bytes are pushed, carrying what has arrived. A producer whose chunks are one and a half PDUs then has each chunk sent as a full segment and a shorter one and released at once, rather than holding half a segment until the next chunk. It costs more segments, PDUs that go out part-filled, and segments short enough that a receiver copies rather than shares them. The cut never goes below half a segment, because that is what keeps every segment but the last at least half a PDU and the count within twice the full-size count; a lower threshold would break the `MaxSegments` allowance and the 32-bit index check above. The strategy is an enum, `Full` by default, so it stays within the range those bounds allow. A transfer whose next segment is waiting on its producer supplies nothing and is passed over (see [A transfer that cannot supply is passed over](#a-transfer-that-cannot-supply-is-passed-over)); `next_pdu` returns `None` when every queued entry is such a transfer, even though `has_pending` is true. + +`SendQueueBytes` bounds the bytes queued and not yet packed, counting pushed chunks and whole bundles alike, so one queue entry that stands for a bundle of any size is still bounded. A segment's bytes leave the count when its last copy is packed. It replaced a bound on queue entries under a new name rather than new units under the old one. `SenderConfig` does not deny unknown fields, so a configuration file written for an earlier build that still says `send-queue-depth` silently gets the new default. The gate is applied as before, by `is_send_queue_full` and `poll_ready`, and neither `enqueue` nor `push` refuses on it. A producer gates `begin` and `enqueue` on it, and each push of a bundle already begun on `is_push_ready(&handle)`. Gating pushes on the full queue alone could deadlock: when every bundle in the queue is waiting on its producer and their bytes have reached the bound, the drain has nothing to send and the producer waits on bytes only it would supply. A bound smaller than one segment does this on the first push. `is_push_ready` is true below the bound, and past it while the handle's bundle cannot go out without more bytes: a transfer whose next segment is waiting, or a bundle that fits one PDU and is not yet whole. A bundle holds less than a segment when such a push is admitted, so the queue exceeds its bound by less than one segment and one chunk per bundle in progress. This is TCP's not-sent low-water mark with the guarantee of progress that TCP gets from byte-granular sending and its cork timer; the sender has neither, so the exception for a waiting bundle provides it without a clock. Nothing wakes a producer when a push becomes ready; it checks again after draining, and the `tower` adapter, which takes whole bundles, does not use it. + +### A transfer that cannot supply is passed over -Instead, the Sender releases a transfer's slot when the last scheduled emission of its Transfer End is packed, and `next_pdu`'s return value reports which transfers drained in that PDU so the CLA can surface completion upward. `complete()` is removed rather than repaired; `cancel()` remains, is a no-op for unknown transfer numbers, and frees a slot only for a transfer that was actually active. +`next_source` scans the queue in order and takes the first entry that can supply a message into the room left. A transfer that cannot, because its next segment is waiting on its producer or does not fit the room (see [Segments are cut at pack time against a fixed PDU size](#segments-are-cut-at-pack-time-against-a-fixed-pdu-size)), is passed over, so messages of different transfers share PDUs, first ready first, as Section 4.1 permits. Any other entry that cannot supply ends the PDU: a Bundle Message too long for the room is not overtaken by a shorter one behind it, so bundles are not reordered to fill PDUs. A Transfer Cancel still goes to the front of the queue. Emission order cannot break Section 5, because the allocator keeps every outstanding number within the window of the newest allocated, whichever transfer supplies the next message. + +Neither reason a transfer is passed over can clear while one PDU is packed, since pushes happen between calls and the room only shrinks, so the entries chosen for a PDU advance through the queue. A transfer supplies at most one segment to a PDU, since a cut either fills the room left, ends the transfer, or under `Half` takes every byte buffered, so each transfer has at most one `Carried` entry. The scan passes over at most the outstanding transfers, so it is bounded by the window size. It is the point where a priority scheduler would plug in later (see [Priority interleaving in place of the single FIFO](#priority-interleaving-in-place-of-the-single-fifo)). + +A PDU can now name several transfers, each ending in it with a short End. The bound behind `CarriedList::with_capacity` is `pdu_size / 32 + window_size`: Bundle Messages of valid bundles are at least 32 bytes, and at most `window_size` transfers are outstanding. It is loose for ordinary traffic, where at most two transfers share a PDU, and the inline size of four is unchanged. + +### A CLA with a clock can flush waiting transfers + +`next_pdu_with` and `next_pdu_into_with` take a `NextPduOptions`, a `Copy` struct of per-call packing choices, and `next_pdu` and `next_pdu_into` pack with its default, which is what most calls want; policy that holds for every PDU stays in `SenderConfig`. Its one field so far is `flush`. A flush packs the PDU as usual, then lets each transfer whose next segment is waiting on its producer, in queue order, cut a segment of what it has buffered into the room left, at least one data byte. It stops at the first queued entry that is not a transfer, as the normal packing does, so bundles are not reordered. A bundle that fits one PDU is queued only when whole, so it is never flushed. + +The sender has no clock, so it never cuts below the `SegmentCutStrategy` by itself. Flushing on every call would make the strategy meaningless, since the `tower` drain is woken by every push, and would give up the half-segment floor that the `MaxSegments` allowance and the index check rely on. The CLA holds the clock, so it decides: it flushes when its producer has been quiet long enough, as TCP's cork timer does, or when a fixed-rate link offers a slot that would otherwise carry only padding. Frequent flushes raise a bundle's segment count, and with it what the receiver must allow. Because a flushed segment can be a single byte, the index check moves to each flushed cut: one is refused if the rest of the bundle, cut normally from then on at one index per half segment at most, could need index `u32::MAX`. + +A transfer packed normally is left either no room, since a full segment fills a PDU, or, under `SegmentCutStrategy::Half`, no buffered bytes, so it is never also flushed in the same PDU, and the PDU still names each transfer once. The `tower` `Stream` packs with the default options. A flush is not a substitute for `is_push_ready`: it lets a CLA send what is buffered, but a producer that stops pushing because the queue is full still has to be told that its own push is the way forward. + +### Link framing is configuration, and bare bundles go through the sender + +BTP-U links differ in whether a PDU must be a fixed size. `LinkFraming::FixedSize`, the default, pads every PDU to the `PduSize` with Definite Padding, which is what CCSDS-style frames need. `LinkFraming::Variable` treats the `PduSize` as a ceiling and pads a PDU only up to its `min_pdu_len`, 0 by default (as `LinkFraming::variable` builds it), which is what a datagram link wants. Ethernet sets the floor to 46, its minimum frame payload: Section 3.1 of the Ethernet draft asks a sender to pad short PDUs with Definite Padding so the receiver never sees the MAC's own padding, and 46 is correct with or without an 802.1Q tag. On any datagram link, QUIC included, a floor also hides the size of small messages from an observer, such as a lone Transfer Cancel or a short Bundle Message, at the cost of the padding bytes, while `FixedSize` hides every PDU's size at the cost of padding them all. A floor above the `PduSize` pads every PDU to the `PduSize`; it is capped rather than refused, so a plain `usize` serves. A bare bundle frame cannot be padded, because the padding would be read as bundle bytes, so under `BundleFraming::Bare` a bundle shorter than the floor goes out as a Bundle Message instead. That costs nothing on Ethernet, where a bundle under 46 bytes and its 4-byte header still fit in the 46 bytes the frame occupies anyway, and it spares the CLA from tracking which bundles it may send bare. A receiver gains nothing it can rely on from this, since its peer may be another implementation, and still needs the `BundleExtent` hook to delimit a short bare bundle. + +A variable-length link may additionally be shared with peers that speak raw bundles, and BTP-U reserves the message-type values that collide with a bundle's first byte (Section 12.1) precisely so a receiver can tell the two apart. `BundleFraming::Bare` lets the sender emit a fitting, hint-free bundle as its own bytes with no BTP-U header. This happens inside the sender's queue, not around it. A bare frame is queued behind whatever preceded it, counts against the `SendQueueBytes`, is handed out by `next_pdu` as the enqueued buffer itself rather than a copy, and is visible to whatever schedules that queue. Writing bare bundles to the link directly would let them race and starve the transfers the sender is pacing, and would hide them from any future prioritisation. Bare framing is only suitable for links that deliver frames at exactly the sent length, or that pad only up to a minimum the `min_pdu_len` floor covers, for the reason given under encapsulated bundles above. + +### Receiver memory is bounded by what is retained + +The receiver holds every segment of an in-progress transfer until it completes, so the cap on a transfer is the cap on memory, and it is enforced as data arrives rather than after reassembly. `MaxTransferSize` is applied as two separate conditions, and a transfer must meet both. The first is a policy on the bundle's true length: a transfer is rejected as `TooLarge` the moment the segment bytes it has stored exceed the cap, or earlier if the sender's Bundle Length hint already promises an oversized bundle. The second is an allowance for segmentation, rejected as `TooFragmented`, and it takes one of two forms. By default each stored segment is charged `SEGMENT_OVERHEAD` (64 bytes, standing in for the `Bytes` handle and map entry, a round-down of the 70 to 100 bytes they cost) and each retained hint its list entry (`HINT_OVERHEAD`, 40 bytes on a 64-bit target) plus its value, except a well-formed Bundle Length, which is held inline and charged nothing; a transfer whose bookkeeping alone exceeds the cap (or `MIN_OVERHEAD_BUDGET`, 4 KiB, when the cap is smaller than that) is rejected. A sender using PDU-sized segments pays a few percent of bookkeeping and never approaches the budget, while a flood of empty or one-byte segments is bounded to roughly `max_transfer_size / 64` entries instead of `max_transfer_size` of them, and a peer filling every hint type with 255-byte values is bounded the same way. When both fail, `TooLarge` is reported. A rejected transfer's number is remembered so a repeat cannot re-create it. + +The per-segment charge assumes segments of at least 64 bytes. On a link whose PDUs are smaller than that, an honest sender's cap-sized bundle needs more than `max_transfer_size / 64` segments and is refused as `TooFragmented`. The receiver cannot infer the sender's segment size from what arrives, since a hostile peer chooses it, so `ReceiverConfig::max_segments_per_transfer` lets the operator set the segment count directly. With a `MaxSegments` set, the per-segment charge is replaced by that limit on the number of distinct segments; a segment over the limit rejects the transfer without being stored, and repeats of stored segments never count. Hint bytes are still charged against the bookkeeping budget. The knob names the quantity it limits rather than deriving it from a link property, because the limit is what sets memory: every stored segment, empty ones included, costs roughly 70 to 100 bytes of heap, so an operator can read the per-transfer cost straight off the value. `MaxSegments::for_link_pdu_size` computes a recommended value from the link's PDU size and the cap: four times the count a cap-sized bundle needs when every segment fills a PDU of that size less 12 bytes of message framing, never fewer than 64. The factor of four leaves room for a sender that interleaves transfers or cuts short segments to fill PDU tails. A small PDU size with a large cap yields a large limit (about 83 million segments for 64-byte PDUs and a 1 GiB cap), which is exactly the memory an honest sender on such a link needs; the operator should see that number and lower the cap if it is too much. + +Two copies keep the charged figures honest. A segment shorter than half its PDU is copied out of the PDU rather than kept as a view, because a view keeps the whole PDU allocation alive and a one-byte segment per 1500-byte frame would otherwise retain fifteen hundred bytes per counted byte; segments of at least half a PDU stay views and pin at most twice their length. Each segment is charged what it keeps alive, a view its whole PDU and a copy its own length, so pinning is in the charged figures rather than beside them. Retained hint values, which live as long as the transfer, are copied for the same reason. The segment copy has a CPU price, about 100 to 300 ns and one allocation per copied segment: a flood of 1500-byte PDUs each carrying 100 one-byte segments was measured at 2 to 3 million segments per second per core, which is below wire rate at 1 Gbit/s. That is accepted. It degrades gracefully, since the receiver falls behind and the link drops frames while memory stays bounded, and `Delivery::Streamed` removes the copy for in-order data (see [Streamed delivery releases the contiguous prefix](#streamed-delivery-releases-the-contiguous-prefix)). + +Without a `MaxSegments`, the resulting bound is three times the cap of charged state per transfer: the segments keep alive at most twice the cap, and the bookkeeping budget adds the cap again. With one, the bookkeeping is roughly the limit times 70 to 100 bytes per transfer, which exceeds the cap when a link-derived limit assumes fewer than a few hundred data bytes per segment; that is the price of accepting a small-PDU link's honest segmentation, and it is what the operator should size for. + +Up to `window_size` transfers can be open at once, so the per-transfer bounds alone would let one peer hold `window_size` times as much: with the defaults (1 GiB and 16), 48 GiB of charged state. The 1 GiB default matches TCPCLv4, but a TCPCLv4 session is a reliable, usually authenticated stream carrying one transfer at a time, while anyone who can put frames on a BTP-U link can open every transfer in the window. `MaxRetainedBytes` therefore bounds the receiver as a whole. Every in-progress transfer's charge (the memory its segments keep alive, `SEGMENT_OVERHEAD` per stored segment, and its hint charge) is summed, and a message that would take the sum over the limit rejects its transfer as `ReceiverFull`. The transfer that grew is the one refused, not an older one: it is simpler, it never discards a transfer that is nearly complete, and under attack either choice loses honest transfers, since on an unauthenticated one-way link availability cannot be defended at this layer (a peer can always advance the window past everything held). Memory is what the limit protects. Segments are charged here even when a `MaxSegments` limit replaces the per-transfer segment charge, so a flood of empty segments counts against the total. The limit is a byte value rather than a multiple of the cap, so the operator reads the receiver's ceiling straight off the configuration, as with `MaxSegments`. The configured value is enforced as given. A receiver could instead raise a value below one transfer's full allowance to that allowance, so any transfer the per-transfer rules admit can always be received on its own, but that allowance is set by the finest segmentation the limits permit and reaches several times the cap on small-PDU links (6.92 GiB for a 1 GiB cap and 64-byte PDUs), so the raised figure would bear little relation to what the operator wrote. A value below it instead refuses such a transfer as `ReceiverFull`, which is accurate: the operator has chosen a smaller effective bundle size. With no value configured the limit is one transfer's full allowance, `MaxRetainedBytes::for_transfers` with a count of one, 3 GiB for the defaults, which keeps TCPCLv4's promise of one cap-sized bundle at a time and brings the default receiver from 48 GiB of charged state to 3 GiB. + +The operator knows the largest bundle, the link's PDU size, and how many transfers to hold, while what the receiver must bound is charged state, whose segment count the sender chooses. `MaxSegments::for_link_pdu_size` and `MaxRetainedBytes::for_transfers` chain the first to the second, one helper per field, and the `ReceiverConfig` rustdoc tabulates the gap between a sender that fills its PDUs and one that segments as finely as the limits allow (1.05 against 2.17 GiB per transfer on 1500-byte PDUs, 2.46 against 6.92 GiB on 64-byte ones) so the operator can place the limit between them. Because the real cost depends on the peer, `Receiver::retained_bytes` exposes the charged total for a CLA to export as a gauge. It is read-only and reveals nothing the peer did not send, and it is the only way to measure a real peer against the bounds. A Bundle Length hint does not reserve budget in advance: it would let one small message claim up to the cap without sending it, making the budget cheaper to exhaust than by sending data. + +These figures are estimates of the receiver's retained state, not a bound on every allocation, and view pinning assumes the CLA hands `receive_pdu` buffers sized to the received frame; real footprint exceeds the charged limit by allocator rounding, up to about half again under a flood of tiny segments (see `SEGMENT_OVERHEAD`). The defaults suit a host, not a constrained device; there the cap and the retention limit should both come down. + +Because entries are charged for their overhead, empty segments can be stored safely, and they are: Section 4 completes a transfer once segments 0 through N are present, an empty segment is still segment k, and a streaming sender that learns of end-of-input only after emitting a full segment has no way to finish except with an empty End at N. Section 8.2 and 8.3 say such messages should not be sent, not that a receiver may refuse them, so a receiver that dropped them would fail to complete a conforming peer's transfer. + +Two further structures could otherwise grow without limit. A transfer that is over must not be re-opened by a late or repeated message: for a cancelled one Section 4.2 requires it, and for a delivered or rejected one the alternative is a second delivery of the same bundle or a phantom transfer that sits in the window until it expires. Their numbers are therefore remembered with the reason, in a map pruned by the same window advance that expires transfers and so bounded by the window size. Both maps are keyed not by the bare transfer number but by a `TransferId`, the number extended to a 64-bit serial whose low 32 bits are the number itself, so a key costs 8 bytes; the extension is taken from the greatest number seen: forward for a new number, which Section 5 accepts up to 2³¹ + W/2 ahead and so beyond the half-space where RFC 1982 comparison is defined, and backward by less than W for one in progress. Keys therefore sort oldest first across the 2³² roll-over, the entries a window advance expires are a leading run of each map, and expiry costs only what it removes rather than a walk of up to W live and W closed entries on every new number; `TransferExpired` events come out oldest first. Only the window makes ids, and only for in-window numbers, so neither map can be indexed by a raw number that skipped the window check. A Cancel for an in-window number the receiver holds no segments for is remembered the same way, because Section 5 makes every number in the window an in-progress transfer and Section 8.4 says the segments that arrive after the Cancel must be discarded too. Hints are kept one per hint type with the value from the most recently received message winning, which is structurally bounded by the hint-type space and the one-byte value length and charged to the bookkeeping budget besides; the codec already folds a message's hint chain to one item per type while decoding, so a chain of half a million repeats costs at most 128 items in flight. + +Messages that contradict a transfer's established segment sequence, such as a second End that disagrees with the recorded final index or a segment beyond it, are dropped rather than applied, because applying them would make completion permanently unsatisfiable and pin the transfer's memory until the window expires it. A bogus End at exactly the highest index seen is undetectable at this layer; that is a job for bundle-level integrity. + +A restarted sender that follows Section 4's advice and starts from a random transfer number is accepted only about half the time, because the Figure 2 test treats a number as new only within half the number space ahead of the greatest seen, and the draft offers no resynchronisation rule. `Receiver::reset` exists for a CLA that learns of a restart out of band; the hazard is documented on `Receiver`. + +### Reassembly copies once and shares when it can + +A completed multi-segment transfer is concatenated into one contiguous buffer. The copy is deliberate: the BPA parses bundles from contiguous bytes, and one copy per delivered bundle is cheap relative to the transfer. An unsegmented Bundle Message hands back its own `Bytes`, a reference-count increment rather than a copy. A single-segment transfer hands back its segment as stored, which is still a view of the PDU when the segment filled at least half of it and otherwise the copy made on arrival (see [Receiver memory is bounded by what is retained](#receiver-memory-is-bounded-by-what-is-retained)); either way it is not copied again. For a very large bundle the concatenation is a burst of work inside one `receive_pdu` call (hundreds of milliseconds per gibibyte, depending on the host), during which frames queue at the link; `Delivery::Streamed` removes it. + +### Streamed delivery releases the contiguous prefix + +`ReceiverConfig::delivery` chooses between `Delivery::Whole`, the default, and `Delivery::Streamed`. Under `Whole` a transfer is held until it is complete and reported as one `Received`. Under `Streamed` the receiver keeps a reorder buffer per transfer and releases its bytes in Segment Index order from segment 0, tracking the first index not yet released. Unsegmented bundles (Bundle messages, bare and encapsulated frames) are reported as `Received` in both modes, since they arrive whole and have no transfer number. + +A streamed transfer is reported by three events: + +```rust +TransferStarted { id: TransferId, hints: Hints }, +TransferData { id: TransferId, data: Bytes, hints: Option }, +TransferFinished { id: TransferId, data: Bytes, hints: Option }, +``` -### The Receiver streams the contiguous prefix +- `TransferStarted` is reported just before the transfer's first released data, so a CLA starts its `dispatch` with bytes to read and the BPA's pre-drain gate sees the bundle's first bytes at once. A transfer that never receives segment 0 never surfaces, as under `Whole`; its memory is bounded by the window and the retention limit. +- `TransferData` carries one released segment, without a copy: an in-order segment is the view the decoder produced, never detached from its PDU because it is released before the call that decoded it returns, and a held one is what `retain_segment` stored. A segment that fills a gap releases the run behind it in the same call, one event per segment. Segments holding no data are not reported. +- `TransferFinished` carries the last segment's bytes and completes the transfer. Its `data` is empty when the last segment is, or when the End names a segment already released. A transfer whose segments all hold no data is rejected as `Empty` and never started, as under `Whole`. +- `TransferStarted` carries the transfer's full hint set so far, empty if it has none, as `Received` does, so both events that first show a consumer a bundle carry the same shape. On `TransferData` and `TransferFinished`, `hints` is `Some`, holding the full current set, on the first event after the set changes, and `None` otherwise. A change carried by a message that releases nothing (an out-of-order segment, an End ahead of a gap, an FEC message) rides the next event; a repeated value is not a change. A consumer starts from the set `TransferStarted` carried and keeps the latest `Some`. The receiver keeps a transfer's hints until the transfer closes. -The initial Receiver buffers every segment of a transfer until completion, then concatenates them into a fresh buffer — unbounded memory under adversarial or just unlucky traffic, plus a full copy of every bundle. The replacement emits reassembled data incrementally: each time the in-order prefix of a transfer extends, the newly contiguous segments (already zero-copy slices of their PDUs) are emitted as chunks, with the final chunk marked as such. A gap in the segment sequence simply stalls emission until repetition fills it. +These map onto the BPA's ingress stream: `TransferStarted` opens it, `TransferData` feeds `Segment::Next`, and `TransferFinished` sends `Segment::Final`. After `TransferStarted`, a `TransferCancelled`, `TransferExpired`, or `TransferRejected` for the same id ends the transfer, and the CLA drops the stream's sender, which the BPA treats as an abort. `Receiver::reset` ends every started transfer without an event; the caller asked for it. -This shape exists because of where the data goes next: it maps one-to-one onto the BPA's `Segment::Next(Bytes)` / `Final(Bytes)` ingress stream, whose `Final`-carries-data form happens to mirror Transfer End carrying the final segment. Memory is bounded to out-of-order segments beyond the contiguous prefix — the receiver-side resource-exhaustion concern largely dissolves as a side effect of the architecture rather than needing a dedicated cap, though a configurable ceiling on buffered out-of-order bytes remains as defence in depth. The Bundle Length hint ([§9.1](https://datatracker.ietf.org/doc/draft-ietf-dtn-btpu/)) shifts role accordingly: from sizing a reassembly preallocation to early-rejecting oversized transfers and advising the BPA's spool. +`TransferId` is the window's 64-bit key: opaque, `Copy`, `Eq`, `Ord`, and `Hash`, with no public constructor, a `transfer_number()` accessor, and a `Debug` that shows the wire number. Every event about a transfer the window admitted carries an id: the three above, and `TransferCancelled`, `TransferExpired`, and `TransferRejected` in both modes, so both modes name transfers the same way. `MessageDropped` carries `transfer_number: u32`, since a message outside the window has no id, and `id: Option`, which is `Some` for every number inside the window, so a CLA keying streams by id can attribute a repeat or a conflict without a second lookup. Ids never repeat within a receiver: `TransferWindow::reset`, which `Receiver::reset` calls, moves the serial's high bits two past the last id issued, so even the oldest id in the window of the first number after the reset is above every earlier id, and an id kept across a reset cannot name a later transfer. An id is scoped to the receiver that issued it, and ids from different receivers are not told apart; a CLA with several channels keys its map by channel first. + +A CLA whose BPA refuses a bundle, or stops reading it, calls `Receiver::refuse(id)`, which discards the transfer's state and remembers it as closed with `DropReason::Refused`, so its later messages are dropped. It returns whether the receiver held the transfer, and does nothing for an id that has expired or whose transfer has already closed. Because ids never repeat, a stale or misdirected id cannot close a transfer whose wire number has since been re-admitted. `refuse` produces no event: the caller knows what it refused. + +Under `Streamed`, a released segment leaves the transfer's map and stops being charged against the retention limit. It still counts toward the transfer's length for `MaxTransferSize`, toward the segment limit, and toward the per-segment bookkeeping charge that stands in for one, all of which count distinct segments whether released or not, so a transfer is accepted or rejected at the same message in both modes. A repeat of a released index is a `Duplicate`, and the highest index seen is tracked explicitly for the End-below-a-seen-segment check. + +The worst case does not change: a peer that sends every segment but segment 0 makes the receiver hold the whole transfer, as under `Whole`, so the per-transfer cap, the segment limit, and the receiver-wide retention limit apply unchanged. What streaming saves is the common case of in-order arrival, where a transfer holds nothing beyond its out-of-order segments and the completion-time copy disappears. + +The receiver cannot push back on the link. Released bytes the BPA has not yet read wait in the CLA's per-transfer stream, outside the receiver's retention limit, so the CLA bounds that stream and calls `refuse` when it is full rather than blocking its frame loop. The `Delivery::Streamed` docs state this rule, and the CLA can charge its buffered bytes to a shared retention budget (see [A retention budget can be shared across receivers](#a-retention-budget-can-be-shared-across-receivers)). + +FEC transfers release nothing: with no scheme, they hold only hints. With a scheme, the scheme decides when source bytes are known-good. A systematic code can release source symbols that arrive in order without decoding, and any code can release a block once it decodes. Either way its output extends the transfer's contiguous prefix, which feeds the same `TransferStarted`, `TransferData`, and `TransferFinished` events, and the source and repair symbols of a released block stop being charged. The release loop therefore takes the next contiguous bytes of a transfer from one method rather than reading the core segment map, so that a scheme plugs into it (see [A pluggable FEC scheme](#a-pluggable-fec-scheme)). + +### A retention budget can be shared across receivers + +The Ethernet draft (Section 4.2) runs one BTP-U instance per logical channel, keyed by interface, VLAN, and the two MAC addresses, so a CLA holds a receiver per peer and `MaxRetainedBytes` multiplies with the peer count. Source addresses are not authenticated, so a peer can open channels at will. A `budget::RetentionBudget` shared by the receivers of a link bounds the total. + +```rust +pub struct RetentionBudget { limit: MaxRetainedBytes, used: AtomicUsize } + +impl RetentionBudget { + pub fn new(limit: MaxRetainedBytes) -> Self; + pub fn limit(&self) -> MaxRetainedBytes; + pub fn used(&self) -> usize; + pub fn try_charge(self: &Arc, bytes: usize) -> Option; +} + +/// Bytes charged against a budget, released when dropped. +pub struct Charge { budget: Arc, bytes: usize } +``` + +- A receiver joins a budget through `Receiver::with_budget(Arc)`, a builder method beside `with_bundle_extent`, which charges what the receiver already holds whatever the limit. `ReceiverConfig` is plain data that can be deserialized, so a shared handle does not belong in it. +- The receiver keeps its own total and passes each change to the budget. Growth is a compare-and-add that fails rather than overshooting, so receivers on different threads cannot together exceed the limit. Growth the budget refuses is recorded as unbudgeted and is the first to be given back when the transfer that grew is rejected, so the receiver's share of the budget is always its total less that. Shrinking, closing a transfer, `reset`, and dropping the receiver release what they free; the receiver's held-transfer state implements `Drop` for the last, so tearing down a channel returns its share. +- A message whose growth the budget refuses rejects its transfer as `RejectReason::BudgetFull`, checked after the transfer's own limits and the receiver's `ReceiverFull`. A separate reason tells an operator that the link is full, not that one peer is over its share. Reject reasons are local diagnostics, for events, logs, and metrics. BTP-U has no channel from receiver to sender today. If a later protocol pairs two sessions in opposite directions and lets a receiver report why it gave up on a transfer, `BudgetFull` and `ReceiverFull` must go out as one indistinguishable reason, so that a peer cannot probe how full the link is. The `RejectReason` docs say so. +- The per-receiver `MaxRetainedBytes` stays, with its default, and is the only fairness between channels: the budget serves whoever asks first, and each channel's limit keeps one channel from taking all of it. A budget below `MaxRetainedBytes::for_transfers(1, ..)` of the receivers it serves can refuse a lone transfer, as the per-receiver limit can. +- A CLA charges the streamed bytes it buffers for the BPA through `try_charge`, one `Charge` per `TransferData`, and sends the charge along with the bytes so that it is released when the BPA reads them. A failed charge is the cue to call `refuse`. The receiver releases a segment's charge when it streams it, and the CLA charges it again a moment later, so another receiver can take the space between the two; the CLA then refuses that transfer, which is the outcome a full link should have. Handing the receiver's charge over in the event would close the gap, but would tie every event to whether a budget is in use. An in-order segment is never charged by the receiver at all, since it is released before the call that decoded it returns. +- `used` and `limit` give the CLA a link-wide gauge beside each receiver's `retained_bytes`. +- `alloc::sync::Arc` needs native atomic compare-and-swap, which thumbv6m lacks, so the `budget` module is compiled only where `target_has_atomic = "ptr"`. A receiver on such a target works without one. `portable-atomic-util`'s `Arc` would lift that if a CLA on such a target ever needs a budget. + +### Enum extensibility follows the trust boundary + +Enums decoded from the wire (`Message`, `HintItem`) carry a faithful `Unknown` catch-all and are otherwise exhaustive. Unknown message types and unknown hints relay byte-exact, and `MessageFlags` keeps the three reserved bits rather than discarding them, because the Message Flags registry (Section 12.3) may assign them and a relay that zeroed them would corrupt a future extension in flight. Encoding refuses a `Message::Unknown` whose type value is defined by the base protocol or bundle-reserved, since the result would decode as something else. The FEC messages carry a single opaque payload for the same reason: the internal boundaries are scheme-defined, so decode followed by encode is the identity even with no scheme registered. + +Enums this crate produces (`ReceiverEvent`, `DropReason`, `RejectReason`, the error types) are plain exhaustive with no `#[non_exhaustive]`, and derive `Clone`, `PartialEq`, and `Eq` so a consumer or a test can compare whole event lists. A consumer that matches per variant should get a compile error when one is added; within the workspace that break is the checklist that new dispositions are handled. + +### The tower adapter stays thin + +The `tower` feature implements `Service` for `Sender` (with `SendRequest: From` for the default-options case, so request-less combinators need no turbofish), an infallible `Service` for `Receiver`, and `Stream` for the sender's PDU drain. `poll_ready` is the sender's admission gate: a window slot must be allocatable and the bytes queued must be below the `SendQueueBytes`. `enqueue` itself checks only the window, and only when it segments; the byte bound is enforced here (and by `is_send_queue_full` for a direct caller) because unsegmented bundles take no window slot, and without it the queue would grow without limit whenever the drain side is slower. The `Service` is the whole-bundle form; a bundle pushed in chunks goes through `begin`, `push`, and `finish`, and a push wakes the parked `Stream`, since it can make a waiting transfer's next segment ready. + +Wake-ups run in both directions: draining a PDU or cancelling a queued bundle wakes every task parked in `poll_ready` once the window and the queue both have room (the predicate `poll_ready` gates on, so a woken task finds the service ready), and enqueueing wakes the task parked in `poll_next`. The enqueue side keeps a list of wakers, deduplicated with `Waker::will_wake`, rather than a single slot, so several producers sharing one `Sender` through a mutex are all woken when capacity frees; a single slot would wake only the last to poll and leave the others asleep with capacity available. Every parked producer is woken rather than one per unit freed: `poll_ready` reserves nothing (below), so a freed unit cannot be handed to one waiter, and a woken producer that then sends nothing would leave the others asleep. The drain side has one consumer and keeps one waker. The sender is a perpetual source and `poll_next` never yields `Ready(None)`. Because the `Service` half, the `Stream` half, and `cancel` all need the same `Sender`, sharing means `Arc>`, and `tower::buffer::Buffer` is documented as unsuitable because it moves the sender into a worker task and strands the drain. `poll_ready` reserves nothing, so a producer sharing the sender must poll and call under one lock acquisition and must release the lock before parking on a `Pending`; otherwise another producer can take the slot it was promised (and `call` fails with `WindowFull`, which then means "poll again"), or the drain task is locked out of packing the End that would free the window. The tower tests step two producers through exactly that pattern, interleaved on one thread. + +### FEC is framing only + +The FEC extension's four message types are encoded, decoded (when enabled), and tracked so the receiver can apply the draft's cancellation rules. A transfer that mixes core and FEC messages (Section 3.2) is rejected as `FecCoreMixing`. One whose FEC configuration changes (Sections 3 and 3.1) is rejected as `FecConfigurationChanged`: the receiver records the form and identifier of the first FEC message, a pre-agreed FEC Instance ID or an explicit FEC Encoding ID, and any later message naming a different form or identifier closes the transfer. The draft names the configuration and the Instance ID but not the message form; closing on a form switch is this crate's reading, because without the pre-agreed instance table the receiver cannot show that an instance and an explicit encoding name the same configuration. That is the part of the configuration visible without a scheme; the scheme-specific information in an explicit message's payload is not compared. The drafts call such a transfer cancelled, but the receiver reports it as `TransferRejected` so a CLA can tell a protocol violation from a sender's Transfer Cancel. No FEC scheme is implemented and the receiver does not attempt FEC reassembly. An earlier `FecScheme` trait that sketched the plugin shape has been removed until a scheme exists to shape it; see [Next steps](#next-steps-not-yet-implemented-or-agreed-in-detail). ## Integration -The crate stays free of `hardy-bpa` dependencies. The shapes above are designed to *rhyme* with the BPA's streaming seams — `next_pdu(max_len)` answers the link's demand the way `dispatch_streamed` answers the BPA's; the Receiver's chunk emission feeds `Segment::Next/Final` through a trivial adapter loop in the CLA — but the coupling lives entirely in the CLA crate that bridges them. This preserves `no_std` portability and keeps the protocol library testable without an async runtime. +A convergence-layer crate owns the link and the async runtime and composes this crate with `hardy-bpa`. On egress it pushes the BPA's forwarded bundle into the sender with `begin`, `push`, and `finish` as its segments arrive (or calls `enqueue` with a whole one), and drives `next_pdu` (or polls the `Stream`) as the link accepts frames, flushing when its own timer says a producer has gone quiet, resolving each bundle's outcome from the `Carried` entries of the PDUs it writes; the window looks after itself, and `cancel` is available for a bundle the CLA decides to abandon while it is still queued, and for one whose producer fails mid-stream. On ingress it hands every received frame, BTP-U PDU or bare bundle alike, to `receive_pdu` and dispatches each `Received` to the BPA, treating the remaining events as telemetry. A CLA that holds `hardy-bpv7` should install a `BundleExtent` hook so encapsulated bundles are delimited properly on links that pad. -Egress integration is staged: today's `Cla::forward(queue, cla_addr, bundle: Bytes)` delivers whole bundles, and a chunk-fed `enqueue` accepts that as a single chunk. When BPA egress streaming lands, the same `enqueue` accepts the storage stream directly and the pull-through pipeline (storage → Sender → link) completes without further API change — pack-time segmentation is the piece that makes this possible. The `queue` parameter of `forward` maps directly onto the Sender's per-enqueue priority. +The crate deliberately has no `hardy-bpa` dependency, so the BPA's `Cla` and `Sink` traits, its queue semantics, and its streaming pipeline are all bridged in the CLA crate. This preserves `no_std` portability and lets the protocol engine be tested without a runtime. There is no BTP-U CLA in this workspace yet; the first one will be the first real test of the interface. ## Standards Compliance -- [draft-ietf-dtn-btpu](https://datatracker.ietf.org/doc/draft-ietf-dtn-btpu/) — §4.1 (interleaving), §5 (transfer window), §6 (repetition, exact-copy rule), §9.1 (Bundle Length hint). Note: the implementation's window-validity check intentionally follows the §5 prose over the published Figure 2 pseudocode, which mis-classifies a repeat of the greatest transfer number as new; a correction to Figure 2 is queued for the next draft revision. -- [draft-ietf-dtn-btpu-fec](https://datatracker.ietf.org/doc/draft-ietf-dtn-btpu-fec/) — message framing only; FEC schemes remain out of scope for this tranche. -- [RFC 9221](https://www.rfc-editor.org/rfc/rfc9221.html) — consumed via QUBICLE; this design deliberately uses no per-datagram acknowledgement signals (see the repetition decision). +- [draft-ietf-dtn-btpu](https://datatracker.ietf.org/doc/draft-ietf-dtn-btpu/): message format, hints, and unrecognized-message handling (Section 7, 7.1, 7.2, 7.3), message definitions (Section 8), the Bundle Length hint (Section 9.1), cancellation semantics (Section 4.2, 8.4), the transfer window (Section 5), the exact-copy rule for repetition (Section 6), and the reserved message-type and flag registries (Section 12.1, 12.3). The target revision is pinned once, in the crate-level rustdoc. +- The receiver's window check follows the Section 5 Figure 2 pseudocode, including its guard that a repeated message for the greatest transfer number is in progress rather than new; without that guard every repeat would re-trigger window expiry. `WINDOW_SIZE / 2` is read as integer division. +- Section 7.3's encapsulated-bundle rule is honoured in both directions: with a `BundleExtent` hook the receiver behaves as one that implements the bundle format, and without one it stops processing the remainder of the PDU as the section requires. +- [draft-ietf-dtn-btpu-fec](https://datatracker.ietf.org/doc/draft-ietf-dtn-btpu-fec/): message framing only; no FEC scheme is implemented. The draft's four message types have no IANA-assigned codes yet (TBD1 to TBD4), so the crate uses provisional values from the Private Use range of the message-type registry, interpreted only when FEC decoding is switched on; they will change when codes are assigned. Mixing core and FEC messages, or changing a transfer's FEC Instance ID, FEC Encoding ID, or FEC message form, cancels the transfer; scheme-specific information is not checked. +- Section 8.4 is read with Section 5's definition of "in progress": a Cancel for any number inside the receive window is applied and remembered, whether or not segments for it have arrived, and a Cancel for a number outside it (ahead of the greatest seen included) is ignored without advancing the window. +- Local policies the drafts permit but do not require: a mandatory per-transfer cap rejects transfers early on the Bundle Length hint and budgets their bookkeeping separately, optionally as a direct segment-count limit; a delivered transfer's number is remembered for the life of the window so a repeat (Section 6) is dropped rather than re-delivered; a Bundle Length hint on a Bundle Message is dropped (Section 9.1 says receivers should ignore it there); an empty Bundle Message, or a transfer that completes with no data, is rejected (Section 8.1 requires a valid bundle); a malformed Bundle Length hint is carried as an unknown hint rather than failing its message. +- Not implemented: message repetition (Section 6), and on the send side any interleaving of transfers (Section 4.1) beyond passing over one that cannot supply. A lost PDU loses the messages it carried. ## Testing -The existing unit and integration tests carry over as the baseline: full-size padding with a repetition count of one reproduces the initial implementation's behaviour exactly. New coverage required by this design: pack-time segmentation under varying per-call capacity (including capacity shrinking mid-transfer, verifying pinned segment boundaries on repeat passes), scheduler fairness and priority ordering under interleaving, automatic window release, and contiguous-prefix emission under reordered and duplicated segment arrival. A crate test plan will follow the Hardy test-plan format once the tranche is scoped into PRs. +Tests of the public API live under `tests/`, one file per subject (`codec`, `codec_header`, `codec_hint`, `codec_message`, `transfer`, `sender`, `send_handle`, `receiver`, `streaming`, `budget`, `tower`, `config`, `tunnel`), with shared fixtures in `tests/common`. A `TransferId` has no public constructor, so `tests/common` mirrors `ReceiverEvent` as an `Event` that names transfers by number, and a `ReceiverEvent` compares equal to the `Event` it maps to. `tunnel` carries non-bundle packets from 40 bytes to 9000 through a variable-framed sender and receiver as an IP-over-UDP tunnel would: over a lossless link, over one that drops every seventh datagram (checking that exactly the undamaged packets arrive and exactly the partly-received transfers expire), and over loopback UDP sockets in lockstep. The only inline tests are those that need private state: the few in `receiver` that inspect reassembly charges and closed-transfer pruning, including a randomised run under both deliveries checking that the receiver-wide total always equals the sum of the transfers' charges and, with a budget joined, the budget's charge, those in `sender` that start the unsegmented bundle ID near `u32::MAX` to check its wrap hold a segment's length across copies, which no public path reaches until repetition exists, check which pushed chunks a segment releases, and set a segment index near `u32::MAX` to check that a flush is refused when the rest of its bundle could need that index, and those in `transfer`, whose window and allocator are crate-internal: the Figure 2 classification, the allocator's span gate, and the ids' boundaries, their agreement with `id` across the wrap, and their order across a reset. Assertions compare whole event lists and typed errors. The tower tests cover every live waker path, including a push readying a waiting transfer's segment, the multi-producer wake, the window opening as the drain packs a Transfer End, and two producers admitted through a shared mutex without either seeing `WindowFull` (interleaved on one thread, which checks the protocol but does not contend for the lock); they run on `futures::executor::block_on`, since every future involved is immediately ready. Three fuzz targets under `fuzz/` drive the decoder (checking that every decoded message re-encodes at its predicted length, relays byte-exact if unknown, and decodes back to itself, across the FEC and extent-hook options), the receiver with a small cap, across the FEC, segment-limit, extent-hook, retention-limit, delivery, shared-budget, and refusal options (checking that no delivered bundle is empty or over the cap, that a `MalformedPdu` only ever ends a PDU's events, that each streamed id starts once and releases data only between its start and its end and never more than the cap, and that a budget's charge equals its receiver's total and stays within its limit), and the sender through arbitrary sequences of `begin`, `push`, `finish`, `enqueue`, `cancel`, and `next_pdu` with and without a flush, under each link framing and cut strategy, into a receiver over a lossless link (checking that every PDU fits its framing, that its `Carried` list names only live bundles, each once, that a producer gated on `is_push_ready` always finds a PDU to drain, and that once every bundle is finished and the queue drained, exactly the bundles neither cancelled nor finished short have arrived, once each, with no queued bytes or window slots left); the crate is built for `thumbv7em-none-eabihf` in CI to guard `no_std` and 32-bit compilation, and for `thumbv6m-none-eabi` with the `critical-section` feature to guard targets without atomic compare-and-swap. + +## Next steps (not yet implemented, or agreed in detail) + +Everything below is direction, not description. It records a sketched second tranche of work on the `Sender`/`Receiver` layer so the reasoning is not lost, and it has been discussed but not adopted as a plan, except where a section says it is agreed: the shapes may change once a real CLA exercises them, and none of the current API is deprecated by it. The codec and transfer-window layers are not expected to change. + +### The links that motivate it + +The current engine assumes one deployment shape: fixed-size link PDUs, a PDU size known at construction, and whole bundles in memory. Four concrete consumers stretch that in different directions. + +- Constant-bit-rate framed links (CCSDS-style): fixed-length PDUs, padding mandatory, blind repetition for loss protection. The current design serves this shape as is. +- Ethernet: variable-length frames with a 46-octet minimum payload, so padding beyond the minimum is wasted; blind repetition for loss protection. `LinkFraming::Variable` with a `min_pdu_len` of 46 covers it, though in practice the NIC's own zero-fill after a BTP-U PDU decodes as Indefinite Padding, so the floor is belt-and-braces. With the floor set, this sender's bare frames are never shorter than 46 bytes, but a receiver of bare frames on Ethernet still needs the `BundleExtent` hook for peers that do not do the same. +- QUIC datagrams ([QUBICLE](https://datatracker.ietf.org/doc/draft-ek-dtn-qubicle/) unreliable service, [RFC 9221](https://www.rfc-editor.org/rfc/rfc9221.html)): self-delimiting datagrams where every padding byte spends congestion-window budget, and where the usable PDU size derives from the negotiated `max_datagram_frame_size` and the live path MTU and can shrink mid-connection. +- A UDP tunnel between two IP hosts, carrying IP packets (a TUN interface) or Ethernet frames (a TAP interface) segmented across datagrams, is the proposed first test consumer: it needs no bundles at all, exercises variable framing and segmentation of inner packets larger than the outer datagram, gives loss a direct metric in expired transfers, and is where FEC, once it exists, can be measured against repetition. + +A further pressure comes from inside Hardy: the BPA's CLA interface is now streamed in both directions (`Sink::dispatch` takes a `Receiver` on ingress; `Cla::forward` supplies one with a `total_len` on egress, see `bpa/docs/streaming_pipeline_design.md`), and a CLA built on this crate would want to bound its memory the same way, with no full-bundle buffering on either path. The receiver's streamed delivery and the sender's send handle (see [A bundle can be pushed in chunks through a send handle](#a-bundle-can-be-pushed-in-chunks-through-a-send-handle)) give it that on both paths. + +### Priority interleaving in place of the single FIFO + +Deferred until the Ethernet convergence layer works without it. Section 4.1 permits interleaving Transfer Messages from different transfers precisely so a large low-priority bundle cannot block a small urgent one; the current queue order makes that head-of-line blocking structural, and it mismatches Hardy's forwarding model, where `Cla::forward` already names a lane per bundle that the CLA currently has nowhere to put. Proposed: per-transfer queues behind a scheduler that chooses which transfer supplies each next message, with unsegmented Bundle Messages and Cancels joining as single-message pseudo-transfers so ordering and priority apply uniformly. Preemption needs no protocol mechanism: it is a policy that takes nothing from a lower transfer while a higher one has data ready, and the lower transfer resumes later with its segment boundaries unchanged. Any such scheduler must still respect the Section 5 span rule at emit time, and it changes what "the End has been packed" means for the self-releasing window: with repetition, the slot is released after the last scheduled emission of the End, not the first, and that emission is the one `Carried::completes` flags. + +This does not make the crate a second priority system beside the BPA's. The BPA decides when each bundle is handed over and on which lane; its lane indices are names whose order is the flow controller's policy. The sender decides only how transfers already queued share PDUs, which nothing above it can do once a large bundle has been handed over. The CLA maps the lane to the sender's priority, and the policy that interprets it is configured where the CLA is. + +#### The shape of the hook + +Policies are out of scope here; what matters is that one can be added without disturbing the API. The intended shape: + +- `SendOptions` gains a `priority` field: a small opaque newtype whose default reproduces FIFO behaviour, given meaning only by the policy, as lane indices are given meaning only by the BPA's flow controller. `SendOptions` is deliberately not `#[non_exhaustive]`, so the field breaks callers' struct literals as a checklist; callers who write `..Default::default()` are unaffected. +- The policy is a trait object passed at construction (`Sender::with_scheduler(config, seed, scheduler)`), with `Sender::new` meaning FIFO. `SenderConfig` stays plain `Copy` data with serde support; a policy's own settings belong to the policy's type. A trait object rather than a `Sender` parameter keeps the sender's type, its impl blocks, the tower impls, and CLA struct fields unchanged; one dynamic call per message choice is immaterial beside packing. +- The trait sees each ready transfer's ID, priority, whether it has started, and its bytes remaining, and returns which one supplies the next message. The sender keeps every protocol invariant (the span rule, exact copies, Cancel placement, End release), so any policy produces valid output by construction. + +The pack loop already asks one private method, `next_source`, which entry supplies each next message given the bytes the PDU holds, and keeps the protocol invariants itself; the trait would answer that question in its place. The other per-PDU decision, whether to fill slack with repeats of recent messages (Section 6) rather than padding, is taken at the same point, so it may be the same trait or a second one. The queue already holds one entry per transfer, which is the set a scheduler would choose among. `Carried { id, completes }` already describes pieces of several transfers in one PDU; the inline size of `CarriedList` would be chosen once interleaving sets how many a PDU names. + +#### Constraints the hook must state + +- **Window slots.** A preempted transfer still holds its window slot, so once W lower-priority transfers are in progress a higher-priority bundle cannot start a transfer whatever the policy says. The sender could reserve slots per priority, cap how many lower-priority transfers it starts, or let the policy cancel one to free a slot. An unsegmented Bundle Message holds no slot, so a small urgent bundle always passes. +- **Admission.** One `SendQueueBytes` across all priorities lets a queue full of low-priority bytes refuse a high-priority enqueue. Admission has to be per priority, or let higher priorities past the bound. +- **Tower.** `Service::poll_ready` runs before the request is seen, so it cannot admit by priority. The per-producer `Lane` on the revisit list answers this as well as wake-one: one lane per priority, each with its own admission and waker, each able to implement `Service`, and each the source of `SendHandle`s (`sender.lane(priority)`, then `lane.begin(total_len)`). Sharing one sender between lanes without `std` is the design work that remains. + +### Repetition as the only loss mechanism + +An earlier sketch proposed an emission ledger: retain emitted messages, consume per-datagram acknowledgement or loss reports from the QUIC stack (RFC 9221 datagram frames are ack-eliciting), and re-emit exactly the messages from lost PDUs. That was rejected as building QUIC inside QUIC: QUBICLE deliberately offers a reliable stream service on the same connection for bundles whose delivery matters, and ack-driven repair in the CLA reconstructs a worse ARQ one layer up, with heuristic loss declaration, roughly an RTT of repair latency, and retention buffers that conflict with the bounded-memory goal. An RFC 9221 acknowledgement also confirms only that the packet arrived, not that anything consumed it. + +What would survive is smaller: a blind repetition count as the single loss knob, set per `enqueue`, which the CLA may tune from aggregate link statistics. Section 6 explicitly anticipates link-layer signalling triggering increased repetition, so that is tuning a protocol-native parameter, not acknowledgement-driven reliability. The protocol-native escalation beyond repetition is the FEC extension. One consequence of the rejection is that messages need be retained only until their last scheduled emission. + +Agreed for the streaming sender: it is designed for repetition, which is implemented later. Each queued segment carries a count of copies still to emit, one for now. Its bytes, its share of `SendQueueBytes`, and, for an End, the transfer's window slot are released when the count reaches zero, not when the segment is first packed, and `Carried::completes` flags the PDU holding the last copy of the End. Adding a repetition count then raises that number without reworking the queues or the window release. How the copies are scheduled is left until then, and it decides whether streaming still bounds memory: copies emitted close together hold only a few segments at a time, while whole-transfer passes, which ride out a burst of loss better, hold the transfer until its last pass begins. + +### Local abandonment of a stalled transfer + +A transfer missing one segment holds its memory until `window_size` newer transfers arrive, which on a quiet link may be never. Under `Streamed`, a CLA can act on that with its own staleness timer, keyed by `TransferId` from `TransferStarted` and ended with `refuse`. A transfer that has not received segment 0 never surfaces, so the timer cannot reach it; the window and the retention limit still bound its cost. A receiver-side idle timeout would cover both, but the receiver has no clock today and would need one passed in. + +### Receiver lifecycle in an Ethernet CLA + +To be designed and implemented with the Ethernet CLA, not in this crate. Because a logical channel (Section 4.2) has its own BTP-U instance, the CLA creates a `Receiver` when the first BTP-U frame arrives on a channel it has not seen, keyed by interface, optional VLAN, source MAC, and destination MAC. That is a policy decision, not a lookup, because the source address is unauthenticated. The policy decides whether to accept the channel at all (configured peers only, or any peer up to a channel limit), which configuration the new receiver gets, when an idle channel's receiver is dropped, and what happens to its open BPA streams when it is. Dropping a receiver ends its started transfers as `reset` does, and its `TransferId`s become meaningless, so the CLA drops its map entries for that channel at the same time. The shared retention budget (see [A retention budget can be shared across receivers](#a-retention-budget-can-be-shared-across-receivers)) bounds what accepted channels hold. The channel policy bounds how many there are. + +### Trailing octets in minimum-size frames + +The Ethernet draft (Section 3.1) has senders pad PDUs to the 46-octet minimum payload with Definite Padding, and has receivers tolerate trailing octets that do not parse in a minimum-size frame, treating trailing zeros as Indefinite Padding. Trailing zeros already decode as Indefinite Padding. Other trailing octets end the PDU with a `MalformedPdu` after the messages before them are kept, so a CLA can ignore `MalformedPdu` on frames of 46 octets or fewer. A receiver option that suppresses the event below a configured length would move that rule into the crate; it is low priority because no bytes are lost either way. + +### Recovering transfers refused while the receiver was full + +A transfer rejected as `ReceiverFull` stays closed for the life of the window even if space is freed later, because its stored segments were discarded and segments sent meanwhile may have been missed. On a link with repetition, a later copy could still fill those gaps. A distinct outcome (a temporarily-full or paused state) that lets such a transfer re-open would need rules for what is kept, what is charged while paused, and how a peer is kept from cycling transfers through it. Deferred until the loss model under [Repetition as the only loss mechanism](#repetition-as-the-only-loss-mechanism) is settled. + +### Suggestion: delivering receiver events to a visitor + +A suggestion only, not agreed as a next step. The sender's `CarriedList` treatment does not carry over to the receiver: a PDU of middle segments yields no events and so already allocates no list, a Transfer End's event sits beside a bundle-sized reassembly copy, a PDU of small bundles yields more events than a few inline slots would hold, and the event count is chosen by the peer (a PDU of Transfer Cancels yields one event per 8 bytes), so no bound exists to size a list by. If the event list ever shows up in profiles, a `receive_pdu_with(pdu, |event| ...)` that hands each event to the caller as it is produced would remove the list whatever the PDU holds. It fits the receiver because a CLA consumes events at once (deliver, count, log), unlike the sender's `Carried` list, which travels with the PDU until it is written. `receive_pdu_into` already serves callers who want one reused buffer. + +### A pluggable FEC scheme + +When a scheme is needed, a trait following the FECFRAME framework (RFC 6363) would give the receiver the payload-ID sizes it needs to split the opaque FEC payloads and the encode/decode operations to produce repair symbols and reconstruct bundles. The earlier sketch returned tuples of `Bytes`, allocated on every FSSI query, conflated ADU segmentation with the scheme, and folded "not enough data yet" into "failed"; the next attempt should be shaped by a concrete scheme rather than in advance of one. + +### API items to settle in the same revision + +These touch the types the streaming work changes, so they are better decided with it than before it. + +- A segmented transfer allocates one hint list at each end: the sender builds segment 0's `Vec` for the Bundle Length, and the decoder returns one that the receiver folds into `Hints` and frees. Making `Message`'s hints a `SmallVec` with one inline item would remove it; wrapped in a `HintList` newtype, as `CarriedList` wraps its `SmallVec`, `smallvec` would stay out of the codec's public API. It would still grow every `Message` from 72 to about 104 bytes, queued Bundle Messages included, and a second hint on segment 0 would spill to the heap anyway. If the allocation ever matters, the pack-loop rewrite can encode segment 0 from `Hints` directly, and the decoder can fill `Hints` in place. +- Priority interleaving changes how many bundles a PDU names, so `CarriedList`'s inline size of four is chosen with it, from measured PDUs. +- Producers sharing one sender are all woken when admission opens and contend for it. A per-producer `Lane` with reserved admission would wake one, and the same lane is the natural source of `SendHandle`s, and of a priority (see [Constraints the hook must state](#constraints-the-hook-must-state)), so the three are one design, deferred with priority. +- The receive fuzz target keys streamed transfers on `TransferId`, but cannot assert one `Received` per transfer, only per wire number between window advances of about 2³¹, since Section 5 lets an advance of that size re-admit a number and `Received` carries no id. Giving `Received` for a segmented transfer an id would close that gap. +- FEC: whether a minimum viable scheme should land with this revision, so that the sender's scheduler and the receiver's reorder buffer are shaped around repair symbols from the start rather than retrofitted (see [A pluggable FEC scheme](#a-pluggable-fec-scheme)). + +### What would not change + +The crate would remain free of `hardy-bpa` dependencies: the shapes above are designed to line up with the BPA's streaming seams, but the coupling lives entirely in the CLA crate that bridges them. `Cla::forward`'s segment stream and `total_len` map onto `begin`, `push`, and `finish` directly, so egress needs no staging. The existing tests would carry over as the baseline, because full-size padding with a repetition count of one reproduces today's behaviour exactly; new coverage would target pack-time segmentation under varying and shrinking capacity, scheduler fairness, window release under repetition, prefix emission under reordering and duplication, and send-handle abort at each point in a transfer. diff --git a/btpu/docs/fuzz_test_plan.md b/btpu/docs/fuzz_test_plan.md new file mode 100644 index 000000000..fb36a29cf --- /dev/null +++ b/btpu/docs/fuzz_test_plan.md @@ -0,0 +1,152 @@ +# Fuzz Test Plan: BTP-U + +| Document Info | Details | + | ----- | ----- | +| **Functional Area** | Convergence Layer Protocol Engine (Robustness) | +| **Module** | `hardy-btpu` | +| **Target Source** | `btpu/fuzz/fuzz_targets/decode.rs`, `btpu/fuzz/fuzz_targets/receive.rs`, `btpu/fuzz/fuzz_targets/send.rs` | +| **Tooling** | `cargo-fuzz` (libFuzzer) + `sanitizers` | +| **Test Suite ID** | FUZZ-BTPU-01 | +| **Version** | 1.2 | + +## 1. Introduction + +This document details the fuzz testing strategy for the `hardy-btpu` module. A BTP-U receiver takes link-layer PDUs from an unauthenticated, unidirectional link with no way to push back on the sender, so every byte it decodes, and every sequence of PDUs it accumulates state from, is attacker-controlled. The sender takes no input from the link, but a CLA drives it through arbitrary interleavings of begin, push, finish, cancel, and drain, and its packing has more state than unit tests enumerate. + +**Primary Objective:** Verify that the codec and the receiver handle arbitrary input without panicking, overrunning a buffer, or looping, that the codec's encode and decode paths agree on everything the decoder yields, that the receiver's event and size guarantees hold for any PDU sequence, and that the sender's call, PDU, and delivery guarantees hold for any sequence of calls. + +## 2. Requirements Mapping + +No Low-Level Requirements are assigned to this crate (see [`UTP-BTPU-01`](unit_test_plan.md) Section 2). The plan verifies: + +| LLR ID | Description | + | ----- | ----- | +| [**REQ-14**](../../docs/requirements.md#req-14-reliability) | Fuzz testing of all external APIs. | + +## 3. Fuzz Target Definition + +### 3.1 Target: `decode` + +The harness decodes the input as one PDU three times: with default options, with FEC decoding on, and with FEC decoding on plus a bundle-extent hook that takes every `0x9F` bundle to be eight bytes long. For every message the decoder yields: + +1. **Re-encoding:** `encode_message` succeeds and writes exactly `encoded_message_len` bytes. + +2. **Byte-exact relay:** an `Unknown` message re-encodes to the bytes it was decoded from. + +3. **Round trip:** decoding the re-encoded bytes yields the same message and nothing else. + +4. **Oversized bundles:** a `Bundle` longer than `MAX_CONTENT_LENGTH` (an encapsulated bundle delimited by the hook or a bare frame) refuses to encode with `LengthOverflow` and leaves the buffer empty. Reaching this branch needs an input over 1 MiB, so it runs only with `-max_len` raised above libFuzzer's default of 4096. + +After iteration the decoder **MUST** report itself exhausted. + +**Coverage Scope:** message header and length field (Section 7), flags (Section 7.1), hint chains and the Bundle Length hint (Sections 7.2, 9.1), unknown types and hints (Section 7.3), every message definition (Section 8) and the FEC message definitions (`draft-ietf-dtn-btpu-fec-02` Section 4), padding (Sections 8.5, 8.6), and bare and encapsulated bundle frames (Section 12.1). + +### 3.2 Target: `receive` + +The input is a flags byte followed by a sequence of PDUs, each prefixed by a one-byte length, fed to one receiver so that window, reassembly, and cancellation state accumulate across PDUs. The receiver has a window of 4 and a transfer cap of 128 bytes, below the 255-byte longest PDU, so the oversize gates are reachable. The flags byte selects: + +| Bit | Configuration | + | ----- | ----- | +| 0 | FEC decoding | +| 1 | A segment limit derived from a 64-byte link PDU, so the limit rather than the per-segment charge bounds segment count | +| 2 | The bundle-extent hook from the `decode` target | +| 3 | A retention limit equal to the cap, below one transfer's allowance, so a lone transfer can be refused as `ReceiverFull` | +| 4 | Streamed delivery | +| 5 | A shared retention budget of half the cap, so a transfer can be refused as `BudgetFull` | +| 6 | Refusal: once a streamed transfer has released more than 32 bytes, the harness refuses it, and a second refusal of the same id is ignored | + +For every PDU: + +1. **Delivered bundles:** every `Received` bundle is non-empty and no larger than the cap. + +2. **PDU faults:** a `MalformedPdu` event is the last event of its PDU. + +3. **Streamed transfers:** each transfer id starts at most once; every `TransferData` is non-empty and follows its `TransferStarted`; the bytes a transfer releases, its `TransferFinished` data included, total no more than the cap; a finished transfer released at least one byte. + +4. **Dropped messages:** a `MessageDropped` carries an id only when its reason is neither `OutsideWindow` nor `UnknownTransfer`, and that id names the dropped transfer number. + +5. **Budget accounting:** with a shared budget, what the budget has charged equals what the receiver reports retained, and never exceeds the limit. + +**Coverage Scope:** reassembly, repeats, and sequence checks (Sections 4, 6, 8.1 to 8.3), Transfer Cancel (Section 8.4), the window across the roll-over (Section 5), the FEC transfer rules (`draft-ietf-dtn-btpu-fec-02` Section 3), the per-transfer, retention, and shared budget limits (Section 10), and streamed release of the contiguous prefix. + +### 3.3 Target: `send` + +The input is two configuration bytes followed by a sequence of operations on one sender, whose PDUs feed a receiver over a lossless link. An operation byte selects one of: `begin` or `enqueue` of a bundle of 5 to 1500 bytes; a push of up to 255 or up to 1020 bytes into an open bundle, so overruns are common; `finish`; a cancel of an open handle or of an enqueued bundle; and `next_pdu`, with or without a flush. The top bit of the operation byte attaches an unknown hint to a new bundle. Every bundle starts with `0x9F` and carries a serial number, so each is distinct and a bare frame may carry it. The first configuration byte selects: + +| Bit | Configuration | + | ----- | ----- | +| 0, 1 | Fixed-size framing, variable framing, variable framing with a 46-byte floor, or variable framing with bare bundle frames | +| 2 | A send queue bound of 16 bytes, below one segment, so pushes past the bound are common | +| 3 | `SegmentCutStrategy::Half` | +| 4 | An initial transfer number of `u32::MAX - 1`, so transfer numbers roll over | + +The second sets the PDU size, 24 to 279 bytes. The sender and receiver windows are both 4. + +1. **Calls:** admission is refused only for a full window, a PDU too small to carry a segment, or a bundle needing too many segments; a push past the bundle's length fails with `Overrun`; `finish` succeeds exactly when every byte was pushed and otherwise fails with `Underrun`; cancelling an enqueued bundle that has not completed succeeds. + +2. **PDUs:** every PDU is no longer than the PDU size and no shorter than the floor, and exactly the PDU size under fixed-size framing. + +3. **Carried lists:** a PDU names each bundle at most once, names only bundles that are live, and marks every Bundle Message and bare frame as completing. + +4. **Push readiness:** whenever an open handle is not push-ready, `next_pdu` returns a PDU. + +5. **Delivery:** the receiver reports only `Received` and `TransferCancelled`; no bundle arrives twice and no cancelled bundle arrives. + +At the end the harness pushes and finishes every open bundle and drains the sender. The sender then has nothing pending, no queued bytes, and a free window slot, and the receiver has delivered exactly the bundles that were neither cancelled nor finished short. + +**Coverage Scope:** segmentation (Section 4), Transfer Cancel (Section 8.4), the window across the roll-over (Section 5), hints on the first segment (Section 7.2), padding and the floor (Sections 8.5, 8.6), bare frames (Section 12.1), and the sender's own policy: pack-time cuts under both strategies, passing over a transfer that cannot supply, the queue bound, and flush. + +## 4. Vulnerability Classes & Mitigation + +| Vulnerability Class | Description | Mitigation Strategy Verified | + | ----- | ----- | ----- | +| **Length Field Overrun** | A message length runs past the PDU, or a hint chain past its message. | Bounds-checked slicing; the fault is terminal for the PDU and earlier messages are kept. | +| **32-bit Truncation** | A wire-derived length or segment index is cast to `usize`. | Lengths compared before conversion; the `decode` round trip catches a truncated re-encode. | +| **Encode/Decode Disagreement** | The encoder writes a different length or different bytes than the decoder read. | Invariants 1 to 3 of the `decode` target. | +| **Unbounded Memory Growth** | Tiny segments, repeated hints, or many concurrent transfers hold more state than configured. | The transfer cap, bookkeeping budget, segment limit, retention limit, and shared budget, reached with the configurations above. | +| **Window Arithmetic** | Transfer numbers near the 2³² roll-over misclassify or never expire. | Accumulated state across arbitrary transfer numbers. | +| **Scheduling State** | A cancel, overrun, or flush leaves a transfer half-sent, holds a window slot, or leaks queued bytes. | The end state and delivery checks of the `send` target. | +| **Hook Misbehaviour** | The bundle-extent hook claims zero bytes or more than remains. | The decoder treats the claim as terminal rather than slicing past the PDU. | + +## 5. Execution & Configuration + +### 5.1 Running the Fuzzer + +From the `btpu/` directory (the fuzz crate pins a nightly toolchain): + +```bash +# Run for a set duration (Regression Mode) +cargo fuzz run decode -- -max_total_time=1800 # 30 Mins +cargo fuzz run receive -- -max_total_time=1800 +cargo fuzz run send -- -max_total_time=1800 + +# Reach the oversized-bundle branch of the decode target +cargo fuzz run decode -- -max_len=1100000 + +# Run indefinitely (Discovery Mode) +cargo fuzz run receive -j $(nproc) +``` + +All three targets are built and run in CI by ClusterFuzzLite, which discovers every `*/fuzz` crate and its `[[bin]]` targets (see [`.clusterfuzzlite/build.sh`](../../.clusterfuzzlite/build.sh)). + +### 5.2 Sanitizer Configuration + +`cargo-fuzz` builds with AddressSanitizer by default, which catches out-of-bounds reads in the slicing paths. + +## 6. Pass/Fail Criteria + +* **PASS:** Each target runs for the defined duration with **zero** crashes. +* **FAIL:** The fuzzer creates a `crash-*`, `timeout-*`, or `oom-*` artifact. + * **Action:** Reproduce with `cargo fuzz run ` and inspect with `cargo fuzz fmt `. + * **Remediation:** Fix the defect and add the input as a regression test in `tests/codec.rs`, `tests/receiver.rs`, `tests/sender.rs`, or `tests/send_handle.rs`. + +## 7. Corpus Management + +No seed corpus is checked in; libFuzzer starts from an empty input. Seeds would shorten discovery, particularly for `receive`, whose interesting states need several well-formed PDUs in sequence. + +* **Location:** `btpu/fuzz/corpus/decode/`, `btpu/fuzz/corpus/receive/`, `btpu/fuzz/corpus/send/` +* **Suggested Seed Data:** + * PDUs produced by `Sender::next_pdu` for small, segmented, and hinted bundles. + * A Transfer Cancel after a partial transfer. + * Bare BPv6 (`0x06`) and BPv7 (`0x9F`) frames. + * FEC messages with both pre-agreed and explicit FEC Instance IDs. diff --git a/btpu/docs/test_coverage_report.md b/btpu/docs/test_coverage_report.md new file mode 100644 index 000000000..a10720aa0 --- /dev/null +++ b/btpu/docs/test_coverage_report.md @@ -0,0 +1,138 @@ +# BTP-U Test Coverage Report + +| Document Info | Details | +| :--- | :--- | +| **Module** | `hardy-btpu` | +| **Crate version** | `0.1.0` | +| **Standard** | `draft-ietf-dtn-btpu-04` — Bundle Transfer Protocol - Unidirectional; `draft-ietf-dtn-btpu-fec-02` — FEC extension (message framing) | +| **Test Plans** | [`UTP-BTPU-01`](unit_test_plan.md), [`FUZZ-BTPU-01`](fuzz_test_plan.md) | + +## 1. LLR Coverage Summary (Requirements Verification Matrix) + +No LLRs are assigned to this crate. BTP-U is a pre-standard IETF extension and falls under the REQ-4 goal, whose Part 4 profiles (4.1 to 4.10) do not include it; fuzzing is under REQ-14 (Part 4 ref 14.1). Verification is traced to the drafts instead: + +| Draft Section | Feature | Result | Test | Plan Section | +| :--- | :--- | :--- | :--- | :--- | +| -04 §3.2, §8.5, §8.6 | Padding | Pass | `codec.rs::pad_pdu_fills_to_target`, `indefinite_padding_skipped`, `sender.rs::a_short_pdu_is_padded_up_to_the_floor` | 3.3, 3.5, 3.8 | +| -04 §4, §8.1–8.3 | Segmentation and reassembly | Pass | `sender.rs::large_bundle_segmented`, `receiver.rs::out_of_order_completes_on_end_recheck`, `streaming.rs::a_filled_gap_releases_the_run_behind_it` | 3.7, 3.10, 3.17 | +| -04 §4.1 | Interleaving | Pass | The sender passes over a transfer that cannot supply (`send_handle.rs::a_transfer_waiting_on_its_producer_is_passed_over`, `sender.rs::a_transfer_without_room_in_the_tail_is_passed_over_but_a_message_is_not`); priority interleaving is not implemented; the receiver accepts interleaved transfers (`receiver.rs::window_wraparound_with_live_transfers`) | 3.8 | +| -04 §4.2, §8.4 | Transfer cancellation | Pass | `sender.rs::cancel_after_partial_emission_discards_the_rest_and_queues_a_cancel`, `receiver.rs::cancel_of_an_in_window_transfer_before_its_segments_is_remembered` | 3.9, 3.11 | +| -04 §5 | Transfer window | Pass | `transfer.rs::new_transfer_boundary_is_half_space_plus_half_window`, `window_gates_on_span_not_count` | 3.6, 3.11 | +| -04 §6 | Repeated messages | Pass (receive) | `receiver.rs::duplicate_segment_dropped_and_first_copy_kept`; the sender does not repeat | 3.10 | +| -04 §7, §7.1 | Message header and flags | Pass | `codec_header.rs::wire_format_layout`, `codec_message.rs::message_flags_nibble_round_trips_all_bits` | 3.1 | +| -04 §7.2, §9.1 | Hint items, Bundle Length hint | Pass | `codec_hint.rs::round_trip_bundle_length_every_width`, `sender.rs::first_segment_has_bundle_length_hint` | 3.2, 3.7 | +| -04 §7.3 | Unrecognised messages and hints | Pass | `codec.rs::unknown_message_with_hints_relays_intact`, `codec_hint.rs::unknown_hint_preserved` | 3.2, 3.3 | +| -04 §10 | Resource limits | Pass | `receiver.rs::tiny_segment_flood_is_rejected_as_too_fragmented`, `transfer_that_would_exceed_the_retention_limit_is_rejected`, `budget.rs::receivers_sharing_a_budget_are_bounded_together` | 3.15, 3.16, 3.18 | +| -04 §12.1 | Bare and encapsulated bundles | Pass | `codec.rs::bare_bpv7_bundle_decoded_as_bundle_message`, `extent_hook_delivers_mid_pdu_bundle_and_iteration_continues` | 3.4, 3.8 | +| fec-02 §3, §4 | FEC message framing and transfer rules | Pass | `codec.rs::round_trip_fec_messages_with_fec_decoding_on`, `receiver.rs::changed_fec_instance_id_rejects_the_transfer` | 3.3, 3.12 | + +## 2. Test Inventory + +### Unit Tests + +Test names are listed per scenario in [`UTP-BTPU-01`](unit_test_plan.md) Section 3; this table groups them by file. + +| File | Tests | Plan Section | Scope | +| :--- | :--- | :--- | :--- | +| `tests/codec_header.rs` | 8 | 3.1 | Header encode/decode, 20-bit length bound | +| `tests/codec_message.rs` | 11 | 3.1 | Flags, type registry, frame classification | +| `tests/codec_hint.rs` | 17 | 3.2 | Hint items, Bundle Length widths, `Hints` set | +| `tests/codec.rs` | 32 | 3.3, 3.4, 3.5 | PDU decoding, fault containment, bare frames, extent hook, padding | +| `tests/transfer.rs` | 4 | 3.19 | `WindowSize`, the window-full error | +| `transfer.rs` (inline) | 21 | 3.6, 3.19 | Window classification, allocator, transfer ids and expiry across the roll-over and across a reset | +| `tests/sender.rs` | 65 | 3.7, 3.8, 3.9, 3.19 | Admission, segmentation, hints, packing, carried list, cancellation, sender configuration | +| `tests/send_handle.rs` | 23 | 3.7, 3.8, 3.9 | Bundles pushed through a `SendHandle`, push errors, segment cut strategy, push readiness, passing over waiting transfers, flush, cancellation, queue bytes | +| `sender.rs` (inline) | 5 | 3.7, 3.8 | Unsegmented ID wrap, segment cutting from pushed chunks, the flush index check, packing progress | +| `tests/receiver.rs` | 102 | 3.10 to 3.16, 3.19, 3.21 | Reassembly, window, cancel, FEC rules, events, hints, limits, receiver configuration, round trips | +| `receiver.rs` (inline) | 10 | 3.11, 3.15, 3.16, 3.18 | Closed-map pruning, reset, overhead and retention accounting, budget accounting | +| `tests/streaming.rs` | 22 | 3.17 | Streamed delivery, its ending events and hints, refusal | +| `tests/budget.rs` | 8 | 3.18 | Shared retention budget and CLA charges | +| `tests/config.rs` | 4 | 3.19 | serde form (`serde` feature) | +| `tests/tower.rs` | 17 | 3.20 | `Service`/`Stream` adapters (`tower` feature) | +| `tests/tunnel.rs` | 3 | 3.21 | Packet tunnel over lossless, lossy, and UDP loopback links | +| Doctests | 3 | 3.21 | README example (`rand` feature), `codec::BundleExtent`, `receiver::ReceiverConfig` | + +**Total: 355 tests (36 inline, 316 integration, 3 doctests) with all features; 330 with default features.** The 25 feature-gated tests are 17 `tower`, 4 `serde`, 3 `rand` seeding tests, and the README doctest. + +### Fuzz Tests + +| Target | File | Status | +| :--- | :--- | :--- | +| `decode` | `fuzz/fuzz_targets/decode.rs` | Implemented — one PDU under three decode configurations; re-encode and round-trip invariants | +| `receive` | `fuzz/fuzz_targets/receive.rs` | Implemented — a PDU sequence into one receiver under 128 configurations; delivery, streaming, budget, and event-order invariants | +| `send` | `fuzz/fuzz_targets/send.rs` | Implemented — begin, push, finish, enqueue, cancel, and drain (with and without flush) on one sender under 32 configurations and 256 PDU sizes, into a receiver over a lossless link; delivery, carried-list, PDU-size, push-readiness, and drain-to-empty invariants | + +**Total: 3 fuzz targets.** + +## 3. Coverage vs Plan + +Cross-reference against [`UTP-BTPU-01`](unit_test_plan.md): + +| Section | Scenario | Planned | Implemented | Status | +| :--- | :--- | :--- | :--- | :--- | +| 3.1 Message Header and Type Space | Header round trip, bounds, layout, flags, registry, frame classification | 7 | 7 | Complete | +| 3.2 Hint Items and Hint Sets | Bounds, widths, chains, unknown hints, folding, `Hints` | 6 | 6 | Complete | +| 3.3 PDU Decoding and Fault Containment | Core, FEC, padding, unknown relay, RFU flags, faults | 8 | 8 | Complete | +| 3.4 Bare and Encapsulated Bundles | Bare frame, hook, undelimitable bundle | 3 | 3 | Complete | +| 3.5 Encoding and Padding | Pad to target, encode failure | 2 | 2 | Complete | +| 3.6 Transfer Window and Number Allocation | Classification, roll-over, expiry, allocator | 4 | 4 | Complete | +| 3.7 Sender: Admission, Segmentation, and Hints | Admission, segmentation, pack-time cutting, send handle, segment cut strategy, push errors, queue bytes, push readiness, hints, identifiers | 10 | 10 | Complete | +| 3.8 Sender: Packing and Link Framing | Framing, padding floor, passing over transfers, flush, progress, carried list, debug output | 9 | 9 | Complete | +| 3.9 Sender: Window Release and Cancellation | Release, cancel transfer, cancel bundle, cancel pushed bundle, ID matching | 5 | 5 | Complete | +| 3.10 Receiver: Reassembly and Repeats | Delivery, conflicts, repeats, empty bundles, copy policy | 5 | 5 | Complete | +| 3.11 Receiver: Window, Cancellation, and Reset | Cancel, expiry, pruning, reset | 4 | 4 | Complete | +| 3.12 Receiver: FEC Transfer Rules | Mixing, configuration change, window and limits | 3 | 3 | Complete | +| 3.13 Receiver: Events and Fault Containment | Prior events kept, event list, debug output | 3 | 3 | Complete | +| 3.14 Receiver: Hints, Bare Frames, and the Extent Hook | Delivered hints, bare frames and hook | 2 | 2 | Complete | +| 3.15 Receiver: Per-Transfer Limits | Cap, budget, link-derived limit, enforcement | 4 | 4 | Complete | +| 3.16 Receiver: Retention Limit | Sizing, enforcement, accounting | 3 | 3 | Complete | +| 3.17 Receiver: Streamed Delivery and Refusal | In-order release, reordering, empty segments, ending, released segments, hints, FEC, drop ids, refusal | 9 | 9 | Complete | +| 3.18 Receiver: Shared Retention Budget | Charges, sharing, reason order, joining and leaving, accounting | 5 | 5 | Complete | +| 3.19 Configuration Types | Sender, receiver, window size, serde | 4 | 4 | Complete | +| 3.20 Tower Adapters | Services, backpressure, wakeups, stream | 4 | 4 | Complete | +| 3.21 End to End | Round trip, tunnel, doc examples | 3 | 3 | Complete | +| **Total** | | **103** | **103** | **100%** | + +## 4. Line Coverage + +> Current figures are generated — see the [coverage summary](../../docs/coverage_summary.md) (refreshed by `scripts/run_lcov.sh`) and the live coverage dashboards (CFLite fuzz coverage on gh-pages; CI-published coverage planned). Figures include the integration tests. + +``` +cargo llvm-cov test --package hardy-btpu --all-features --lcov --output-path lcov.info --html +lcov --summary lcov.info +``` + +### Fuzz Coverage + +``` +cd btpu +cargo +nightly fuzz coverage decode +cargo +nightly fuzz coverage receive +cargo +nightly fuzz coverage send +cargo +nightly cov -- export --format=lcov ... +lcov --summary ./fuzz/coverage//lcov.info +``` + +The two layers check different things. The unit tests pin specified behaviour to known inputs: what the receiver delivers, drops, or rejects, and why. The fuzz targets check that no input panics the codec or receiver, that the codec's encoder and decoder agree on everything the decoder yields, that the receiver's size and event-order guarantees hold for any PDU sequence, and that any sequence of sender calls delivers exactly the bundles neither cancelled nor finished short, once each, without a producer gated on push readiness ever waiting forever. + +## 5. Test Infrastructure + +* **Fixtures:** `tests/common/mod.rs` holds the shared builders: configurations and endpoints (`sender_config`, `receiver_config`, `sender`, `receiver`), messages (`bundle_msg`, `segment`, `end`, `cancel`), expected events (`received`, `received_with`, `dropped`, `rejected`, `expired`, `cancelled`, `none`) as an `Event` mirror of `ReceiverEvent` that names transfers by number, since a `TransferId` cannot be constructed outside the crate, payloads (`bundle`, `bpv7_like`), `encode`, and `is_within`. Each integration-test binary imports it with `mod common;`. +* **Inline tests** cover crate-private state that the public API cannot observe: the receiver's closed-transfer map, retention charges, and budget share, the window's transfer ids, and the sender's packing loop and pushed-chunk release. +* **Determinism:** no test sleeps or depends on timing. The `tower` tests poll by hand and observe wakeups through a waker that raises a flag; `tests/tunnel.rs` runs its UDP loopback in lockstep, with one datagram in flight, and uses a read timeout only to bound a regression. +* **Feature gates:** tests that need `tower`, `serde`, or `rand` are gated on that feature, so `cargo test -p hardy-btpu` passes without them. + +## 6. Key Gaps + +| Area | Gap | Severity | Notes | +| :--- | :--- | :--- | :--- | +| Priority interleaving (-04 §4.1) | Sender interleaves only by passing over a transfer that cannot supply; a ready transfer goes out ahead of everything behind it | Low | Optional in the draft; the receiver side is covered | +| Repetition (-04 §6) | Sender never repeats messages | Low | Optional in the draft; the receiver's duplicate handling is covered | +| FEC schemes | No FEC encoding or repair | Low | Out of scope for fec-02's framing; only framing and transfer rules are tested | +| `no_std` targets | Thumb builds run in CI only | Low | `thumbv7em-none-eabihf`, and `thumbv6m-none-eabi` with `critical-section` | +| Fuzz corpus | No seed corpus checked in | Low | See [`FUZZ-BTPU-01`](fuzz_test_plan.md) Section 7 | +| Interoperability | No test against another BTP-U implementation | Medium | No other implementation is available to test against | + +## 7. Conclusion + +The BTP-U crate has 355 tests (316 integration, 36 inline, 3 doctests) and 3 fuzz targets, covering 103 of 103 planned scenarios (100%). No LLRs are assigned; every implemented section of `draft-ietf-dtn-btpu-04` and the framing and transfer rules of `draft-ietf-dtn-btpu-fec-02` trace to passing tests. Line coverage is generated into the [coverage summary](../../docs/coverage_summary.md). The strongest coverage is fault containment in the codec and the receiver's bounded-memory limits, which are also the fuzz targets' focus. The remaining gaps are the optional sender behaviours (interleaving and repetition), FEC schemes, and the lack of an interoperability peer. diff --git a/btpu/docs/unit_test_plan.md b/btpu/docs/unit_test_plan.md new file mode 100644 index 000000000..b5646ec9b --- /dev/null +++ b/btpu/docs/unit_test_plan.md @@ -0,0 +1,303 @@ +# Unit Test Plan: BTP-U + +| Document Info | Details | + | ----- | ----- | +| **Functional Area** | Convergence Layer Protocol Engine | +| **Module** | `hardy-btpu` | +| **Requirements Ref** | [REQ-4](../../docs/requirements.md#req-4-alignment-with-on-going-dtn-standardisation), [REQ-14](../../docs/requirements.md#req-14-reliability) | +| **Standard Ref** | `draft-ietf-dtn-btpu-04`, `draft-ietf-dtn-btpu-fec-02` | +| **Test Suite ID** | UTP-BTPU-01 | +| **Version** | 1.1 | + +## 1. Introduction + +This document details the unit testing strategy for the `hardy-btpu` module, a `no_std` implementation of the Bundle Transfer Protocol - Unidirectional and the message framing of its FEC extension. The crate is a pure protocol library: a sender that segments and packs bundles into link-layer PDUs, a receiver that reassembles them under bounded memory, the codec both share, and optional `tower` adapters. + +**Scope:** + +* **Codec:** message header, type registry, hint items, PDU decoding with fault containment, bare and encapsulated bundle frames, padding. + +* **Transfer Window:** the Section 5 window and the sender's transfer-number allocator. + +* **Sender:** admission, segmentation, hints, PDU packing under fixed-size and variable link framing with an optional padding floor, the carried-bundle list, window release, and cancellation. + +* **Receiver:** reassembly, repeats, cancellation, window expiry, FEC transfer rules, event reporting, hints, the per-transfer and receiver-wide memory limits, streamed delivery, refusal, and a retention budget shared between receivers. + +* **Configuration:** the validated configuration newtypes and their serde form. + +* **Adapters and end to end:** the `tower` `Service`/`Stream` adapters and sender-to-receiver scenarios, including a packet tunnel over lossless, lossy, and UDP loopback links. + +No FEC scheme is implemented, so FEC coverage is limited to framing and the receiver's cancellation rules. Message repetition and priority interleaving are not implemented and have no tests; the sender interleaves transfers only by passing over one that cannot supply (3.8). + +## 2. Requirements Mapping + +No Low-Level Requirements are assigned to this crate in **[requirements.md](../../docs/requirements.md)**. BTP-U is a pre-standard IETF convergence-layer protocol, which places it under REQ-4; the requirements document's REQ-4 profiles (4.1 to 4.5) do not name it. Verification is therefore traced to the draft sections below, and the crate's reading of points the drafts leave open is recorded under Standards Compliance in [design.md](design.md). + +| Ref | Description | Draft Reference | + | ----- | ----- | ----- | +| **REQ-4** | Alignment with on-going DTN standardisation: pre-standard extensions are verified by automated unit or component testing. | `draft-ietf-dtn-btpu-04` Sections 3 to 9, 12; `draft-ietf-dtn-btpu-fec-02` Section 3 | +| **REQ-14** | Every external API is checked so that incorrect or malicious input does not cause a component to abort. | Sections 7.3, 10; see also [`FUZZ-BTPU-01`](fuzz_test_plan.md) | + +## 3. Unit Test Cases + +Tests live in the crate's `tests/` directory (public API) and in inline `#[cfg(test)]` modules (crate-private state). Integration tests are named `tests/.rs::`; inline tests are named `.rs::`, relative to `src/`. Shared fixtures are in `tests/common/mod.rs`. + +### 3.1 Message Header and Type Space (Sections 7, 7.1, 12.1, 12.3) + +*Objective: Verify the 4-byte message header and the classification of type values and frame first bytes.* + +| Test Scenario | Test | Expected Output | + | ----- | ----- | ----- | +| **Header round trip** | `tests/codec_header.rs::round_trip_basic`, `round_trip_with_hint_flag`, `round_trip_zero_length`, `round_trip_max_length`, `all_message_types_round_trip` | Decoded header equals the encoded one | +| **Content length over 20 bits** | `tests/codec_header.rs::length_above_20_bits_is_refused` | **Error** | +| **Truncated header** | `tests/codec_header.rs::decode_insufficient_data` | `InsufficientData` | +| **Wire layout** | `tests/codec_header.rs::wire_format_layout` | Bytes match the Section 7 layout | +| **Flags nibble** | `tests/codec_message.rs::message_flags_nibble_round_trips_all_bits` | All 16 values round-trip | +| **Type registry** | `tests/codec_message.rs::from_byte_accepts_known_types`, `from_byte_rejects_reserved_and_unassigned_values`, `is_fec_covers_exactly_the_four_extension_types`, `is_reserved_bpv6_covers_value`, `is_reserved_bpv7_covers_range` | Defined, FEC, reserved, and unassigned values classified as Section 12.1 assigns them | +| **Frame first byte** | `tests/codec_message.rs::frame_kind_empty_is_btpu`, `frame_kind_bpv6`, `frame_kind_bpv7_full_range`, `frame_kind_known_btpu_types_classify_as_pdu`, `frame_kind_unallocated_btpu_space_classifies_as_pdu` | 6 and 0x80..0x9F classify as bundles, everything else as a BTP-U PDU | + +### 3.2 Hint Items and Hint Sets (Sections 7.2, 7.3, 9.1) + +*Objective: Verify hint encoding, the Bundle Length hint, unknown-hint preservation, and one-item-per-type folding.* + +| Test Scenario | Test | Expected Output | + | ----- | ----- | ----- | +| **Type and value bounds** | `tests/codec_hint.rs::hint_value_holds_at_most_what_the_length_field_declares`, `hint_type_holds_at_most_seven_bits`, `hint_type_reports_the_wire_type` | Out-of-range values refused | +| **Bundle Length widths** | `tests/codec_hint.rs::round_trip_bundle_length_every_width`, `bundle_length_uses_the_shortest_width_either_side_of_each_boundary` | Round-trips; shortest width chosen | +| **Hint chains** | `tests/codec_hint.rs::round_trip_chained_hints`, `encoded_len_matches_actual`, `truncated_chain_errors` | Chains round-trip at the predicted length; truncation is an **Error** | +| **Unknown and malformed hints** | `tests/codec_hint.rs::unknown_hint_preserved`, `malformed_bundle_length_size_is_carried_as_unknown` | Carried as unknown, byte-exact | +| **Repeat folding** | `tests/codec_hint.rs::repeated_types_fold_latest_wins_in_first_appearance_order`, `long_chain_of_repeats_folds_to_one_item` | One item per type, latest value | +| **`Hints` set** | `tests/codec_hint.rs::hints_keep_one_item_per_type_latest_wins_in_type_order`, `hints_compare_by_items_not_insertion_order`, `malformed_bundle_length_and_bundle_length_replace_each_other`, `hints_get_and_remove_by_type`, `hints_encoded_len_matches_the_encoder` | Set semantics by type; encoded length agrees with the encoder | + +### 3.3 PDU Decoding and Fault Containment (Sections 3.1, 3.2, 7.3, 8) + +*Objective: Verify that a PDU decodes to its messages, that unknown messages relay intact, and that a fault costs only what it must.* + +| Test Scenario | Test | Expected Output | + | ----- | ----- | ----- | +| **Core messages** | `tests/codec.rs::round_trip_core_messages`, `multiple_messages_in_pdu` | Every core type round-trips, several to a PDU | +| **FEC messages** | `tests/codec.rs::round_trip_fec_messages_with_fec_decoding_on`, `fec_types_relay_as_unknown_by_default` | Decoded when enabled, relayed as unknown otherwise | +| **Padding** | `tests/codec.rs::indefinite_padding_skipped`, `all_zeros_pdu` | Padding yields no messages | +| **Unknown message relay** | `tests/codec.rs::unknown_type_preserved`, `unknown_message_with_hints_relays_intact`, `unknown_message_rfu_flag_bits_relay_intact`, `malformed_hints_in_unknown_message_do_not_poison_pdu` | Re-encodes byte-exact | +| **Unknown construction** | `tests/codec.rs::unknown_cannot_carry_defined_or_reserved_type`, `unknown_encodes_exactly_the_types_the_decoder_reads_as_unknown` | Only unassigned types accepted | +| **RFU flags on a known type** | `tests/codec.rs::rfu_flag_bits_on_known_type_are_ignored` | Message decoded normally | +| **Interior fault** | `tests/codec.rs::malformed_interior_skips_only_that_message`, `short_bodies_are_contained_to_their_message`, `malformed_bundle_length_hint_keeps_the_segment` | That message skipped, iteration continues | +| **Framing fault** | `tests/codec.rs::length_past_buffer_is_terminal`, `truncation_at_every_offset_keeps_the_whole_messages_before_it` | Iteration ends; the prefix is kept | + +### 3.4 Bare and Encapsulated Bundles (Sections 7.3, 12.1) + +*Objective: Verify bundles whose first byte is a reserved type value, with and without the bundle-extent hook.* + +| Test Scenario | Test | Expected Output | + | ----- | ----- | ----- | +| **Bare frame** | `tests/codec.rs::bare_bpv6_bundle_decoded_as_bundle_message`, `bare_bpv7_bundle_decoded_as_bundle_message`, `bare_bundle_after_indefinite_padding_is_the_rest_of_the_frame`, `bare_frame_zero_fill_is_delivered_as_bundle_bytes_without_hook` | Rest of the frame delivered as one bundle | +| **Extent hook** | `tests/codec.rs::extent_hook_trims_bare_frame_padding`, `extent_hook_delivers_mid_pdu_bundle_and_iteration_continues` | Bundle delimited by the hook; iteration continues | +| **Undelimitable bundle** | `tests/codec.rs::mid_pdu_encapsulated_bundle_without_hook_is_terminal`, `extent_hook_declining_is_terminal`, `extent_hook_claiming_zero_bytes_is_terminal`, `extent_hook_overrunning_the_pdu_is_terminal` | Terminal **Error**; the prefix is kept | + +### 3.5 Encoding and Padding (Sections 3.2, 8.5, 8.6) + +*Objective: Verify PDU padding and encoder error behaviour.* + +| Test Scenario | Test | Expected Output | + | ----- | ----- | ----- | +| **Pad to target** | `tests/codec.rs::pad_pdu_fills_to_target`, `pad_pdu_small_remainder`, `pad_pdu_beyond_max_content_length_chains_messages` | Exactly the target length, chaining padding messages past the length-field maximum | +| **Encode failure** | `tests/codec.rs::encode_errors_leave_the_buffer_untouched` | **Error**; buffer unchanged | + +### 3.6 Transfer Window and Number Allocation (Section 5) + +*Objective: Verify the receiver's window classification and the sender's span-gated allocator, across the 2³² roll-over.* + +| Test Scenario | Test | Expected Output | + | ----- | ----- | ----- | +| **Classification** | `transfer.rs::first_transfer_is_new`, `same_transfer_is_in_progress`, `sequential_transfers_advance`, `old_transfer_outside_window`, `new_transfer_boundary_is_half_space_plus_half_window`, `odd_window_size_rounds_the_margin_down` | New, in progress, or outside, per Section 5 | +| **Roll-over** | `transfer.rs::wraparound`, `ids_cover_exactly_the_window_behind_a_greatest_of_u32_max`, `an_id_expires_once_the_window_is_a_full_width_past_it`, `expiry_agrees_with_validity_across_the_wrap` | Same results across the wrap | +| **Expiry and reset** | `transfer.rs::expired_ids_are_those_a_full_window_behind`, `reset_forgets_the_greatest`, `ids_after_a_reset_are_above_every_id_before_it`, `a_reset_before_any_transfer_changes_nothing` | Expired ids reported; reset forgets the greatest, and ids keep counting across it | +| **Allocator** | `transfer.rs::allocate_sequential`, `allocator_wraps`, `release_of_oldest_frees_slot`, `window_gates_on_span_not_count`, `span_gate_survives_wraparound`, `release_of_unknown_number_is_ignored` | Allocation gated on the span of outstanding numbers | + +### 3.7 Sender: Admission, Segmentation, and Hints (Sections 4, 8.1 to 8.3, 9.1) + +*Objective: Verify what `enqueue` and `begin` accept, how bundles are cut and pushed, and where hints travel.* + +| Test Scenario | Test | Expected Output | + | ----- | ----- | ----- | +| **Admission** | `tests/sender.rs::empty_bundle_rejected_at_enqueue`, `minimum_pdu_size_cannot_queue_an_undrainable_message`, `segmenting_floor_grows_with_the_bundle_length_hint`, `pdu_too_small_to_segment_leaves_window_untouched`, `max_pdu_size_bundle_encodes_without_panic`; `tests/send_handle.rs::begin_refuses_a_bundle_whose_last_index_could_reach_u32_max` | Undeliverable bundles refused before taking a window slot, including one whose last Segment Index could reach `u32::MAX` | +| **Segmentation** | `tests/sender.rs::small_bundle_no_segmentation`, `large_bundle_segmented` | A fitting bundle is one Bundle Message; a larger one is segments and an End | +| **Pack-time cutting** | `tests/sender.rs::a_segment_is_cut_short_to_fill_the_tail_of_a_pdu`, `a_tail_under_half_a_segment_is_left_to_padding`; `sender.rs::a_last_segment_takes_any_tail_it_fits_and_a_cut_keeps_its_length` | A segment fills a PDU's tail only if it holds half a segment; a last segment takes any tail it fits; a segment's copies share its length | +| **Send handle** | `tests/send_handle.rs::a_pushed_bundle_goes_out_as_the_same_bundle_enqueued`, `a_fitting_bundle_is_queued_when_its_last_byte_is_pushed`, `a_segment_goes_out_once_its_bytes_are_pushed`, `an_empty_chunk_does_nothing`, `begin_refuses_what_enqueue_refuses`, `a_bare_bundle_must_start_with_a_bundle_reserved_byte`; `sender.rs::a_segment_waits_for_all_of_its_bytes` | Pushed chunks go out as the enqueued bundle would; a segment waits for all its bytes | +| **Segment cut strategy** | `tests/send_handle.rs::a_half_cut_sends_half_a_segment_without_waiting_for_the_rest`; `tests/config.rs::sender_config_round_trips_as_kebab_case_integers` | Under `Half`, a segment goes out once half its bytes are pushed and not before; under `Full` it waits for all of them | +| **Push errors** | `tests/send_handle.rs::an_overrun_is_refused_and_leaves_the_bundle_as_it_was`, `an_underrun_at_finish_cancels_a_started_transfer`, `an_underrun_at_finish_drops_a_partly_pushed_fitting_bundle`, `a_push_after_cancelling_by_id_is_not_in_progress` | `Overrun` changes nothing; `Underrun` cancels; a cancelled bundle is `NotInProgress` | +| **Queue bytes** | `tests/sender.rs::a_segmented_transfer_frees_queue_bytes_as_its_segments_are_packed`; `tests/send_handle.rs::queued_bytes_bound_reports_full_on_pushed_bytes` | Pushed and enqueued bytes counted until packed; never refused | +| **Push readiness** | `tests/send_handle.rs::a_producer_gated_on_push_readiness_never_waits_on_itself`, `push_readiness_past_the_bound_follows_whether_the_bundle_waits`, `push_readiness_admits_a_fitting_bundle_and_one_not_in_progress` | Past the bound, a push is ready only while its bundle cannot go out without more bytes; a producer gated on it completes under a bound smaller than a segment | +| **Hints** | `tests/sender.rs::first_segment_has_bundle_length_hint`, `caller_hints_ride_first_segment_with_derived_bundle_length`, `repeated_caller_hint_types_go_out_once_latest_wins`, `first_segment_capacity_reduced_by_exactly_the_hint_bytes`, `caller_hints_ride_unsegmented_bundle_message`, `caller_hint_of_the_bundle_length_type_is_discarded_whatever_its_shape` | Hints on segment 0 or the Bundle Message; Bundle Length derived by the sender | +| **Identifiers** | `sender.rs::unsegmented_ids_wrap_across_message_and_bare`; `tests/sender.rs::from_rng_seeds_initial_transfer_number`, `try_from_rng_seeds_initial_transfer_number`, `try_from_rng_returns_the_rng_error` | IDs wrap; seeding as 3.6 | + +### 3.8 Sender: Packing and Link Framing (Sections 3, 3.2, 7.3, 8.5) + +*Objective: Verify PDU contents under each link framing, packing progress, and the carried-bundle list.* + +| Test Scenario | Test | Expected Output | + | ----- | ----- | ----- | +| **Variable framing** | `tests/sender.rs::variable_link_pdus_are_not_padded`, `variable_link_segmented_pdus_fill_the_pdu_except_the_last` | Unpadded PDUs sized to content | +| **Padding floor** | `tests/sender.rs::a_short_pdu_is_padded_up_to_the_floor`, `a_gap_too_small_for_a_padding_header_is_zero_filled`, `a_pdu_at_or_over_the_floor_is_not_padded`, `a_floor_above_the_pdu_size_pads_to_the_pdu_size`, `a_bundle_shorter_than_the_floor_is_a_padded_message_not_a_bare_frame`, `a_bare_frame_at_the_floor_is_sent_unpadded`, `a_floor_above_the_pdu_size_still_allows_a_full_size_bare_frame` | Short PDUs padded to the floor, capped at the PDU size; a bundle under the floor is a Bundle Message, never a bare frame | +| **Bare framing** | `tests/sender.rs::bare_framing_emits_the_bundle_bytes_alone_using_the_whole_pdu`, `bare_frames_keep_queue_order_and_never_share_a_pdu`, `bare_framing_keeps_hinted_bundles_framed`, `bare_framing_requires_a_bundle_reserved_first_byte`, `bare_bundles_count_against_send_queue_bytes` | Eligible bundles alone in a PDU, in queue order | +| **Passing over transfers** | `tests/send_handle.rs::a_transfer_waiting_on_its_producer_is_passed_over`; `tests/sender.rs::a_transfer_without_room_in_the_tail_is_passed_over_but_a_message_is_not` | A transfer waiting on its producer or without room is passed over; a message that does not fit ends the PDU | +| **Flush** | `tests/send_handle.rs::a_flush_sends_what_a_waiting_transfer_has_buffered`, `a_flush_fills_the_room_left_with_waiting_transfers_in_queue_order`, `a_flush_does_not_reorder_past_a_message_that_does_not_fit`; `sender.rs::a_flush_is_refused_if_the_rest_could_need_index_u32_max` | After the ready messages, each waiting transfer in queue order sends what it has buffered, cut to the room; a partly pushed fitting bundle is not flushed; the flush stops at a message that does not fit; a cut that could leave the rest of the bundle needing index `u32::MAX` is refused | +| **Packing progress** | `sender.rs::oversized_queued_message_goes_out_alone_and_the_queue_moves_on` | Every PDU consumes at least one message | +| **Carried list** | `tests/sender.rs::pdu_lists_every_bundle_it_carries_and_flags_those_it_completes`, `middle_segments_list_their_transfer_as_incomplete`, `bare_frame_pdu_lists_its_bundle_as_complete`, `cancelled_transfer_is_never_listed_as_complete` | Every bundle with bytes listed; completion flagged once | +| **Carried list storage** | `tests/sender.rs::next_pdu_into_replaces_the_callers_list`, `next_pdu_list_moves_to_the_heap_only_past_the_inline_entries`, `reused_list_keeps_its_heap_buffer_for_a_pdu_that_would_fit_inline`, `with_capacity_below_the_inline_entries_stays_inline`, `list_sized_by_the_bundle_bound_holds_the_fullest_pdu_of_valid_bundles`, `lists_compare_by_entries_not_storage`; `tests/send_handle.rs::every_outstanding_transfer_can_end_in_one_pdu_within_the_list_bound` | Inline up to four entries; reused buffers kept; the documented bound holds a PDU ending every outstanding transfer | +| **Debug output** | `tests/sender.rs::debug_summarises_the_queue_instead_of_printing_it` | No bundle bytes printed | + +### 3.9 Sender: Window Release and Cancellation (Sections 4.2, 5, 8.4) + +*Objective: Verify the self-releasing window and every `cancel` outcome.* + +| Test Scenario | Test | Expected Output | + | ----- | ----- | ----- | +| **Window release** | `tests/sender.rs::window_slot_is_released_when_the_transfer_end_is_packed`, `cancelling_the_newest_transfer_keeps_the_window_span` | Slot freed when the End is packed; span rule kept | +| **Cancel a transfer** | `tests/sender.rs::cancel_before_any_emission_queues_no_cancel_message`, `cancel_after_partial_emission_discards_the_rest_and_queues_a_cancel`, `cancel_goes_ahead_of_the_queued_backlog`, `cancel_after_the_last_bytes_are_packed_changes_nothing`, `bogus_cancel_is_a_noop` | Transfer Cancel only if part was emitted, at the queue front | +| **Cancel an unsegmented bundle** | `tests/sender.rs::cancel_removes_a_queued_bundle_message_and_sends_nothing_for_it`, `cancel_removes_a_queued_bare_frame`, `cancel_leaves_queued_bare_bundles_intact`, `cancelling_a_queued_bundle_frees_send_queue_bytes` | Entry removed; nothing sent | +| **Cancel a pushed bundle** | `tests/send_handle.rs::cancelling_a_handle_before_any_push_sends_nothing`, `cancelling_a_handle_drops_its_pushed_bytes` | Pushed bytes dropped; no Cancel unless part was emitted | +| **ID matching** | `tests/sender.rs::cancel_matches_the_id_variant_as_well_as_the_number`, `cancel_by_one_variant_leaves_the_others_with_that_number` | Only the named ID cancelled | + +### 3.10 Receiver: Reassembly and Repeats (Sections 4, 6, 8.1 to 8.3) + +*Objective: Verify delivery from any arrival order, sequence checks, repeat handling, and the copy policy.* + +| Test Scenario | Test | Expected Output | + | ----- | ----- | ----- | +| **Delivery** | `tests/receiver.rs::bundle_message_immediate`, `two_segment_transfer`, `out_of_order_completes_on_end_recheck`, `end_before_late_segment_completes_on_the_segment`, `empty_end_completes_the_transfer`, `empty_middle_segment_counts_toward_completion`, `final_segment_index_of_u32_max_leaves_the_transfer_open` | `Received` once all segments are held | +| **Sequence conflicts** | `tests/receiver.rs::conflicting_end_dropped_and_transfer_still_completes`, `segment_beyond_final_index_dropped`, `end_below_seen_segment_dropped` | `MessageDropped`; transfer unaffected | +| **Repeats** | `tests/receiver.rs::duplicate_segment_dropped_and_first_copy_kept`, `duplicate_end_dropped`, `duplicate_hints_are_not_applied`, `repeated_messages_of_a_delivered_transfer_do_not_redeliver` | `DropReason::Duplicate` or the closing reason; no second delivery | +| **Empty bundles** | `tests/receiver.rs::empty_bundle_message_rejected`, `transfer_with_no_data_is_rejected`, `empty_single_end_is_rejected_through_receive_pdu` | `BundleRejected` or `TransferRejected` (`Empty`); no bundle is zero bytes | +| **Copy policy** | `tests/receiver.rs::single_segment_transfer_shares_the_segment_bytes`, `segments_shorter_than_half_the_pdu_are_copied_out_of_it`, `retained_hint_values_are_copied_out_of_the_pdu` | Small fragments copied so no PDU is pinned | + +### 3.11 Receiver: Window, Cancellation, and Reset (Sections 4.2, 5, 8.4) + +*Objective: Verify Transfer Cancel handling, window expiry order, and the closed-transfer memory.* + +| Test Scenario | Test | Expected Output | + | ----- | ----- | ----- | +| **Transfer Cancel** | `tests/receiver.rs::repeated_cancel_is_idempotent`, `cancel_of_unknown_transfer_ignored`, `cancel_of_an_in_window_transfer_before_its_segments_is_remembered` | Applied in the window, remembered, ignored outside | +| **Window and expiry** | `tests/receiver.rs::outside_window_drop_reported`, `window_wraparound_with_live_transfers`, `transfers_straddling_the_wrap_expire_oldest_first`, `number_behind_a_just_wrapped_greatest_expires_before_it`, `number_more_than_half_the_space_ahead_advances_the_window` | `TransferExpired` oldest first, across the wrap | +| **Closed-transfer pruning** | `receiver.rs::closed_map_pruned_by_window_advance`, `closed_map_pruned_across_the_wrap` | Bounded by the window | +| **Reset** | `tests/receiver.rs::reset_forgets_delivered_transfers`, `reset_accepts_a_restarted_sender`; `receiver.rs::reset_clears_all_state` | All state cleared | + +### 3.12 Receiver: FEC Transfer Rules (`draft-ietf-dtn-btpu-fec-02` Sections 3, 3.1, 3.2) + +*Objective: Verify the cancellation rules that apply without an FEC scheme.* + +| Test Scenario | Test | Expected Output | + | ----- | ----- | ----- | +| **Core and FEC mixing** | `tests/receiver.rs::fec_message_on_core_transfer_rejects_it`, `core_message_on_fec_transfer_rejects_it`, `core_fec_mixing_rejects_through_receive_pdu` | `TransferRejected` (`FecCoreMixing`) | +| **Configuration change** | `tests/receiver.rs::fec_messages_with_one_configuration_keep_the_transfer_open`, `changed_fec_instance_id_rejects_the_transfer`, `changed_fec_encoding_id_rejects_the_transfer`, `switching_between_pre_agreed_and_explicit_fec_rejects_the_transfer` | `TransferRejected` (`FecConfigurationChanged`) | +| **Window, cancel, and limits** | `tests/receiver.rs::fec_transfer_expires_like_a_core_one`, `cancel_closes_an_fec_transfer`, `fec_types_do_not_touch_the_window_unless_enabled`, `fec_transfer_promising_an_oversized_bundle_is_rejected`, `fec_hint_bytes_count_against_retention` | FEC transfers share the core window and limits | + +### 3.13 Receiver: Events and Fault Containment (Section 7.3) + +*Objective: Verify that `receive_pdu` is infallible and that a fault never discards earlier events.* + +| Test Scenario | Test | Expected Output | + | ----- | ----- | ----- | +| **Faults keep prior events** | `tests/receiver.rs::malformed_message_mid_pdu_keeps_prior_events_and_continues`, `malformed_pdu_keeps_prior_events` | `MalformedMessage` or `MalformedPdu` after the earlier events | +| **Event list** | `tests/receiver.rs::every_message_in_a_pdu_can_produce_an_event`, `receive_pdu_into_replaces_the_callers_list`, `receive_pdu_into_reuses_the_callers_allocation` | One event per message at most; caller's allocation reused | +| **Debug output** | `tests/receiver.rs::debug_summarises_held_and_closed_transfers` | No bundle bytes printed | + +### 3.14 Receiver: Hints, Bare Frames, and the Extent Hook (Sections 7.3, 9.1) + +*Objective: Verify the hints reported with a bundle and the receive side of bare and encapsulated bundles.* + +| Test Scenario | Test | Expected Output | + | ----- | ----- | ----- | +| **Delivered hints** | `tests/receiver.rs::bundle_received_surfaces_transfer_hints`, `bundle_message_hints_deduped_latest_wins`, `bundle_length_hint_on_bundle_message_is_ignored`, `bundle_length_hint_smaller_than_the_data_still_delivers`, `malformed_bundle_length_hint_does_not_discard_the_segment` | One hint per type, latest wins; Bundle Length advisory | +| **Bare frames and the hook** | `tests/receiver.rs::bare_frame_through_receive_pdu`, `bundle_extent_hook_trims_padding_and_steps_over_mid_pdu_bundles`, `bundle_extent_hook_survives_its_own_panic`, `mid_pdu_bundle_without_hook_ends_the_pdu` | Delivered as `Received`; a panicking hook ends only the PDU | + +### 3.15 Receiver: Per-Transfer Limits (Section 10; local policy) + +*Objective: Verify the transfer cap, the bookkeeping budget, and the optional segment limit.* + +| Test Scenario | Test | Expected Output | + | ----- | ----- | ----- | +| **Transfer cap** | `tests/receiver.rs::transfer_is_rejected_as_its_data_passes_the_cap`, `oversized_bundle_message_rejected_with_event`, `transfer_at_exactly_the_cap_is_accepted_and_one_byte_over_rejected`, `bundle_of_exactly_the_cap_is_delivered_in_one_byte_segments`, `bundle_length_hint_rejects_before_accumulation` | `TooLarge` past the cap, delivered at it | +| **Bookkeeping budget** | `tests/receiver.rs::tiny_segment_flood_is_rejected_as_too_fragmented`, `retained_hint_bytes_count_against_the_bookkeeping_budget`; `receiver.rs::segments_and_hints_are_charged_to_overhead_not_data`, `malformed_bundle_length_is_charged_and_bundle_length_is_not` | `TooFragmented` past the budget | +| **Link-derived segment limit** | `tests/receiver.rs::link_derived_limit_is_four_times_the_reference_count`, `link_derived_limit_never_falls_below_64`, `link_derived_limit_of_a_pdu_no_larger_than_the_framing_assumes_one_byte_segments`, `link_derived_limit_caps_segment_data_at_the_content_length_ceiling`, `link_derived_limit_saturates_at_u32_max` | Documented formula and bounds | +| **Segment limit enforcement** | `tests/receiver.rs::small_pdu_link_delivers_a_bundle_the_per_segment_charge_rejects`, `bundle_of_exactly_the_cap_is_delivered_with_a_link_derived_limit`, `transfer_of_exactly_the_segment_limit_completes`, `segment_beyond_the_limit_rejects_the_transfer_as_too_fragmented`, `repeated_segments_do_not_consume_the_allowance`, `transfer_end_cannot_bypass_the_segment_limit`, `too_large_takes_precedence_over_too_fragmented`, `bundle_length_hint_does_not_raise_the_segment_limit`, `large_pdus_do_not_raise_the_segment_limit`, `without_a_segment_limit_the_per_segment_charge_still_applies`; `receiver.rs::segment_over_the_limit_is_counted_but_never_stored` | Limit enforced on distinct segments; the over-limit segment is not stored | + +### 3.16 Receiver: Retention Limit (Section 10; local policy) + +*Objective: Verify the receiver-wide bound on retained state and its sizing helper.* + +| Test Scenario | Test | Expected Output | + | ----- | ----- | ----- | +| **Sizing** | `tests/receiver.rs::retention_for_transfers_is_the_finest_segmentation_charge`, `max_retained_bytes_reports_the_limit_in_effect`, `smallest_retention_limit_admits_only_what_is_charged_nothing`, `receiver_config_sizing_table_figures`, `filled_pdus_charge_at_most_the_documented_figure`, `one_transfer_can_be_charged_exactly_for_transfers` | Matches the `ReceiverConfig` rustdoc figures | +| **Enforcement** | `tests/receiver.rs::transfer_that_would_exceed_the_retention_limit_is_rejected`, `retention_limit_below_one_transfers_allowance_is_enforced`, `retention_limit_above_one_transfers_allowance_is_enforced`, `empty_segments_count_against_the_retention_limit_under_a_segment_limit` | `ReceiverFull` for the transfer that grew | +| **Accounting** | `tests/receiver.rs::retention_is_released_by_cancel_expiry_and_rejection`, `retained_bytes_reports_the_charged_state`; `receiver.rs::retained_segment_is_charged_what_it_keeps_alive`, `duplicate_segment_is_not_charged_twice`, `retained_always_equals_the_sum_of_held_charges` | Charged total equals the sum of held charges under either delivery, released on close | + +### 3.17 Receiver: Streamed Delivery and Refusal (Sections 4, 6, 8.1 to 8.4; local policy) + +*Objective: Verify in-order release under `Delivery::Streamed`, the events that end a streamed transfer, and `Receiver::refuse`.* + +| Test Scenario | Test | Expected Output | + | ----- | ----- | ----- | +| **In-order release** | `tests/streaming.rs::in_order_segments_are_released_as_views_of_their_pdus`, `a_lone_end_starts_and_finishes`, `bundle_messages_are_received_whole` | `TransferStarted`, then `TransferData` and `TransferFinished` holding views of their PDUs, nothing retained; Bundle messages still `Received` | +| **Reordering** | `tests/streaming.rs::a_filled_gap_releases_the_run_behind_it`, `an_end_arriving_before_the_gap_fills_finishes_with_the_run`, `an_end_naming_a_released_segment_finishes_with_no_data` | One event per segment of a released run, in index order | +| **Empty segments** | `tests/streaming.rs::empty_segments_are_not_reported`, `a_transfer_with_no_data_is_rejected_without_starting` | No event for an empty segment; an all-empty transfer `TransferRejected` (`Empty`) and never started | +| **Ending a started transfer** | `tests/streaming.rs::a_started_transfer_can_be_cancelled`, `a_started_transfer_can_expire`, `released_bytes_count_toward_the_transfer_size_cap` | `TransferCancelled`, `TransferExpired`, or `TransferRejected` for the started transfer | +| **Released segments** | `tests/streaming.rs::a_repeat_of_a_released_segment_is_a_duplicate`, `an_end_below_a_released_segment_conflicts`, `both_deliveries_reject_at_the_same_segment` | `Duplicate` or `SegmentIndexConflict`; the segment limit and bookkeeping charge reject at the same message under either delivery | +| **Hints** | `tests/streaming.rs::hints_are_reported_when_they_change`, `hints_before_the_first_data_go_out_on_started` | `TransferStarted` carries the full set, empty if none; later events carry it on the first event after a change, `None` otherwise | +| **FEC transfers** | `tests/streaming.rs::fec_transfers_release_nothing` | No events | +| **Drop ids** | `tests/streaming.rs::a_dropped_message_names_its_transfer_by_id_inside_the_window` | `MessageDropped` carries the id `TransferStarted` gave for `Duplicate` and `Delivered`, and none for a number outside the window | +| **Refusal** | `tests/streaming.rs::refuse_closes_a_held_transfer`, `refuse_ignores_a_finished_transfer`, `refuse_ignores_an_expired_transfer`, `refuse_ignores_an_id_from_before_a_reset` | State discarded and later messages dropped as `Refused`; `false` for an id the receiver no longer holds | + +### 3.18 Receiver: Shared Retention Budget (Section 10; local policy) + +*Objective: Verify that receivers and the CLA sharing a `RetentionBudget` are bounded together and that every charge is returned.* + +| Test Scenario | Test | Expected Output | + | ----- | ----- | ----- | +| **Charges** | `tests/budget.rs::a_charge_is_released_when_dropped`, `concurrent_charges_never_exceed_the_limit` | `try_charge` fails rather than overshooting, across threads; dropping a `Charge` releases it | +| **Sharing** | `tests/budget.rs::receivers_sharing_a_budget_are_bounded_together`, `a_cla_charge_leaves_less_for_receivers` | `TransferRejected` (`BudgetFull`) once the budget is spent; dropping a receiver or a charge frees its share | +| **Reason order** | `tests/budget.rs::receiver_full_is_reported_before_budget_full` | `ReceiverFull` when both limits are exceeded | +| **Joining and leaving** | `tests/budget.rs::joining_a_budget_charges_what_is_already_held`, `a_reset_releases_the_receivers_share` | What is held is moved to the new budget whatever its limit, and released by close and reset | +| **Accounting** | `tests/budget.rs::released_segments_are_not_charged`; `receiver.rs::a_budget_below_the_receiver_limit_tracks_it_exactly` | The budget's charge equals the receiver's total under either delivery, released segments uncharged | + +### 3.19 Configuration Types + +*Objective: Verify the validated configuration newtypes, their defaults, and their serde form.* + +| Test Scenario | Test | Expected Output | + | ----- | ----- | ----- | +| **Sender configuration** | `tests/sender.rs::config_defaults`, `pdu_size_boundaries`, `pdu_size_default_is_the_const`, `pdu_size_parses_and_formats_as_its_integer`, `send_queue_bytes_zero_rejected`, `send_queue_bytes_converts_to_and_from_non_zero`, `send_queue_bytes_parses_and_formats_as_its_integer` | Range-checked; parse and format as integers | +| **Receiver configuration** | `tests/receiver.rs::config_defaults`, `max_transfer_size_zero_rejected`, `max_transfer_size_converts_to_and_from_non_zero`, `max_transfer_size_parses_and_formats_as_its_integer`, `max_segments_zero_rejected`, `max_segments_converts_to_and_from_non_zero`, `max_segments_parses_and_formats_as_its_integer`, `max_retained_bytes_zero_rejected`, `max_retained_bytes_converts_parses_and_formats_as_its_integer` | Range-checked; parse and format as integers; whole delivery by default | +| **Window size** | `tests/transfer.rs::window_size_boundaries`, `window_size_default_is_recommended`, `window_size_parses_and_formats_as_its_integer`, `window_full_names_the_window_size`; `transfer.rs::window_and_allocator_report_their_size` | Range-checked; default is the draft's recommendation | +| **Serde** (`serde`) | `tests/config.rs::sender_config_round_trips_as_kebab_case_integers`, `receiver_config_round_trips_as_kebab_case_integers`, `missing_fields_take_their_defaults`, `out_of_range_values_are_rejected_on_deserialize` | Kebab-case integers; defaults filled; out-of-range refused | + +### 3.20 Tower Adapters (`tower`) + +*Objective: Verify the `Service` and `Stream` adapters, their backpressure, and their wakeups.* + +| Test Scenario | Test | Expected Output | + | ----- | ----- | ----- | +| **Services** | `tests/tower.rs::receiver_service_round_trip`, `sender_service_enqueue_then_stream_drain`, `sender_service_with_layer`, `sender_service_send_request_carries_hints` | Requests reach the sender and receiver unchanged | +| **Backpressure** | `tests/tower.rs::sender_service_poll_ready_blocks_until_the_oldest_end_drains`, `sender_service_poll_ready_blocks_when_send_queue_full`, `producers_admitted_under_one_lock_hold_never_see_window_full` | `poll_ready` pending while the window or queue is full | +| **Wakeups** | `tests/tower.rs::sender_drain_wakes_pending_enqueue_task_when_window_full`, `sender_cancel_wakes_pending_enqueue_task`, `every_parked_producer_is_woken`, `cancelling_a_queued_bundle_wakes_a_producer_parked_on_queue_bytes`, `draining_a_bare_frame_wakes_a_producer_parked_on_queue_bytes`, `sender_enqueue_wakes_pending_drain_task`, `a_push_that_readies_a_segment_wakes_the_pending_drain_task` | Every capacity change wakes the parked side | +| **Stream** | `tests/tower.rs::sender_stream_pending_when_idle`, `sender_stream_yields_the_bundles_each_pdu_carries`, `stream_stays_pending_after_cancel_empties_the_queue` | Pending when idle, never finishes | + +### 3.21 End to End + +*Objective: Verify the sender and receiver together, including over a real socket.* + +| Test Scenario | Test | Expected Output | + | ----- | ----- | ----- | +| **Round trip** | `tests/receiver.rs::sender_receiver_round_trip`, `sender_receiver_round_trip_small` | Every bundle delivered intact | +| **Packet tunnel** | `tests/tunnel.rs::lossless_link_delivers_every_packet_in_order`, `lossy_link_delivers_exactly_the_packets_it_did_not_damage`, `udp_loopback_carries_every_packet` | Undamaged packets delivered in order; damaged transfers expire | +| **Documentation examples** | Doctests: the README example (`rand`), `codec::BundleExtent`, `receiver::ReceiverConfig` | Compile and run | + +## 4. Execution & Pass Criteria + +* **Command:** `cargo test -p hardy-btpu --all-features`. The `tower`, `serde`, and `rand` scenarios need their features; `cargo test -p hardy-btpu` runs the rest. + +* **`no_std` builds:** CI builds the crate for `thumbv7em-none-eabihf`, and for `thumbv6m-none-eabi` with `critical-section`. + +* **Pass Criteria:** All tests listed above must return `ok`. + +* **Coverage Target:** > 90% line coverage for `src/codec/`, `src/transfer.rs`, `src/sender.rs`, and `src/receiver.rs`. diff --git a/btpu/fuzz/Cargo.toml b/btpu/fuzz/Cargo.toml new file mode 100644 index 000000000..da5e29010 --- /dev/null +++ b/btpu/fuzz/Cargo.toml @@ -0,0 +1,35 @@ +[package] +name = "hardy-btpu-fuzz" +version = "0.0.0" +publish = false +edition.workspace = true +rust-version.workspace = true + +[package.metadata] +cargo-fuzz = true + +[dependencies] +libfuzzer-sys = "0.4" +bytes = "1" +hardy-btpu = { path = ".." } + +[[bin]] +name = "decode" +path = "fuzz_targets/decode.rs" +test = false +bench = false +doc = false + +[[bin]] +name = "receive" +path = "fuzz_targets/receive.rs" +test = false +bench = false +doc = false + +[[bin]] +name = "send" +path = "fuzz_targets/send.rs" +test = false +bench = false +doc = false diff --git a/btpu/fuzz/fuzz_targets/decode.rs b/btpu/fuzz/fuzz_targets/decode.rs new file mode 100644 index 000000000..2c7fccbbe --- /dev/null +++ b/btpu/fuzz/fuzz_targets/decode.rs @@ -0,0 +1,67 @@ +#![no_main] + +use bytes::{Bytes, BytesMut}; +use hardy_btpu::codec::{ + DecodeOptions, Error, decode_pdu_with, encode_message, encoded_message_len, + header::{HEADER_SIZE, MAX_CONTENT_LENGTH}, + message::Message, +}; +use libfuzzer_sys::fuzz_target; + +// Every 0x9F "bundle" is taken to be eight bytes long, so the extent hook +// path is exercised alongside the hookless one. +fn eight_byte_bundles(bytes: &[u8]) -> Option { + (bytes.first() == Some(&0x9F) && bytes.len() >= 8).then_some(8) +} + +fuzz_target!(|data: &[u8]| { + let pdu = Bytes::copy_from_slice(data); + let hook = &eight_byte_bundles; + for options in [ + DecodeOptions::default(), + DecodeOptions { + fec: true, + bundle_extent: None, + }, + DecodeOptions { + fec: true, + bundle_extent: Some(hook), + }, + ] { + let mut iter = decode_pdu_with(pdu.clone(), options); + for item in iter.by_ref() { + let Ok(msg) = item else { + continue; + }; + // Everything the decoder yields must re-encode, at exactly the + // length it predicts. An unknown message re-encodes to the + // bytes it was decoded from (byte-exact relay). The one + // exception is an encapsulated bundle longer than a message's + // content field can hold: it was never a message, and encoding + // it as one must refuse rather than truncate. Reaching this + // branch takes an input over 1 MiB, so it runs only with a + // `-max_len` raised above libFuzzer's default of 4096. + let mut buf = BytesMut::new(); + if let Message::Bundle { data, .. } = &msg + && data.len() > MAX_CONTENT_LENGTH + { + assert!(matches!( + encode_message(&msg, &mut buf), + Err(Error::LengthOverflow { .. }) + )); + assert!(buf.is_empty()); + continue; + } + encode_message(&msg, &mut buf).expect("decoded messages re-encode"); + assert_eq!(buf.len(), encoded_message_len(&msg)); + if let Message::Unknown { data, .. } = &msg { + assert_eq!(&buf[HEADER_SIZE..], data.as_ref()); + } + // And the encoding decodes back to the same message, alone. + let mut again = decode_pdu_with(buf.freeze(), options); + assert_eq!(again.next(), Some(Ok(msg))); + assert_eq!(again.next(), None); + } + assert!(iter.is_exhausted()); + } +}); diff --git a/btpu/fuzz/fuzz_targets/receive.rs b/btpu/fuzz/fuzz_targets/receive.rs new file mode 100644 index 000000000..d7202c99e --- /dev/null +++ b/btpu/fuzz/fuzz_targets/receive.rs @@ -0,0 +1,146 @@ +#![no_main] + +use std::{collections::HashMap, num::NonZeroUsize, sync::Arc}; + +use bytes::Bytes; +use hardy_btpu::{ + budget::RetentionBudget, + receiver::{ + Delivery, DropReason, MaxRetainedBytes, MaxSegments, MaxTransferSize, Receiver, + ReceiverConfig, ReceiverEvent, + }, + transfer::{TransferId, WindowSize}, +}; +use libfuzzer_sys::fuzz_target; + +// Every 0x9F "bundle" is taken to be eight bytes long, as in the decode +// target, so the receiver's hook path is exercised too. +fn eight_byte_bundles(bytes: &[u8]) -> Option { + (bytes.first() == Some(&0x9F) && bytes.len() >= 8).then_some(8) +} + +// The input is a sequence of PDUs, each prefixed by a one-byte length, fed to +// one receiver so window, reassembly, and abandonment state accumulate. A +// cap of 128, below the 255-byte longest PDU, makes the oversize gates +// reachable, the single-message Bundle gate included. The first byte selects FEC +// decoding (bit 0), a segment limit derived from a 64-byte link PDU, so the +// limit rather than the per-segment charge bounds segment count (bit 1), the +// bundle-extent hook (bit 2), and a receiver-wide retention limit of the cap, +// below one transfer's allowance, so a transfer can be refused as the receiver +// being full while it is the only one held (bit 3). Without it the default +// limit, one transfer's allowance, still makes concurrent transfers contend. +// Bit 4 selects streamed delivery, bit 5 a shared budget of half the cap, +// so a transfer can be refused as the budget being full, and bit 6 refuses +// each streamed transfer once it has released more than 32 bytes. +fuzz_target!(|data: &[u8]| { + let flags = data.first().copied().unwrap_or_default(); + let max_transfer_size = MaxTransferSize::try_from(128).unwrap(); + let mut receiver = Receiver::new(ReceiverConfig { + window_size: WindowSize::try_from(4).unwrap(), + max_transfer_size, + max_segments_per_transfer: (flags & 2 != 0) + .then(|| MaxSegments::for_link_pdu_size(64, max_transfer_size)), + max_retained_bytes: (flags & 8 != 0).then_some(MaxRetainedBytes::from(NonZeroUsize::from( + max_transfer_size, + ))), + fec: flags & 1 != 0, + delivery: if flags & 16 != 0 { + Delivery::Streamed + } else { + Delivery::Whole + }, + }); + if flags & 4 != 0 { + receiver = receiver.with_bundle_extent(eight_byte_bundles); + } + let budget = (flags & 32 != 0).then(|| { + Arc::new(RetentionBudget::new( + MaxRetainedBytes::try_from(max_transfer_size.get() / 2).unwrap(), + )) + }); + if let Some(budget) = &budget { + receiver = receiver.with_budget(Arc::clone(budget)); + } + let refuse_after = (flags & 64 != 0).then_some(32); + // Bytes released by each started, unfinished streamed transfer. + let mut streams: HashMap = HashMap::new(); + let mut events = Vec::new(); + let mut rest = data.get(1..).unwrap_or_default(); + while let Some((&len, tail)) = rest.split_first() { + let len = usize::from(len).min(tail.len()); + let (pdu, tail) = tail.split_at(len); + receiver.receive_pdu_into(Bytes::copy_from_slice(pdu), &mut events); + for (i, event) in events.iter().enumerate() { + match event { + // A delivered bundle is never empty and never over the cap. + ReceiverEvent::Received { data, .. } => { + assert!(!data.is_empty()); + assert!(data.len() <= max_transfer_size.get()); + } + // A PDU-level fault ends the PDU, so it is reported once, + // last. + ReceiverEvent::MalformedPdu { .. } => assert_eq!(i, events.len() - 1), + // A streamed transfer starts once, releases data only + // between its start and its end, and never more than the + // cap. Ids are never reused, even across a reset. + ReceiverEvent::TransferStarted { id, .. } => { + assert!(streams.insert(*id, 0).is_none()); + } + ReceiverEvent::TransferData { id, data, .. } => { + assert!(!data.is_empty()); + let released = streams.get_mut(id).unwrap(); + *released += data.len(); + assert!(*released <= max_transfer_size.get()); + } + ReceiverEvent::TransferFinished { id, data, .. } => { + let released = streams.remove(id).unwrap() + data.len(); + assert!(released > 0 && released <= max_transfer_size.get()); + } + // A dropped message has an id exactly when its number is + // inside the window, and the id names that number. + ReceiverEvent::MessageDropped { + transfer_number, + id, + reason, + } => { + let outside = matches!( + reason, + DropReason::OutsideWindow | DropReason::UnknownTransfer + ); + match id { + None => assert!(outside), + Some(id) => { + assert!(!outside); + assert_eq!(id.transfer_number(), *transfer_number); + } + } + } + ReceiverEvent::TransferCancelled { id } + | ReceiverEvent::TransferExpired { id } + | ReceiverEvent::TransferRejected { id, .. } => { + streams.remove(id); + } + _ => {} + } + } + if let Some(limit) = refuse_after { + let refused: Vec<_> = streams + .iter() + .filter(|&(_, &released)| released > limit) + .map(|(&id, _)| id) + .collect(); + for id in refused { + assert!(receiver.refuse(id)); + assert!(!receiver.refuse(id)); + streams.remove(&id); + } + } + // The receiver is the budget's only user, so it holds the budget's + // whole charge, which never exceeds the limit. + if let Some(budget) = &budget { + assert_eq!(budget.used(), receiver.retained_bytes()); + assert!(budget.used() <= budget.limit().get()); + } + rest = tail; + } +}); diff --git a/btpu/fuzz/fuzz_targets/send.rs b/btpu/fuzz/fuzz_targets/send.rs new file mode 100644 index 000000000..d410bcc08 --- /dev/null +++ b/btpu/fuzz/fuzz_targets/send.rs @@ -0,0 +1,380 @@ +#![no_main] + +use std::collections::HashSet; + +use bytes::Bytes; +use hardy_btpu::{ + codec::hint::{HintItem, HintType, HintValue, Hints}, + receiver::{ + MaxRetainedBytes, MaxSegments, MaxTransferSize, Receiver, ReceiverConfig, ReceiverEvent, + }, + sender::{ + BundleFraming, Error, LinkFraming, NextPduOptions, PduSize, SegmentCutStrategy, SendHandle, + SendId, SendKind, SendOptions, SendQueueBytes, Sender, SenderConfig, + }, + transfer::WindowSize, +}; +use libfuzzer_sys::fuzz_target; + +const WINDOW: u16 = 4; +const MAX_BUNDLE: usize = 1500; + +// The input's bytes, read front to back; reading past the end yields `None`. +struct Input<'a>(&'a [u8]); + +impl Input<'_> { + fn byte(&mut self) -> Option { + let (&b, rest) = self.0.split_first()?; + self.0 = rest; + Some(b) + } + + // A bundle length from two bytes, at least the five `bundle` needs. + fn bundle_len(&mut self) -> Option { + let n = u16::from_be_bytes([self.byte()?, self.byte()?]); + Some(5 + usize::from(n) % (MAX_BUNDLE - 4)) + } +} + +// A bundle of `len` bytes, distinct from every other the run makes: a +// bundle-reserved first byte, so a bare frame may carry it, then `serial`. +fn bundle(serial: u32, len: usize) -> Bytes { + let mut data = vec![0x9F]; + data.extend_from_slice(&serial.to_be_bytes()); + data.extend((5..len).map(|i| i as u8)); + Bytes::from(data) +} + +// Options carrying an unknown hint, so segment 0 has hints besides the +// Bundle Length, or none. +fn options(hinted: bool) -> SendOptions { + let mut hints = Hints::new(); + if hinted { + hints.insert(HintItem::Unknown { + hint_type: HintType::new(0x41).unwrap(), + value: HintValue::new(Bytes::from_static(b"hint")).unwrap(), + }); + } + SendOptions { hints } +} + +// A bundle begun and not yet finished or cancelled. +struct Open { + handle: SendHandle, + data: Bytes, +} + +// What the run expects of the sender, checked as PDUs are drained into a +// receiver over a lossless link. +struct Run { + sender: Sender, + receiver: Receiver, + pdu_size: usize, + min_pdu_len: usize, + fixed_size: bool, + serial: u32, + open: Vec, + // Bundles finished or enqueued whose last bytes are not yet packed, + // which `cancel` may still reach by ID. + queued: Vec<(SendId, Bytes)>, + // IDs that may appear in a PDU's carried list: begun or enqueued, not + // cancelled, and not yet completed. + live: HashSet, + // Bundles that must be delivered by the end of the run. + expected: HashSet, + cancelled: HashSet, + delivered: HashSet, +} + +impl Run { + fn new(flags: u8, pdu_byte: u8) -> Self { + let (link_framing, fixed_size) = match flags & 3 { + 0 => (LinkFraming::FixedSize, true), + framing => ( + LinkFraming::Variable { + bundle_framing: if framing == 3 { + BundleFraming::Bare + } else { + BundleFraming::Message + }, + min_pdu_len: if framing == 2 { 46 } else { 0 }, + }, + false, + ), + }; + let pdu_size = 24 + usize::from(pdu_byte); + let min_pdu_len = match link_framing { + LinkFraming::FixedSize => pdu_size, + LinkFraming::Variable { min_pdu_len, .. } => min_pdu_len.min(pdu_size), + }; + let config = SenderConfig { + pdu_size: PduSize::try_from(pdu_size).unwrap(), + window_size: WindowSize::try_from(WINDOW).unwrap(), + // Below one segment, so pushes past the bound are common. + send_queue_bytes: SendQueueBytes::try_from(if flags & 4 != 0 { 16 } else { 4096 }) + .unwrap(), + link_framing, + segment_cut_strategy: if flags & 8 != 0 { + SegmentCutStrategy::Half + } else { + SegmentCutStrategy::Full + }, + }; + // Transfer numbers roll over within the run. + let initial = if flags & 16 != 0 { u32::MAX - 1 } else { 0 }; + let receiver = Receiver::new(ReceiverConfig { + window_size: WindowSize::try_from(WINDOW).unwrap(), + max_transfer_size: MaxTransferSize::try_from(2 * MAX_BUNDLE).unwrap(), + // Flushed segments may carry one byte each. + max_segments_per_transfer: Some(MaxSegments::MAX), + max_retained_bytes: Some(MaxRetainedBytes::try_from(1 << 24).unwrap()), + ..ReceiverConfig::default() + }); + Self { + sender: Sender::new(config, initial), + receiver, + pdu_size, + min_pdu_len, + fixed_size, + serial: 0, + open: Vec::new(), + queued: Vec::new(), + live: HashSet::new(), + expected: HashSet::new(), + cancelled: HashSet::new(), + delivered: HashSet::new(), + } + } + + fn next_bundle(&mut self, len: usize) -> Bytes { + self.serial += 1; + bundle(self.serial, len) + } + + // Admission refuses only for want of a window slot or a PDU able to + // carry the segments. + fn check_refusal(e: Error) { + assert!( + matches!( + e, + Error::Window(_) | Error::PduTooSmall { .. } | Error::TooManySegments { .. } + ), + "{e:?}" + ); + } + + fn begin(&mut self, len: usize, hinted: bool) { + let data = self.next_bundle(len); + match self.sender.begin(len, options(hinted)) { + Ok(handle) => { + assert!(self.live.insert(handle.id())); + self.open.push(Open { handle, data }); + } + Err(e) => Self::check_refusal(e), + } + } + + fn enqueue(&mut self, len: usize, hinted: bool) { + let data = self.next_bundle(len); + match self.sender.enqueue(data.clone(), options(hinted)) { + Ok(id) => { + assert!(self.live.insert(id)); + self.queued.push((id, data.clone())); + self.expected.insert(data); + } + Err(e) => Self::check_refusal(e), + } + } + + fn push(&mut self, k: usize, n: usize) { + let open = &mut self.open[k]; + let (pushed, total) = (open.handle.pushed(), open.handle.total_len()); + let overrun = pushed + n > total; + let chunk = if overrun { + Bytes::from(vec![0; n]) + } else { + open.data.slice(pushed..pushed + n) + }; + let result = self.sender.push(&mut open.handle, chunk); + if overrun { + assert!(matches!(result, Err(Error::Overrun { .. })), "{result:?}"); + assert_eq!(open.handle.pushed(), pushed); + } else { + assert_eq!(result, Ok(())); + assert_eq!(open.handle.pushed(), pushed + n); + } + } + + fn finish(&mut self, k: usize) { + let Open { handle, data } = self.open.swap_remove(k); + let (id, complete) = (handle.id(), handle.pushed() == handle.total_len()); + match self.sender.finish(handle) { + Ok(finished) => { + assert!(complete); + assert_eq!(finished, id); + if self.live.contains(&id) { + self.queued.push((id, data.clone())); + } + self.expected.insert(data); + } + Err(e) => { + assert!(!complete); + assert!(matches!(e, Error::Underrun { .. }), "{e:?}"); + self.cancel_bundle(id, data); + } + } + } + + // Record `data`, whose ID was `id`, as cancelled: it must never arrive. + fn cancel_bundle(&mut self, id: SendId, data: Bytes) { + assert!(!self.delivered.contains(&data)); + self.live.remove(&id); + self.expected.remove(&data); + self.cancelled.insert(data); + } + + fn cancel_open(&mut self, k: usize) { + let Open { handle, data } = self.open.swap_remove(k); + let id = handle.id(); + if self.sender.cancel(handle) { + self.cancel_bundle(id, data); + } else { + // Only a bundle whose last bytes are packed is beyond reach. + assert!(!self.live.contains(&id)); + self.expected.insert(data); + } + } + + fn cancel_queued(&mut self, k: usize) { + let (id, data) = self.queued.swap_remove(k); + assert!(self.sender.cancel(id), "a queued bundle is still in reach"); + self.cancel_bundle(id, data); + } + + fn next_pdu(&mut self, options: NextPduOptions) -> bool { + // A producer that drains whenever a push is not ready always finds + // a PDU to drain. + let blocked = self + .open + .iter() + .any(|o| !self.sender.is_push_ready(&o.handle)); + let Some(pdu) = self.sender.next_pdu_with(options) else { + assert!( + !blocked, + "a producer gated on push readiness would wait forever" + ); + return false; + }; + assert!(pdu.data.len() <= self.pdu_size); + assert!(pdu.data.len() >= self.min_pdu_len); + if self.fixed_size { + assert_eq!(pdu.data.len(), self.pdu_size); + } + + let mut seen = HashSet::new(); + for c in &pdu.carried { + assert!(seen.insert(c.id), "{:?} listed twice", c.id); + assert!(self.live.contains(&c.id), "{:?} is not live", c.id); + if c.id.kind() != SendKind::Transfer { + assert!(c.completes); + } + if c.completes { + self.live.remove(&c.id); + self.queued.retain(|(id, _)| *id != c.id); + } + } + + for event in self.receiver.receive_pdu(pdu.data) { + match event { + ReceiverEvent::Received { data, .. } => { + assert!( + !self.cancelled.contains(&data), + "a cancelled bundle arrived" + ); + assert!(self.delivered.insert(data), "a bundle arrived twice"); + } + // A transfer cancelled after its first segment was sent. + ReceiverEvent::TransferCancelled { .. } => {} + other => panic!("unexpected event on a lossless link: {other:?}"), + } + } + true + } + + // Push and finish every open bundle, then drain the queue. + fn complete(mut self) { + while let Some(open) = self.open.last() { + let n = open.handle.total_len() - open.handle.pushed(); + self.push(self.open.len() - 1, n); + self.finish(self.open.len() - 1); + } + while self.next_pdu(NextPduOptions::default()) {} + assert!(!self.sender.has_pending()); + assert_eq!(self.sender.queued_bytes(), 0); + assert!(self.sender.is_window_available()); + assert!(self.live.is_empty()); + assert_eq!(self.delivered, self.expected); + } +} + +// The input is two configuration bytes and a sequence of operations on one +// sender, whose PDUs are fed to a receiver over a lossless link. The first +// byte selects the link framing (bits 0 and 1: fixed-size, variable, +// variable with a 46-byte floor, or variable with bare bundle frames), a +// send queue bound below one segment (bit 2), `SegmentCutStrategy::Half` +// (bit 3), and an initial transfer number that rolls over (bit 4); the +// second sets the PDU size, 24 to 279 bytes. At the end every open bundle +// is pushed and finished and the queue drained, and exactly the bundles +// neither cancelled nor finished short must have arrived, once each. +fuzz_target!(|data: &[u8]| { + let mut input = Input(data); + let (Some(flags), Some(pdu_byte)) = (input.byte(), input.byte()) else { + return; + }; + let mut run = Run::new(flags, pdu_byte); + while let Some(op) = input.byte() { + let hinted = op & 0x80 != 0; + match op % 8 { + 0 => { + let Some(len) = input.bundle_len() else { break }; + run.begin(len, hinted); + } + 1 => { + let Some(len) = input.bundle_len() else { break }; + run.enqueue(len, hinted); + } + 2 | 3 if !run.open.is_empty() => { + let (Some(k), Some(n)) = (input.byte(), input.byte()) else { + break; + }; + // Chunks up to 1020 bytes, often enough to overrun. + let n = if op % 8 == 3 { + usize::from(n) * 4 + } else { + usize::from(n) + }; + run.push(usize::from(k) % run.open.len(), n); + } + 4 if !run.open.is_empty() => { + let Some(k) = input.byte() else { break }; + run.finish(usize::from(k) % run.open.len()); + } + 5 => { + let Some(k) = input.byte() else { break }; + if hinted && !run.open.is_empty() { + run.cancel_open(usize::from(k) % run.open.len()); + } else if !run.queued.is_empty() { + run.cancel_queued(usize::from(k) % run.queued.len()); + } + } + 6 => { + run.next_pdu(NextPduOptions::default()); + } + 7 => { + run.next_pdu(NextPduOptions { flush: true }); + } + _ => {} + } + } + run.complete(); +}); diff --git a/btpu/fuzz/rust-toolchain.toml b/btpu/fuzz/rust-toolchain.toml new file mode 100644 index 000000000..5d56faf9a --- /dev/null +++ b/btpu/fuzz/rust-toolchain.toml @@ -0,0 +1,2 @@ +[toolchain] +channel = "nightly" diff --git a/btpu/src/budget.rs b/btpu/src/budget.rs new file mode 100644 index 000000000..dd612871b --- /dev/null +++ b/btpu/src/budget.rs @@ -0,0 +1,133 @@ +//! A retention limit shared by several receivers, and by whatever else a +//! CLA charges against it. +//! +//! Compiled only on targets with pointer-width atomic compare-and-swap, +//! which `alloc::sync::Arc` needs. + +use alloc::sync::Arc; +use core::{ + fmt, + sync::atomic::{AtomicUsize, Ordering}, +}; + +use crate::receiver::MaxRetainedBytes; + +/// A limit on the state several [`Receiver`](crate::receiver::Receiver)s +/// retain together, in the units [`MaxRetainedBytes`] counts. +/// +/// A link that runs one receiver per peer, as the Ethernet convergence +/// layer runs one per logical channel, multiplies each receiver's +/// [`MaxRetainedBytes`] by the number of peers, and an unauthenticated peer +/// can add channels at will. Receivers joined to one budget with +/// [`Receiver::with_budget`](crate::receiver::Receiver::with_budget) are +/// bounded together: a message whose growth the budget cannot take rejects +/// its transfer as +/// [`RejectReason::BudgetFull`](crate::receiver::RejectReason::BudgetFull). +/// Each receiver's own limit still applies, and is the only fairness +/// between them: the budget serves whoever asks first. +/// +/// A CLA can charge what it holds for the same link, such as streamed +/// bytes not yet read by the BPA, with [`Self::try_charge`], so that one +/// limit covers everything the link holds. +/// +/// Charges are exact across threads: growth that would exceed the limit +/// fails rather than overshooting. +pub struct RetentionBudget { + limit: MaxRetainedBytes, + used: AtomicUsize, +} + +impl RetentionBudget { + /// A budget of `limit`, with nothing charged. + pub fn new(limit: MaxRetainedBytes) -> Self { + Self { + limit, + used: AtomicUsize::new(0), + } + } + + /// The configured limit. + pub fn limit(&self) -> MaxRetainedBytes { + self.limit + } + + /// What is charged now, by receivers and [`Charge`]s together. + pub fn used(&self) -> usize { + self.used.load(Ordering::Relaxed) + } + + /// Charge `bytes`, or return `None` if that would exceed the limit. + /// The charge is released when the returned [`Charge`] is dropped. + pub fn try_charge(self: &Arc, bytes: usize) -> Option { + self.try_add(bytes).then(|| Charge { + budget: Arc::clone(self), + bytes, + }) + } + + /// Add `bytes` to what is charged if the total stays within the limit, + /// and report whether it did. + pub(crate) fn try_add(&self, bytes: usize) -> bool { + self.update(|used| used.checked_add(bytes).filter(|&n| n <= self.limit.get())) + } + + /// Add `bytes` whatever the limit. + pub(crate) fn add(&self, bytes: usize) { + self.update(|used| Some(used.saturating_add(bytes))); + } + + /// Release `bytes` charged earlier. + pub(crate) fn release(&self, bytes: usize) { + self.update(|used| Some(used.saturating_sub(bytes))); + } + + /// Replace what is charged with `f` of it, unless `f` returns `None`, + /// and report whether it was replaced. + fn update(&self, f: impl FnMut(usize) -> Option) -> bool { + // Relaxed: the counter guards no other memory, and the + // read-modify-write alone keeps concurrent charges exact. + self.used + .fetch_update(Ordering::Relaxed, Ordering::Relaxed, f) + .is_ok() + } +} + +impl fmt::Debug for RetentionBudget { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.debug_struct("RetentionBudget") + .field("limit", &self.limit) + .field("used", &self.used()) + .finish() + } +} + +/// Bytes charged against a [`RetentionBudget`], released when dropped. +/// +/// Not `Clone`: each charge is released once. A CLA keeps a charge with +/// what it accounts for, so that dropping one releases the other. +#[must_use = "dropping a charge releases it at once"] +pub struct Charge { + budget: Arc, + bytes: usize, +} + +impl Charge { + /// The bytes charged. + pub fn bytes(&self) -> usize { + self.bytes + } +} + +impl Drop for Charge { + fn drop(&mut self) { + self.budget.release(self.bytes); + } +} + +impl fmt::Debug for Charge { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.debug_struct("Charge") + .field("bytes", &self.bytes) + .finish() + } +} diff --git a/btpu/src/codec/error.rs b/btpu/src/codec/error.rs new file mode 100644 index 000000000..7172314a1 --- /dev/null +++ b/btpu/src/codec/error.rs @@ -0,0 +1,77 @@ +//! The codec's error type. + +use bytes::TryGetError; + +/// Shorthand for results whose error is [`enum@Error`]. +pub type Result = core::result::Result; + +/// Errors from message encoding and decoding. +#[derive(Debug, Clone, PartialEq, Eq, thiserror::Error)] +pub enum Error { + /// The bytes at a message boundary begin an encapsulated bundle in its + /// native format (`first_byte` 0x06 for BPv6 or 0x80..=0x9F for BPv7, + /// Section 12.1) whose extent could not be determined: no + /// [`BundleExtent`](crate::codec::BundleExtent) was supplied, or it + /// returned `None`. Per Section 7.3 decoding stops here and the + /// remainder of the PDU, from `offset`, is left unprocessed. + /// + /// `offset` counts from the start of the PDU. The decoder does not + /// hand the PDU back, so a caller that wants to inspect `pdu[offset..]` + /// keeps a clone of the `Bytes` it passed in (a reference-count + /// increment, not a copy). + #[error( + "Encapsulated bundle (first byte {first_byte:#04x}) at offset {offset} of undeterminable extent" + )] + EncapsulatedBundle { + /// The bundle's first byte, which no BTP-U message type uses. + first_byte: u8, + /// Where the bundle starts. + offset: usize, + }, + + /// A message content length exceeds the 20-bit maximum. + #[error("Message content length {length} exceeds 20-bit maximum ({max})")] + LengthOverflow { + /// The content length that would have been encoded. + length: usize, + /// The largest content length the header can carry. + max: usize, + }, + + /// Not enough data to decode a message, header, hint, or encapsulated + /// bundle. + /// + /// When a message or encapsulated bundle runs past the end of a PDU + /// (the fault that ends [`decode_pdu`](crate::codec::decode_pdu) + /// iteration), both counts are from the start of the PDU. A hint + /// chain that runs past its message's content, or one decoded alone, + /// counts from the start of the chain; a lone header counts from its + /// first byte. A fixed-size field (a transfer number, segment index, + /// or FEC identifier) cut short describes that one read: `needed` is + /// the field's size and `available` what remained of the content. + #[error("Insufficient data: need {needed} bytes, have {available}")] + InsufficientData { + /// The bytes the failed read required. + needed: usize, + /// The bytes there were. + available: usize, + }, + + /// A [`Message::Unknown`](crate::codec::message::Message::Unknown) + /// carries a type value that is not unknown: one defined by the base + /// protocol or reserved for encapsulated bundles. Encoding it would + /// produce a message the decoder reads as something else. + #[error("Message type {0:#04x} is defined or reserved and cannot be relayed as unknown")] + NotAnUnknownType(u8), +} + +/// A fixed-size field read past the end of a message's content: `needed` +/// and `available` describe the read that failed. +impl From for Error { + fn from(e: TryGetError) -> Self { + Error::InsufficientData { + needed: e.requested, + available: e.available, + } + } +} diff --git a/btpu/src/codec/header.rs b/btpu/src/codec/header.rs new file mode 100644 index 000000000..91a9aa552 --- /dev/null +++ b/btpu/src/codec/header.rs @@ -0,0 +1,101 @@ +//! The four-byte message header (Section 7). + +use crate::codec::{Error, Result, message::MessageFlags}; + +/// Size of the standard message header in bytes. +pub const HEADER_SIZE: usize = 4; + +/// Maximum value of the 20-bit content-length field. +/// +/// Held as a `usize` because it bounds in-memory buffer sizes; see +/// [`ContentLength`] for the field's value type. +pub const MAX_CONTENT_LENGTH: usize = 0xF_FFFF; // 1,048,575 + +/// A message content length: a value of the header's 20-bit Length field +/// (Section 7), so at most [`MAX_CONTENT_LENGTH`]. +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] +pub struct ContentLength(usize); + +impl ContentLength { + /// The largest content length the field can declare. + pub const MAX: Self = Self(MAX_CONTENT_LENGTH); + + /// Returns the content length `len`, or `None` if it exceeds + /// [`MAX_CONTENT_LENGTH`]. + pub const fn new(len: usize) -> Option { + if len <= MAX_CONTENT_LENGTH { + Some(Self(len)) + } else { + None + } + } + + /// The length in bytes. + pub const fn get(self) -> usize { + self.0 + } +} + +impl TryFrom for ContentLength { + type Error = Error; + + /// Errors with [`Error::LengthOverflow`] if `len` exceeds + /// [`MAX_CONTENT_LENGTH`]. + fn try_from(len: usize) -> Result { + Self::new(len).ok_or(Error::LengthOverflow { + length: len, + max: MAX_CONTENT_LENGTH, + }) + } +} + +/// A decoded BTP-U message header (Section 7). +/// +/// Layout (4 bytes, network byte order): +/// ```text +/// 0 1 2 3 +/// 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +/// +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +/// | Type | Flags | Length (20-bit unsigned int) | +/// +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +/// ``` +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct MessageHeader { + /// The message type value (Section 12.1 registry). + pub message_type: u8, + /// The four flag bits (Section 7.1). + pub flags: MessageFlags, + /// The content length in bytes. + pub length: ContentLength, +} + +/// Encode a message header into its 4-byte wire form. +pub fn encode_header(header: &MessageHeader) -> [u8; HEADER_SIZE] { + // At most 20 bits by construction, so the low three bytes hold it. + let [.., high, mid, low] = header.length.get().to_be_bytes(); + [ + header.message_type, + (header.flags.to_nibble() << 4) | high, + mid, + low, + ] +} + +/// Decode a message header from a byte slice. +/// +/// Errors with [`Error::InsufficientData`] if `src` is shorter than +/// [`HEADER_SIZE`]. +pub fn decode_header(src: &[u8]) -> Result { + let [message_type, flags_high, mid, low, ..] = *src else { + return Err(Error::InsufficientData { + needed: HEADER_SIZE, + available: src.len(), + }); + }; + let length = usize::from(flags_high & 0x0F) << 16 | usize::from(mid) << 8 | usize::from(low); + Ok(MessageHeader { + message_type, + flags: MessageFlags::from_nibble(flags_high >> 4), + length: ContentLength(length), + }) +} diff --git a/btpu/src/codec/hint.rs b/btpu/src/codec/hint.rs new file mode 100644 index 000000000..e09894e15 --- /dev/null +++ b/btpu/src/codec/hint.rs @@ -0,0 +1,439 @@ +//! Hint items (Section 7.2): typed, length-prefixed values that may lead +//! a message's content. + +use alloc::vec::{self, Vec}; +use core::ops::Deref; + +use bytes::{Buf, BufMut, Bytes, BytesMut}; + +use crate::codec::{Error, Result}; + +/// Size of a single hint item header (type+H byte, length byte). +pub const HINT_HEADER_SIZE: usize = 2; + +/// Maximum hint value length representable by the 8-bit length field. +pub const MAX_HINT_VALUE_LEN: usize = u8::MAX as usize; + +/// Number of distinct hint types the 7-bit type field can express, and so +/// the most items [`decode_hints`] ever returns for one message. +pub(crate) const HINT_TYPE_COUNT: usize = HintType::MAX.0 as usize + 1; + +/// A hint type: a value of the 7-bit Hint Type field (Section 12.2 +/// registry). +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] +pub struct HintType(u8); + +impl HintType { + /// The Bundle Length hint (Section 9.1). + pub const BUNDLE_LENGTH: Self = Self(0); + + /// The largest hint type: the type occupies the upper 7 bits of the + /// first hint header byte. + pub const MAX: Self = Self(0x7F); + + /// Returns the hint type `value`, or `None` if it exceeds + /// [`Self::MAX`]. + pub const fn new(value: u8) -> Option { + if value <= Self::MAX.0 { + Some(Self(value)) + } else { + None + } + } + + /// The type code. + pub const fn get(self) -> u8 { + self.0 + } +} + +/// A hint value: at most [`MAX_HINT_VALUE_LEN`] bytes, what the 8-bit +/// Hint Length field can declare (Section 7.2). +/// +/// Dereferences to the value's bytes. +#[derive(Debug, Clone, PartialEq, Eq, Hash)] +pub struct HintValue(Bytes); + +impl HintValue { + /// Returns the hint value `bytes`, or `None` if it is longer than + /// [`MAX_HINT_VALUE_LEN`]. + pub fn new(bytes: Bytes) -> Option { + (bytes.len() <= MAX_HINT_VALUE_LEN).then_some(Self(bytes)) + } + + /// The value's bytes. + pub fn into_bytes(self) -> Bytes { + self.0 + } + + /// A copy of the value in an allocation of its own, so holding it does + /// not keep alive the buffer it was decoded from. + pub(crate) fn detached(&self) -> Self { + Self(Bytes::copy_from_slice(&self.0)) + } + + /// The Hint Length field for this value. + fn len_byte(&self) -> u8 { + // At most MAX_HINT_VALUE_LEN by construction. + self.0.len() as u8 + } +} + +impl Deref for HintValue { + type Target = Bytes; + + fn deref(&self) -> &Bytes { + &self.0 + } +} + +/// A decoded BTP-U hint item. +/// +/// Every value of this type is encodable: the type and value newtypes hold +/// the field limits. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum HintItem { + /// Bundle Length hint: total length of the bundle being transferred + /// (Section 9.1). + BundleLength(u64), + /// A hint this implementation does not interpret, preserved for forward + /// compatibility (Section 7.3: skipped via its length, never an error). + /// Also produced for a Bundle Length hint whose value length is not one + /// the draft allows: the item is still fully framed, so it is carried + /// opaquely rather than failing the message. + Unknown { + /// The hint type. + hint_type: HintType, + /// The raw value bytes. + value: HintValue, + }, +} + +impl HintItem { + /// The item's wire type code (Section 12.2 registry). + pub fn hint_type(&self) -> HintType { + match self { + HintItem::BundleLength(_) => HintType::BUNDLE_LENGTH, + HintItem::Unknown { hint_type, .. } => *hint_type, + } + } +} + +/// A transfer's hints, or one message's: at most one item per hint type +/// (Section 7.2: hints are transfer-scoped and repeatable, and the value +/// most recently received supersedes an earlier one of the same type). +/// +/// [`ReceiverEvent::Received`](crate::receiver::ReceiverEvent::Received) +/// delivers one, and [`SendOptions`](crate::sender::SendOptions) takes one, +/// so a relay can pass a received set straight back to a sender. +/// +/// Inserting an item replaces any item of the same type. A well-formed +/// Bundle Length is held inline, so a set holding only that does not +/// allocate. A type-0 item whose value length Section 9.1 does not allow is +/// an [`HintItem::Unknown`] of type [`HintType::BUNDLE_LENGTH`], and it and +/// a [`HintItem::BundleLength`] replace each other like any two items of +/// one type. +/// +/// Items iterate in ascending hint-type order. Equality and hashing follow +/// the items, which have one representation per set. +#[derive(Debug, Clone, Default, PartialEq, Eq, Hash)] +pub struct Hints { + bundle_length: Option, + /// Every other item, sorted by hint type, one entry per type: at most + /// 128 entries of at most 255 value bytes each. + other: Vec<(HintType, HintValue)>, +} + +impl Hints { + /// An empty set. + pub const fn new() -> Self { + Self { + bundle_length: None, + other: Vec::new(), + } + } + + /// Add `item`, replacing any item of the same type. + pub fn insert(&mut self, item: HintItem) { + let slot = self + .other + .binary_search_by_key(&item.hint_type(), |&(t, _)| t); + match item { + HintItem::BundleLength(len) => { + self.bundle_length = Some(len); + if let Ok(i) = slot { + self.other.remove(i); + } + } + HintItem::Unknown { hint_type, value } => { + if hint_type == HintType::BUNDLE_LENGTH { + self.bundle_length = None; + } + match slot { + Ok(i) => self.other[i].1 = value, + Err(i) => self.other.insert(i, (hint_type, value)), + } + } + } + } + + /// Remove and return the item of type `hint_type`, if any. + pub fn remove(&mut self, hint_type: HintType) -> Option { + if hint_type == HintType::BUNDLE_LENGTH + && let Some(len) = self.bundle_length.take() + { + return Some(HintItem::BundleLength(len)); + } + let i = self + .other + .binary_search_by_key(&hint_type, |&(t, _)| t) + .ok()?; + let (hint_type, value) = self.other.remove(i); + Some(HintItem::Unknown { hint_type, value }) + } + + /// The well-formed Bundle Length hint's value, if the set holds one. + pub fn bundle_length(&self) -> Option { + self.bundle_length + } + + /// The item of type `hint_type`, if any. A clone: an unknown item's + /// value is a reference-counted [`Bytes`]. + pub fn get(&self, hint_type: HintType) -> Option { + if hint_type == HintType::BUNDLE_LENGTH + && let Some(len) = self.bundle_length + { + return Some(HintItem::BundleLength(len)); + } + let i = self + .other + .binary_search_by_key(&hint_type, |&(t, _)| t) + .ok()?; + let (hint_type, value) = &self.other[i]; + Some(HintItem::Unknown { + hint_type: *hint_type, + value: value.clone(), + }) + } + + /// The number of items. + pub fn len(&self) -> usize { + usize::from(self.bundle_length.is_some()) + self.other.len() + } + + /// Whether the set holds no items. + pub fn is_empty(&self) -> bool { + self.bundle_length.is_none() && self.other.is_empty() + } + + /// The items in ascending hint-type order, cloned as for [`Self::get`]. + pub fn iter(&self) -> impl Iterator + '_ { + self.bundle_length + .map(HintItem::BundleLength) + .into_iter() + .chain( + self.other + .iter() + .map(|(hint_type, value)| HintItem::Unknown { + hint_type: *hint_type, + value: value.clone(), + }), + ) + } + + /// The encoded size of the items (headers and values). + pub fn encoded_len(&self) -> usize { + self.bundle_length.map_or(0, |len| { + HINT_HEADER_SIZE + usize::from(bundle_length_width(len)) + }) + self + .other + .iter() + .map(|(_, value)| HINT_HEADER_SIZE + value.len()) + .sum::() + } + + /// The values held as [`HintItem::Unknown`], which the receiver charges + /// against its limits. + pub(crate) fn unknown_values(&self) -> impl Iterator { + self.other.iter().map(|(_, value)| value) + } + + /// The items in ascending hint-type order, as a message carries them. + pub fn into_vec(self) -> Vec { + let mut items = Vec::with_capacity(self.len()); + items.extend(self.bundle_length.map(HintItem::BundleLength)); + items.extend( + self.other + .into_iter() + .map(|(hint_type, value)| HintItem::Unknown { hint_type, value }), + ); + items + } +} + +impl Extend for Hints { + fn extend>(&mut self, items: I) { + for item in items { + self.insert(item); + } + } +} + +impl FromIterator for Hints { + fn from_iter>(items: I) -> Self { + let mut hints = Self::new(); + hints.extend(items); + hints + } +} + +impl From> for Hints { + fn from(items: Vec) -> Self { + items.into_iter().collect() + } +} + +impl IntoIterator for Hints { + type Item = HintItem; + type IntoIter = vec::IntoIter; + + fn into_iter(self) -> Self::IntoIter { + self.into_vec().into_iter() + } +} + +/// Returns the total encoded size of a slice of hint items (headers + values). +pub fn encoded_hints_len(hints: &[HintItem]) -> usize { + hints + .iter() + .map(|h| HINT_HEADER_SIZE + hint_value_len(h)) + .sum() +} + +/// Encode a chain of hint items into `dst`. +/// +/// Sets the H flag on all items except the last, per Section 7.2. +pub fn encode_hints(hints: &[HintItem], dst: &mut BytesMut) { + let count = hints.len(); + for (i, item) in hints.iter().enumerate() { + let more = i + 1 < count; + write_hint(item, more, dst); + } +} + +fn write_hint(item: &HintItem, more: bool, dst: &mut BytesMut) { + let h_bit = u8::from(more); + match item { + HintItem::BundleLength(len) => { + let width = bundle_length_width(*len); + dst.put_u8((HintType::BUNDLE_LENGTH.0 << 1) | h_bit); + dst.put_u8(width); + dst.put_uint(*len, usize::from(width)); + } + HintItem::Unknown { hint_type, value } => { + dst.put_u8((hint_type.0 << 1) | h_bit); + dst.put_u8(value.len_byte()); + dst.put_slice(value); + } + } +} + +/// The width of the shortest Bundle Length encoding that holds `len` +/// (Section 9.1: 1, 2, 4, or 8 bytes). +fn bundle_length_width(len: u64) -> u8 { + match len { + 0..=0xFF => 1, + 0x100..=0xFFFF => 2, + 0x1_0000..=0xFFFF_FFFF => 4, + _ => 8, + } +} + +fn hint_value_len(item: &HintItem) -> usize { + match item { + // Sized by the encoder itself so the two can never disagree. + HintItem::BundleLength(len) => usize::from(bundle_length_width(*len)), + HintItem::Unknown { value, .. } => value.len(), + } +} + +/// Decode the chain of hint items at the start of `content`. +/// +/// Returns the decoded items and the number of bytes the chain occupied. +/// Unknown hint values are zero-copy [`Bytes`] views into `content`. +/// +/// The result holds at most one item per hint type, in order of first +/// appearance on the wire, with a later repeat of a type replacing the +/// earlier value. Section 7.2 permits repeats, and a chain may hold up to +/// half a million two-byte items, so folding while decoding keeps the +/// returned `Vec` bounded by the 128-value type space rather than by the +/// message length. The order is not normalised here; collecting the items +/// into [`Hints`] orders them by type. +/// +/// Errors with [`Error::InsufficientData`] if the chain runs past the end of +/// `content`. +pub fn decode_hints(content: &Bytes) -> Result<(Vec, usize)> { + let mut items: Vec = Vec::new(); + // Index into `items` of the entry for each hint type, or NONE. + const NONE: u8 = u8::MAX; + let mut slot = [NONE; HINT_TYPE_COUNT]; + let mut offset = 0; + + loop { + if offset + HINT_HEADER_SIZE > content.len() { + return Err(Error::InsufficientData { + needed: offset + HINT_HEADER_SIZE, + available: content.len(), + }); + } + + let type_h_byte = content[offset]; + let hint_type = HintType(type_h_byte >> 1); + let more = type_h_byte & 1 != 0; + let value_len = usize::from(content[offset + 1]); + offset += HINT_HEADER_SIZE; + + if offset + value_len > content.len() { + return Err(Error::InsufficientData { + needed: offset + value_len, + available: content.len(), + }); + } + + let value = content.slice(offset..offset + value_len); + offset += value_len; + + let item = decode_hint_item(hint_type, value); + match slot[usize::from(hint_type.0)] { + NONE => { + // Fewer than 128 distinct types can exist, so the index fits. + slot[usize::from(hint_type.0)] = items.len() as u8; + items.push(item); + } + i => items[usize::from(i)] = item, + } + + if !more { + break; + } + } + + Ok((items, offset)) +} + +fn decode_hint_item(hint_type: HintType, value: Bytes) -> HintItem { + if hint_type == HintType::BUNDLE_LENGTH { + // Section 9.1: the value length MUST be 1, 2, 4, or 8. Any other + // length is a sender fault, but the item is still fully framed, so + // it is carried as an unknown item instead of failing the message + // (hints are ignorable by definition, Section 7.3). + if let 1 | 2 | 4 | 8 = value.len() { + return HintItem::BundleLength(value.as_ref().get_uint(value.len())); + } + } + HintItem::Unknown { + hint_type, + // The length came from an 8-bit field. + value: HintValue(value), + } +} diff --git a/btpu/src/codec/message.rs b/btpu/src/codec/message.rs new file mode 100644 index 000000000..ac10601c7 --- /dev/null +++ b/btpu/src/codec/message.rs @@ -0,0 +1,310 @@ +//! The messages of Section 8 and of the FEC extension, as the codec decodes +//! and encodes them. + +use alloc::vec::Vec; + +use bytes::Bytes; + +use crate::{ + codec::{header::HEADER_SIZE, hint::HintItem}, + fec::{ExplicitFecMessage, PreAgreedFecMessage}, +}; + +/// Size of the transfer number field, the whole content of a Transfer +/// Cancel and the start of every other transfer and FEC message's fields. +pub(crate) const TRANSFER_NUMBER_SIZE: usize = 4; + +/// Size of a Transfer Segment or End message's fields after its hints: the +/// transfer number and the segment index. +pub(crate) const SEGMENT_FIELDS_SIZE: usize = TRANSFER_NUMBER_SIZE + 4; + +/// Size of an FEC message's fields after its hints: the transfer number and +/// the instance or encoding ID byte. +pub(crate) const FEC_FIELDS_SIZE: usize = TRANSFER_NUMBER_SIZE + 1; + +/// What a Transfer Segment or End message spends before its data, hints +/// aside: the header and [`SEGMENT_FIELDS_SIZE`]. +pub(crate) const SEGMENT_FRAMING: usize = HEADER_SIZE + SEGMENT_FIELDS_SIZE; + +/// BTP-U message type, as encoded in the first byte of the message header. +/// +/// Only the type values this crate implements are representable; every +/// other value, reserved or unassigned, decodes as +/// [`Message::Unknown`] (see [`MessageType::from_byte`]). +/// +/// The four FEC extension types have no IANA-assigned values yet: the FEC +/// draft lists them as TBD1..TBD4. This crate uses 0x70..=0x73, from the +/// Private Use range of the BTPU Message Types registry (Section 12.1), as +/// provisional values. They will change when the codes are assigned, and +/// peers must agree on them out of band until then. Because the range is +/// Private Use, a decoder interprets these four values only when +/// [`DecodeOptions::fec`](crate::codec::DecodeOptions::fec) is set; a +/// deployment with its own private types is otherwise unaffected. +#[repr(u8)] +#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)] +pub enum MessageType { + /// Indefinite Padding Message (Section 8.6). Special format: a single + /// zero byte with no length field, so it never has a header of its own. + /// Kept as the registry value: the decoder skips zero bytes and yields + /// no [`Message`] for them. + IndefinitePadding = 0x00, + /// Definite Padding Message (Section 8.5). + DefinitePadding = 0x01, + /// Bundle Message (Section 8.1). Complete bundle in a single message. + Bundle = 0x02, + /// Transfer Segment Message (Section 8.2). + TransferSegment = 0x03, + /// Transfer End Message (Section 8.3). + TransferEnd = 0x04, + /// Transfer Cancel Message (Section 8.4). + TransferCancel = 0x05, + /// Pre-agreed FEC Source Message (Section 4.1 of btpu-fec, TBD1; + /// provisional value). + PreAgreedFecSource = 0x70, + /// Explicit FEC Source Message (Section 4.2 of btpu-fec, TBD2; + /// provisional value). + ExplicitFecSource = 0x71, + /// Pre-agreed FEC Repair Message (Section 4.3 of btpu-fec, TBD3; + /// provisional value). + PreAgreedFecRepair = 0x72, + /// Explicit FEC Repair Message (Section 4.4 of btpu-fec, TBD4; + /// provisional value). + ExplicitFecRepair = 0x73, +} + +impl MessageType { + /// The message type for a wire type byte, or `None` if this crate does + /// not implement it. + /// + /// `None` is the normal outcome for every unassigned, Private Use, or + /// bundle-reserved value; the caller decides what that means (the + /// decoder relays such messages as [`Message::Unknown`]). + pub fn from_byte(b: u8) -> Option { + Some(match b { + 0x00 => Self::IndefinitePadding, + 0x01 => Self::DefinitePadding, + 0x02 => Self::Bundle, + 0x03 => Self::TransferSegment, + 0x04 => Self::TransferEnd, + 0x05 => Self::TransferCancel, + 0x70 => Self::PreAgreedFecSource, + 0x71 => Self::ExplicitFecSource, + 0x72 => Self::PreAgreedFecRepair, + 0x73 => Self::ExplicitFecRepair, + _ => return None, + }) + } + + /// Whether this is one of the four FEC extension message types. + pub fn is_fec(self) -> bool { + matches!( + self, + Self::PreAgreedFecSource + | Self::ExplicitFecSource + | Self::PreAgreedFecRepair + | Self::ExplicitFecRepair + ) + } +} + +impl From for u8 { + fn from(t: MessageType) -> u8 { + t as u8 + } +} + +/// Returns `true` if the type byte is the BPv6 reserved value (0x06), the +/// initial octet of a BPv6 bundle (Section 12.1). +pub fn is_reserved_bpv6(message_type: u8) -> bool { + message_type == 0x06 +} + +/// Returns `true` if the type byte falls in the BPv7 reserved range +/// (0x80..=0x9F), the possible initial octets of a BPv7 bundle's CBOR array +/// (Section 12.1). +pub fn is_reserved_bpv7(message_type: u8) -> bool { + (0x80..=0x9F).contains(&message_type) +} + +/// Classification of a received link-layer frame's first byte. +/// +/// BTP-U deliberately avoids the byte values that begin BPv6 (`0x06`) and +/// BPv7 (`0x80..=0x9F`) bundles, so a CLA carrying a mix of BTP-U PDUs and +/// bare bundle frames on the same link can route each frame by inspecting a +/// single byte. See [`frame_kind`]. +/// +/// All other first-byte values, including unallocated ranges, classify as +/// [`FrameKind::BtpuPdu`]. Future BTP-U message types may be assigned to +/// those unallocated bytes; a current decoder parses them as +/// [`Message::Unknown`] for forward compatibility. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)] +pub enum FrameKind { + /// A BTP-U PDU (or an empty frame, which decodes to zero messages). + BtpuPdu, + /// A BPv6 bundle (first byte = `0x06`). + Bpv6Bundle, + /// A BPv7 bundle (first byte in `0x80..=0x9F`, the CBOR array header range). + Bpv7Bundle, +} + +/// Classify a received frame by its first byte. +/// +/// Returns [`FrameKind::Bpv6Bundle`] if the frame begins with `0x06`, +/// [`FrameKind::Bpv7Bundle`] if it begins with a byte in `0x80..=0x9F`, and +/// [`FrameKind::BtpuPdu`] otherwise (including the empty-frame case). The +/// same two predicates, [`is_reserved_bpv6`] and [`is_reserved_bpv7`], +/// define the ranges. +pub fn frame_kind(frame: &[u8]) -> FrameKind { + match frame.first() { + Some(&b) if is_reserved_bpv6(b) => FrameKind::Bpv6Bundle, + Some(&b) if is_reserved_bpv7(b) => FrameKind::Bpv7Bundle, + _ => FrameKind::BtpuPdu, + } +} + +/// The 4-bit message flags field (Section 7.1). +#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)] +pub struct MessageFlags { + /// Bit 0 (MSB): when set, Hint Items follow the header. + pub hint: bool, + /// The three currently-unassigned bits (the low 3 bits of the nibble), + /// preserved verbatim. The flags registry is Standards Action (Section + /// 12.3), so future documents may assign them; zeroing them here would + /// corrupt a relayed [`Message::Unknown`] from such a sender. Locally + /// originated messages leave this zero (Section 7.1: the sender MUST). + pub rfu: u8, +} + +impl MessageFlags { + /// Encode into the 4-bit nibble (bits 7..4 of the second header byte). + pub fn to_nibble(self) -> u8 { + (if self.hint { 0x8 } else { 0 }) | (self.rfu & 0x7) + } + + /// Decode from the 4-bit nibble. + pub fn from_nibble(nibble: u8) -> Self { + Self { + hint: nibble & 0x8 != 0, + rfu: nibble & 0x7, + } + } +} + +/// A decoded BTP-U message. +/// +/// There is no variant for Indefinite Padding: it is a run of zero bytes +/// with no header, which the decoder consumes silently and +/// [`pad_pdu`](crate::codec::pad_pdu) writes directly. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum Message { + /// Definite Padding (type 1). Content is ignored. + DefinitePadding { + /// The declared content length; the content itself is never read. + len: usize, + }, + + /// Complete bundle (type 2), or an encapsulated bundle in its native + /// format found by the decoder (Section 7.3). + /// + /// The two are not told apart after decoding, and re-encoding always + /// frames the bundle as a type 2 message: an encapsulated bundle does + /// not relay byte-exact, and one longer than + /// [`MAX_CONTENT_LENGTH`](crate::codec::header::MAX_CONTENT_LENGTH) is + /// refused with [`Error::LengthOverflow`](crate::codec::Error::LengthOverflow). + Bundle { + /// Hint items carried by the message (always empty for an + /// encapsulated bundle). + hints: Vec, + /// The bundle bytes. + data: Bytes, + }, + + /// Transfer Segment (type 3, Section 8.2). + TransferSegment(TransferSegmentMessage), + + /// Transfer End (type 4, Section 8.3): the transfer's final segment, + /// with the same content layout as a Transfer Segment. + TransferEnd(TransferSegmentMessage), + + /// Transfer Cancel (type 5). + /// + /// The decoder ignores any hints and any content after the Transfer + /// Number: Section 8.4 gives the content as exactly the number but does + /// not say what a receiver does with any other length, and the header + /// length frames the message so the extra octets can be skipped. The + /// encoder writes neither. + TransferCancel { + /// The transfer to abandon (Section 8.4). + transfer_number: u32, + }, + + /// Pre-agreed FEC Source (provisional type 0x70, see [`MessageType`]): + /// the payload carries an ADU. + PreAgreedFecSource(PreAgreedFecMessage), + + /// Explicit FEC Source (provisional type 0x71, see [`MessageType`]): + /// the payload carries an ADU. + ExplicitFecSource(ExplicitFecMessage), + + /// Pre-agreed FEC Repair (provisional type 0x72, see [`MessageType`]): + /// the payload carries repair symbols. + PreAgreedFecRepair(PreAgreedFecMessage), + + /// Explicit FEC Repair (provisional type 0x73, see [`MessageType`]): + /// the payload carries repair symbols. + ExplicitFecRepair(ExplicitFecMessage), + + /// A message whose type this decoder does not interpret (an unassigned + /// value, a Private Use value, or an FEC value with FEC decoding off). + /// Preserved for forward compatibility. + /// + /// `data` is the raw, uninterpreted message content (any hint bytes + /// included) and `flags` preserves the decoded flags, so re-encoding + /// relays the message intact; in particular the H flag still frames + /// the hint bytes sitting at the front of `data`. + Unknown { + /// The wire type byte. + message_type: u8, + /// The flags nibble, verbatim. + flags: MessageFlags, + /// The message content, verbatim. + data: Bytes, + }, +} + +impl Message { + /// The wire type byte the message is encoded with. + pub(crate) fn type_byte(&self) -> u8 { + let mt = match self { + Message::DefinitePadding { .. } => MessageType::DefinitePadding, + Message::Bundle { .. } => MessageType::Bundle, + Message::TransferSegment(_) => MessageType::TransferSegment, + Message::TransferEnd(_) => MessageType::TransferEnd, + Message::TransferCancel { .. } => MessageType::TransferCancel, + Message::PreAgreedFecSource(_) => MessageType::PreAgreedFecSource, + Message::ExplicitFecSource(_) => MessageType::ExplicitFecSource, + Message::PreAgreedFecRepair(_) => MessageType::PreAgreedFecRepair, + Message::ExplicitFecRepair(_) => MessageType::ExplicitFecRepair, + Message::Unknown { message_type, .. } => return *message_type, + }; + mt.into() + } +} + +/// Content of a Transfer Segment message (type 3, Section 8.2) or a +/// Transfer End message (type 4, Section 8.3). The two have the same +/// layout: a Transfer End is the transfer's final segment, and only the +/// message type says so. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct TransferSegmentMessage { + /// The transfer this segment belongs to. + pub transfer_number: u32, + /// The segment's position in the transfer, counted from zero. In a + /// Transfer End this is the final index `N`, and the transfer is + /// complete once segments `0..=N` have all arrived. + pub segment_index: u32, + /// Hint items carried by the message. + pub hints: Vec, + /// The segment's bytes. + pub data: Bytes, +} diff --git a/btpu/src/codec/mod.rs b/btpu/src/codec/mod.rs new file mode 100644 index 000000000..ddc734fa1 --- /dev/null +++ b/btpu/src/codec/mod.rs @@ -0,0 +1,676 @@ +//! The BTP-U wire format: walking a PDU into messages (Section 7) and +//! encoding messages and padding back into one. + +use alloc::vec::Vec; +use core::{fmt, iter::FusedIterator}; + +use bytes::{Buf, BufMut, Bytes, BytesMut}; + +use self::{ + header::{ContentLength, HEADER_SIZE, MessageHeader, decode_header, encode_header}, + hint::{HintItem, decode_hints, encode_hints, encoded_hints_len}, + message::{ + FEC_FIELDS_SIZE, FrameKind, Message, MessageFlags, MessageType, SEGMENT_FIELDS_SIZE, + TRANSFER_NUMBER_SIZE, TransferSegmentMessage, frame_kind, + }, +}; +use crate::fec::{ExplicitFecMessage, PreAgreedFecMessage}; + +mod error; +pub mod header; +pub mod hint; +pub mod message; + +pub use self::error::{Error, Result}; + +/// Finds the extent of an encapsulated bundle so the decoder can step over +/// it: the caller's "peek" into a bundle format this crate does not parse. +/// +/// BTP-U reserves the message-type values that begin a bundle (0x06 for +/// BPv6, 0x80..=0x9F for BPv7, Section 12.1) so that a bundle in its native +/// format can appear where a message is expected (Section 7.3). Both bundle +/// formats are self-delimiting, but only to a receiver that implements them. +/// A CLA that holds a bundle parser supplies one of these through +/// [`DecodeOptions::bundle_extent`]; the decoder then delivers exactly the +/// bundle's bytes and continues with whatever follows, which is what +/// Section 7.3 asks of a receiver that implements the format. +/// +/// Any `Fn(&[u8]) -> Option` implements the trait, and a reference +/// to one coerces to the `&dyn BundleExtent` the options hold: +/// +/// ``` +/// # use bytes::Bytes; +/// # use hardy_btpu::codec::{DecodeOptions, decode_pdu_with, message::Message}; +/// // A stand-in for a bundle parser: every BPv7 bundle here is 3 bytes. +/// let extent = |bytes: &[u8]| (bytes.len() >= 3).then_some(3); +/// let options = DecodeOptions { +/// bundle_extent: Some(&extent), +/// ..DecodeOptions::default() +/// }; +/// // The bundle, then link padding the hook lets the decoder skip. +/// let pdu = Bytes::from_static(&[0x9F, 0xAA, 0xFF, 0, 0, 0]); +/// let mut messages = decode_pdu_with(pdu, options); +/// assert_eq!( +/// messages.next(), +/// Some(Ok(Message::Bundle { +/// hints: vec![], +/// data: Bytes::from_static(&[0x9F, 0xAA, 0xFF]), +/// })) +/// ); +/// assert_eq!(messages.next(), None); +/// ``` +/// +/// Once a hook is supplied, the decoder relies on it alone: a bundle it +/// declines is never taken to fill the rest of the PDU, even where the +/// decoder would do so without a hook, since a hook that cannot delimit +/// the bundle is better evidence than the bare-frame guess. +pub trait BundleExtent { + /// The length in bytes of the bundle that begins at `bytes[0]`, or + /// `None` if `bytes` does not begin a bundle this implementation can + /// delimit (including a bundle truncated by the end of `bytes`). + /// + /// `bytes` runs from the bundle's first byte to the end of the PDU, so + /// it may hold further messages or link padding after the bundle. + /// + /// The decoder ends the PDU with [`Error::EncapsulatedBundle`] on + /// `None` or `Some(0)` (a zero-length bundle cannot advance it), and + /// with [`Error::InsufficientData`] on a length past the end of + /// `bytes`. If this panics, the [`MessageIter`] is left where it was, + /// so advancing it again calls the hook on the same bytes. + fn bundle_extent(&self, bytes: &[u8]) -> Option; +} + +impl Option> BundleExtent for F { + fn bundle_extent(&self, bytes: &[u8]) -> Option { + self(bytes) + } +} + +/// Options for [`decode_pdu_with`]. The default decodes the base protocol +/// only and has no bundle-extent hook. +#[derive(Clone, Copy, Default)] +pub struct DecodeOptions<'a> { + /// Interpret the four provisional FEC message types (0x70..=0x73, see + /// [`MessageType`]). Off by default: the values are in the Private Use + /// range, so a link whose peer assigns them privately must be left to + /// relay them as [`Message::Unknown`]. + pub fec: bool, + /// How to find the extent of an encapsulated bundle (Section 7.3). + /// Without one, a bundle-reserved first byte makes the whole PDU the + /// bundle, and one found after a message is a terminal + /// [`Error::EncapsulatedBundle`]; see [`decode_pdu`] for the padding + /// consequences. + pub bundle_extent: Option<&'a dyn BundleExtent>, +} + +impl fmt::Debug for DecodeOptions<'_> { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.debug_struct("DecodeOptions") + .field("fec", &self.fec) + .field("bundle_extent", &self.bundle_extent.is_some()) + .finish() + } +} + +/// Lazily decode the messages in a single convergence layer PDU with the +/// default [`DecodeOptions`]: base protocol only, no bundle-extent hook. +/// +/// Returns a [`MessageIter`] yielding one [`Result`](Result) per +/// message; nothing is parsed until the iterator is advanced. Faults are +/// contained to the smallest unit the wire format allows: the Section 7 +/// header length bounds every message, so a message whose extent is known +/// but whose interior is malformed yields an [`Err`] and iteration continues +/// at the next message boundary (the skip-and-continue rule of Section 7.3), +/// while a framing fault (truncated header, length past the buffer, or an +/// encapsulated bundle of unknown extent) yields a final [`Err`] and stops +/// iteration permanently, since without a boundary the remaining bytes +/// cannot be walked. [`MessageIter::is_exhausted`] distinguishes the two +/// after an error. +/// +/// Indefinite Padding (zero bytes) is consumed silently. Unknown message +/// types are preserved as [`Message::Unknown`]. +/// +/// # Encapsulated bundles and padding +/// +/// A bundle in its native format may appear wherever a message may +/// (Section 7.3); it is recognised by its first byte (0x06 for BPv6, +/// 0x80..=0x9F for BPv7, Section 12.1), and a receiver that can delimit it +/// delivers it as a [`Message::Bundle`] and continues. This crate does not +/// parse bundle formats, so how far it gets depends on whether the caller +/// supplied a [`DecodeOptions::bundle_extent`] via [`decode_pdu_with`]: +/// +/// - **With a hook**, the bundle is exactly the bytes the hook measures, +/// whatever precedes or follows it: leading Indefinite Padding, further +/// messages after it, and trailing link padding are all handled. +/// - **Without one**, a bundle found before any message has been parsed +/// (at the start of the PDU, or after nothing but Indefinite Padding) is +/// taken to run to the end of the PDU and is yielded as a single +/// [`Message::Bundle`]; one found after a message is a terminal +/// [`Error::EncapsulatedBundle`], and the rest of the PDU is discarded, +/// as Section 7.3 requires of a receiver that cannot find the extent. +/// +/// The hookless rule is only correct when the link delivers the frame at +/// exactly the bundle's length. A link that pads frames to a minimum or +/// fixed size (Ethernet's 46-octet minimum payload, fixed-length CCSDS +/// frames) hands the padding to the consumer as bundle bytes, and a peer +/// that follows a bare bundle with padding or further messages loses them. +/// Such links need the hook; the sender-side counterpart is documented on +/// [`BundleFraming::Bare`](crate::sender::BundleFraming::Bare). A BTP-U +/// PDU has neither problem: link zero-fill after its last message decodes +/// as Indefinite Padding. +pub fn decode_pdu(pdu: Bytes) -> MessageIter<'static> { + decode_pdu_with(pdu, DecodeOptions::default()) +} + +/// Lazily decode the messages in a single convergence layer PDU with +/// explicit [`DecodeOptions`]. See [`decode_pdu`] for the iteration and +/// fault-containment rules. +pub fn decode_pdu_with(pdu: Bytes, options: DecodeOptions<'_>) -> MessageIter<'_> { + MessageIter { + pdu, + offset: 0, + options, + parsed_message: false, + done: false, + } +} + +/// Lazy message iterator over a PDU, returned by [`decode_pdu`] and +/// [`decode_pdu_with`]. +/// +/// Owns the PDU [`Bytes`], so yielded messages hold zero-copy views into it. +/// An [`Err`] for a message whose extent was known is recoverable: iteration +/// resumes at the next message boundary. An [`Err`] from the framing itself +/// exhausts the iterator: the stream position is unreliable, so no further +/// messages are parsed. +pub struct MessageIter<'a> { + pdu: Bytes, + offset: usize, + options: DecodeOptions<'a>, + /// Whether a message header has been framed yet. Until one has, a + /// bundle-reserved byte with no extent hook is read as a bare bundle + /// frame running to the end of the PDU. + parsed_message: bool, + done: bool, +} + +impl fmt::Debug for MessageIter<'_> { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.debug_struct("MessageIter") + .field("pdu_len", &self.pdu.len()) + .field("offset", &self.offset) + .field("options", &self.options) + .field("parsed_message", &self.parsed_message) + .field("done", &self.done) + .finish() + } +} + +impl MessageIter<'_> { + /// Whether the iterator has stopped permanently: either the PDU was + /// fully consumed, or a framing fault made the remainder undecodable + /// and it was discarded. + /// + /// Checked immediately after an [`Err`] item, this tells whether the + /// fault was contained to one skipped message (`false`) or cost the rest + /// of the PDU (`true`). + pub fn is_exhausted(&self) -> bool { + self.done + } + + /// Decode the BTP-U message at `offset`. + /// + /// Framing errors (a truncated header or a length past the buffer) are + /// terminal for the whole PDU: the next message boundary cannot be + /// determined. Once the extent is known, an interior fault is contained + /// to this one message: the offset moves to the next boundary the header + /// length gives and iteration continues. + fn next_message(&mut self) -> Result { + let hdr = match decode_header(&self.pdu[self.offset..]) { + Ok(hdr) => hdr, + // Counted from the start of the PDU, as the other framing + // errors are, rather than from the start of the header. + Err(Error::InsufficientData { .. }) => { + self.done = true; + return Err(Error::InsufficientData { + needed: self.offset + HEADER_SIZE, + available: self.pdu.len(), + }); + } + Err(e) => { + self.done = true; + return Err(e); + } + }; + let content_end = self.offset + HEADER_SIZE + hdr.length.get(); + if content_end > self.pdu.len() { + self.done = true; + return Err(Error::InsufficientData { + needed: content_end, + available: self.pdu.len(), + }); + } + self.parsed_message = true; + let content = self.pdu.slice(self.offset + HEADER_SIZE..content_end); + self.offset = content_end; + decode_message(hdr, content, self.options) + } + + /// Deliver the encapsulated bundle at `offset` (Section 7.3), whose + /// first byte is bundle-reserved. + /// + /// The extent comes from the caller's [`BundleExtent`] hook if there is + /// one; failing that, a bundle that precedes every message is taken to + /// fill the rest of the PDU (the bare-frame case). Anything else is + /// terminal: the extent is unknowable and Section 7.3 forbids processing + /// the remainder. + fn next_encapsulated_bundle(&mut self) -> Result { + let first_byte = self.pdu[self.offset]; + let remaining = self.pdu.len() - self.offset; + let extent = match self.options.bundle_extent { + Some(hook) => hook.bundle_extent(&self.pdu[self.offset..]), + None if !self.parsed_message => Some(remaining), + None => None, + }; + match extent { + Some(n) if n > 0 && n <= remaining => { + let data = self.pdu.slice(self.offset..self.offset + n); + self.offset += n; + Ok(Message::Bundle { + hints: Vec::new(), + data, + }) + } + Some(n) if n > remaining => { + self.done = true; + Err(Error::InsufficientData { + needed: self.offset.saturating_add(n), + available: self.pdu.len(), + }) + } + _ => { + self.done = true; + Err(Error::EncapsulatedBundle { + first_byte, + offset: self.offset, + }) + } + } + } +} + +impl Iterator for MessageIter<'_> { + type Item = Result; + + fn next(&mut self) -> Option { + if self.done { + return None; + } + + // Indefinite padding: skip a run of zero bytes (Section 8.6). It + // has no semantic content and the receiver MUST ignore it, so no + // item is yielded for it. + match self.pdu[self.offset..].iter().position(|&b| b != 0) { + Some(n) => self.offset += n, + None => { + self.offset = self.pdu.len(); + self.done = true; + return None; + } + } + + Some(match frame_kind(&self.pdu[self.offset..]) { + FrameKind::BtpuPdu => self.next_message(), + FrameKind::Bpv6Bundle | FrameKind::Bpv7Bundle => self.next_encapsulated_bundle(), + }) + } +} + +impl FusedIterator for MessageIter<'_> {} + +/// Decode one message whose header and content have been framed. +fn decode_message( + hdr: MessageHeader, + content: Bytes, + options: DecodeOptions<'_>, +) -> Result { + // Resolve the message type BEFORE parsing anything from the content. + // Section 7.3 requires an unrecognized type to be skipped via the + // header length field and processing to continue; it is preserved + // opaquely, its content (hint bytes included) uninterpreted, so a + // malformed or extension-defined hint chain in an unknown message + // cannot error the rest of the PDU. With FEC decoding off, the four + // provisional FEC codes are Private Use values like any other. + let Some(mt) = known_type(hdr.message_type, options.fec) else { + return Ok(Message::Unknown { + message_type: hdr.message_type, + flags: hdr.flags, + data: content, + }); + }; + + match mt { + // Receivers MUST ignore Definite Padding content (Section 8.5), so + // it is never parsed. Type 0 never arrives here: the iterator + // consumes zero runs before framing a header, and a zero byte is + // headerless (Section 8.6). Were one framed anyway, its declared + // content could only be padding. + MessageType::IndefinitePadding | MessageType::DefinitePadding => { + Ok(Message::DefinitePadding { len: content.len() }) + } + + MessageType::Bundle => { + let (hints, data) = split_hints(hdr.flags, content)?; + Ok(Message::Bundle { hints, data }) + } + + MessageType::TransferSegment | MessageType::TransferEnd => { + let (hints, mut data) = split_hints(hdr.flags, content)?; + let m = TransferSegmentMessage { + transfer_number: data.try_get_u32()?, + segment_index: data.try_get_u32()?, + hints, + data, + }; + Ok(if mt == MessageType::TransferEnd { + Message::TransferEnd(m) + } else { + Message::TransferSegment(m) + }) + } + + // Section 8.4 gives the content as exactly the Transfer Number, but + // not what a receiver does with any other length. The header + // frames the message, so octets after the number are skipped, as + // are any hints (the H flag is header-level, Section 7.1). + MessageType::TransferCancel => { + let (_, mut data) = split_hints(hdr.flags, content)?; + Ok(Message::TransferCancel { + transfer_number: data.try_get_u32()?, + }) + } + + // The FEC payload stays opaque: its scheme-defined internal + // boundaries are not knowable here (see the struct docs). + MessageType::PreAgreedFecSource | MessageType::PreAgreedFecRepair => { + let (hints, mut payload) = split_hints(hdr.flags, content)?; + let m = PreAgreedFecMessage { + transfer_number: payload.try_get_u32()?, + fec_instance_id: payload.try_get_u8()?, + hints, + payload, + }; + Ok(if mt == MessageType::PreAgreedFecSource { + Message::PreAgreedFecSource(m) + } else { + Message::PreAgreedFecRepair(m) + }) + } + + MessageType::ExplicitFecSource | MessageType::ExplicitFecRepair => { + let (hints, mut payload) = split_hints(hdr.flags, content)?; + let m = ExplicitFecMessage { + transfer_number: payload.try_get_u32()?, + fec_encoding_id: payload.try_get_u8()?, + hints, + payload, + }; + Ok(if mt == MessageType::ExplicitFecSource { + Message::ExplicitFecSource(m) + } else { + Message::ExplicitFecRepair(m) + }) + } + } +} + +/// Split a message's content into its hint items (if the H flag is set) and +/// the type-specific data that follows them. +fn split_hints(flags: MessageFlags, content: Bytes) -> Result<(Vec, Bytes)> { + if !flags.hint { + return Ok((Vec::new(), content)); + } + let (items, consumed) = decode_hints(&content)?; + Ok((items, content.slice(consumed..))) +} + +/// Returns the total encoded size of a message, exactly matching what +/// [`encode_message`] writes: header + hints + content. +pub fn encoded_message_len(message: &Message) -> usize { + HEADER_SIZE + Content::of(message).len() +} + +/// The encoded size of a Transfer Segment or Transfer End message carrying +/// `hints` and `data_len` bytes of segment data. +pub(crate) fn segment_message_len(hints: &[HintItem], data_len: usize) -> usize { + let content = Content { + hints, + fields: Fields::Segment(0, 0), + body: Body::Zeros(data_len), + }; + HEADER_SIZE + content.len() +} + +/// Encode a Transfer Segment message, or a Transfer End if `end`, up to its +/// data, which the caller appends: exactly `data_len` bytes, so that the +/// header's length is true. Lets the sender write a segment's data +/// straight from the chunks it was pushed in. +/// +/// Errors, leaving `dst` untouched, if the content would not fit the +/// 20-bit length field. +pub(crate) fn encode_segment_head( + end: bool, + transfer_number: u32, + segment_index: u32, + hints: &[HintItem], + data_len: usize, + dst: &mut BytesMut, +) -> Result<()> { + let message_type = if end { + MessageType::TransferEnd + } else { + MessageType::TransferSegment + }; + let content = Content { + hints, + fields: Fields::Segment(transfer_number, segment_index), + body: Body::Zeros(data_len), + }; + content.encode_head(message_type as u8, content.flags(), dst) +} + +/// Encode a single message into `dst`. +/// +/// Errors leave `dst` untouched: the content length is checked before any +/// byte is written. A [`Message::Unknown`] whose type +/// value is defined by the base protocol or bundle-reserved is refused with +/// [`Error::NotAnUnknownType`]; the four provisional FEC values are Private +/// Use and may be relayed as unknown. +pub fn encode_message(message: &Message, dst: &mut BytesMut) -> Result<()> { + let content = Content::of(message); + let flags = match message { + Message::Unknown { + message_type, + flags, + .. + } => { + if !is_relayable_as_unknown(*message_type) { + return Err(Error::NotAnUnknownType(*message_type)); + } + *flags + } + _ => content.flags(), + }; + content.encode_head(message.type_byte(), flags, dst)?; + match content.body { + Body::Zeros(len) => dst.put_bytes(0, len), + Body::Bytes(bytes) => dst.put_slice(bytes), + } + Ok(()) +} + +/// The message type a framed message of type `message_type` decodes as, or +/// `None` if it is carried as [`Message::Unknown`]. With `fec` off, the +/// four provisional FEC codes are Private Use values like any other. +fn known_type(message_type: u8, fec: bool) -> Option { + MessageType::from_byte(message_type).filter(|mt| fec || !mt.is_fec()) +} + +/// Whether a type value may be carried by [`Message::Unknown`]: anything a +/// decoder with FEC off would itself have produced as unknown, which +/// excludes the base-protocol types and the bundle-reserved values (those +/// never frame as a message, so the decoder never asks [`known_type`]). +fn is_relayable_as_unknown(message_type: u8) -> bool { + frame_kind(&[message_type]) == FrameKind::BtpuPdu && known_type(message_type, false).is_none() +} + +/// The fixed-size fields between a message's hints and its body. +enum Fields { + None, + Cancel(u32), + Segment(u32, u32), + Fec(u32, u8), +} + +impl Fields { + fn len(&self) -> usize { + match self { + Fields::None => 0, + Fields::Cancel(_) => TRANSFER_NUMBER_SIZE, + Fields::Segment(..) => SEGMENT_FIELDS_SIZE, + Fields::Fec(..) => FEC_FIELDS_SIZE, + } + } + + fn put(&self, dst: &mut BytesMut) { + match *self { + Fields::None => {} + Fields::Cancel(transfer_number) => dst.put_u32(transfer_number), + Fields::Segment(transfer_number, segment_index) => { + dst.put_u32(transfer_number); + dst.put_u32(segment_index); + } + Fields::Fec(transfer_number, id) => { + dst.put_u32(transfer_number); + dst.put_u8(id); + } + } + } +} + +/// The variable-length end of a message's content. +enum Body<'a> { + /// Padding: this many zero bytes. + Zeros(usize), + Bytes(&'a [u8]), +} + +impl Body<'_> { + fn len(&self) -> usize { + match self { + Body::Zeros(len) => *len, + Body::Bytes(bytes) => bytes.len(), + } + } +} + +/// A message's content, everything after the header, in wire order. +struct Content<'a> { + hints: &'a [HintItem], + fields: Fields, + body: Body<'a>, +} + +impl<'a> Content<'a> { + /// The flags of a message the crate defines with this content. + fn flags(&self) -> MessageFlags { + MessageFlags { + hint: !self.hints.is_empty(), + rfu: 0, + } + } + + /// Write the header, hints, and fields of a message with this content, + /// everything but the body. Errors, leaving `dst` untouched, if the + /// content would not fit the 20-bit length field. + fn encode_head(&self, message_type: u8, flags: MessageFlags, dst: &mut BytesMut) -> Result<()> { + let length = ContentLength::try_from(self.len())?; + write_header(message_type, flags, length, dst); + encode_hints(self.hints, dst); + self.fields.put(dst); + Ok(()) + } + + fn of(message: &'a Message) -> Self { + let (hints, fields, body): (&[HintItem], _, _) = match message { + Message::DefinitePadding { len } => (&[], Fields::None, Body::Zeros(*len)), + Message::Bundle { hints, data } => (hints, Fields::None, Body::Bytes(data)), + Message::TransferSegment(m) | Message::TransferEnd(m) => ( + &m.hints, + Fields::Segment(m.transfer_number, m.segment_index), + Body::Bytes(&m.data), + ), + Message::TransferCancel { transfer_number } => { + (&[], Fields::Cancel(*transfer_number), Body::Bytes(&[])) + } + Message::PreAgreedFecSource(m) | Message::PreAgreedFecRepair(m) => ( + &m.hints, + Fields::Fec(m.transfer_number, m.fec_instance_id), + Body::Bytes(&m.payload), + ), + Message::ExplicitFecSource(m) | Message::ExplicitFecRepair(m) => ( + &m.hints, + Fields::Fec(m.transfer_number, m.fec_encoding_id), + Body::Bytes(&m.payload), + ), + Message::Unknown { data, .. } => (&[], Fields::None, Body::Bytes(data)), + }; + Self { + hints, + fields, + body, + } + } + + fn len(&self) -> usize { + encoded_hints_len(self.hints) + self.fields.len() + self.body.len() + } +} + +/// Write a message header for `length` content bytes. +fn write_header(message_type: u8, flags: MessageFlags, length: ContentLength, dst: &mut BytesMut) { + dst.put_slice(&encode_header(&MessageHeader { + message_type, + flags, + length, + })); +} + +/// Pad `dst` to `target_len` bytes. +/// +/// Uses Definite Padding for >= 4 bytes of remaining space, then Indefinite +/// Padding (zeros) for any remaining 1-3 bytes, per spec recommendation. +/// Space beyond a single message's 20-bit length field is filled with a +/// chain of maximum-size Definite Padding messages (padding is valid at any +/// point in a PDU, Section 3.2), so every target length is reachable and the +/// emitted headers are always truthful. +pub fn pad_pdu(dst: &mut BytesMut, target_len: usize) { + while dst.len() < target_len { + let remaining = target_len - dst.len(); + if remaining >= HEADER_SIZE { + // Definite Padding: header (4 bytes) + zero-filled content, + // capped to what the 20-bit length field can declare. + let length = ContentLength::new(remaining - HEADER_SIZE).unwrap_or(ContentLength::MAX); + write_header( + MessageType::DefinitePadding.into(), + MessageFlags::default(), + length, + dst, + ); + dst.put_bytes(0, length.get()); + } else { + // Indefinite Padding: just zero bytes + dst.put_bytes(0, remaining); + } + } +} diff --git a/btpu/src/fec.rs b/btpu/src/fec.rs new file mode 100644 index 000000000..a6ed5eb1a --- /dev/null +++ b/btpu/src/fec.rs @@ -0,0 +1,66 @@ +//! FEC extension message types (draft-ietf-dtn-btpu-fec). +//! +//! This module defines the content of the four message types introduced by +//! the BTP-U FEC extension. The crate frames them (encode, decode, and +//! window tracking) but implements no FEC scheme and performs no FEC +//! reassembly; the payload of each message is carried opaquely. +//! +//! The four types share two layouts. A pre-agreed message names a +//! scheme instance configured out of band; an explicit one names an FEC +//! Encoding ID and carries the scheme's inline parameters in its payload. +//! Whether the payload holds source data or repair symbols is decided by +//! the [`Message`](crate::codec::message::Message) variant that wraps the +//! content, not by the content itself. +//! +//! The message type codes are not yet IANA-assigned (the FEC draft lists +//! them as TBD1..TBD4); the values used here are provisional Private Use +//! codes, see [`MessageType`](crate::codec::message::MessageType). Because +//! the range is Private Use, a decoder interprets them only when asked to +//! via [`DecodeOptions::fec`](crate::codec::DecodeOptions::fec) or +//! [`ReceiverConfig::fec`](crate::receiver::ReceiverConfig::fec); otherwise +//! they relay as [`Message::Unknown`](crate::codec::message::Message::Unknown). + +use alloc::vec::Vec; + +use bytes::Bytes; + +use crate::codec::hint::HintItem; + +/// Content of a Pre-agreed FEC Source or Repair message (provisional types +/// 0x70 and 0x72), for a transfer using a pre-configured FEC scheme. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct PreAgreedFecMessage { + /// The transfer this message belongs to. + pub transfer_number: u32, + /// Identifies the pre-agreed FEC scheme instance. + pub fec_instance_id: u8, + /// Hint items carried by the message. + pub hints: Vec, + /// The Explicit Source FEC Payload ID and Source Data (Source message), + /// or the Repair FEC Payload ID and Repair Symbol Data (Repair message), + /// as raw bytes. + /// + /// The boundary between the two is defined by the pre-agreed scheme and + /// cannot be determined here. + pub payload: Bytes, +} + +/// Content of an Explicit FEC Source or Repair message (provisional types +/// 0x71 and 0x73), which carries its FEC-Scheme-Specific Information +/// (FSSI) inline. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct ExplicitFecMessage { + /// The transfer this message belongs to. + pub transfer_number: u32, + /// The FEC Encoding ID of the scheme (RFC 6363 Section 5.6). + pub fec_encoding_id: u8, + /// Hint items carried by the message. + pub hints: Vec, + /// The FSSI elements, then the Explicit Source FEC Payload ID and + /// Source Data (Source message) or the Repair FEC Payload ID and Repair + /// Symbol Data (Repair message), as raw bytes. + /// + /// The boundaries are defined by the scheme identified by + /// `fec_encoding_id` and cannot be determined here. + pub payload: Bytes, +} diff --git a/btpu/src/lib.rs b/btpu/src/lib.rs new file mode 100644 index 000000000..a49aae18a --- /dev/null +++ b/btpu/src/lib.rs @@ -0,0 +1,270 @@ +#![no_std] +#![forbid(unsafe_code)] +#![warn(missing_docs)] +/*! +Bundle Transfer Protocol - Unidirectional (BTP-U) codec and transfer logic. + +This crate implements the protocol defined in [draft-ietf-dtn-btpu] for the +unidirectional, unreliable transfer of binary objects (typically BPv7 +bundles) over frame-based convergence layer protocols. + +It also frames the messages of the FEC extension defined in +[draft-ietf-dtn-btpu-fec]; no FEC scheme is implemented. + +[draft-ietf-dtn-btpu]: https://datatracker.ietf.org/doc/draft-ietf-dtn-btpu/ +[draft-ietf-dtn-btpu-fec]: https://datatracker.ietf.org/doc/draft-ietf-dtn-btpu-fec/ + +Section references throughout this crate's documentation and comments follow +draft-ietf-dtn-btpu-04 and draft-ietf-dtn-btpu-fec-02; this is the only place +the target revisions are pinned. + +# Overview + +BTP-U sits between the Bundle Protocol and a convergence layer protocol, +providing segmentation and transfer windowing without requiring IP services +or a return channel. The protocol additionally permits message interleaving +and repetition-based loss protection; this crate's sender interleaves only +by passing over a transfer that cannot supply its next segment, and does not +repeat messages (see [`sender::Sender`]). + +```text ++----------------------+ +| DTN Application | ++----------------------+ +| BPv7 / BPv6 | ++----------------------+ +| BTP-U | <-- this crate ++----------------------+ +| CL Protocol | ++----------------------+ +``` + +# Architecture + +The crate is a **pure protocol library** with no dependency on `hardy-bpa`, +`hardy-bpv7`, `std`, or any async runtime. It is `#![no_std]` and uses only +`alloc`, suitable for embedded and CCSDS-frame deployments. A +convergence-layer-specific CLA crate would use this library alongside +`hardy-bpa` to integrate with the BPA. + +Two layers of abstraction are provided: + +- **Low-level codec**: [`codec::decode_pdu`], [`codec::encode_message`], + [`codec::pad_pdu`] for direct PDU manipulation. [`codec::decode_pdu_with`] + takes [`codec::DecodeOptions`]: FEC decoding, and a + [`codec::BundleExtent`] hook through which a caller that parses bundles + lets the decoder delimit encapsulated bundles. +- **High-level sender/receiver**: [`sender::Sender`] and + [`receiver::Receiver`] for typical CLA implementations, configured by + [`sender::SenderConfig`] and [`receiver::ReceiverConfig`]. A receiver + delivers each bundle whole, or streams it in order as it arrives. + Receivers serving one link can share a `budget::RetentionBudget` (on + targets with pointer-width atomics). + +The [`transfer`] module holds the Section 5 types both layers above share: +[`transfer::WindowSize`] and the receiver's [`transfer::TransferId`]. + +# Features + +- **`serde`**: `Serialize`/`Deserialize` for the configuration types + ([`sender::SenderConfig`], [`receiver::ReceiverConfig`], and the validated + newtypes they hold, which deserialize from plain integers with their range + checks applied). +- **`rand`**: Adds `try_from_rng` and `from_rng` constructors for + [`sender::Sender`] that seed the + initial transfer number from a `rand_core::TryRng` (such as the operating + system's `rand::rngs::SysRng`) or `rand_core::Rng`. Without this feature, + callers pass an explicit `initial_transfer_number: u32`. +- **`tower`**: Implements `tower::Service` for + [`sender::Sender`] (enqueue; `SendRequest: From`) and + `tower::Service` for [`receiver::Receiver`] (PDU dispatch), and + `futures_core::Stream` (item: [`sender::Pdu`]) for [`sender::Sender`] + (outgoing PDU drain). Enables composition with `tower::ServiceBuilder` for rate + limiting, metrics, etc. Requires `std` at the consumer level (the `tower` + crate itself is std-only). + + The sender's `Service::poll_ready` applies backpressure from the + transfer window and the [`sender::SendQueueBytes`]; the receiver's + service is always ready and never fails. How to run the `Stream` drain + alongside producers, and how several producers may share one sender, is + set out under Concurrency in [`sender::Sender`]. +- **`critical-section`**: For targets without native atomic + compare-and-swap (such as `thumbv6m`, Cortex-M0). The crate itself uses + no atomics, but `bytes` reference-counts with them; this feature switches + `bytes` to `portable-atomic` with its critical-section fallback, which + needs a `critical-section` implementation from the HAL or runtime. +*/ + +extern crate alloc; + +use core::{error::Error, fmt, num::ParseIntError}; + +/// Implements the conversions every configuration newtype shares, as +/// `core::num::NonZero` does for its integer: `Display`, `Binary`, `Octal`, +/// `LowerHex`, and `UpperHex` forwarded to the field, `FromStr` returning +/// [`ParseError`], `TryFrom<$int>` returning [`OutOfRange`], and +/// `From<$ty> for $int`. Naming a `NonZero` type as well adds `From` in +/// both directions, for newtypes that accept every non-zero value. +/// +/// The type must have a `NAME` constant, `MIN` and `MAX` constants, and +/// `const fn new($int) -> Option` and `const fn get(self) -> $int`. +/// The [`OutOfRange`] carries no `max` when `MAX` is the integer's maximum. +/// Compile-time assertions check that `new` accepts exactly `MIN..=MAX` +/// at the boundaries, that `Option<$ty>` costs no space, and, for the +/// `NonZero` form, that `MIN..=MAX` is every non-zero value, since its +/// `From` bypasses `new`. +/// +/// Defined above the `mod` declarations because `macro_rules!` scoping is +/// textual: only modules declared after it can use it. +macro_rules! config_newtype { + ($ty:ident: $int:ty, $nz:ty) => { + config_newtype!($ty: $int); + + // `From<$nz>` admits every non-zero value without calling `new`. + const _: () = assert!($ty::MIN.get() == 1 && $ty::MAX.get() == <$int>::MAX); + + impl From<$nz> for $ty { + fn from(n: $nz) -> Self { + Self(n) + } + } + + impl From<$ty> for $nz { + fn from(v: $ty) -> $nz { + v.0 + } + } + }; + ($ty:ident: $int:ty) => { + config_newtype!(@fmt $ty: Display, Binary, Octal, LowerHex, UpperHex); + + // `MIN` and `MAX` are what `OutOfRange` reports, so they must be + // the bounds `new` applies. + const _: () = assert!( + $ty::new($ty::MIN.get()).is_some() + && $ty::new($ty::MAX.get()).is_some() + && ($ty::MIN.get() == 0 || $ty::new($ty::MIN.get() - 1).is_none()) + && ($ty::MAX.get() == <$int>::MAX || $ty::new($ty::MAX.get() + 1).is_none()) + ); + const _: () = assert!(size_of::>() == size_of::<$ty>()); + + impl ::core::str::FromStr for $ty { + type Err = $crate::ParseError; + + fn from_str(s: &str) -> ::core::result::Result { + let v = s.parse::<$int>().map_err(|source| $crate::ParseError::Syntax { + name: Self::NAME, + source, + })?; + Ok(Self::try_from(v)?) + } + } + + impl TryFrom<$int> for $ty { + type Error = $crate::OutOfRange; + + // `as u64` widens: there is no `From for u64`, and every + // integer used here fits. + fn try_from(v: $int) -> ::core::result::Result { + Self::new(v).ok_or($crate::OutOfRange { + name: Self::NAME, + value: v as u64, + min: Self::MIN.get() as u64, + max: (Self::MAX.get() != <$int>::MAX).then_some(Self::MAX.get() as u64), + }) + } + } + + impl From<$ty> for $int { + fn from(v: $ty) -> $int { + v.get() + } + } + }; + (@fmt $ty:ty: $($tr:ident),*) => { + $( + impl ::core::fmt::$tr for $ty { + fn fmt(&self, f: &mut ::core::fmt::Formatter<'_>) -> ::core::fmt::Result { + ::core::fmt::$tr::fmt(&self.0, f) + } + } + )* + }; +} + +#[cfg(target_has_atomic = "ptr")] +pub mod budget; +pub mod codec; +pub mod fec; +pub mod receiver; +pub mod sender; +pub mod transfer; + +#[cfg(feature = "tower")] +mod service; + +/// A configuration value outside the range its type accepts. +/// +/// The error of every configuration newtype's `TryFrom`, and of its `FromStr` +/// through [`ParseError::OutOfRange`]: [`sender::PduSize`], +/// [`sender::SendQueueBytes`], [`transfer::WindowSize`], +/// [`receiver::MaxTransferSize`], [`receiver::MaxSegments`], and +/// [`receiver::MaxRetainedBytes`]. Values are +/// widened to `u64` so one type covers all of them. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct OutOfRange { + /// What the value configures, in the words the message uses. + pub name: &'static str, + /// The rejected value. + pub value: u64, + /// The least valid value. + pub min: u64, + /// The greatest valid value, or `None` when every value from `min` up + /// to the integer type's maximum is valid. + pub max: Option, +} + +// Hand-written rather than derived with `thiserror`: the message takes a +// different form when there is no `max`, which one `#[error]` format string +// cannot express. +impl fmt::Display for OutOfRange { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + let Self { + name, + value, + min, + max, + } = self; + match max { + Some(max) => write!(f, "Invalid {name} {value} (must be {min}..={max})"), + None => write!(f, "Invalid {name} {value} (must be at least {min})"), + } + } +} + +impl Error for OutOfRange {} + +/// A configuration value that could not be parsed from a string. +/// +/// The error of every configuration newtype's `FromStr`. +#[derive(Debug, Clone, PartialEq, Eq, thiserror::Error)] +pub enum ParseError { + /// The string is not an unsigned integer of the newtype's width. + #[error("Invalid {name}: {source}")] + Syntax { + /// What the value configures, in the words the message uses. + name: &'static str, + /// Why the integer did not parse. + source: ParseIntError, + }, + /// The integer parsed but is outside the newtype's range. + #[error(transparent)] + OutOfRange(#[from] OutOfRange), +} + +/// Compiles the README's code blocks as doctests so the example there +/// cannot drift from the API. The example seeds its sender from the +/// operating system RNG, so it needs the `rand` feature. +#[cfg(all(doctest, feature = "rand"))] +#[doc = include_str!("../README.md")] +pub struct ReadmeDoctests; diff --git a/btpu/src/receiver.rs b/btpu/src/receiver.rs new file mode 100644 index 000000000..71d2936cf --- /dev/null +++ b/btpu/src/receiver.rs @@ -0,0 +1,2298 @@ +//! The receiving end: decodes PDUs, reassembles transfers within the +//! Section 5 window under configured memory limits, and reports every +//! outcome as a [`ReceiverEvent`]. + +#[cfg(target_has_atomic = "ptr")] +use alloc::sync::Arc; +use alloc::{ + boxed::Box, + collections::{BTreeMap, btree_map::Entry}, + vec::Vec, +}; +use core::{ + fmt, + mem::{replace, take}, + num::{NonZeroU32, NonZeroUsize}, +}; + +use bytes::{BufMut, Bytes, BytesMut}; + +#[cfg(target_has_atomic = "ptr")] +use crate::budget::RetentionBudget; +use crate::{ + codec::{ + BundleExtent, DecodeOptions, Error, decode_pdu_with, + header::MAX_CONTENT_LENGTH, + hint::{HINT_TYPE_COUNT, HintItem, HintType, HintValue, Hints, MAX_HINT_VALUE_LEN}, + message::{Message, SEGMENT_FIELDS_SIZE, SEGMENT_FRAMING, TransferSegmentMessage}, + }, + transfer::{Admission, TransferId, TransferWindow, WindowSize}, +}; + +/// Bookkeeping bytes charged for every stored segment, in addition to the +/// segment's own length, when no [`MaxSegments`] limit is configured. +/// +/// It is an estimate of what a stored segment costs beyond its data (the +/// [`Bytes`] handle and its map entry), not an exact heap figure, so that a +/// peer sending many tiny segments cannot hold far more memory than the +/// [`MaxTransferSize`] suggests. It is rounded down: measured on a 64-bit +/// host, a tiny stored segment costs 70 to 100 bytes after allocator +/// rounding, so a flood can hold up to about half as much again as the +/// charged figure. The charge counts against a transfer's +/// bookkeeping budget, not against the bundle-size cap itself (see +/// [`MaxTransferSize`]); a flood of empty or one-byte segments is bounded to +/// roughly `max_transfer_size / SEGMENT_OVERHEAD` entries. The same bound +/// applies to an honest sender, so a link whose segments carry fewer than +/// about `SEGMENT_OVERHEAD` data bytes each needs a [`MaxSegments`] limit +/// to deliver bundles near the cap. +pub const SEGMENT_OVERHEAD: usize = 64; + +/// The least bookkeeping budget a transfer has, whatever its +/// [`MaxTransferSize`]: room for 64 stored segments, or the equivalent in +/// hint bytes. +/// +/// The budget is otherwise the cap itself, which for a cap of a few hundred +/// bytes would leave room for no segments at all; the floor keeps a small +/// cap from refusing every segmented transfer while still bounding what a +/// peer can hold. +pub const MIN_OVERHEAD_BUDGET: usize = 64 * SEGMENT_OVERHEAD; + +/// The least limit [`MaxSegments::for_link_pdu_size`] yields, whatever the +/// cap: the same 64 segments [`MIN_OVERHEAD_BUDGET`] allows. +const MIN_SEGMENT_ALLOWANCE: MaxSegments = + MaxSegments(NonZeroU32::new((MIN_OVERHEAD_BUDGET / SEGMENT_OVERHEAD) as u32).unwrap()); + +/// How many times the reference segment count +/// [`MaxSegments::for_link_pdu_size`] allows. +/// +/// Headroom for segments that do not fill their PDU: the first segment's +/// hints, packing remainders, and a sender interleaving transfers within a +/// PDU (Section 4.1), where two-way interleaving alone halves every +/// segment. +const SEGMENT_ALLOWANCE_MULTIPLIER: u32 = 4; + +/// What a retained hint item counts against the limits beyond its value +/// bytes: its entry in its transfer's sorted list. +/// +/// Charged the same way and to the same bookkeeping budget as +/// [`SEGMENT_OVERHEAD`], except for a well-formed Bundle Length hint, which +/// is held inline and charged nothing. The figure is the entry's size, so +/// it differs between 32- and 64-bit targets. +pub const HINT_OVERHEAD: usize = size_of::<(HintType, HintValue)>(); + +/// The most hint charge one transfer can hold: one item of the longest +/// value for every hint type. +const MAX_HINT_CHARGE: usize = HINT_TYPE_COUNT * (HINT_OVERHEAD + MAX_HINT_VALUE_LEN); + +/// The most segment data one message can carry, hints aside. +const MAX_SEGMENT_DATA: usize = MAX_CONTENT_LENGTH - SEGMENT_FIELDS_SIZE; + +/// A validated cap, in bytes (non-zero), on what a receiver accepts in one +/// transfer or unsegmented message. +/// +/// The cap is a policy on the bundle's true length and a budget on the +/// state a transfer may hold, applied as data arrives. They are separate +/// acceptance conditions: a bundle within the cap can still be refused for +/// arriving in too many segments. +/// +/// - A transfer is rejected as [`RejectReason::TooLarge`] as soon as the +/// segment bytes received so far exceed the cap, or earlier if the +/// sender's Bundle Length hint promises a bundle above it. +/// - A transfer is rejected as [`RejectReason::TooFragmented`] when it holds +/// more segments, or more hint data, than the bundle it could be carrying +/// justifies. With a [`MaxSegments`] limit configured, the number of +/// distinct segments is limited to it. Without one, each stored +/// segment is charged [`SEGMENT_OVERHEAD`] instead. Either way every +/// retained hint is charged [`HINT_OVERHEAD`] plus its value, except a +/// well-formed Bundle Length, which the transfer holds inline and is +/// charged nothing. The charges count against a bookkeeping budget of the cap, +/// or [`MIN_OVERHEAD_BUDGET`] if the cap is smaller. +/// - An unsegmented Bundle message is compared against the cap by its +/// length alone, since it is never stored. +/// +/// When both conditions fail on the same message, the transfer is reported +/// as `TooLarge`. Construct via [`MaxTransferSize::new`], [`TryFrom`], +/// or [`From`], which enforce the bound at the edge; every +/// consumer of a `MaxTransferSize` can then rely on it. +/// There is no "unlimited" value: a receiver reassembles bundles in +/// memory, so an unbounded cap hands a remote peer a memory-exhaustion +/// lever. Derive the cap from the deployment, the largest bundle the node +/// is meant to accept. At or near [`Self::MAX`] the size gates stop +/// working: the receiver's counters saturate at `usize::MAX`, so neither +/// [`RejectReason::TooLarge`] nor, under the default [`MaxRetainedBytes`], +/// [`RejectReason::ReceiverFull`] can fire, and on a 32-bit target memory +/// runs out first. The cap is per transfer; +/// see [`Receiver`] for the bound on the receiver as a whole. +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] +#[repr(transparent)] +#[cfg_attr( + feature = "serde", + derive(serde::Serialize, serde::Deserialize), + serde(try_from = "usize", into = "usize") +)] +pub struct MaxTransferSize(NonZeroUsize); + +impl MaxTransferSize { + /// What the value configures, as error messages name it. + const NAME: &str = "max transfer size"; + + /// The smallest cap, one byte. + pub const MIN: Self = Self(NonZeroUsize::MIN); + + /// The largest cap, `usize::MAX` bytes. + pub const MAX: Self = Self(NonZeroUsize::MAX); + + /// 1 GiB, matching the Hardy TCPCLv4 transfer-MRU default. + pub const DEFAULT: Self = Self(NonZeroUsize::new(0x4000_0000).unwrap()); + + /// Returns the cap for `bytes`, or `None` if it is zero. + pub const fn new(bytes: usize) -> Option { + match NonZeroUsize::new(bytes) { + Some(n) => Some(Self(n)), + None => None, + } + } + + /// Returns the cap as a plain integer. + pub const fn get(self) -> usize { + self.0.get() + } +} + +impl Default for MaxTransferSize { + fn default() -> Self { + Self::DEFAULT + } +} + +config_newtype!(MaxTransferSize: usize, NonZeroUsize); + +/// A validated limit on the distinct segments one transfer may hold +/// (non-zero). +/// +/// A transfer that receives a new segment with the limit already reached is +/// rejected as [`RejectReason::TooFragmented`]; repeats of a stored segment +/// never count. The limit replaces the [`SEGMENT_OVERHEAD`] charge, so a +/// link whose segments carry few data bytes can deliver bundles up to the +/// [`MaxTransferSize`], which that charge would refuse. +/// +/// Each stored segment, empty ones included, costs up to about 100 bytes of +/// heap beyond its data (see [`SEGMENT_OVERHEAD`]), so a limit of `N` lets +/// one transfer hold about `N × 100` bytes of bookkeeping, up to +/// `window_size` transfers at once, all within the [`MaxRetainedBytes`]. +/// Size it for the link rather than setting it large: a peer can fill it +/// with empty segments. +/// +/// [`Self::for_link_pdu_size`] derives a limit from the link's PDU size and +/// the cap; [`MaxSegments::new`], [`TryFrom`], and +/// [`From`] take a value directly. +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] +#[repr(transparent)] +#[cfg_attr( + feature = "serde", + derive(serde::Serialize, serde::Deserialize), + serde(try_from = "u32", into = "u32") +)] +pub struct MaxSegments(NonZeroU32); + +impl MaxSegments { + /// What the value configures, as error messages name it. + const NAME: &str = "max segments per transfer"; + + /// The smallest limit, one segment. + pub const MIN: Self = Self(NonZeroU32::MIN); + + /// The largest limit, `u32::MAX` segments. + pub const MAX: Self = Self(NonZeroU32::MAX); + + /// Returns the limit for `segments`, or `None` if it is zero. + pub const fn new(segments: u32) -> Option { + match NonZeroU32::new(segments) { + Some(n) => Some(Self(n)), + None => None, + } + } + + /// Returns the limit as a plain integer. + pub const fn get(self) -> u32 { + self.0.get() + } + + /// The limit for a link that delivers PDUs of at most `link_pdu_size` + /// bytes, lower-layer headers removed, to a receiver with + /// `max_transfer_size`. + /// + /// With `S` the segment data one such PDU carries (the PDU less 12 bytes + /// of message header, transfer number, and segment index; at least 1, at + /// most the 20-bit content-length ceiling), the limit is four times + /// `ceil(max_transfer_size / S)`, never less than 64 and never more than + /// `u32::MAX`. Hints are ignored in `S`; the factor of four absorbs + /// them along with packing remainders and a sender interleaving + /// transfers within a PDU. For a 1 MiB cap and 1036-byte PDUs (`S` = + /// 1024) the limit is 4096 segments. + /// + /// The result is only as good as the reference: a sender whose PDUs are + /// much smaller, that interleaves more than a few transfers per PDU, or + /// that repeats large hints on every segment can have a valid transfer + /// rejected. A small `link_pdu_size` with a large cap gives a large + /// limit; check the result against the memory budget above. + pub fn for_link_pdu_size(link_pdu_size: usize, max_transfer_size: MaxTransferSize) -> Self { + let segment_data = link_pdu_size + .saturating_sub(SEGMENT_FRAMING) + .clamp(1, MAX_SEGMENT_DATA); + let reference = + u32::try_from(max_transfer_size.get().div_ceil(segment_data)).unwrap_or(u32::MAX); + let limit = reference.saturating_mul(SEGMENT_ALLOWANCE_MULTIPLIER); + Self::new(limit).map_or(MIN_SEGMENT_ALLOWANCE, |l| l.max(MIN_SEGMENT_ALLOWANCE)) + } +} + +config_newtype!(MaxSegments: u32, NonZeroU32); + +/// A validated limit on the state a [`Receiver`] retains across all of its +/// in-progress transfers, in bytes (non-zero). +/// +/// The retained state is what each transfer is charged: the memory its +/// stored segments keep alive, [`SEGMENT_OVERHEAD`] per stored segment, and +/// its retained hints as [`MaxTransferSize`] charges them. A segment copied +/// out of its PDU keeps alive its own length; one kept as a view keeps +/// alive the whole PDU, at most twice its length (see [`Receiver`]), and is +/// charged that. Segments are charged here whether or not a [`MaxSegments`] +/// limit replaces the per-transfer segment charge, so empty segments count +/// against the receiver's total too. An FEC transfer stores no payload +/// while no FEC scheme is implemented, so it is charged only its hints; +/// the number of such transfers is bounded by the window size. +/// +/// A message that would take the total over the limit rejects the transfer +/// it belongs to as [`RejectReason::ReceiverFull`], and the transfer stays +/// closed when space is freed later; transfers already held are kept. A +/// transfer's charge is released when it is delivered, cancelled, +/// rejected, or expired. +/// +/// The configured value is enforced as given. Below `for_transfers(1, ..)` +/// (see [`Self::for_transfers`]), the most one transfer can be charged, a +/// transfer the per-transfer limits admit can be rejected as +/// [`RejectReason::ReceiverFull`] even when nothing else is held; see +/// [`ReceiverConfig`] for sizing. +/// +/// The limit is on charged state, which estimates heap rather than +/// measuring it: allocator rounding adds to each entry (see +/// [`SEGMENT_OVERHEAD`]). +/// +/// Construct via [`MaxRetainedBytes::new`], [`TryFrom`], or +/// [`From`]. +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] +#[repr(transparent)] +#[cfg_attr( + feature = "serde", + derive(serde::Serialize, serde::Deserialize), + serde(try_from = "usize", into = "usize") +)] +pub struct MaxRetainedBytes(NonZeroUsize); + +impl MaxRetainedBytes { + /// What the value configures, as error messages name it. + const NAME: &str = "max retained bytes"; + + /// The smallest limit, one byte. + pub const MIN: Self = Self(NonZeroUsize::MIN); + + /// The largest limit, `usize::MAX` bytes. + pub const MAX: Self = Self(NonZeroUsize::MAX); + + /// Returns the limit for `bytes`, or `None` if it is zero. + pub const fn new(bytes: usize) -> Option { + match NonZeroUsize::new(bytes) { + Some(n) => Some(Self(n)), + None => None, + } + } + + /// Returns the limit as a plain integer. + pub const fn get(self) -> usize { + self.0.get() + } + + /// The limit that holds `transfers` transfers at once however their + /// sender segments them: `transfers` times the most one transfer under + /// `max_transfer_size` and `max_segments` may be charged, saturating at + /// `usize::MAX`. With `transfers` of one it is the default for + /// [`ReceiverConfig::max_retained_bytes`]. + /// + /// A transfer's segments hold at most `max_transfer_size` bytes of data + /// and keep alive at most twice that, the PDUs of segments kept as + /// views. Without a segment limit, one transfer's most is that twice + /// the cap plus the bookkeeping budget its segments and hints share (the + /// cap again, or [`MIN_OVERHEAD_BUDGET`] if the cap is smaller): three + /// times the cap for caps of at least 4 KiB, so 3 GiB for the default + /// 1 GiB cap. With one, it is twice the cap, plus [`SEGMENT_OVERHEAD`] + /// for each segment the limit allows, plus the most hint charge a + /// transfer can retain (about 37 KiB on a 64-bit target, or the + /// bookkeeping budget if that is smaller). + /// + /// This is what a peer that segments as finely as the limits allow can + /// make a transfer cost, not what a typical one does; see + /// [`ReceiverConfig`] for the difference. + pub const fn for_transfers( + transfers: NonZeroUsize, + max_transfer_size: MaxTransferSize, + max_segments: Option, + ) -> Self { + const TWO: NonZeroUsize = NonZeroUsize::new(2).unwrap(); + let cap = max_transfer_size.0; + let budget = overhead_budget(cap.get()); + let bookkeeping = match max_segments { + None => budget, + Some(limit) => { + let hints = if budget < MAX_HINT_CHARGE { + budget + } else { + MAX_HINT_CHARGE + }; + (limit.get() as usize) + .saturating_mul(SEGMENT_OVERHEAD) + .saturating_add(hints) + } + }; + let held = cap.saturating_mul(TWO); + Self(held.saturating_add(bookkeeping).saturating_mul(transfers)) + } +} + +config_newtype!(MaxRetainedBytes: usize, NonZeroUsize); + +/// The bookkeeping budget of one transfer under a cap of `max_transfer_size` +/// bytes: the cap itself, or [`MIN_OVERHEAD_BUDGET`] if the cap is smaller. +const fn overhead_budget(max_transfer_size: usize) -> usize { + if max_transfer_size < MIN_OVERHEAD_BUDGET { + MIN_OVERHEAD_BUDGET + } else { + max_transfer_size + } +} + +/// Configuration for a [`Receiver`]. Every field has a default, so a +/// configuration file may set only what it changes. +/// +/// # Sizing +/// +/// Three fields set the receiver's memory, each from something the +/// operator knows: [`max_transfer_size`](Self::max_transfer_size) from the +/// largest bundle expected, +/// [`max_segments_per_transfer`](Self::max_segments_per_transfer) from the +/// link's PDU size, and [`max_retained_bytes`](Self::max_retained_bytes) +/// from how many transfers should be held at once. Set them in that +/// order, deriving the second with [`MaxSegments::for_link_pdu_size`] and +/// the third with [`MaxRetainedBytes::for_transfers`]. +/// +/// On a lossy link carrying segmented traffic, hold the window. A +/// transfer that lost a segment stays held until the window expires it, so +/// under the default limit of one transfer's worth, a single damaged +/// transfer gets the next ones refused as [`RejectReason::ReceiverFull`] +/// until it expires. `for_transfers` with the window size avoids that, +/// and costs little when the cap is small: 432 kB for a window of 16 and a +/// 9000-byte cap. +/// +/// The bundle size alone does not fix what a transfer costs. Each stored +/// segment is charged [`SEGMENT_OVERHEAD`] on top of the memory it keeps +/// alive, and the sender, not the receiver, decides how many segments a +/// bundle is cut into. A sender that fills PDUs of `P` bytes, each +/// carrying `S = P - 12` data bytes after the framing, is charged at most +/// `ceil((max_transfer_size + 10) / S) * (P + 64)`: every segment keeps its +/// PDU alive, and the 10 bytes allow for the Bundle Length hint this +/// crate's sender puts on the first segment. One that segments as finely +/// as the segment limit allows, by design or not, is charged what +/// `for_transfers` returns for one transfer. For the default 1 GiB cap: +/// +/// | Link PDU | Segment limit | Filled PDUs | Finest segmentation | +/// |----------|---------------|-------------|---------------------| +/// | 1500 bytes | 2,886,404 | 1.05 GiB | 2.17 GiB | +/// | 64 bytes | 82,595,528 | 2.46 GiB | 6.92 GiB | +/// +/// Without a segment limit, the finest segmentation is charged three times +/// the cap, and a link whose PDUs carry fewer than 64 data bytes cannot +/// deliver a cap-sized bundle at all (see [`SEGMENT_OVERHEAD`]). +/// +/// A retention limit at or above the first figure and below the second, +/// times the transfers to be held, admits senders that fill their PDUs and +/// refuses the finest segmentation as [`RejectReason::ReceiverFull`]. The figures are +/// charged state, not heap (see [`MaxRetainedBytes`]). +/// [`Receiver::retained_bytes`] reports what is +/// held, so a CLA can measure real traffic against these bounds. +/// +/// ``` +/// use core::num::NonZeroUsize; +/// +/// use hardy_btpu::receiver::{MaxTransferSize, MaxRetainedBytes, MaxSegments, ReceiverConfig}; +/// +/// let cap = MaxTransferSize::DEFAULT; +/// let segments = MaxSegments::for_link_pdu_size(1500, cap); +/// let transfers = NonZeroUsize::new(2).unwrap(); +/// let config = ReceiverConfig { +/// max_transfer_size: cap, +/// max_segments_per_transfer: Some(segments), +/// max_retained_bytes: Some(MaxRetainedBytes::for_transfers(transfers, cap, Some(segments))), +/// ..ReceiverConfig::default() +/// }; +/// ``` +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)] +#[cfg_attr( + feature = "serde", + derive(serde::Serialize, serde::Deserialize), + serde(default, rename_all = "kebab-case") +)] +pub struct ReceiverConfig { + /// The Section 5 transfer window. It MUST NOT be smaller than the + /// sender's window (Section 5); configuring the same size at both ends + /// is RECOMMENDED. Default: 16. + pub window_size: WindowSize, + /// The per-transfer reassembly cap. Default: 1 GiB. + pub max_transfer_size: MaxTransferSize, + /// The most distinct segments one transfer may hold (see + /// [`MaxSegments`], and [`MaxSegments::for_link_pdu_size`] to derive it + /// from the link). Default: `None`, which charges each segment + /// [`SEGMENT_OVERHEAD`] against the bookkeeping budget instead. + pub max_segments_per_transfer: Option, + /// The most state the receiver retains across all in-progress + /// transfers (see [`MaxRetainedBytes`]). Default: `None`, which + /// enforces [`MaxRetainedBytes::for_transfers`] one transfer at the + /// `max_transfer_size` and `max_segments_per_transfer`, not an unlimited + /// total. That default grows with the segment limit: 3 GiB for the + /// defaults, and the "Finest segmentation" figures of the sizing table + /// above with a limit derived from the link. + /// [`Receiver::max_retained_bytes`] reports the value in effect. + pub max_retained_bytes: Option, + /// Interpret the four provisional FEC message types (see + /// [`DecodeOptions::fec`]). Default: `false`, so they are left to relay + /// as unknown messages. + pub fec: bool, + /// How a segmented transfer's bundle is handed over. Default: + /// [`Delivery::Whole`]. + pub delivery: Delivery, +} + +/// How a [`Receiver`] hands over the bundle of a segmented transfer. +/// +/// Unsegmented bundles (Bundle messages and bare or encapsulated frames) +/// arrive whole and are reported as [`ReceiverEvent::Received`] either way. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)] +#[cfg_attr( + feature = "serde", + derive(serde::Serialize, serde::Deserialize), + serde(rename_all = "kebab-case") +)] +pub enum Delivery { + /// Hold a transfer until every segment has arrived, then report the + /// reassembled bundle as one [`ReceiverEvent::Received`]. + #[default] + Whole, + /// Release a transfer's bytes as soon as they are contiguous from its + /// start: [`ReceiverEvent::TransferStarted`], then + /// [`ReceiverEvent::TransferData`] for each released segment, then + /// [`ReceiverEvent::TransferFinished`]. A released segment is no + /// longer held, so in-order arrival costs the receiver little beyond + /// the segment being released, and the bundle is never copied into + /// one buffer. + /// + /// The receiver cannot push back on the link. Released bytes that the + /// consumer has not yet read are outside the receiver's limits, so a + /// CLA that buffers them bounds the buffer and calls + /// [`Receiver::refuse`] when it is full, rather than blocking the loop + /// that feeds the receiver. + Streamed, +} + +/// Why an otherwise well-formed message was not applied to a transfer. +/// +/// Deliberately exhaustive: values are produced by this crate, never decoded +/// from the wire, and a consumer that acts per-variant should get a compile +/// error when one is added. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum DropReason { + /// The transfer was previously cancelled by the sender (Section 4.2). + Cancelled, + /// The transfer number is outside the current receive window (Section 5). + OutsideWindow, + /// A Cancel referenced a transfer that is not in progress (Section 8.4). + UnknownTransfer, + /// The transfer already completed and its bundle was delivered; a + /// message for it now is a repeat (Section 6) and must not re-open it. + Delivered, + /// The receiver rejected the transfer earlier, for the reason carried, + /// and reported it then as [`ReceiverEvent::TransferRejected`]. + Rejected(RejectReason), + /// The message's segment index contradicts the transfer's established + /// segment sequence (Section 4: segments run 0..=N with exactly one + /// final index): a second End disagreeing with the recorded final index, + /// an End claiming a final index below a segment already seen, or a + /// segment beyond the final index. Applying it would make completion + /// permanently unsatisfiable. + SegmentIndexConflict, + /// The transfer is in progress and already holds a segment at this + /// index: a repeat (Section 6). The first copy is kept, and the + /// repeat's hints are not applied. A repeat after the transfer closed + /// is reported with the reason it closed instead. + Duplicate, + /// The caller gave up on the transfer with [`Receiver::refuse`]. + Refused, +} + +/// Why the receiver rejected an in-progress transfer, reported by +/// [`ReceiverEvent::TransferRejected`] and then carried by +/// [`DropReason::Rejected`] for the transfer's later messages. +/// +/// Deliberately exhaustive, as [`DropReason`] is. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum RejectReason { + /// The transfer's bundle exceeds the [`MaxTransferSize`]: the segment + /// bytes stored, or the sender's Bundle Length hint, are above the cap. + TooLarge, + /// The transfer holds far more segments, or more hint data, than the + /// bundle it could be carrying justifies: it exceeded its + /// [`MaxSegments`] limit, or its bookkeeping ([`SEGMENT_OVERHEAD`] per + /// stored segment when no limit is configured, plus retained hint + /// bytes) exceeds the [`MaxTransferSize`] on its own. + TooFragmented, + /// The transfer mixed core Transfer Segment or End messages with FEC + /// messages, which the FEC extension forbids (Section 3.2 of + /// draft-ietf-dtn-btpu-fec); the receiver treats the transfer as + /// cancelled. + FecCoreMixing, + /// An FEC transfer's FEC configuration changed mid-transfer: a different + /// pre-agreed FEC Instance ID, a different explicit FEC Encoding ID, or a + /// switch between the pre-agreed and explicit forms (Sections 3 and 3.1 + /// of draft-ietf-dtn-btpu-fec). Counting a form switch as a change is + /// this crate's reading: without the instance table the two forms + /// cannot be shown to name the same configuration. Scheme-specific + /// information is not compared. The receiver treats the transfer as + /// cancelled. + FecConfigurationChanged, + /// Every segment of the transfer arrived but none held data. Zero bytes + /// cannot be a valid bundle: an empty Bundle Message is rejected under + /// Section 8.1, and the same policy applies to a reassembled transfer. + Empty, + /// The message would have taken the state the receiver retains across + /// all transfers over its [`MaxRetainedBytes`]. The transfer the + /// message belongs to is the one refused; transfers already held are + /// kept. Like every rejection it closes the transfer for the life of + /// the window, even if space is freed later: its stored segments are + /// discarded, segments sent meanwhile may have been missed, and a + /// re-opened transfer could not complete without them. + ReceiverFull, + /// The message would have taken the [`RetentionBudget`] the receiver + /// shares with others over its limit. Handled as + /// [`Self::ReceiverFull`] is: the transfer is closed and transfers + /// already held are kept. Reported first as `ReceiverFull` when both + /// limits are exceeded. + /// + /// Reject reasons are local diagnostics, for events, logs, and metrics. + /// A protocol that reports to a peer why its transfer was given up must + /// send this and `ReceiverFull` as one reason the peer cannot tell + /// apart, so that the peer cannot probe how full the link is. + BudgetFull, +} + +impl From for DropReason { + fn from(reason: RejectReason) -> Self { + Self::Rejected(reason) + } +} + +/// Events emitted by the receiver for the calling CLA to act on. +/// +/// Deliberately exhaustive, as [`DropReason`] is. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum ReceiverEvent { + /// A complete bundle has been reassembled, or received whole as a Bundle + /// message or an encapsulated bundle. + Received { + /// The bundle bytes. + data: Bytes, + /// The transfer's hint items (Section 7.2), including hint types + /// this implementation does not recognise, so extension metadata (a + /// correlator, say) reaches the caller without an API change. One + /// item per hint type: hints are transfer-scoped and repeatable, + /// and the value from the most recently received message carrying + /// a type supersedes any earlier one, whether across the messages + /// of a transfer or within a single Bundle message. A Bundle + /// Length hint on a Bundle message is omitted (Section 9.1: + /// receivers SHOULD ignore it there). + /// + /// A transfer's Bundle Length hint is passed on as the sender wrote + /// it. It is advisory: the receiver uses it only to reject an + /// oversized transfer early, and it can disagree with `data`, + /// whose length is the authoritative one. + hints: Hints, + }, + + /// Under [`Delivery::Streamed`], a segmented transfer's first bytes are + /// ready: [`Self::TransferData`] events follow, then + /// [`Self::TransferFinished`]. + /// + /// Reported when the transfer's bytes from its start first include + /// data, not when its first message arrives, so a consumer starting a + /// bundle here has bytes to read at once. A transfer that never + /// receives segment 0 is never started. Once started, a transfer ends + /// with exactly one of [`Self::TransferFinished`], + /// [`Self::TransferCancelled`], [`Self::TransferExpired`], or + /// [`Self::TransferRejected`] for the same id, or silently by + /// [`Receiver::reset`]; any but the first means the bytes already + /// released are not a whole bundle. + TransferStarted { + /// The transfer, as the events that follow name it. + id: TransferId, + /// The transfer's hint items so far (see [`Self::Received`]), empty + /// if it has none. The events that follow carry the set again only + /// when it changes. + hints: Hints, + }, + + /// Under [`Delivery::Streamed`], the next segment of a started + /// transfer, in order. Segments holding no data are not reported. + TransferData { + /// The transfer. + id: TransferId, + /// The segment's bytes: a view of the PDU it arrived in, or a copy + /// if it was held and short (see [`Receiver`]). + data: Bytes, + /// The transfer's full hint set if it has changed since the last + /// event for this transfer, else `None`. A consumer starts from + /// the set [`Self::TransferStarted`] carried and keeps the latest + /// `Some`. A change carried by a message that released + /// nothing is reported on the next event. + hints: Option, + }, + + /// Under [`Delivery::Streamed`], the last segment of a started + /// transfer: the bundle is complete. Later messages for the transfer + /// are dropped as [`DropReason::Delivered`]. + TransferFinished { + /// The transfer. + id: TransferId, + /// The final segment's bytes, empty if it held none. + data: Bytes, + /// As for [`Self::TransferData`]. + hints: Option, + }, + + /// A transfer was cancelled by the sender. + /// + /// Also reported for a Cancel of an in-window number the receiver has + /// seen nothing of: Section 5 counts every number in the window as in + /// progress, and Section 8.4 has segments arriving after the Cancel + /// discarded, so the number is remembered as cancelled either way. + TransferCancelled { + /// The cancelled transfer. + id: TransferId, + }, + + /// A transfer was evicted from the window (incomplete). One window + /// advance can evict several; they are reported oldest first. + TransferExpired { + /// The expired transfer. + id: TransferId, + }, + + /// A message was dropped without being applied to any transfer. + /// Informational: the caller decides whether this matters (statistics, + /// logging, or nothing at all). + MessageDropped { + /// The transfer the message named. + transfer_number: u32, + /// The transfer's id, as the receiver's other events for it name + /// it, or `None` if the number is outside the receive window + /// ([`DropReason::OutsideWindow`], [`DropReason::UnknownTransfer`]), + /// where no id is assigned. + id: Option, + /// Why the message was dropped. + reason: DropReason, + }, + + /// An in-progress transfer was rejected and its state discarded; later + /// messages for it are dropped as [`DropReason::Rejected`] with the + /// same reason. Distinct from + /// [`Self::TransferCancelled`], which reports a sender's Transfer Cancel, + /// although a transfer rejected for a protocol violation is one the + /// drafts call cancelled. + TransferRejected { + /// The rejected transfer. + id: TransferId, + /// Why it was rejected. + reason: RejectReason, + }, + + /// An unsegmented Bundle message was rejected by local policy: its + /// content exceeds the configured [`MaxTransferSize`], or it is empty + /// (`len == 0`), which cannot be the valid bundle Section 8.1 requires. + /// The counterpart of [`Self::TransferRejected`] for bundles that never + /// had a transfer number. + BundleRejected { + /// The rejected content length in bytes. + len: usize, + }, + + /// One message could not be decoded and was skipped; processing + /// continued at the next message boundary given by the Section 7 header + /// length (the skip-and-continue rule of Section 7.3). + MalformedMessage { + /// Why the message could not be decoded. + error: Error, + }, + + /// The PDU could not be walked further (no message boundary could be + /// determined, or an encapsulated bundle of unknown extent was reached, + /// Section 7.3) and the remainder was discarded. Always the final event + /// of its PDU. + /// + /// Positions in `error` count from the start of the PDU passed to + /// [`Receiver::receive_pdu`] or [`Receiver::receive_pdu_into`]; a caller + /// that wants to inspect the discarded bytes keeps a clone of that + /// `Bytes` (a reference-count increment, not a copy). + MalformedPdu { + /// Why the walk stopped. + error: Error, + }, +} + +/// Whether a transfer uses core segmentation or FEC, and for FEC the +/// configuration its first message named. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum TransferKind { + Core, + Fec(FecConfig), +} + +/// The part of an FEC transfer's configuration visible without an FEC +/// scheme: which message form it uses and the identifier that form carries. +/// A pre-agreed FEC Instance ID and an explicit FEC Encoding ID name +/// different things, so the two never compare equal. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum FecConfig { + PreAgreed { instance_id: u8 }, + Explicit { encoding_id: u8 }, +} + +/// The content of a Transfer Segment or Transfer End, annotated with what +/// the receiver needs to store it. +struct CoreSegment { + index: u32, + data: Bytes, + /// A Transfer End, whose index is the transfer's final index `N`. + end: bool, + /// The length of the PDU `data` was decoded from, if any, for the copy + /// rule in [`retain_segment`]. + pdu_len: Option, +} + +impl CoreSegment { + /// Whether the segment contradicts the sequence the transfer has + /// already established (Section 4: one transfer is segments `0..=N`). + /// Such a segment would make completion unsatisfiable forever. + fn conflicts(&self, transfer: &InProgressTransfer) -> bool { + if self.end { + // One transfer has exactly one final segment: a second End + // disagreeing with the recorded final index, or an End claiming + // a final index below a segment already seen. A repeated + // identical End is normal repetition and stays idempotent. + transfer + .final_segment_index + .is_some_and(|n| n != self.index) + || transfer + .highest_index + .is_some_and(|highest| highest > self.index) + } else { + // A segment beyond the established final index would leave the + // highest index above N. + transfer.final_segment_index.is_some_and(|n| self.index > n) + } + } + + /// Whether the segment adds nothing to the transfer: its index is + /// already stored or released and, for an End, already recorded as the + /// final index. An End at the index of a stored or released Segment + /// still records `N`. + fn is_duplicate(&self, transfer: &InProgressTransfer) -> bool { + transfer.has_segment(self.index) + && (!self.end || transfer.final_segment_index == Some(self.index)) + } +} + +/// The bookkeeping charge for `segments` segments: [`SEGMENT_OVERHEAD`] +/// each. +fn segment_overhead(segments: usize) -> usize { + segments.saturating_mul(SEGMENT_OVERHEAD) +} + +/// What a transfer's retained hints count against the limits: +/// [`HINT_OVERHEAD`] plus the value length for each item held as +/// [`HintItem::Unknown`]. A well-formed Bundle Length is held inline and +/// charged nothing. +fn hint_charge(hints: &Hints) -> usize { + hints + .unknown_values() + .map(|value| HINT_OVERHEAD + value.len()) + .sum() +} + +/// Detach a hint item's value from the PDU it was decoded from. +/// +/// Hints are retained for the life of a transfer, and an unknown hint's +/// value is a view into its PDU; copying the value (at most 255 bytes) means +/// a retained hint never pins a whole PDU. +fn own_hint(hint: HintItem) -> HintItem { + match hint { + HintItem::Unknown { hint_type, value } => HintItem::Unknown { + hint_type, + value: value.detached(), + }, + other => other, + } +} + +/// Detach a segment from its PDU when keeping the view would pin far more +/// memory than the segment is worth, and return the segment with the +/// memory it keeps alive. +/// +/// A stored segment that is a view into its PDU keeps the whole PDU +/// allocation alive. Segments of at least half the PDU are kept as views +/// and keep alive the PDU, at most twice their length; shorter ones are +/// copied and keep alive only themselves. A message not decoded from a PDU +/// (`pdu_len` is `None`) is stored as given and counted at its own length. +fn retain_segment(data: Bytes, pdu_len: Option) -> (Bytes, usize) { + match pdu_len { + Some(pdu_len) if data.len() < pdu_len.div_ceil(2) => { + let len = data.len(); + (Bytes::copy_from_slice(&data), len) + } + Some(pdu_len) => (data, pdu_len), + None => { + let len = data.len(); + (data, len) + } + } +} + +/// A segment held by its transfer. +struct Stored { + data: Bytes, + /// The memory `data` keeps alive, as [`retain_segment`] reported it. + held: usize, +} + +/// What [`InProgressTransfer::release_next`] found. +enum Release { + /// The next segment in order, removed from the transfer; `last` if its + /// index is the final index. + Segment { data: Bytes, last: bool }, + /// Every segment through the final index was released earlier: the + /// End named the index of a segment already released. + Complete, + /// The next segment has not arrived, or the final index is unknown. + Waiting, +} + +/// What [`InProgressTransfer::next_step`] released. +enum Step { + /// The next segment has not arrived, or the final index is unknown. + Waiting, + /// The transfer ended without a data byte. + Empty, + /// The next contiguous bytes: the transfer's first event if `start` + /// holds its full hint set, its last if `last`, with `hints` if they + /// changed since last taken. + Out { + start: Option, + data: Bytes, + last: bool, + hints: Option, + }, +} + +struct InProgressTransfer { + kind: TransferKind, + /// Segments arrived and not released, by index. + segments: BTreeMap, + final_segment_index: Option, + /// The highest segment index stored or released. + highest_index: Option, + /// Segments released under [`Delivery::Streamed`], which are indices + /// `0..released`. Wider than an index, since releasing index + /// `u32::MAX` makes it 2^32. + released: u64, + /// [`ReceiverEvent::TransferStarted`] has been reported. + started: bool, + hints: Hints, + /// `hints` changed since it was last reported. + hints_changed: bool, + /// Segment bytes received so far, including a segment refused for + /// exceeding the segment limit and segments released: the bundle's + /// length as far as it is known, compared against the + /// [`MaxTransferSize`]. + data_bytes: usize, + /// The memory the stored segments keep alive, the sum of their + /// [`Stored::held`]. + held_bytes: usize, + /// A new segment arrived with the segment limit already reached, and + /// was not stored. + over_segment_limit: bool, +} + +impl InProgressTransfer { + fn new(kind: TransferKind) -> Self { + Self { + kind, + segments: BTreeMap::new(), + final_segment_index: None, + highest_index: None, + released: 0, + started: false, + hints: Hints::new(), + hints_changed: false, + data_bytes: 0, + held_bytes: 0, + over_segment_limit: false, + } + } + + /// Whether the segment at `index` is stored or was released. + fn has_segment(&self, index: u32) -> bool { + u64::from(index) < self.released || self.segments.contains_key(&index) + } + + /// The distinct segments the transfer has had, stored or released. + fn distinct_segments(&self) -> u64 { + // usize is at most 64 bits on every supported target. + self.released + self.segments.len() as u64 + } + + /// Whether `index` is the next segment to release under + /// [`Delivery::Streamed`]. + fn is_next(&self, index: u32) -> bool { + u64::from(index) == self.released + } + + /// Insert a segment unless its index is already stored or released. + /// Repeats are dropped before this; an occupied index here is an End + /// naming a stored or released Segment's index as final, which keeps + /// the first copy. + /// + /// With `retain`, the segment is detached from its PDU per + /// [`retain_segment`] and charged what it keeps alive. Without, it is + /// stored as given and charged nothing, for a segment released before + /// the call that stored it returns. + /// + /// With a `segment_limit`, a new segment that would exceed it is not + /// stored but its bytes are still counted, so [`Self::exceeds`] can + /// prefer [`RejectReason::TooLarge`] when the bundle is over both limits. + fn insert_segment( + &mut self, + index: u32, + data: Bytes, + pdu_len: Option, + retain: bool, + segment_limit: Option, + ) { + if u64::from(index) < self.released { + return; + } + let distinct = self.distinct_segments(); + let Entry::Vacant(e) = self.segments.entry(index) else { + return; + }; + self.data_bytes = self.data_bytes.saturating_add(data.len()); + if segment_limit.is_some_and(|limit| distinct >= limit) { + self.over_segment_limit = true; + return; + } + let (data, held) = if retain { + retain_segment(data, pdu_len) + } else { + (data, 0) + }; + self.held_bytes = self.held_bytes.saturating_add(held); + self.highest_index = self.highest_index.max(Some(index)); + e.insert(Stored { data, held }); + } + + /// Remove the next segment in order, if it has arrived, and count it + /// released. + fn release_next(&mut self) -> Release { + let released = self.released; + if let Some(entry) = self.segments.first_entry() + && u64::from(*entry.key()) == released + { + let (index, stored) = entry.remove_entry(); + self.held_bytes = self.held_bytes.saturating_sub(stored.held); + self.released += 1; + return Release::Segment { + data: stored.data, + last: self.final_segment_index == Some(index), + }; + } + if self + .final_segment_index + .is_some_and(|n| u64::from(n) < self.released) + { + Release::Complete + } else { + Release::Waiting + } + } + + /// Release the next contiguous bytes under [`Delivery::Streamed`], as + /// the event they make. + fn next_step(&mut self) -> Step { + loop { + let (data, last) = match self.release_next() { + Release::Segment { data, last } => (data, last), + Release::Complete => (Bytes::new(), true), + Release::Waiting => return Step::Waiting, + }; + if data.is_empty() && !last { + continue; + } + // The size limits were enforced on every insert. Zero bytes + // cannot be a valid bundle, as for an empty Bundle Message + // (Section 8.1); a transfer with no data was never started. + if last && self.data_bytes == 0 { + return Step::Empty; + } + // The start reports the full set, so no change is pending. + let start = (!replace(&mut self.started, true)).then(|| { + self.hints_changed = false; + self.hints.clone() + }); + return Step::Out { + start, + data, + last, + hints: self.take_hints(), + }; + } + } + + /// The full hint set if it changed since last taken, else `None`. + fn take_hints(&mut self) -> Option { + take(&mut self.hints_changed).then(|| self.hints.clone()) + } + + /// Which draft-ietf-dtn-btpu-fec rule a message of `kind` breaks on + /// this transfer, if any: mixing core and FEC messages (Section 3.2) is + /// [`RejectReason::FecCoreMixing`], and changing the FEC configuration + /// mid-transfer (Sections 3 and 3.1) is + /// [`RejectReason::FecConfigurationChanged`]. Either MUST cancel the + /// transfer. + fn kind_mismatch(&self, kind: TransferKind) -> Option { + match (self.kind, kind) { + (established, kind) if established == kind => None, + (TransferKind::Fec(_), TransferKind::Fec(_)) => { + Some(RejectReason::FecConfigurationChanged) + } + _ => Some(RejectReason::FecCoreMixing), + } + } + + /// Which [`MaxTransferSize`] rule the transfer provably breaks, if any: + /// [`RejectReason::TooLarge`] when the bytes received or the sender's + /// Bundle Length hint exceed `max`, [`RejectReason::TooFragmented`] when a + /// segment was refused by the segment limit or the bookkeeping exceeds + /// its budget. The bookkeeping is the hint charge, plus + /// [`SEGMENT_OVERHEAD`] per distinct segment, stored or released, when + /// there is no `segment_limit`, so that a transfer passes or fails + /// these rules the same way under either [`Delivery`]. + fn exceeds(&self, max: usize, segment_limit: Option) -> Option { + let segments = match segment_limit { + Some(_) => 0, + None => { + segment_overhead(usize::try_from(self.distinct_segments()).unwrap_or(usize::MAX)) + } + }; + if self.data_bytes > max || self.hints.bundle_length().is_some_and(|h| h > max as u64) { + Some(RejectReason::TooLarge) + } else if self.over_segment_limit + || segments.saturating_add(hint_charge(&self.hints)) > overhead_budget(max) + { + Some(RejectReason::TooFragmented) + } else { + None + } + } + + /// What the transfer counts against the receiver's [`MaxRetainedBytes`]: + /// the memory its stored segments keep alive, [`SEGMENT_OVERHEAD`] per + /// stored segment whatever the segment limit, and its hints. Released + /// segments are not held, so not charged. + fn charge(&self) -> usize { + self.held_bytes + .saturating_add(segment_overhead(self.segments.len())) + .saturating_add(hint_charge(&self.hints)) + } + + /// Record hints from a message, keeping the latest value per hint type, + /// and note whether the set changed. + fn apply_hints(&mut self, hints: Vec) { + for hint in hints { + if self.hints.get(hint.hint_type()).as_ref() != Some(&hint) { + self.hints.insert(own_hint(hint)); + self.hints_changed = true; + } + } + } + + /// Check whether all segments 0..=N have been received. + fn is_complete(&self) -> bool { + let Some(n) = self.final_segment_index else { + return false; + }; + // `n + 1` distinct indices, the greatest `n`, can only be 0..=n. + // Counted in u64: `n` is wire-supplied, so `n + 1` overflows u32 + // when a hostile End claims a final index of u32::MAX. + self.distinct_segments() == u64::from(n) + 1 && self.highest_index == Some(n) + } + + /// Concatenate segments in order and return the reassembled bundle. + /// + /// A single-segment transfer hands back its lone [`Bytes`] as stored, + /// with no further copy: a view of its PDU if the segment was at least + /// half the PDU, otherwise the copy [`retain_segment`] made on arrival. + /// Multi-segment reassembly deliberately copies once + /// into a contiguous buffer: the BPA parses bundles from contiguous + /// bytes, and one copy per delivered bundle is cheap relative to the + /// transfer itself. + fn reassemble(&self) -> Bytes { + if let Some((_, only)) = self.segments.first_key_value() + && self.segments.len() == 1 + { + return only.data.clone(); + } + let total = self.segments.values().map(|s| s.data.len()).sum(); + let mut buf = BytesMut::with_capacity(total); + for stored in self.segments.values() { + buf.put_slice(&stored.data); + } + buf.freeze() + } +} + +/// The in-progress transfers and the sum of their charges. +/// +/// The sum equals the transfers' charges by construction: every change to a +/// held transfer goes through [`Self::modify`], which re-charges it, and +/// every removal releases the charge of what it removes. With a +/// [`RetentionBudget`], every change to the sum is passed on to it, so the +/// budget holds `retained - unbudgeted` for this receiver, and dropping the +/// receiver releases that. +#[derive(Default)] +struct Held { + /// Keyed in window order, oldest first, so a window advance expires a + /// leading run of entries and costs only what it expires. + transfers: BTreeMap, + /// The sum of [`InProgressTransfer::charge`] over `transfers`. + retained: usize, + /// Growth in `retained` the budget refused. Non-zero only between the + /// change that grew a transfer and that transfer's rejection, which + /// releases at least as much. + unbudgeted: usize, + #[cfg(target_has_atomic = "ptr")] + budget: Option>, +} + +impl Held { + fn get(&self, id: TransferId) -> Option<&InProgressTransfer> { + self.transfers.get(&id) + } + + fn len(&self) -> usize { + self.transfers.len() + } + + /// The newest transfer's id, if any transfer is held. + fn newest(&self) -> Option { + self.transfers.last_key_value().map(|(&id, _)| id) + } + + /// The transfer at `id`, opened as a new transfer of `kind` if none is + /// held. A new transfer holds nothing, so it is charged nothing. + fn get_or_open(&mut self, id: TransferId, kind: TransferKind) -> &InProgressTransfer { + self.transfers + .entry(id) + .or_insert_with(|| InProgressTransfer::new(kind)) + } + + /// Apply `f` to the transfer at `id`, if one is held, and re-charge it. + fn modify( + &mut self, + id: TransferId, + f: impl FnOnce(&mut InProgressTransfer) -> R, + ) -> Option { + let transfer = self.transfers.get_mut(&id)?; + let before = transfer.charge(); + let result = f(transfer); + let after = transfer.charge(); + if after > before { + self.grow(after - before); + } else { + self.shrink(before - after); + } + Some(result) + } + + /// Remove the transfer at `id`, if one is held, releasing its charge. + fn remove(&mut self, id: TransferId) -> Option { + let transfer = self.transfers.remove(&id)?; + self.shrink(transfer.charge()); + Some(transfer) + } + + /// Remove the oldest transfer if `window` has moved past it, releasing + /// its charge, and return its id. + fn pop_expired(&mut self, window: &TransferWindow) -> Option { + let entry = self + .transfers + .first_entry() + .filter(|entry| window.is_expired(*entry.key()))?; + let (id, transfer) = entry.remove_entry(); + self.shrink(transfer.charge()); + Some(id) + } + + fn clear(&mut self) { + self.transfers.clear(); + self.shrink(self.retained); + } + + /// Add `bytes` to the sum, and to the budget if it has room. + fn grow(&mut self, bytes: usize) { + self.retained = self.retained.saturating_add(bytes); + #[cfg(target_has_atomic = "ptr")] + if let Some(budget) = &self.budget + && !budget.try_add(bytes) + { + self.unbudgeted = self.unbudgeted.saturating_add(bytes); + } + } + + /// Take `bytes` from the sum, and from the budget what it was charged. + fn shrink(&mut self, bytes: usize) { + let bytes = bytes.min(self.retained); + self.retained -= bytes; + let absorbed = bytes.min(self.unbudgeted); + self.unbudgeted -= absorbed; + #[cfg(target_has_atomic = "ptr")] + if let Some(budget) = &self.budget { + budget.release(bytes - absorbed); + } + } + + /// What the budget holds for this receiver. + #[cfg(target_has_atomic = "ptr")] + fn budgeted(&self) -> usize { + self.retained - self.unbudgeted + } + + /// Charge everything held to `budget` instead of the current budget, + /// whatever its limit. + #[cfg(target_has_atomic = "ptr")] + fn set_budget(&mut self, budget: Arc) { + if let Some(old) = self.budget.take() { + old.release(self.budgeted()); + } + budget.add(self.retained); + self.unbudgeted = 0; + self.budget = Some(budget); + } +} + +#[cfg(target_has_atomic = "ptr")] +impl Drop for Held { + fn drop(&mut self) { + if let Some(budget) = &self.budget { + budget.release(self.budgeted()); + } + } +} + +/// Manages inbound PDU processing, transfer window, and segment reassembly. +/// +/// # Memory +/// +/// Each in-progress transfer is bounded by the [`MaxTransferSize`]: its +/// segment bytes may not exceed the cap, and its bookkeeping (segments and +/// hints) has a budget of its own; with a [`MaxSegments`] limit the segment +/// count is limited directly instead. A segment shorter than half its PDU +/// is copied out; a longer one stays a view that keeps alive at most twice +/// its length, and is charged that, provided each PDU arrives in a buffer +/// of its own size (a `Bytes` split from a larger receive buffer pins all +/// of it). So a transfer is charged at most three times the cap. Up to +/// `window_size` transfers can be in progress, charged together at most the +/// [`MaxRetainedBytes`], by default one transfer's full allowance. Charged +/// state estimates retained heap (see [`SEGMENT_OVERHEAD`]); receive +/// buffers, returned events, and allocator overhead are separate. Closed +/// transfer numbers are remembered until the window passes them, so +/// in-progress and closed transfers together number at most the window +/// size. +/// +/// # Sender restarts +/// +/// A restarted sender SHOULD begin from a random transfer number (Section +/// 4). The Figure 2 acceptance test treats a number as new only if it lies +/// within half the number space ahead of the greatest seen, so a random +/// restart lands in the accepted region with probability about one half; +/// otherwise every transfer from the new sender is reported outside the +/// window until its numbers catch up. The draft offers no resynchronisation +/// rule. A CLA that learns of a restart out of band should call +/// [`Self::reset`]. +pub struct Receiver { + state: Reassembly, + /// Held apart from `state` so that [`Self::receive_pdu`] can lend it to + /// the decoder while mutating `state`: the two are disjoint borrows, so + /// the hook is neither shared nor taken out and put back (which a + /// panicking hook would leave undone). A `Box` rather than an `Arc` + /// keeps the crate buildable on targets without pointer-width atomics. + bundle_extent: Option>, +} + +/// A [`Receiver`]'s configuration and reassembly state: everything but the +/// extent hook. +struct Reassembly { + max_transfer_size: MaxTransferSize, + /// [`ReceiverConfig::max_segments_per_transfer`], widened once for the + /// comparison against a segment count. + segment_limit: Option, + /// [`ReceiverConfig::max_retained_bytes`], or its default. + max_retained_bytes: MaxRetainedBytes, + fec: bool, + delivery: Delivery, + window: TransferWindow, + held: Held, + /// In-window transfers that are over, and why: delivered, cancelled by + /// the sender, or rejected by local policy. A message for one of them + /// is a repeat or a straggler and must not re-open it (Section 4.2 for + /// cancelled transfers; the same trap for the rest). Keys are always + /// in-window (pruned by [`Self::expire_old_transfers`]), so the map is + /// bounded by the window size. + closed: BTreeMap, +} + +impl fmt::Debug for Receiver { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + let state = &self.state; + let mut f = f.debug_struct("Receiver"); + f.field("max_transfer_size", &state.max_transfer_size) + .field("segment_limit", &state.segment_limit) + .field("max_retained_bytes", &state.max_retained_bytes) + .field("retained", &state.held.retained) + .field("fec", &state.fec) + .field("delivery", &state.delivery); + #[cfg(target_has_atomic = "ptr")] + f.field("budget", &state.held.budget); + f.field("bundle_extent", &self.bundle_extent.is_some()) + .field("window", &state.window) + .field("transfers", &state.held.len()) + .field("closed", &state.closed.len()) + .finish() + } +} + +impl Receiver { + /// Create a new receiver. + /// + /// Transfers that provably exceed the configured [`MaxTransferSize`] or + /// segment allowance, that break the FEC extension's rules, or that + /// complete with no data are rejected with + /// [`ReceiverEvent::TransferRejected`], and oversized or empty Bundle + /// messages with [`ReceiverEvent::BundleRejected`]. + pub fn new(config: ReceiverConfig) -> Self { + let default_retention = MaxRetainedBytes::for_transfers( + NonZeroUsize::MIN, + config.max_transfer_size, + config.max_segments_per_transfer, + ); + Self { + state: Reassembly { + max_transfer_size: config.max_transfer_size, + segment_limit: config.max_segments_per_transfer.map(|m| u64::from(m.get())), + max_retained_bytes: config.max_retained_bytes.unwrap_or(default_retention), + fec: config.fec, + delivery: config.delivery, + window: TransferWindow::new(config.window_size), + held: Held::default(), + closed: BTreeMap::new(), + }, + bundle_extent: None, + } + } + + /// Supply the caller's way of finding the extent of an encapsulated + /// bundle (see [`BundleExtent`] and [`DecodeOptions::bundle_extent`]). + /// + /// Without one, a bare bundle frame is taken to be the whole PDU, link + /// padding included, and a bundle found after a message ends the PDU + /// with [`ReceiverEvent::MalformedPdu`]; see + /// [`decode_pdu`](crate::codec::decode_pdu) for the padding pitfalls. + pub fn with_bundle_extent( + mut self, + bundle_extent: impl BundleExtent + Send + Sync + 'static, + ) -> Self { + self.bundle_extent = Some(Box::new(bundle_extent)); + self + } + + /// Join `budget`, a limit shared with other receivers (see + /// [`RetentionBudget`]). Anything already held is charged to it, + /// whatever its limit, and released from a budget joined earlier. + #[cfg(target_has_atomic = "ptr")] + pub fn with_budget(mut self, budget: Arc) -> Self { + self.state.held.set_budget(budget); + self + } + + /// Give up on the transfer `id`: discard what it holds and drop its + /// later messages as [`DropReason::Refused`]. For a CLA whose + /// consumer refused the bundle or stopped reading it, or that stopped + /// waiting for it. + /// + /// Returns whether the receiver held state for `id`. An id whose + /// transfer has closed or left the window is ignored, so a stale id + /// cannot close a later transfer that reuses its number. No event is + /// produced. + pub fn refuse(&mut self, id: TransferId) -> bool { + let state = &mut self.state; + if state.held.get(id).is_none() { + return false; + } + state.close(id, DropReason::Refused); + true + } + + /// Discard every in-progress transfer, forget closed transfer numbers, + /// and return the window to its initial state, as if the receiver were + /// newly constructed, except that [`TransferId`]s keep counting, so + /// none issued before the reset names a transfer after it. + /// Configuration and any budget are kept. No events are produced, not + /// even for transfers [`ReceiverEvent::TransferStarted`] reported: the + /// caller knows what it discarded. + pub fn reset(&mut self) { + let state = &mut self.state; + state.window.reset(); + state.held.clear(); + state.closed.clear(); + } + + /// The state held across all in-progress transfers, charged as + /// [`MaxRetainedBytes`] counts it. + /// + /// Read between calls, it lets a CLA export a gauge and compare what + /// its peers cost against the configured limit (see + /// [`ReceiverConfig`] for sizing). + pub fn retained_bytes(&self) -> usize { + self.state.held.retained + } + + /// The [`MaxRetainedBytes`] in effect: the configured value, or the + /// default [`ReceiverConfig::max_retained_bytes`] derives from the other + /// limits. + pub fn max_retained_bytes(&self) -> MaxRetainedBytes { + self.state.max_retained_bytes + } + + /// Process a received convergence layer PDU. Returns zero or more events. + /// + /// Infallible at the PDU level: every framing and semantic fault is + /// expressed as an event ([`ReceiverEvent::MalformedMessage`], + /// [`ReceiverEvent::MalformedPdu`], [`ReceiverEvent::MessageDropped`], + /// ...) alongside whatever the well-formed messages produced, so a + /// fault late in a PDU never discards the events of the prefix before + /// it. + /// + /// Taking `pdu` by value (rather than `&[u8]`) lets the codec extract + /// message data as zero-copy [`Bytes`] views into the original buffer. + /// A `Bytes` made from a fresh `Vec` allocates its shared header the + /// first time it is sliced; with [`Self::receive_pdu_into`] that is the + /// only allocation a PDU of whole Bundle messages costs. + /// + /// The list grows with the number of messages in the PDU, which the + /// peer chooses: most messages can produce an event, and one that + /// advances the window also reports each transfer it expires. A PDU + /// of 8-byte Transfer Cancels for unknown transfers yields one + /// [`ReceiverEvent::MessageDropped`] per 8 bytes, a list several times + /// the PDU's size. [`Self::receive_pdu_into`] fills a list the caller + /// keeps instead, so a CLA reusing one list pays for that growth once + /// rather than on every PDU. + pub fn receive_pdu(&mut self, pdu: Bytes) -> Vec { + let mut events = Vec::new(); + self.receive_pdu_into(pdu, &mut events); + events + } + + /// [`Self::receive_pdu`], writing the events into `events` rather than + /// a new list. + /// + /// `events` is cleared first, so after the call it holds exactly this + /// PDU's events; its capacity is kept, so a list reused across PDUs is + /// reallocated only when a PDU produces more events than any before it. + pub fn receive_pdu_into(&mut self, pdu: Bytes, events: &mut Vec) { + events.clear(); + let pdu_len = pdu.len(); + // The decoder borrows the hook for the whole PDU while `state` is + // mutated; the fields are disjoint, so both borrows hold at once. + let Self { + state, + bundle_extent, + } = self; + let options = DecodeOptions { + fec: state.fec, + bundle_extent: bundle_extent.as_deref().map(|e| e as &dyn BundleExtent), + }; + let mut messages = decode_pdu_with(pdu, options); + while let Some(item) = messages.next() { + match item { + Ok(msg) => state.process_into(msg, Some(pdu_len), events), + Err(error) if messages.is_exhausted() => { + events.push(ReceiverEvent::MalformedPdu { error }); + } + Err(error) => { + events.push(ReceiverEvent::MalformedMessage { error }); + } + } + } + } + + /// Process a single decoded message. + /// + /// A message given here is not associated with a PDU, so its segment + /// data is stored as supplied rather than copied (see [`Receiver`] for + /// the copy rule `receive_pdu` applies). + pub fn process_message(&mut self, message: Message) -> Vec { + let mut events = Vec::new(); + self.state.process_into(message, None, &mut events); + events + } +} + +impl Reassembly { + /// Process one message, appending its events to `events`. `pdu_len` is + /// the length of the PDU the message was decoded from, if any. + fn process_into( + &mut self, + message: Message, + pdu_len: Option, + events: &mut Vec, + ) { + match message { + Message::DefinitePadding { .. } | Message::Unknown { .. } => {} + + Message::Bundle { data, hints } => { + // Section 8.1: the content MUST be a valid bundle, which zero + // bytes cannot be; and the size cap applies unstored. + if data.is_empty() || data.len() > self.max_transfer_size.get() { + events.push(ReceiverEvent::BundleRejected { len: data.len() }); + return; + } + // Section 9.1: a Bundle Length hint is only meaningful on + // Transfer Segment and End messages and SHOULD be ignored + // elsewhere. + let mut hints = Hints::from(hints); + hints.remove(HintType::BUNDLE_LENGTH); + events.push(ReceiverEvent::Received { data, hints }); + } + + Message::TransferSegment(m) => self.process_segment(m, false, pdu_len, events), + Message::TransferEnd(m) => self.process_segment(m, true, pdu_len, events), + + Message::TransferCancel { transfer_number } => { + self.process_transfer_cancel(transfer_number, events) + } + + // FEC messages are tracked but not decoded (no FEC scheme is + // implemented). They are stored with their FecConfig to detect + // mixing and configuration changes. + Message::PreAgreedFecSource(m) | Message::PreAgreedFecRepair(m) => self + .process_transfer_message( + m.transfer_number, + TransferKind::Fec(FecConfig::PreAgreed { + instance_id: m.fec_instance_id, + }), + m.hints, + None, + events, + ), + Message::ExplicitFecSource(m) | Message::ExplicitFecRepair(m) => self + .process_transfer_message( + m.transfer_number, + TransferKind::Fec(FecConfig::Explicit { + encoding_id: m.fec_encoding_id, + }), + m.hints, + None, + events, + ), + } + } + + /// [`Self::process_transfer_message`] for a Transfer Segment, or for a + /// Transfer End if `end`. + fn process_segment( + &mut self, + m: TransferSegmentMessage, + end: bool, + pdu_len: Option, + events: &mut Vec, + ) { + let segment = CoreSegment { + index: m.segment_index, + data: m.data, + end, + pdu_len, + }; + self.process_transfer_message( + m.transfer_number, + TransferKind::Core, + m.hints, + Some(segment), + events, + ); + } + + /// Admission check: is `transfer_number` eligible for processing at all? + /// It must be inside the receive window and not already closed + /// (delivered, cancelled, or rejected). + /// + /// Returns the transfer's id, or `None` once the message is reported + /// dropped. A number that advances the window expires the transfers it + /// moves past, and their [`ReceiverEvent::TransferExpired`] events are + /// pushed onto `events` whatever the outcome. + fn admit( + &mut self, + transfer_number: u32, + events: &mut Vec, + ) -> Option { + let id = match self.window.admit(transfer_number) { + Admission::OutsideWindow => { + Self::drop_message(transfer_number, None, DropReason::OutsideWindow, events); + return None; + } + Admission::New(id) => { + self.expire_old_transfers(events); + id + } + Admission::InProgress(id) => id, + }; + + // A repeated message for a closed transfer MUST NOT re-open it + // (Section 4.2 for cancelled; the same trap applies to delivered + // and locally rejected transfers). Checked after the window (the + // traffic is still window-relevant) but before any transfer entry + // is inserted. + match self.closed.get(&id) { + Some(&reason) => { + Self::drop_message(transfer_number, Some(id), reason, events); + None + } + None => Some(id), + } + } + + /// Which limit, if any, a state change has taken the transfer at `id` + /// or the receiver over: the transfer's [`MaxTransferSize`] rules (see + /// [`InProgressTransfer::exceeds`]), then the receiver's + /// [`MaxRetainedBytes`] as [`RejectReason::ReceiverFull`], then the + /// shared budget as [`RejectReason::BudgetFull`]. + /// + /// The receiver's total was within its limits before the change, so the + /// transfer that grew is the one to reject. + fn over_limit(&self, id: TransferId) -> Option { + self.held + .get(id) + .and_then(|t| t.exceeds(self.max_transfer_size.get(), self.segment_limit)) + .or_else(|| { + (self.held.retained > self.max_retained_bytes.get()) + .then_some(RejectReason::ReceiverFull) + }) + .or_else(|| (self.held.unbudgeted > 0).then_some(RejectReason::BudgetFull)) + } + + /// Report a message that is dropped with no state touched. Drops are + /// expected traffic (repetition, reordering, a moved window), not + /// faults, so they surface as [`ReceiverEvent::MessageDropped`]. + fn drop_message( + transfer_number: u32, + id: Option, + reason: DropReason, + events: &mut Vec, + ) { + events.push(ReceiverEvent::MessageDropped { + transfer_number, + id, + reason, + }); + } + + /// [`Self::close`] the transfer at `id` as rejected for `reason`, and + /// report it. + fn reject_transfer( + &mut self, + id: TransferId, + reason: RejectReason, + events: &mut Vec, + ) { + self.close(id, reason.into()); + events.push(ReceiverEvent::TransferRejected { id, reason }); + } + + /// Shared pipeline for every message that opens or extends a transfer + /// (Segment, End, and the four FEC messages): admission, transfer-kind + /// check, sequence-conflict check, state application, oversize gate, + /// completion check. + /// + /// `kind` is what this message would make a new transfer. `segment` is + /// the content of a Segment or End, and `None` for an FEC message, + /// whose payload is not stored while no FEC scheme is implemented; its + /// hints are still kept, and the Bundle Length hint still policed. + fn process_transfer_message( + &mut self, + transfer_number: u32, + kind: TransferKind, + hints: Vec, + segment: Option, + events: &mut Vec, + ) { + let Some(id) = self.admit(transfer_number, events) else { + return; + }; + + let transfer = self.held.get_or_open(id, kind); + + if let Some(reason) = transfer.kind_mismatch(kind) { + return self.reject_transfer(id, reason, events); + } + + // A conflicting or duplicate message is dropped with no state + // touched, hints included. + if segment.as_ref().is_some_and(|s| s.conflicts(transfer)) { + return Self::drop_message( + transfer_number, + Some(id), + DropReason::SegmentIndexConflict, + events, + ); + } + if segment.as_ref().is_some_and(|s| s.is_duplicate(transfer)) { + return Self::drop_message(transfer_number, Some(id), DropReason::Duplicate, events); + } + + let segment_limit = self.segment_limit; + let streamed = self.delivery == Delivery::Streamed; + self.held.modify(id, |transfer| { + transfer.apply_hints(hints); + if let Some(CoreSegment { + index, + data, + end, + pdu_len, + }) = segment + { + if end { + transfer.final_segment_index = Some(index); + } + // An empty segment or End is still stored: Section 4 + // completes a transfer once indices 0..=N are present, and a + // streaming sender may have no other way to finish. The + // segment charge bounds a flood of them. The next segment + // of a streamed transfer is released below, before this + // call returns, so it is not copied; until then it counts + // only its bookkeeping charge against the limits. + let retain = !(streamed && transfer.is_next(index)); + transfer.insert_segment(index, data, pdu_len, retain, segment_limit); + } + }); + + if let Some(reason) = self.over_limit(id) { + return self.reject_transfer(id, reason, events); + } + + // A late segment may fill the last gap after the End arrived. + if streamed { + self.release_prefix(id, events); + } else { + self.complete_if_ready(id, events); + } + } + + /// Under [`Delivery::Streamed`], release the transfer's segments that + /// are now contiguous from its start, reporting + /// [`ReceiverEvent::TransferStarted`] before the first data and + /// [`ReceiverEvent::TransferFinished`] for the last segment, which + /// closes the transfer. A no-op if the next segment has not arrived, + /// as for an FEC transfer, whose payload is not stored. + /// + /// The release path takes the next contiguous bytes from + /// [`InProgressTransfer::release_next`]; an FEC scheme would feed it + /// decoded source bytes the same way. + fn release_prefix(&mut self, id: TransferId, events: &mut Vec) { + loop { + let (start, data, last, hints) = + match self.held.modify(id, InProgressTransfer::next_step) { + Some(Step::Out { + start, + data, + last, + hints, + }) => (start, data, last, hints), + Some(Step::Empty) => { + return self.reject_transfer(id, RejectReason::Empty, events); + } + Some(Step::Waiting) | None => return, + }; + if let Some(hints) = start { + events.push(ReceiverEvent::TransferStarted { id, hints }); + } + if last { + // Closed rather than forgotten, as for a whole delivery. + self.close(id, DropReason::Delivered); + events.push(ReceiverEvent::TransferFinished { id, data, hints }); + return; + } + events.push(ReceiverEvent::TransferData { id, data, hints }); + } + } + + /// If the transfer's segments are all present (and its final index is + /// known), reassemble it, remove it from the window, and push a + /// `Received` event. A no-op otherwise. Called after every segment + /// or End insert so out-of-order completion is detected regardless of which + /// message arrives last. + fn complete_if_ready(&mut self, id: TransferId, events: &mut Vec) { + let Some(transfer) = self.held.get(id) else { + return; + }; + if !transfer.is_complete() { + return; + } + // The size limits were enforced on every insert. Zero bytes cannot + // be a valid bundle, as for an empty Bundle Message (Section 8.1). + if transfer.data_bytes == 0 { + return self.reject_transfer(id, RejectReason::Empty, events); + } + // Reassembled while the segments are still charged, so the copy is + // never made against budget accounted free. + let data = transfer.reassemble(); + // Closed rather than forgotten: a sender may repeat any message + // (Section 6), and a repeat must not deliver the bundle again. + let hints = self.close(id, DropReason::Delivered); + events.push(ReceiverEvent::Received { data, hints }); + } + + fn process_transfer_cancel(&mut self, transfer_number: u32, events: &mut Vec) { + // Section 8.4 ignores a Cancel for a transfer not in progress, and + // Section 5 defines in progress by the window alone. So a Cancel + // never advances the window, and one inside it is recorded even + // before any segment arrives, so later segments are discarded. + let Some(id) = self.window.id(transfer_number) else { + return Self::drop_message(transfer_number, None, DropReason::UnknownTransfer, events); + }; + + // A repeated Cancel of a transfer already closed is idempotent and + // reported with the original reason. + if let Some(&reason) = self.closed.get(&id) { + return Self::drop_message(transfer_number, Some(id), reason, events); + } + + // A closed transfer is never also in progress, so this discards the + // transfer's segments if any have arrived, and otherwise records + // the Cancel ahead of them. + self.close(id, DropReason::Cancelled); + events.push(ReceiverEvent::TransferCancelled { id }); + } + + /// Drop every transfer, live or closed, that the window has moved past, + /// reporting the live ones as [`ReceiverEvent::TransferExpired`] oldest + /// first. Both maps are keyed in window order, so the expired entries + /// are a leading run and the cost is O(log n) per entry expired, not a + /// walk of the window. + fn expire_old_transfers(&mut self, events: &mut Vec) { + debug_assert!( + self.held + .newest() + .is_none_or(|k| self.window.is_behind_greatest(k)) + && self + .closed + .last_key_value() + .is_none_or(|(&k, _)| self.window.is_behind_greatest(k)), + "a held transfer is ahead of the window" + ); + while let Some(id) = self.held.pop_expired(&self.window) { + events.push(ReceiverEvent::TransferExpired { id }); + } + + // Prune the closed map the same way; this is what keeps it bounded + // by the window size. No events: these were already reported as + // Received / TransferCancelled / TransferRejected when they + // closed. + while let Some(entry) = self.closed.first_entry() + && self.window.is_expired(*entry.key()) + { + entry.remove(); + } + + // Every id in either map is in the window, and no id is in both. + debug_assert!( + self.held.len() + self.closed.len() <= usize::from(self.window.window_size().get()), + "more transfers remembered than the window holds" + ); + } + + /// Remember the transfer at `id` as closed with `reason`, so later + /// messages for it are dropped rather than re-opening it, and remove its + /// in-progress state, if any, releasing its charge. Returns the + /// transfer's hints, empty if it had no state. + fn close(&mut self, id: TransferId, reason: DropReason) -> Hints { + self.closed.insert(id, reason); + self.held + .remove(id) + .map(|transfer| transfer.hints) + .unwrap_or_default() + } + + /// The in-progress transfer numbered `transfer_number`, looked up + /// through the window as production code does. + #[cfg(test)] + fn transfer(&self, transfer_number: u32) -> Option<&InProgressTransfer> { + self.held.get(self.window.id(transfer_number)?) + } + + /// Why the transfer numbered `transfer_number` closed, if it has. + #[cfg(test)] + fn closed_reason(&self, transfer_number: u32) -> Option<&DropReason> { + self.closed.get(&self.window.id(transfer_number)?) + } +} + +#[cfg(test)] +mod tests { + use alloc::vec; + + use super::*; + use crate::{codec::encode_message, fec::PreAgreedFecMessage}; + + fn receiver(window_size: u16, max_transfer_size: usize) -> Receiver { + Receiver::new(ReceiverConfig { + window_size: WindowSize::try_from(window_size).unwrap(), + max_transfer_size: MaxTransferSize::try_from(max_transfer_size).unwrap(), + max_segments_per_transfer: None, + max_retained_bytes: None, + fec: false, + delivery: Delivery::Whole, + }) + } + + fn segment(transfer_number: u32, segment_index: u32, data: &'static [u8]) -> Message { + Message::TransferSegment(TransferSegmentMessage { + transfer_number, + segment_index, + hints: vec![], + data: Bytes::from_static(data), + }) + } + + fn end(transfer_number: u32, segment_index: u32, data: &'static [u8]) -> Message { + Message::TransferEnd(TransferSegmentMessage { + transfer_number, + segment_index, + hints: vec![], + data: Bytes::from_static(data), + }) + } + + #[test] + fn closed_map_pruned_by_window_advance() { + let mut r = receiver(4, usize::MAX); + r.process_message(segment(0, 0, b"x")); + r.process_message(Message::TransferCancel { transfer_number: 0 }); + assert_eq!(r.state.closed_reason(0), Some(&DropReason::Cancelled)); + + // Advance the window until 0 falls out of it. + for t in 1..=4u32 { + r.process_message(segment(t, 0, b"x")); + } + assert!(r.state.closed.is_empty()); + + // A really late segment for 0 is now dropped as out-of-window. + let events = r.process_message(segment(0, 0, b"x")); + assert_eq!( + events, + vec![ReceiverEvent::MessageDropped { + transfer_number: 0, + id: None, + reason: DropReason::OutsideWindow, + }] + ); + assert!(r.state.transfer(0).is_none()); + } + + #[test] + fn closed_map_pruned_across_the_wrap() { + let mut r = receiver(4, usize::MAX); + for t in [u32::MAX - 1, u32::MAX, 0] { + r.process_message(end(t, 0, b"x")); + } + assert_eq!(r.state.closed.len(), 3); + + // 1 is new and leaves MAX - 1 three behind the next number; 4 moves + // the window past everything delivered. + r.process_message(segment(1, 0, b"x")); + assert_eq!(r.state.closed.len(), 3); + r.process_message(segment(4, 0, b"x")); + assert!(r.state.closed.is_empty()); + } + + fn unknown(hint_type: u8, value: &'static [u8]) -> HintItem { + HintItem::Unknown { + hint_type: HintType::new(hint_type).unwrap(), + value: HintValue::new(Bytes::from_static(value)).unwrap(), + } + } + + fn segment_with( + transfer_number: u32, + segment_index: u32, + hints: Vec, + data: &'static [u8], + ) -> Message { + Message::TransferSegment(TransferSegmentMessage { + transfer_number, + segment_index, + hints, + data: Bytes::from_static(data), + }) + } + + #[test] + fn segments_and_hints_are_charged_to_overhead_not_data() { + let mut r = Receiver::new(ReceiverConfig::default()); + r.process_message(segment(0, 0, b"a")); + r.process_message(segment(0, 1, b"")); + r.process_message(segment_with( + 0, + 2, + vec![HintItem::BundleLength(3), unknown(0x41, b"corr")], + b"b", + )); + let t = r.state.transfer(0).unwrap(); + assert_eq!(t.segments.len(), 3); + assert_eq!(t.data_bytes, 2); + assert_eq!(t.held_bytes, 2); + // The Bundle Length is held inline and charged nothing. + assert_eq!(t.hints.bundle_length(), Some(3)); + assert_eq!(hint_charge(&t.hints), HINT_OVERHEAD + 4); + + // Superseding a hint releases the old value's charge. + r.process_message(segment_with(0, 3, vec![unknown(0x41, b"c")], b"c")); + assert_eq!( + hint_charge(&r.state.transfer(0).unwrap().hints), + HINT_OVERHEAD + 1 + ); + + // And a larger value raises it again. + r.process_message(segment_with(0, 4, vec![unknown(0x41, b"longer")], b"d")); + let t = r.state.transfer(0).unwrap(); + assert_eq!(hint_charge(&t.hints), HINT_OVERHEAD + 6); + assert_eq!( + r.retained_bytes(), + t.held_bytes + 5 * SEGMENT_OVERHEAD + hint_charge(&t.hints) + ); + } + + #[test] + fn malformed_bundle_length_is_charged_and_bundle_length_is_not() { + let hints = Hints::from(vec![HintItem::BundleLength(10), unknown(0, b"abc")]); + assert_eq!(hint_charge(&hints), HINT_OVERHEAD + 3); + let hints = Hints::from(vec![unknown(0, b"abc"), HintItem::BundleLength(10)]); + assert_eq!(hint_charge(&hints), 0); + } + + #[test] + fn retained_segment_is_charged_what_it_keeps_alive() { + let pdu = Bytes::from(vec![0; 100]); + // At least half the PDU: kept as a view, charged the whole PDU. + let (data, held) = retain_segment(pdu.slice(..50), Some(pdu.len())); + assert_eq!(data.as_ptr(), pdu.as_ptr()); + assert_eq!(held, 100); + // Shorter: copied, charged its own length. + let (data, held) = retain_segment(pdu.slice(..49), Some(pdu.len())); + assert_ne!(data.as_ptr(), pdu.as_ptr()); + assert_eq!(held, 49); + // An odd PDU length rounds the half up, so a view never keeps alive + // more than twice its length. + let (data, held) = retain_segment(pdu.slice(..50), Some(101)); + assert_ne!(data.as_ptr(), pdu.as_ptr()); + assert_eq!(held, 50); + // Not from a PDU: stored as given. + let (data, held) = retain_segment(pdu.slice(..10), None); + assert_eq!(data.as_ptr(), pdu.as_ptr()); + assert_eq!(held, 10); + } + + #[test] + fn duplicate_segment_is_not_charged_twice() { + let mut r = Receiver::new(ReceiverConfig::default()); + r.process_message(segment(0, 0, b"abc")); + r.process_message(segment(0, 0, b"abc")); + let t = r.state.transfer(0).unwrap(); + assert_eq!(t.data_bytes, 3); + assert_eq!(t.held_bytes, 3); + assert_eq!(r.retained_bytes(), 3 + SEGMENT_OVERHEAD); + } + + #[test] + fn segment_over_the_limit_is_counted_but_never_stored() { + let mut t = InProgressTransfer::new(TransferKind::Core); + t.insert_segment(0, Bytes::from_static(b"ab"), None, true, Some(1)); + t.insert_segment(0, Bytes::from_static(b"ab"), None, true, Some(1)); + assert!(!t.over_segment_limit, "a duplicate is not a new segment"); + + t.insert_segment(1, Bytes::from_static(b"cde"), None, true, Some(1)); + assert!(t.over_segment_limit); + assert_eq!(t.segments.len(), 1); + assert_eq!(t.data_bytes, 5); + assert_eq!(t.held_bytes, 2); + } + + #[test] + fn reset_clears_all_state() { + let mut r = receiver(4, usize::MAX); + r.process_message(segment(7, 0, b"x")); + r.process_message(segment(8, 0, b"x")); + r.process_message(Message::TransferCancel { transfer_number: 8 }); + assert_eq!(r.state.window.greatest(), Some(8)); + + r.reset(); + assert_eq!(r.state.held.len(), 0); + assert_eq!(r.retained_bytes(), 0); + assert!(r.state.closed.is_empty()); + assert_eq!(r.state.window.greatest(), None); + } + + /// A xorshift generator for the randomised ledger test. Not + /// cryptographic, and need not be: it picks messages, not secrets, and + /// a fixed seed keeps the test deterministic. + struct XorShift(u64); + + impl XorShift { + fn below(&mut self, n: u64) -> u64 { + self.0 ^= self.0 << 13; + self.0 ^= self.0 >> 7; + self.0 ^= self.0 << 17; + self.0 % n + } + } + + /// Feeds 20,000 random messages to `r`, checking after each that its + /// retained total is the sum of its transfers' charges, and that a + /// budget it has joined holds exactly that total. Returns how many + /// transfers the budget refused. + #[cfg(target_has_atomic = "ptr")] + fn check_ledger(mut r: Receiver, budget: &RetentionBudget) -> usize { + let mut budget_full = 0; + static DATA: [u8; 64] = [0xAB; 64]; + let mut rng = XorShift(0x9E37_79B9_7F4A_7C15); + for step in 0..20_000 { + // The numbers drift upward, so the window keeps advancing and + // transfers keep opening rather than all closing for good. + let transfer_number = step / 64 + rng.below(6) as u32; + let hints = match rng.below(4) { + 0 => vec![HintItem::BundleLength(rng.below(400))], + 1 => vec![unknown(rng.below(3) as u8, &DATA[..rng.below(9) as usize])], + _ => vec![], + }; + let m = TransferSegmentMessage { + transfer_number, + segment_index: rng.below(6) as u32, + hints, + data: Bytes::from_static(&DATA[..rng.below(65) as usize]), + }; + let message = match rng.below(9) { + 0..=3 => Message::TransferSegment(m), + 4 | 5 => Message::TransferEnd(m), + 6 => Message::TransferCancel { transfer_number }, + _ => Message::PreAgreedFecSource(PreAgreedFecMessage { + transfer_number, + fec_instance_id: rng.below(2) as u8, + hints: m.hints, + payload: m.data, + }), + }; + // Half the messages arrive in a PDU of their own, so segments + // are both kept as views and copied. + let events = if rng.below(2) == 0 { + let mut pdu = BytesMut::new(); + encode_message(&message, &mut pdu).unwrap(); + r.receive_pdu(pdu.freeze()) + } else { + r.process_message(message) + }; + budget_full += events + .iter() + .filter(|e| { + matches!( + e, + ReceiverEvent::TransferRejected { + reason: RejectReason::BudgetFull, + .. + } + ) + }) + .count(); + let held = &r.state.held; + assert_eq!( + held.retained, + held.transfers + .values() + .map(InProgressTransfer::charge) + .sum::(), + "after step {step}" + ); + assert_eq!(held.unbudgeted, 0, "after step {step}"); + assert_eq!(budget.used(), held.retained, "after step {step}"); + } + drop(r); + assert_eq!(budget.used(), 0, "dropping the receiver releases its share"); + budget_full + } + + #[cfg(target_has_atomic = "ptr")] + fn ledger_config(delivery: Delivery) -> ReceiverConfig { + ReceiverConfig { + window_size: WindowSize::MIN, + max_transfer_size: MaxTransferSize::try_from(256).unwrap(), + max_segments_per_transfer: None, + max_retained_bytes: MaxRetainedBytes::new(2048), + fec: true, + delivery, + } + } + + #[cfg(target_has_atomic = "ptr")] + #[test] + fn retained_always_equals_the_sum_of_held_charges() { + for delivery in [Delivery::Whole, Delivery::Streamed] { + let budget = Arc::new(RetentionBudget::new(MaxRetainedBytes::MAX)); + let r = Receiver::new(ledger_config(delivery)).with_budget(Arc::clone(&budget)); + assert_eq!(check_ledger(r, &budget), 0, "{delivery:?}"); + } + } + + #[cfg(target_has_atomic = "ptr")] + #[test] + fn a_budget_below_the_receiver_limit_tracks_it_exactly() { + for delivery in [Delivery::Whole, Delivery::Streamed] { + let budget = Arc::new(RetentionBudget::new(MaxRetainedBytes::new(300).unwrap())); + let r = Receiver::new(ledger_config(delivery)).with_budget(Arc::clone(&budget)); + assert_ne!( + check_ledger(r, &budget), + 0, + "{delivery:?}: the budget refused nothing" + ); + } + } +} diff --git a/btpu/src/sender.rs b/btpu/src/sender.rs new file mode 100644 index 000000000..ffa863b9f --- /dev/null +++ b/btpu/src/sender.rs @@ -0,0 +1,2228 @@ +//! The sending end: queues bundles, allocates their transfer numbers within +//! the Section 5 window, and packs their messages into PDUs. + +use alloc::{collections::VecDeque, vec::Vec}; +#[cfg(feature = "tower")] +use core::task::Waker; +use core::{fmt, num::NonZeroUsize, ops::Deref, slice}; + +use bytes::{Buf, Bytes, BytesMut}; +#[cfg(feature = "rand")] +use rand_core::{Rng, TryRng}; +use smallvec::SmallVec; + +use crate::{ + codec::{ + encode_message, encode_segment_head, encoded_message_len, + header::{HEADER_SIZE, MAX_CONTENT_LENGTH}, + hint::{HintItem, HintType, Hints}, + message::{FrameKind, Message, SEGMENT_FRAMING, frame_kind}, + pad_pdu, segment_message_len, + }, + // Aliased: this module has its own `Error`. + transfer::{Error as TransferError, TransferNumberAllocator, WindowSize}, +}; + +/// Shorthand for results whose error is [`enum@Error`] unless stated. +pub type Result = core::result::Result; + +/// Errors from queuing bundles for transmission. +#[derive(Debug, Clone, PartialEq, Eq, thiserror::Error)] +pub enum Error { + /// The bundle is empty. A Bundle Message's content MUST be a valid + /// bundle (Section 8.1), which an empty payload can never be; rejecting + /// it here keeps the sender from emitting the zero-content Bundle + /// Message the draft says SHOULD NOT be used. + #[error("Empty data")] + Empty, + + /// No transfer window slot is available. + #[error(transparent)] + Window(#[from] TransferError), + + /// The configured PDU is too small to carry even one byte of a segment + /// of this bundle alongside its headers and hints. The likeliest cause + /// is a large caller-supplied hint chain. + /// + /// The first segment always carries the derived Bundle Length hint, + /// whose value takes 1, 2, 4, or 8 bytes by the bundle's length, so + /// with no caller hints the least PDU that can segment a bundle is 16, + /// 17, 19, or 23 bytes, for bundles of up to 255 bytes, 64 KiB, 4 GiB, + /// and beyond. + #[error( + "PDU size {pdu_size} cannot carry a segment: {required} bytes needed for its framing and one data byte" + )] + PduTooSmall { + /// The least PDU size that would carry the first segment: its + /// framing and hints, plus one data byte. + required: usize, + /// The configured PDU size. + pdu_size: usize, + }, + + /// The bundle would need more segments than the 32-bit segment index + /// can number at this PDU size (Section 8.2). + /// + /// The count allows for segments cut short to fill the tail of a PDU, + /// which carry at least half of a full segment, so a bundle is refused + /// once twice its full-size segment count would not fit. + /// + /// Theoretical in practice: a bundle over 4 GiB needs a PDU of at least + /// 23 bytes to segment (see [`Self::PduTooSmall`]), whose later + /// segments carry at least 11 bytes each, or 6 when cut short, so only + /// a bundle of more than 24 GiB in memory can reach it, and none can on + /// a 32-bit target. + #[error("Bundle of {len} bytes needs more than 2^32 segments at PDU size {pdu_size}")] + TooManySegments { + /// The bundle's length in bytes. + len: usize, + /// The configured PDU size. + pdu_size: usize, + }, + + /// A [`Sender::push`] would take the bundle past the `total_len` given + /// to [`Sender::begin`]. Nothing is pushed. + #[error("Pushing {chunk} bytes would overrun the bundle: {pushed} of {total_len} bytes pushed")] + Overrun { + /// The bundle's length, as given to [`Sender::begin`]. + total_len: usize, + /// The bytes pushed before the refused chunk. + pushed: usize, + /// The length of the refused chunk. + chunk: usize, + }, + + /// [`Sender::finish`] was called before the bundle's last byte was + /// pushed. The bundle is cancelled, as [`Sender::cancel`] would. + #[error("Bundle finished short: {pushed} of {total_len} bytes pushed")] + Underrun { + /// The bundle's length, as given to [`Sender::begin`]. + total_len: usize, + /// The bytes pushed. + pushed: usize, + }, + + /// The handle names no bundle in progress: the bundle was cancelled by + /// its ID, or the handle came from another sender. + #[error("No bundle in progress for this send handle")] + NotInProgress, + + /// The first chunk of a bundle [`Sender::begin`] chose to send as a + /// bare bundle frame ([`BundleFraming::Bare`]) does not start with a + /// bundle-reserved byte, so a receiver would not take it for a bundle. + /// Nothing is pushed. `begin` cannot see the first byte, so it + /// assumes a bundle; [`Sender::enqueue`] can, and frames such data as + /// a Bundle Message instead. + #[error("Bare bundle frame does not start with a bundle-reserved byte")] + NotABundle, +} + +/// A validated convergence layer PDU size in bytes +/// ([`PduSize::MIN`]..=[`PduSize::MAX`]). +/// +/// Construct via [`PduSize::new`] or [`TryFrom`], which enforce the +/// bounds at the edge; every consumer of a `PduSize` can then rely on them. +/// Together they guarantee that anything [`Sender::enqueue`] accepts can +/// eventually be drained by [`Sender::next_pdu`]: no queued message is ever +/// larger than a PDU. +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] +#[repr(transparent)] +#[cfg_attr( + feature = "serde", + derive(serde::Serialize, serde::Deserialize), + serde(try_from = "usize", into = "usize") +)] +pub struct PduSize(NonZeroUsize); + +impl PduSize { + /// What the value configures, as error messages name it. + const NAME: &str = "PDU size"; + + /// Minimum supported convergence layer PDU size: one message header. + /// + /// Below this, `enqueue` could accept a message (the header alone is + /// [`HEADER_SIZE`] bytes) that no PDU can ever carry, and `next_pdu` + /// would emit pure padding forever without draining it. + /// + /// The minimum only keeps the sender live; a usable PDU is larger (see + /// [`Error::PduTooSmall`] for what segmenting needs). + pub const MIN: Self = Self(NonZeroUsize::new(HEADER_SIZE).unwrap()); + + /// Maximum supported convergence layer PDU size. + /// + /// A PDU of this size can be exactly filled by a single message carrying + /// the maximum 20-bit content length. Any message or padding content the + /// sender derives from a `PduSize` is guaranteed encodable; beyond this + /// bound, segment capacities and padding lengths would overflow the + /// 20-bit length field. + pub const MAX: Self = Self(NonZeroUsize::new(HEADER_SIZE + MAX_CONTENT_LENGTH).unwrap()); + + /// A common Ethernet-MTU-sized PDU (1500 bytes). + pub const DEFAULT: Self = Self(NonZeroUsize::new(1500).unwrap()); + + /// Returns the PDU size for `bytes`, or `None` if it is outside + /// [`MIN`](Self::MIN)..=[`MAX`](Self::MAX). + pub const fn new(bytes: usize) -> Option { + match NonZeroUsize::new(bytes) { + Some(n) if bytes >= Self::MIN.get() && bytes <= Self::MAX.get() => Some(Self(n)), + _ => None, + } + } + + /// Returns the PDU size as a plain integer. + pub const fn get(self) -> usize { + self.0.get() + } +} + +impl Default for PduSize { + fn default() -> Self { + Self::DEFAULT + } +} + +config_newtype!(PduSize: usize); + +/// A bound on the bytes a [`Sender`] holds queued and not yet packed into +/// a PDU: whole bundles from [`Sender::enqueue`] and chunks from +/// [`Sender::push`] alike, so one queue entry standing for a bundle of any +/// size is still bounded. +/// +/// The bound drives backpressure, not errors: the `tower` +/// `Service::poll_ready` returns `Pending` while the queue is at the bound, +/// and direct callers pace themselves with [`Sender::is_send_queue_full`] +/// and by draining [`Sender::next_pdu`]; `enqueue` and `push` never refuse +/// on it, so a bundle larger than the bound is still taken when the queue +/// is below it. Construct via [`SendQueueBytes::new`], [`TryFrom`], +/// or [`From`]; a zero bound would park `poll_ready` forever. +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] +#[repr(transparent)] +#[cfg_attr( + feature = "serde", + derive(serde::Serialize, serde::Deserialize), + serde(try_from = "usize", into = "usize") +)] +pub struct SendQueueBytes(NonZeroUsize); + +impl SendQueueBytes { + /// What the value configures, as error messages name it. + const NAME: &str = "send queue bytes"; + + /// The smallest bound, one byte. + pub const MIN: Self = Self(NonZeroUsize::MIN); + + /// The largest bound, `usize::MAX` bytes. + pub const MAX: Self = Self(NonZeroUsize::MAX); + + /// 1 MiB. + pub const DEFAULT: Self = Self(NonZeroUsize::new(1 << 20).unwrap()); + + /// Returns the bound for `bytes`, or `None` if it is zero. + pub const fn new(bytes: usize) -> Option { + match NonZeroUsize::new(bytes) { + Some(n) => Some(Self(n)), + None => None, + } + } + + /// Returns the bound as a plain integer. + pub const fn get(self) -> usize { + self.0.get() + } +} + +impl Default for SendQueueBytes { + fn default() -> Self { + Self::DEFAULT + } +} + +config_newtype!(SendQueueBytes: usize, NonZeroUsize); + +/// The framing discipline of the link a [`Sender`] feeds. +/// +/// It decides how far [`Sender::next_pdu`] pads each PDU and whether a +/// fitting bundle may travel without a BTP-U header. The combination +/// "fixed-size frames with bare bundles" is unrepresentable: a bare frame is +/// the bundle's own bytes, so it cannot be padded. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)] +#[cfg_attr( + feature = "serde", + derive(serde::Serialize, serde::Deserialize), + serde(rename_all = "kebab-case", rename_all_fields = "kebab-case") +)] +pub enum LinkFraming { + /// Fixed-size frames (for example CCSDS frames): every PDU is padded to + /// exactly the configured [`PduSize`] and a fitting bundle is always a + /// Bundle Message. + #[default] + FixedSize, + /// Variable-length PDUs (for example UDP datagrams or Ethernet + /// frames): the [`PduSize`] is a ceiling, a PDU is padded only up to + /// `min_pdu_len`, and `bundle_framing` chooses how a fitting bundle is + /// put on the wire. + Variable { + /// How a bundle that fits in one PDU is framed. Default: a Bundle + /// Message, so a configuration file may say `variable: {}`. + #[cfg_attr(feature = "serde", serde(default))] + bundle_framing: BundleFraming, + /// The length a shorter PDU is padded up to, with Definite Padding + /// (and Indefinite Padding for a gap of under four bytes). Default: + /// 0, no padding. + /// + /// Ethernet wants 46, its minimum frame payload, so that the + /// receiver never sees the MAC's own padding (Section 3.1 of the + /// Ethernet convergence layer draft); 42 is enough behind an + /// 802.1Q tag, and 46 is correct either way. On any datagram link + /// a floor also hides the size of small messages, such as a lone + /// Transfer Cancel, from an observer, at the cost of the padding; + /// [`LinkFraming::FixedSize`] hides every PDU's size. + /// + /// A floor above the [`PduSize`] pads every PDU to the `PduSize`. + /// A bare bundle frame ([`BundleFraming::Bare`]) cannot be padded, + /// since the padding would be read as bundle bytes, so a bundle + /// shorter than the floor is sent as a Bundle Message and padded. + #[cfg_attr(feature = "serde", serde(default))] + min_pdu_len: usize, + }, +} + +impl LinkFraming { + /// Variable-length PDUs framing a fitting bundle as `bundle_framing` + /// says, with no `min_pdu_len` floor. + pub const fn variable(bundle_framing: BundleFraming) -> Self { + Self::Variable { + bundle_framing, + min_pdu_len: 0, + } + } +} + +/// How a [`Sender`] on a variable-length link frames a bundle that fits in +/// one PDU (see [`LinkFraming::Variable`]). +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)] +#[cfg_attr( + feature = "serde", + derive(serde::Serialize, serde::Deserialize), + serde(rename_all = "kebab-case") +)] +pub enum BundleFraming { + /// A type-2 Bundle Message (Section 8.1): the 4-byte header is the only + /// overhead, and it lets the bundle share a PDU with other messages, + /// carry hints, and tolerate link padding after it (zero-fill decodes + /// as Indefinite Padding). + #[default] + Message, + /// The bundle's own bytes, with no BTP-U header, for a peer that accepts + /// bare bundle frames. Such a frame cannot be packed with other + /// messages or carry hints, so it occupies a PDU of its own, and a + /// bundle enqueued with hints is still sent as a Bundle Message. + /// + /// A receiver tells a bare frame from a PDU by its first byte, which the + /// bundle-reserved message-type values (Section 12.1) guarantee; the + /// sender therefore only emits bare frames whose first byte + /// [`frame_kind`] classifies as a bundle, and frames anything else as a + /// Bundle Message. It also frames a bundle shorter than the + /// `min_pdu_len` floor as a Bundle Message, which can be padded. + /// + /// **Padding links.** A bare frame carries nothing that tells the + /// receiver where the bundle ends, so a link that pads frames to a + /// minimum size (Ethernet's 46-octet minimum payload, for instance) + /// delivers the padding as bundle bytes. Setting `min_pdu_len` to the + /// link's minimum keeps this sender's bare frames clear of it. A + /// receiver cannot rely on that, since its peer may be another + /// implementation, so it still needs to delimit a bare bundle itself + /// (see [`BundleExtent`](crate::codec::BundleExtent) for this crate's + /// receive side). A link that pads to a fixed size should use + /// [`LinkFraming::FixedSize`]. When in doubt, use + /// [`BundleFraming::Message`]. + Bare, +} + +/// When a [`Sender`] cuts a segment whose bytes are still being pushed +/// (see [`Sender::push`]). +/// +/// A segment carries what remains of the bundle up to a full segment, or +/// fills the room left in a PDU if that is at least half a segment. The +/// policy decides what happens when fewer of those bytes have been pushed. +/// Under either policy every segment but the last carries at least half a +/// full segment, so a bundle's segment count stays within twice its +/// full-size count. A bundle given whole to [`Sender::enqueue`] is cut the +/// same way under both. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)] +#[cfg_attr( + feature = "serde", + derive(serde::Serialize, serde::Deserialize), + serde(rename_all = "kebab-case") +)] +pub enum SegmentCutStrategy { + /// Wait until all of the segment's bytes are pushed. A slow producer + /// does not multiply the segment count, and segments are as long as + /// the PDU allows. + #[default] + Full, + /// Cut once at least half a full segment's bytes are pushed, carrying + /// what has been pushed. A producer whose chunks are not a multiple of + /// the segment size, say one and a half PDUs, then has each chunk sent + /// as one full segment and one shorter one, and released, without + /// waiting for the next chunk. The cost is more segments, PDUs that go + /// out part-filled, and segments short enough that a receiver may copy + /// rather than share them. + Half, +} + +/// Configuration for a [`Sender`]. Every field has a default, so a +/// configuration file may set only what it changes. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)] +#[cfg_attr( + feature = "serde", + derive(serde::Serialize, serde::Deserialize), + serde(default, rename_all = "kebab-case") +)] +pub struct SenderConfig { + /// The link's PDU size. Default: 1500. + pub pdu_size: PduSize, + /// The Section 5 transfer window. Default: 16. + pub window_size: WindowSize, + /// The pending-queue admission bound, in bytes. Default: 1 MiB. + pub send_queue_bytes: SendQueueBytes, + /// The link's framing discipline. Default: fixed-size frames. + pub link_framing: LinkFraming, + /// When a segment of a bundle still being pushed is cut. Default: + /// once all of its bytes are pushed. + pub segment_cut_strategy: SegmentCutStrategy, +} + +/// Options for [`Sender::enqueue`] and [`Sender::begin`]. +/// +/// `Default`-constructible; `SendOptions::default()` sends with no +/// caller-supplied hints. +// Not `#[non_exhaustive]`: `hints` carries anything the sender need not +// understand, so a new field means a new sender capability, and the +// compile break on callers' struct literals is the useful checklist. +#[derive(Debug, Clone, Default)] +pub struct SendOptions { + /// Hint items to attach to the transfer, carried on the Bundle message + /// (unsegmented) or the first segment (segmented) alongside the + /// sender-derived Bundle Length hint, in ascending hint-type order. A + /// caller-supplied item of type [`HintType::BUNDLE_LENGTH`] is + /// discarded: the sender derives the truthful value itself. + pub hints: Hints, +} + +/// A bundle plus its [`SendOptions`], the request type of the `tower` +/// `Service` impl. `From` builds one with default options. +#[derive(Debug, Clone)] +pub struct SendRequest { + /// The bundle to send. + pub data: Bytes, + /// How to send it. + pub options: SendOptions, +} + +impl From for SendRequest { + fn from(data: Bytes) -> Self { + Self { + data, + options: SendOptions::default(), + } + } +} + +/// Options for [`Sender::next_pdu_with`] and [`Sender::next_pdu_into_with`]: how one +/// PDU is packed. Policy that holds for every PDU belongs in +/// [`SenderConfig`]. +/// +/// `Default`-constructible; `NextPduOptions::default()` packs only what is +/// ready to send, as [`Sender::next_pdu`] and the `tower` `Stream` impl do. +// Not `#[non_exhaustive]`: a new field is a new packing policy, and the +// compile break on callers' struct literals is the useful checklist. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)] +pub struct NextPduOptions { + /// Also send what bundles waiting on their producers have buffered. + /// + /// The PDU is packed as usual, then each segmented bundle whose next + /// segment waits on bytes not yet pushed (see [`SegmentCutStrategy`]), + /// in queue order, cuts a segment of what it has buffered into the room + /// left, at least one data byte. A PDU that would otherwise be empty + /// carries those segments alone. The packing stops at the first queued + /// message that is not part of such a bundle, as it does without a + /// flush, so bundles are not reordered. + /// + /// The sender has no clock, so it never cuts short of the + /// [`SegmentCutStrategy`] on its own. A CLA that has one asks for a + /// flush when its producer has been quiet for long enough, as TCP's + /// cork timer does, or when the link offers a transmission slot that + /// would otherwise carry only padding. + /// + /// A flushed segment may be far shorter than half a segment, so + /// frequent flushes raise a bundle's segment count, and with it what + /// the receiver must allow (see `MaxSegments` in the receiver). A cut + /// is refused if the rest of the bundle, segmented normally, could then + /// need the segment index `u32::MAX`. A bundle that fits one PDU is + /// queued only once fully pushed, so it is never flushed. + pub flush: bool, +} + +/// A bundle being pushed into a [`Sender`], from [`Sender::begin`] until +/// it is ended by [`Sender::finish`] or [`Sender::cancel`]. +/// +/// **Dropping a handle does not cancel its bundle.** The handle is a token, +/// not a borrow of the sender, so it cannot reach the sender when dropped. +/// A bundle whose producer gives up stays queued, supplying nothing, and a +/// segmented one holds its window slot, until the CLA calls +/// `cancel(handle)`. A CLA that keeps the sender in `Arc>` can +/// wrap the handle in a guard of its own that cancels on drop. +/// +/// The handle counts the bytes pushed through it, so [`Sender::push`] +/// refuses an overrun and [`Sender::finish`] detects an underrun. It +/// converts into the bundle's [`SendId`], consuming it, which is how +/// `cancel(handle)` ends it. Use a handle only with the sender that issued +/// it: another sender may hold a bundle under the same ID. +#[must_use = "dropping a send handle does not cancel its bundle; finish or cancel it"] +#[derive(Debug, PartialEq, Eq)] +pub struct SendHandle { + id: SendId, + total_len: usize, + pushed: usize, +} + +impl SendHandle { + /// The ID under which [`Pdu::carried`] reports the bundle. + pub fn id(&self) -> SendId { + self.id + } + + /// The bundle's length, as given to [`Sender::begin`]. + pub fn total_len(&self) -> usize { + self.total_len + } + + /// The bytes pushed so far. + pub fn pushed(&self) -> usize { + self.pushed + } +} + +impl From for SendId { + fn from(handle: SendHandle) -> Self { + handle.id + } +} + +/// Identifies a bundle from [`Sender::enqueue`] or [`Sender::begin`] until +/// its last bytes leave in a PDU, naming it in the [`Carried`] entries of +/// every PDU that carries part of it. +/// +/// [`Self::kind`] reports how the bundle travels. A segmented bundle's ID +/// holds its transfer number, which is outstanding in the window until its +/// End is packed or it is cancelled, so no two queued transfers share one. +/// Bundle Messages and bare frames draw from one separate `u32` counter +/// that advances with every such enqueue and wraps at `u32::MAX`; their +/// IDs are unique while queued unless 2³² unsegmented bundles are queued +/// at once. +/// +/// Any ID may be passed back to [`Sender::cancel`] or +/// [`Sender::is_outstanding`]. The ID is opaque, with no public +/// constructor, so a caller names a bundle only by an ID the sender issued, +/// never by a wire value such as a transfer number. Its `Debug` output +/// shows the kind and the number, for logs. +#[derive(Clone, Copy, PartialEq, Eq, Hash, PartialOrd, Ord)] +pub struct SendId(Repr); + +#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, PartialOrd, Ord)] +enum Repr { + /// The transfer number. + Transfer(u32), + /// A value of the unsegmented-bundle counter. + Message(u32), + /// As for `Message`. + Bare(u32), +} + +/// Where [`Sender::find_unsegmented`] found a bundle: its index in the send +/// queue, or among the bundles still being pushed. +enum Unsegmented { + Queued(usize), + Assembling(usize), +} + +impl SendId { + /// How the bundle travels. + pub const fn kind(self) -> SendKind { + match self.0 { + Repr::Transfer(_) => SendKind::Transfer, + Repr::Message(_) => SendKind::Message, + Repr::Bare(_) => SendKind::Bare, + } + } +} + +impl fmt::Debug for SendId { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + self.0.fmt(f) + } +} + +/// How a bundle travels, as [`SendId::kind`] reports it. Fixed when the +/// bundle is enqueued or begun. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)] +pub enum SendKind { + /// Segmented into a transfer. Its window slot is released by the + /// sender itself once the transfer's End message has been packed into + /// a PDU. + Transfer, + /// Sent as a single Bundle Message. + Message, + /// Sent as a bare bundle frame in a PDU of its own (see + /// [`BundleFraming::Bare`]). + Bare, +} + +// The tag fits the padding beside the `u32`, so an ID costs what a `u64` +// would; `Carried` lists rely on that staying true. +const _: () = assert!(size_of::() == 8); + +/// A bundle with bytes in a PDU, as listed by [`Pdu::carried`]. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)] +pub struct Carried { + /// The bundle, as [`Sender::enqueue`] reported it. + pub id: SendId, + /// Whether this PDU carries the bundle's last bytes: always for a + /// Bundle Message or bare frame, and for a segmented bundle when the + /// PDU holds its Transfer End. Once the PDU holding it is written, + /// the sender will emit nothing further for the bundle. + pub completes: bool, +} + +/// The [`Carried`] entries of one PDU, held inline up to +/// [`Self::INLINE`] entries and on the heap beyond that. +/// +/// Dereferences to `[Carried]`. Equality, hashing and `Debug` follow the +/// entries, not where they are stored. +/// +/// A list keeps its heap buffer once it has one: when refilled by +/// [`Sender::next_pdu_into`] it writes into that buffer even for a PDU +/// whose entries would fit inline, so a reused list stops allocating once +/// it has grown to the largest PDU seen. [`Self::with_capacity`] starts +/// with a heap buffer of a chosen size. +/// +/// # Sizing +/// +/// A PDU of valid BPv7 bundles carries at most `pdu_size / 32 + +/// window_size` entries. The smallest valid bundle is 28 bytes (RFC 9171: +/// a 20-byte primary block with its mandatory CRC and every EID +/// `dtn:none`, an empty payload block, and the indefinite-length array +/// around them), so each Bundle Message is at least 32 bytes with its +/// header, and a PDU holds at most `pdu_size / 32` of them. Each transfer +/// with segments in the PDU adds one entry, and at most `window_size` +/// transfers are outstanding. The sender does not parse bundles; one +/// enqueued below 28 bytes can exceed the bound, at the cost of the list +/// growing. +/// +/// The bound is loose for ordinary traffic. Transfers are packed in queue +/// order unless one is passed over, so a PDU names more than two transfers +/// only when bundles are pushed in chunks and several wait on their +/// producers, or when transfers end within one PDU. +#[derive(Clone, PartialEq, Eq, Hash)] +pub struct CarriedList(SmallVec<[Carried; CarriedList::INLINE]>); + +impl CarriedList { + /// Entries held without allocating. A PDU lists one entry for each + /// transfer with segments in it and one for each Bundle Message, so + /// traffic of bundles longer than a quarter of the PDU, given whole to + /// [`Sender::enqueue`], stays inline. + pub const INLINE: usize = 4; + + /// An empty list, held inline. + pub const fn new() -> Self { + Self(SmallVec::new_const()) + } + + /// An empty list that holds `capacity` entries without allocating + /// again: inline up to [`Self::INLINE`], otherwise in a heap buffer + /// allocated now and kept for the list's life. + pub fn with_capacity(capacity: usize) -> Self { + Self(SmallVec::with_capacity(capacity)) + } + + /// Entries the list holds without allocating. + pub fn capacity(&self) -> usize { + self.0.capacity() + } + + /// Empty the list, keeping any heap buffer. + fn clear(&mut self) { + self.0.clear(); + } + + /// Append `item`, moving the entries to the heap if the inline slots + /// are full. A PDU names each bundle once. + fn push(&mut self, item: Carried) { + debug_assert!( + self.iter().all(|c| c.id != item.id), + "a PDU names each bundle once" + ); + self.0.push(item); + } +} + +impl Default for CarriedList { + fn default() -> Self { + Self::new() + } +} + +impl Deref for CarriedList { + type Target = [Carried]; + + fn deref(&self) -> &[Carried] { + &self.0 + } +} + +impl<'a> IntoIterator for &'a CarriedList { + type Item = &'a Carried; + type IntoIter = slice::Iter<'a, Carried>; + + fn into_iter(self) -> Self::IntoIter { + self.iter() + } +} + +impl fmt::Debug for CarriedList { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.debug_list().entries(self.iter()).finish() + } +} + +/// A packed PDU and the bundles it carries, returned by [`Sender::next_pdu`]. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct Pdu { + /// The PDU, ready for the link. + pub data: Bytes, + /// Every bundle with bytes in `data`, once each, in the order packed. + /// Empty for a PDU holding only a Transfer Cancel. + pub carried: CarriedList, +} + +/// One unit of the pending send queue. No `Debug`: it holds bundle bytes. +enum QueueEntry { + /// A Bundle Message and its ID, packed with its neighbours into PDUs. + Message { id: SendId, message: Message }, + /// A Transfer Cancel for the transfer number, which carries no bundle. + Cancel(u32), + /// A segmented transfer, cut into Transfer Segment and Transfer End + /// messages as PDUs are packed. Its ID is its transfer number. + Transfer(QueuedTransfer), + /// A bare bundle frame ([`BundleFraming::Bare`]) and the counter value + /// of its [`SendKind::Bare`]: emitted as a PDU of its own, since + /// nothing can precede, follow, or pad it. + BareBundle { id: u32, data: Bytes }, +} + +impl QueueEntry { + /// The ID of the unsegmented bundle this entry holds, if it holds one. + fn unsegmented_id(&self) -> Option { + match self { + Self::Message { id, .. } => Some(*id), + Self::BareBundle { id, .. } => Some(SendId(Repr::Bare(*id))), + Self::Cancel(_) | Self::Transfer(_) => None, + } + } + + /// The segmented transfer the entry holds, if it holds one. + fn as_transfer(&self) -> Option<&QueuedTransfer> { + match self { + Self::Transfer(t) => Some(t), + _ => None, + } + } + + /// [`Self::as_transfer`], mutably. + fn as_transfer_mut(&mut self) -> Option<&mut QueuedTransfer> { + match self { + Self::Transfer(t) => Some(t), + _ => None, + } + } + + /// Whether the entry can supply its next message into `room` bytes of + /// a PDU, `empty` if nothing has been packed into it yet. + /// + /// An empty PDU takes a whole message whatever its size, which is what + /// guarantees progress, and is the only PDU a bare bundle frame, which + /// is not a message and travels alone, can go in. A transfer supplies + /// a segment only once its bytes have been pushed (see + /// [`QueuedTransfer::chunk_for`]). + fn supplies(&self, room: usize, empty: bool) -> bool { + match self { + Self::Message { message, .. } => empty || encoded_message_len(message) <= room, + Self::Cancel(transfer_number) => empty || cancel_message_len(*transfer_number) <= room, + Self::Transfer(t) => t.next_chunk(room).is_some(), + Self::BareBundle { .. } => empty, + } + } + + /// The bundle bytes the entry holds, as [`SendQueueBytes`] counts them. + fn queued_bytes(&self) -> usize { + match self { + Self::Message { message, .. } => match message { + Message::Bundle { data, .. } => data.len(), + _ => 0, + }, + Self::Cancel(_) => 0, + Self::Transfer(t) => t.buffered, + Self::BareBundle { data, .. } => data.len(), + } + } +} + +/// The copies emitted of each segment. Repetition (Section 6) would make +/// this a per-transfer setting; the queue already keeps a segment's +/// boundaries until its last copy. +const COPIES: u32 = 1; + +/// A segmented bundle in the queue, from [`Sender::begin`] (or +/// [`Sender::enqueue`]) until the last copy of its End is packed. +/// +/// Its segments are cut as PDUs are packed, each sized to the room left in +/// the PDU (see [`Self::chunk_for`]) and written straight from the chunks +/// pushed into it, so a queued transfer costs one queue entry and handles +/// on the pushed buffers rather than one message per segment. +struct QueuedTransfer { + transfer_number: u32, + /// The bundle's length, all pushed or not. + total_len: usize, + /// The pushed bytes from the cursor on, the segment with copies left + /// included, in order. + chunks: VecDeque, + /// The bytes in `chunks`. + buffered: usize, + /// The hints for segment 0: the sender-derived Bundle Length followed + /// by the caller's. Dropped once segment 0's last copy is cut. + first_segment_hints: Vec, + /// Data bytes segment 0 may carry, reduced by its hints. + first_capacity: usize, + /// Data bytes every later segment may carry. + capacity: usize, + /// When a segment whose bytes are not all pushed is cut. + segment_cut_strategy: SegmentCutStrategy, + /// Bytes before the segment at the cursor. + offset: usize, + /// The index of the segment at the cursor. + next_index: u32, + /// The segment at the cursor, once its first copy is cut and while + /// copies remain: its boundaries are fixed by the first copy, so every + /// copy is byte-identical (Section 6). + cut: Option, +} + +/// A segment with copies still to emit. +struct Cut { + /// Its data bytes, from the transfer's cursor. + len: usize, + /// Copies still to emit, at least one. + copies_left: u32, +} + +impl QueuedTransfer { + /// A transfer with nothing pushed yet. + fn new( + transfer_number: u32, + total_len: usize, + first_segment_hints: Vec, + first_capacity: usize, + capacity: usize, + segment_cut_strategy: SegmentCutStrategy, + ) -> Self { + Self { + transfer_number, + total_len, + chunks: VecDeque::new(), + buffered: 0, + first_segment_hints, + first_capacity, + capacity, + segment_cut_strategy, + offset: 0, + next_index: 0, + cut: None, + } + } + + /// Whether the next segment is waiting on bytes not yet pushed: it + /// could not go even in an empty PDU of `pdu_size` bytes. + fn waiting(&self, pdu_size: usize) -> bool { + self.cut.is_none() + && self + .chunk_for(self.offset, self.next_index, pdu_size) + .is_none() + } + + /// Whether any segment has been cut (and so packed into a PDU). + fn started(&self) -> bool { + self.offset > 0 || self.cut.is_some() + } + + /// Whether the last copy of every segment has been cut, the Transfer + /// End included. + fn finished(&self) -> bool { + self.offset >= self.total_len + } + + /// Append pushed bytes. + fn push(&mut self, chunk: Bytes) { + self.buffered += chunk.len(); + self.chunks.push_back(chunk); + } + + /// The hints segment `index` carries. + fn hints_at(&self, index: u32) -> &[HintItem] { + if index == 0 { + &self.first_segment_hints + } else { + &[] + } + } + + /// The most data bytes segment `index` may carry. + fn capacity_at(&self, index: u32) -> usize { + if index == 0 { + self.first_capacity + } else { + self.capacity + } + } + + /// The encoded length of segment `index` carrying `len` data bytes. + fn segment_len(&self, index: u32, len: usize) -> usize { + segment_message_len(self.hints_at(index), len) + } + + /// The data bytes segment `index` fits in `room` bytes of a PDU, or + /// `None` if not even its framing fits. + fn data_room(&self, index: u32, room: usize) -> Option { + room.checked_sub(self.segment_len(index, 0)) + } + + /// The data bytes of a segment not yet cut, at `offset` with index + /// `index`, placed in `room` bytes of a PDU, or `None` if it cannot go + /// there. + /// + /// A segment carries what remains of the bundle up to its capacity. If + /// that does not fit, the segment fills the room instead, but only if + /// the room holds at least half its capacity: the segment count then + /// stays within twice the full-size count, and every segment but the + /// last fills at least half a PDU. If not all of those bytes have been + /// pushed, the segment waits for them, or under [`SegmentCutStrategy::Half`] + /// carries what has been pushed if that is at least half its capacity, + /// which keeps the same bounds. An empty PDU always has room for a + /// full-size segment, since `plan` sizes the capacities to fit one. + fn chunk_for(&self, offset: usize, index: u32, room: usize) -> Option { + let fits = self.data_room(index, room)?; + let capacity = self.capacity_at(index); + let whole = (self.total_len - offset).min(capacity); + let len = if whole <= fits { + whole + } else if fits >= capacity.div_ceil(2) { + fits + } else { + return None; + }; + let pushed = self.offset + self.buffered - offset; + if len <= pushed { + Some(len) + } else if self.segment_cut_strategy == SegmentCutStrategy::Half + && pushed >= capacity.div_ceil(2) + { + Some(pushed) + } else { + None + } + } + + /// The data bytes the next segment message carries in `room` bytes, or + /// `None` if it cannot go there. + fn next_chunk(&self, room: usize) -> Option { + match &self.cut { + Some(cut) => (self.segment_len(self.next_index, cut.len) <= room).then_some(cut.len), + None => self.chunk_for(self.offset, self.next_index, room), + } + } + + /// The data bytes a flush cuts from this transfer into `room` bytes of + /// a PDU of `pdu_size`, or `None` if it cuts nothing (see + /// [`NextPduOptions::flush`]). + /// + /// Only a transfer whose next segment is waiting on its producer is + /// flushed. The segment carries what has been pushed, as much as fits, + /// at least one byte. The rest of the bundle, cut normally from then + /// on, needs at most one index per half segment (see `chunk_for`), so + /// the cut is refused if that could reach `u32::MAX`, as `plan` refuses + /// a bundle. The rest is never empty: a waiting segment has fewer + /// bytes pushed than it would carry. + fn flush_chunk(&self, room: usize, pdu_size: usize) -> Option { + if !self.waiting(pdu_size) { + return None; + } + let len = self.buffered.min(self.data_room(self.next_index, room)?); + if len == 0 { + return None; + } + let rest = (self.total_len - self.offset - len) as u64; + let last_index = + u64::from(self.next_index) + rest.div_ceil(self.capacity.div_ceil(2) as u64); + (last_index < u64::from(u32::MAX)).then_some(len) + } + + /// The encoded length of the segment messages the transfer supplies to + /// `room` bytes of a PDU, as [`Sender::pack`] would cut them. + fn planned_len(&self, room: usize) -> usize { + let (mut offset, mut index, mut total) = (self.offset, self.next_index, 0); + if let Some(cut) = &self.cut { + total = self.segment_len(index, cut.len); + if total > room { + return 0; + } + offset += cut.len; + index = index.wrapping_add(1); + } + while offset < self.total_len { + let Some(chunk) = self.chunk_for(offset, index, room - total) else { + break; + }; + total += self.segment_len(index, chunk); + offset += chunk; + index = index.wrapping_add(1); + } + total + } + + /// Write the next segment message, carrying `len` data bytes, into + /// `dst`, and return the pushed bytes it releases. The caller sizes + /// `len` with [`Self::next_chunk`] or [`Self::flush_chunk`]. The last + /// segment is a Transfer End. Bytes are released only with a segment's + /// last copy. + fn cut(&mut self, len: usize, dst: &mut BytesMut) -> usize { + let index = self.next_index; + let end = self.offset + len == self.total_len; + encode_segment_head( + end, + self.transfer_number, + index, + self.hints_at(index), + len, + dst, + ) + .expect("segments are sized against the PDU"); + let mut left = len; + for chunk in &self.chunks { + let n = left.min(chunk.len()); + dst.extend_from_slice(&chunk[..n]); + left -= n; + if left == 0 { + break; + } + } + + let copies_left = self.cut.take().map_or(COPIES, |cut| cut.copies_left) - 1; + if copies_left > 0 { + self.cut = Some(Cut { len, copies_left }); + return 0; + } + if index == 0 { + self.first_segment_hints = Vec::new(); + } + self.release(len); + self.offset += len; + self.next_index = index.wrapping_add(1); + len + } + + /// Drop `len` bytes from the front of `chunks`. + fn release(&mut self, len: usize) { + self.buffered -= len; + let mut left = len; + while left > 0 { + let front = self + .chunks + .front_mut() + .expect("released bytes are buffered"); + if front.len() > left { + front.advance(left); + break; + } + left -= front.len(); + self.chunks.pop_front(); + } + } +} + +/// A bundle that fits one PDU, being pushed through a [`SendHandle`]: +/// queued as a Bundle Message or bare frame when its last byte arrives. +struct Assembly { + id: SendId, + hints: Vec, + /// The pushed chunks, gathered into one buffer when the last arrives. + chunks: SmallVec<[Bytes; 1]>, + /// The bytes in `chunks`. + len: usize, +} + +/// The encoded length of a Transfer Cancel. +fn cancel_message_len(transfer_number: u32) -> usize { + encoded_message_len(&Message::TransferCancel { transfer_number }) +} + +/// Manages outbound BTP-U transfers, segmentation, and PDU packing. +/// +/// The sender is convergence-layer agnostic: a CLA calls [`Sender::enqueue`] to +/// submit bundles and [`Sender::next_pdu`] to obtain packed PDUs ready for +/// transmission, each listing the bundles it carries. +/// +/// # Transfer window +/// +/// A segmented bundle takes a Section 5 window slot when it is enqueued and +/// gives it back when its Transfer End is packed into a PDU by +/// [`Self::next_pdu`]: a unidirectional link offers no acknowledgement to +/// anchor an explicit completion call to, and once the End has left the +/// queue the sender has nothing further to emit for the transfer. The only +/// other way out of the window is [`Self::cancel`]. The window is enforced +/// on the span of outstanding numbers, not their count: a new transfer is +/// refused while its number would push the oldest outstanding transfer out +/// of the window, so draining the queue in order is what frees it. +/// +/// # Loss protection +/// +/// This sender emits each message exactly once and packs the queue in +/// arrival order, except that a Transfer Cancel goes to the front (see +/// [`Self::cancel`]) and a transfer that cannot supply its next segment is +/// passed over (see [`Self::next_pdu`]). The repetition (Section 6) the +/// protocol permits is not implemented here, so a lost PDU loses the +/// messages it carried; a bundle that fits one PDU is lost outright, and a +/// segmented one is lost when its transfer expires at the receiver. Those +/// are properties of the link to weigh when choosing it. +/// +/// # Link framing +/// +/// [`SenderConfig::link_framing`] fixes the link's framing discipline at +/// construction. By default the sender assumes fixed-size frames: every +/// PDU is padded to the configured [`PduSize`], and a bundle that fits in +/// one PDU is emitted as a type-2 Bundle Message (Section 8.1), whose 4-byte +/// header lets it share a PDU with another transfer's segments, be followed +/// by padding, and carry hints. [`LinkFraming::Variable`] pads only up to +/// a floor, if one is set, and may additionally emit fitting bundles as +/// bare bundle frames for a peer that accepts them (see +/// [`BundleFraming::Bare`] for the padding caveat). +/// +/// Every outbound unit, bare frames included, passes through the one +/// pending queue: a bare frame is emitted in arrival order behind the +/// messages queued before it, counts against the [`SendQueueBytes`], and is +/// visible to whatever schedules that queue. Writing bare bundles to the +/// link around the sender would instead let them race and starve the +/// transfers queued here. +/// +/// # Concurrency +/// +/// `Sender` is single-owner: it is mutated through `&mut self`, and the +/// `tower` impls follow suit. Under the `tower` feature, +/// `Service::poll_ready` parks the caller while the transfer window is +/// saturated or the send queue is at its [`SendQueueBytes`]; the window gate +/// applies to unsegmented bundles too, since `poll_ready` cannot see the +/// request. Every parked task is woken when `Stream::poll_next` drains a +/// PDU or [`Self::cancel`] frees a slot or queued bytes. `poll_next` +/// parks while nothing is ready to pack and never yields `Ready(None)`. +/// Only the drain frees a full window or queue, so run producers and the +/// drain from separate tasks, or from one task that selects over both; a +/// task that awaits `ready()` before polling the drain stops for good once +/// the window fills. +/// +/// Several producers may share a `Sender` through `Arc>`. +/// Admission is not reserved between `poll_ready` and `call` (or between +/// [`Self::is_window_available`] and [`Self::enqueue`]), so a producer +/// takes the lock, polls, and if ready calls before releasing it; and it +/// releases the lock before parking, so the drain can take it. A +/// `WindowFull` from `enqueue` after a positive check means another +/// producer got there first; check again. The drain has one consumer. Do +/// **not** use `tower::buffer::Buffer`: it moves the `Sender` into a worker +/// task and exposes only the `Service` half, so the drain and +/// [`Self::cancel`] become unreachable, PDUs never leave, and window slots +/// never free. +pub struct Sender { + pdu_size: PduSize, + /// Admission bound on `queued_bytes`; enforced by the `tower` + /// `Service::poll_ready` rather than by `enqueue` or `push` themselves. + send_queue_bytes: SendQueueBytes, + /// The bundle bytes held in `pending` and `assembling`. + queued_bytes: usize, + /// Owns the set of outstanding transfer numbers and the Section 5 window + /// rule. [`Self::cancel`] and the End-packing release in + /// [`Self::next_pdu`] only act on numbers it reports as outstanding. + allocator: TransferNumberAllocator, + link_framing: LinkFraming, + segment_cut_strategy: SegmentCutStrategy, + pending: VecDeque, + /// Bundles that fit one PDU, begun and not yet fully pushed. Each + /// joins `pending` when its last byte is pushed. + assembling: Vec, + /// The counter value for the next [`SendKind::Message`] or + /// [`SendKind::Bare`]; wraps. + next_bundle_id: u32, + /// Every task parked in the `tower` `Service::poll_ready`, woken when a + /// window slot frees or the send queue drains below its bound. A list + /// rather than a slot so that several producers sharing the sender + /// through a mutex are all woken; re-registration by a task already + /// present is deduplicated with `Waker::will_wake`. + #[cfg(feature = "tower")] + enqueue_wakers: Vec, + /// The task parked in `Stream::poll_next`, woken when an entry joins + /// `pending` or a queued transfer is pushed more bytes. Single-slot: + /// the drain has one consumer. + #[cfg(feature = "tower")] + drain_waker: Option, +} + +/// Summarises the queue rather than printing it: the queued bundles' +/// bytes would make the output as large as the backlog. +impl fmt::Debug for Sender { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + let mut d = f.debug_struct("Sender"); + d.field("pdu_size", &self.pdu_size) + .field("send_queue_bytes", &self.send_queue_bytes) + .field("link_framing", &self.link_framing) + .field("segment_cut_strategy", &self.segment_cut_strategy) + .field("window_size", &self.allocator.window_size()) + .field("transfers_outstanding", &self.allocator.in_progress()) + .field("window_available", &self.is_window_available()) + .field("queued", &self.pending.len()) + .field("queued_bytes", &self.queued_bytes) + .field("assembling", &self.assembling.len()) + .field("next_bundle_id", &self.next_bundle_id); + #[cfg(feature = "tower")] + d.field("enqueue_wakers", &self.enqueue_wakers.len()) + .field("drain_waker", &self.drain_waker.is_some()); + d.finish() + } +} + +impl Sender { + /// Create a new sender that will allocate `initial_transfer_number` as + /// its first transfer number. + /// + /// The BTP-U spec recommends choosing this value unpredictably (typically + /// from a random source) to reduce the likelihood of a receiver mistaking + /// the new sender for an old one after a restart; see `Sender::try_from_rng` + /// and `Sender::from_rng` (under the `rand` feature) for the common case + /// of seeding from an RNG. + pub fn new(config: SenderConfig, initial_transfer_number: u32) -> Self { + Self { + pdu_size: config.pdu_size, + send_queue_bytes: config.send_queue_bytes, + queued_bytes: 0, + allocator: TransferNumberAllocator::new(config.window_size, initial_transfer_number), + link_framing: config.link_framing, + segment_cut_strategy: config.segment_cut_strategy, + pending: VecDeque::new(), + assembling: Vec::new(), + next_bundle_id: 0, + #[cfg(feature = "tower")] + enqueue_wakers: Vec::new(), + #[cfg(feature = "tower")] + drain_waker: None, + } + } + + /// Create a new sender with the initial transfer number seeded from `rng`. + /// Convenience wrapper over [`Self::new`]. + #[cfg(feature = "rand")] + pub fn from_rng(config: SenderConfig, rng: &mut R) -> Self { + Self::new(config, rng.next_u32()) + } + + /// Create a new sender with the initial transfer number seeded from a + /// fallible `rng`, such as the operating system's `rand::rngs::SysRng`. + /// + /// # Errors + /// + /// Returns the RNG's error if it cannot produce a value. + #[cfg(feature = "rand")] + pub fn try_from_rng(config: SenderConfig, rng: &mut R) -> Result { + Ok(Self::new(config, rng.try_next_u32()?)) + } + + /// Append `entry` to the pending queue and wake the drain. + fn queue(&mut self, entry: QueueEntry) { + self.pending.push_back(entry); + self.wake_drain(); + } + + /// Remove the entry at queue position `at`, uncounting the bytes it + /// holds. + fn dequeue(&mut self, at: usize) -> Option { + let entry = self.pending.remove(at)?; + self.queued_bytes -= entry.queued_bytes(); + Some(entry) + } + + /// The queue position of the transfer numbered `transfer_number`, and + /// the transfer. Costs a scan of the send queue. + fn find_transfer(&self, transfer_number: u32) -> Option<(usize, &QueuedTransfer)> { + self.pending.iter().enumerate().find_map(|(at, e)| { + e.as_transfer() + .filter(|t| t.transfer_number == transfer_number) + .map(|t| (at, t)) + }) + } + + /// Wake every task parked on a `Service::poll_ready` that returned + /// `Pending` because the window was full or the send queue was at its + /// bound, once both have room: the same predicate `poll_ready` gates + /// on, so a woken task finds the service ready. No-op without the + /// `tower` feature. + #[cfg(feature = "tower")] + fn wake_enqueue(&mut self) { + if self.is_window_available() && !self.is_send_queue_full() { + for w in self.enqueue_wakers.drain(..) { + w.wake(); + } + } + } + #[cfg(not(feature = "tower"))] + fn wake_enqueue(&mut self) {} + + /// Wake any task parked on a `Stream::poll_next` that returned `Pending` + /// because nothing was ready. No-op without the `tower` feature. + #[cfg(feature = "tower")] + fn wake_drain(&mut self) { + if let Some(w) = self.drain_waker.take() { + w.wake(); + } + } + #[cfg(not(feature = "tower"))] + fn wake_drain(&mut self) {} + + /// Whether a segmented bundle could currently be admitted without + /// violating the transfer window: whether its transfer number would keep + /// the oldest outstanding transfer inside the window. + /// + /// The `tower` `Service::poll_ready` uses this as its window gate, so it + /// is exactly the predicate [`Self::enqueue`] applies when segmenting. + pub fn is_window_available(&self) -> bool { + self.allocator.can_allocate() + } + + /// Whether the bytes queued and not yet packed have reached the + /// configured [`SendQueueBytes`]. + /// + /// The `tower` `Service::poll_ready` uses this as its admission gate: + /// unsegmented bundles take no window slot, so without it the queue + /// would grow without bound whenever the drain side is slower. Direct + /// [`Self::enqueue`] and [`Self::push`] callers can poll it to pace + /// themselves the same way, draining [`Self::next_pdu`] when it reports + /// full. + /// + /// Gate [`Self::begin`] and [`Self::enqueue`] on it, and each push of a + /// bundle already begun on [`Self::is_push_ready`]. A segment goes out + /// only once its bytes are pushed (see [`SegmentCutStrategy`]), so when + /// every bundle in the queue is waiting on its producer, only pushes can + /// free it; a producer that waits here mid-bundle can then wait on + /// bytes only it would supply. + pub fn is_send_queue_full(&self) -> bool { + self.queued_bytes >= self.send_queue_bytes.get() + } + + /// Whether a producer pacing itself on the [`SendQueueBytes`] should + /// push the next chunk of the bundle `handle` was begun for now, rather + /// than drain [`Self::next_pdu`] first. + /// + /// True while the send queue is below its bound, and otherwise while + /// the bundle cannot go out without more bytes: a segmented bundle whose + /// next segment is waiting on its producer, or a bundle that fits one + /// PDU and is not yet fully pushed. A producer that waits only when + /// this is false therefore never waits on bytes only it would supply. + /// A bundle holds less than one segment (or one PDU) when a push past + /// the bound is admitted for it, so the queue exceeds its bound by less + /// than one segment and one chunk per bundle in progress. + /// + /// Also true for a handle whose bundle is not in progress, so that + /// [`Self::push`] reports why. Nothing wakes a producer when this + /// turns true; it is checked again after draining. + /// + /// Costs a scan of the send queue for a segmented bundle while the + /// queue is at its bound. + pub fn is_push_ready(&self, handle: &SendHandle) -> bool { + if !self.is_send_queue_full() { + return true; + } + let SendId(Repr::Transfer(transfer_number)) = handle.id else { + return true; + }; + self.find_transfer(transfer_number) + .is_none_or(|(_, t)| t.waiting(self.pdu_size.get())) + } + + /// The bundle bytes queued and not yet packed into a PDU, as + /// [`SendQueueBytes`] bounds them. + pub fn queued_bytes(&self) -> usize { + self.queued_bytes + } + + /// Register a task to be woken when a window slot frees or the send + /// queue drains. Used by the `tower` Service impl from `poll_ready`. + /// Costs a scan of the tasks already parked, to skip one that is. + /// + /// Deduplication relies on `Waker::will_wake`. An executor that polls + /// a pending task again without waking it, with a waker that is not + /// `will_wake`-equal to the last, would add one entry per such poll + /// until the next wake clears the list; polling without a wake breaks + /// the executor contract, and one task's wakers compare equal in the + /// common executors, so the list stays one entry per producer. + #[cfg(feature = "tower")] + pub(crate) fn register_enqueue_waker(&mut self, waker: &Waker) { + if !self.enqueue_wakers.iter().any(|w| w.will_wake(waker)) { + self.enqueue_wakers.push(waker.clone()); + } + } + + /// Register a waker to be notified when a new PDU becomes available. + /// Used by the `tower` Stream impl from `poll_next`. + #[cfg(feature = "tower")] + pub(crate) fn register_drain_waker(&mut self, waker: Waker) { + self.drain_waker = Some(waker); + } + + /// Queue a bundle for transmission, returning the ID under which + /// [`Pdu::carried`] will report it. + /// + /// If the bundle fits in a single PDU (as a Bundle message), it is emitted + /// without segmentation and the ID is a [`SendKind::Message`]. + /// Otherwise, it is split into Transfer Segment and Transfer End + /// messages under a newly allocated transfer number, and the ID is a + /// [`SendKind::Transfer`]. + /// + /// Under [`BundleFraming::Bare`], a bundle that fits in a PDU, carries no + /// caller hints, and begins with a bundle-reserved byte is instead queued + /// as a bare frame and the ID is a [`SendKind::Bare`]. + /// + /// Caller hints from `options` ride on the Bundle message or the first + /// segment (hints are transfer-scoped, Section 7.2); the sender derives + /// and attaches the Bundle Length hint itself when segmenting. + /// + /// An empty `data` is rejected with [`Error::Empty`]: it cannot be + /// a valid bundle (Section 8.1), and nothing is queued. + /// + /// Segments are copied out of `data` into the PDUs that carry them, + /// and `data` is released as its last segment is packed. + pub fn enqueue(&mut self, data: Bytes, options: SendOptions) -> Result { + let bare_ok = frame_kind(&data) != FrameKind::BtpuPdu; + let len = data.len(); + let id = match self.plan(len, options, bare_ok)? { + Planned::Whole { id, hints } => { + self.queue_whole(id, hints, data); + id + } + Planned::Transfer(mut t) => { + t.push(data); + let id = SendId(Repr::Transfer(t.transfer_number)); + self.queue(QueueEntry::Transfer(t)); + id + } + }; + self.queued_bytes += len; + Ok(id) + } + + /// Begin a bundle of `total_len` bytes whose bytes are pushed later, + /// returning the handle they are pushed through. + /// + /// The bundle is framed as [`Self::enqueue`] would frame `total_len` + /// bytes with these `options`, and is refused for the same reasons + /// before anything is queued; a segmented bundle takes its window slot + /// here. Under [`BundleFraming::Bare`], `begin` cannot see the first + /// byte, so a fitting bundle without caller hints is begun as a + /// [`SendKind::Bare`] and its first chunk must start with a + /// bundle-reserved byte (see [`Error::NotABundle`]). + /// + /// Push the bytes with [`Self::push`] and end the bundle with + /// [`Self::finish`], or abandon it with [`Self::cancel`]. A bundle that + /// fits one PDU joins the queue when its last byte is pushed, so such + /// bundles go out in the order they are completed. A segmented bundle + /// is queued here, and each segment goes out once its bytes are pushed + /// (see [`SegmentCutStrategy`]); until then the bundles queued behind it + /// go ahead of it. + pub fn begin(&mut self, total_len: usize, options: SendOptions) -> Result { + let id = match self.plan(total_len, options, true)? { + Planned::Whole { id, hints } => { + self.assembling.push(Assembly { + id, + hints, + chunks: SmallVec::new(), + len: 0, + }); + id + } + Planned::Transfer(t) => { + let id = SendId(Repr::Transfer(t.transfer_number)); + self.queue(QueueEntry::Transfer(t)); + id + } + }; + Ok(SendHandle { + id, + total_len, + pushed: 0, + }) + } + + /// Push the next bytes of the bundle `handle` was begun for. + /// + /// The chunk is held, not copied, until the PDUs carrying it are + /// packed. An empty chunk does nothing. + /// + /// # Errors + /// + /// Nothing is pushed if the chunk would take the bundle past its + /// length ([`Error::Overrun`]), the bundle has been cancelled or `handle` + /// is another sender's ([`Error::NotInProgress`]), or the first chunk of + /// a bare bundle frame is not a bundle ([`Error::NotABundle`]). The + /// bundle stays as it was in each case. + pub fn push(&mut self, handle: &mut SendHandle, chunk: Bytes) -> Result<()> { + if chunk.is_empty() { + return Ok(()); + } + let len = chunk.len(); + let left = handle.total_len - handle.pushed; + if len > left { + return Err(Error::Overrun { + total_len: handle.total_len, + pushed: handle.pushed, + chunk: len, + }); + } + + if let SendId(Repr::Transfer(transfer_number)) = handle.id { + let (at, _) = self + .find_transfer(transfer_number) + .ok_or(Error::NotInProgress)?; + self.pending[at] + .as_transfer_mut() + .expect("find_transfer names a transfer") + .push(chunk); + self.wake_drain(); + } else { + let at = self + .assembling + .iter() + .position(|a| a.id == handle.id) + .ok_or(Error::NotInProgress)?; + let assembly = &mut self.assembling[at]; + if assembly.len == 0 + && matches!(assembly.id, SendId(Repr::Bare(_))) + && frame_kind(&chunk) == FrameKind::BtpuPdu + { + return Err(Error::NotABundle); + } + assembly.chunks.push(chunk); + assembly.len += len; + if len == left { + let Assembly { + id, hints, chunks, .. + } = self.assembling.swap_remove(at); + self.queue_whole(id, hints, gather(chunks, handle.total_len)); + } + } + self.queued_bytes += len; + handle.pushed += len; + Ok(()) + } + + /// End the bundle `handle` was begun for, returning its ID. + /// + /// The bundle's last bytes may still be queued; [`Pdu::carried`] reports + /// when they leave. + /// + /// # Errors + /// + /// [`Error::Underrun`] if fewer than `total_len` bytes were pushed. The + /// bundle is then cancelled, as by [`Self::cancel`], since its missing + /// bytes can never arrive. + pub fn finish(&mut self, handle: SendHandle) -> Result { + if handle.pushed < handle.total_len { + let error = Error::Underrun { + total_len: handle.total_len, + pushed: handle.pushed, + }; + self.cancel(handle); + return Err(error); + } + Ok(handle.id) + } + + /// Decide how a bundle of `total_len` bytes is framed, taking its ID + /// and, if it is segmented, its window slot. `bare_ok` is whether its + /// first byte allows a bare bundle frame. + fn plan(&mut self, total_len: usize, options: SendOptions, bare_ok: bool) -> Result { + if total_len == 0 { + return Err(Error::Empty); + } + let pdu_size = self.pdu_size.get(); + + // The sender owns the Bundle Length hint, so a caller-supplied one + // is discarded. + let mut hints = options.hints; + hints.remove(HintType::BUNDLE_LENGTH); + + if self.bare_bundles() + && hints.is_empty() + && (self.padded_len()..=pdu_size).contains(&total_len) + && bare_ok + { + // A bare frame needs no header, so it may use the whole PDU. + // One shorter than the floor cannot be padded, since padding + // after it would read as bundle bytes, so it goes out as a + // Bundle Message instead. + return Ok(Planned::Whole { + id: SendId(Repr::Bare(self.take_bundle_id())), + hints: Vec::new(), + }); + } + + if HEADER_SIZE + hints.encoded_len() + total_len <= pdu_size { + // Fits in a single Bundle message. + return Ok(Planned::Whole { + id: SendId(Repr::Message(self.take_bundle_id())), + hints: hints.into_vec(), + }); + } + + // Segment the bundle. Size the segments before taking a transfer + // number so a PDU too small to carry them never touches the window. + let capacity = pdu_size.saturating_sub(SEGMENT_FRAMING); + hints.insert(HintItem::BundleLength(total_len as u64)); + let first_segment_framing = SEGMENT_FRAMING + hints.encoded_len(); + let first_capacity = pdu_size.saturating_sub(first_segment_framing); + + if capacity == 0 || first_capacity == 0 { + return Err(Error::PduTooSmall { + required: first_segment_framing + 1, + pdu_size, + }); + } + // Every segment between the first and the last carries at least + // half of `capacity` (see `QueuedTransfer::chunk_for`), and the + // last index must fit the 32-bit field (Section 8.2) short of + // `u32::MAX`, which the receiver treats as a count of segments it + // can never hold. + let last_index = 1 + total_len / capacity.div_ceil(2); + if u32::try_from(last_index).map_or(true, |i| i == u32::MAX) { + return Err(Error::TooManySegments { + len: total_len, + pdu_size, + }); + } + + let transfer_number = self.allocator.allocate()?; + Ok(Planned::Transfer(QueuedTransfer::new( + transfer_number, + total_len, + hints.into_vec(), + first_capacity, + capacity, + self.segment_cut_strategy, + ))) + } + + /// Queue a whole bundle as `plan` framed it. The caller counts its + /// bytes. + fn queue_whole(&mut self, id: SendId, hints: Vec, data: Bytes) { + let entry = match id { + SendId(Repr::Bare(id)) => QueueEntry::BareBundle { id, data }, + _ => self.message_entry(id, Message::Bundle { hints, data }), + }; + self.queue(entry); + } + + /// Abandon a bundle the sender has not finished emitting. + /// + /// For a [`SendKind::Transfer`], the transfer's window slot is + /// freed and its segments not yet packed are discarded. If any had + /// already been emitted, a Transfer Cancel message is queued at the + /// front, ahead of everything already waiting, so the receiver discards + /// what it holds (Section 4.2) as soon as the next PDU arrives; if none + /// had, the receiver never learned of the transfer and no Cancel is + /// sent. + /// + /// A [`SendKind::Message`] or [`SendKind::Bare`] still + /// in the queue is removed from it. Such a bundle travels whole, so the + /// receiver has seen none of it and nothing is sent in its place. + /// + /// A bundle begun with [`Self::begin`] may be cancelled at any point, + /// by its handle or its ID, and the chunks pushed for it are dropped. + /// + /// Returns whether the bundle was cancelled. Returns `false`, changing + /// nothing, if the sender has nothing left to emit for `id`: it was + /// never issued, was already cancelled, or its last bytes have been + /// packed, in which case a PDU has listed it with + /// [`Carried::completes`] set. + /// + /// Costs a scan of the send queue, and for a transfer a scan of the + /// outstanding transfer numbers as well (at most the window size). + pub fn cancel(&mut self, id: impl Into) -> bool { + let id = id.into(); + let cancelled = match id.0 { + Repr::Transfer(transfer_number) => self.cancel_transfer(transfer_number), + Repr::Message(_) | Repr::Bare(_) => match self.find_unsegmented(id) { + Some(Unsegmented::Queued(at)) => { + self.dequeue(at); + true + } + Some(Unsegmented::Assembling(at)) => { + self.queued_bytes -= self.assembling.swap_remove(at).len; + true + } + None => false, + }, + }; + if cancelled { + // A window slot or queued bytes freed. No drain wake is + // needed: cancelling only ever removes or replaces entries. + self.wake_enqueue(); + } + cancelled + } + + /// The [`SendKind::Transfer`] case of [`Self::cancel`]. + fn cancel_transfer(&mut self, transfer_number: u32) -> bool { + if !self.allocator.release(transfer_number) { + return false; + } + + // An outstanding transfer is one queue entry until its End is + // packed. Segments are cut in index order, so a transfer that has + // not started means nothing of it has been emitted. + let at = self.find_transfer(transfer_number).map(|(at, _)| at); + let nothing_emitted = match at.and_then(|at| self.dequeue(at)) { + Some(QueueEntry::Transfer(t)) => !t.started(), + _ => false, + }; + if !nothing_emitted { + // At the front, so the receiver can drop what it holds without + // waiting out the backlog. Emitting a smaller number early + // cannot raise the greatest emitted, so Section 5 still holds. + self.pending.push_front(QueueEntry::Cancel(transfer_number)); + } + true + } + + /// Pack pending messages into a PDU of at most `pdu_size` bytes. + /// + /// Returns `None` if nothing is ready: the queue is empty, or holds + /// only transfers whose next segments wait on bytes not yet pushed (and + /// with [`NextPduOptions::flush`], none of them has bytes buffered). + /// Entries are packed in queue order, but a transfer that cannot supply + /// its next segment, for want of bytes or of room, is passed over, so + /// a bundle still being pushed does not hold up the queue behind it. + /// Under [`LinkFraming::FixedSize`] the PDU is padded to exactly `pdu_size` + /// bytes; under [`LinkFraming::Variable`] it holds the packed messages, + /// padded up to `min_pdu_len` if they are shorter. A queued bare + /// bundle frame, never shorter than `min_pdu_len`, is returned as-is, + /// on its own, sharing the enqueued buffer. + /// + /// [`Pdu::carried`] names every bundle with bytes in the PDU and flags + /// those whose last bytes it carries. A CLA that reports per-bundle + /// outcomes can treat a bundle as sent once the PDU flagging it + /// [`Carried::completes`] is written, and as failed if it + /// [`Self::cancel`]s it first (a cancelled bundle is never flagged) or + /// a write of any PDU carrying it fails. [`Self::next_pdu_into`] does + /// the same into a reused list. + /// + /// The list starts inline, so it allocates only for a PDU carrying + /// more than [`CarriedList::INLINE`] bundles. + /// + /// Packing a Transfer End releases its transfer's window slot (see + /// [`Sender`]). + pub fn next_pdu(&mut self) -> Option { + self.next_pdu_with(NextPduOptions::default()) + } + + /// [`Self::next_pdu`], packed as `options` asks. + pub fn next_pdu_with(&mut self, options: NextPduOptions) -> Option { + let mut carried = CarriedList::new(); + let data = self.next_pdu_into_with(&mut carried, options)?; + Some(Pdu { data, carried }) + } + + /// [`Self::next_pdu`], writing the carried bundles into `carried` + /// rather than a new list, so a caller draining a busy link allocates + /// nothing for the list once it has grown to the largest PDU seen, or + /// nothing at all if created with + /// [`CarriedList::with_capacity`]`(pdu_size / 32 + window_size)` (see + /// [`CarriedList`] for that bound). + /// + /// `carried` is cleared first, keeping its heap buffer if it has one, + /// so after the call it holds exactly this PDU's entries, and is left + /// empty when `None` is returned. + pub fn next_pdu_into(&mut self, carried: &mut CarriedList) -> Option { + self.next_pdu_into_with(carried, NextPduOptions::default()) + } + + /// [`Self::next_pdu_into`], packed as `options` asks. + pub fn next_pdu_into_with( + &mut self, + carried: &mut CarriedList, + options: NextPduOptions, + ) -> Option { + carried.clear(); + let pdu = self.pack(carried, options)?; + + // Draining frees send-queue capacity (and possibly a window slot); + // wake any task parked on `poll_ready`. + self.wake_enqueue(); + + Some(pdu) + } + + /// Pack the queue's next PDU, recording the bundles it carries in + /// `carried`, or return `None` if nothing is ready. + /// + /// The loop is driven by the queue, not by a count planned up front: + /// [`Self::next_source`] names the entry that supplies each message, + /// and an entry leaves the queue only once its last message is packed. + /// The first message always goes in, so every PDU makes progress. + /// Nothing larger than an empty PDU is ever queued (a Bundle Message is + /// only made for a bundle that fits with its hints, `plan` sizes a + /// transfer's full segments to fit or refuses the bundle, and a + /// Transfer Cancel is smaller than any segment of the transfer it + /// cancels), but were it otherwise the message would go out as one + /// oversized PDU rather than stall the queue. + fn pack(&mut self, carried: &mut CarriedList, options: NextPduOptions) -> Option { + let pdu_size = self.pdu_size.get(); + if self.next_source(0, pdu_size).is_none() + && !(options.flush && self.flush_source(0, 0, pdu_size).is_some()) + { + return None; + } + let padded_len = self.padded_len(); + let mut buf = BytesMut::with_capacity(if padded_len == pdu_size { + pdu_size + } else { + self.planned_len(pdu_size).max(padded_len) + }); + while let Some(at) = self.next_source(buf.len(), pdu_size) { + let consumed = match &mut self.pending[at] { + QueueEntry::BareBundle { id, data } => { + // Chosen only for an empty PDU; returned as-is, alone. + carried.push(Carried { + id: SendId(Repr::Bare(*id)), + completes: true, + }); + let data = data.clone(); + self.dequeue(at); + return Some(data); + } + QueueEntry::Message { id, message } => { + carried.push(Carried { + id: *id, + completes: true, + }); + encode_queued(message, &mut buf); + true + } + QueueEntry::Cancel(transfer_number) => { + let transfer_number = *transfer_number; + encode_queued(&Message::TransferCancel { transfer_number }, &mut buf); + true + } + QueueEntry::Transfer(t) => { + let len = t + .next_chunk(pdu_size.saturating_sub(buf.len())) + .expect("next_source checked that it fits"); + self.queued_bytes -= t.cut(len, &mut buf); + let completes = t.finished(); + // A transfer supplies one segment to a PDU: the cut + // either fills the room, ends the transfer, or under + // `SegmentCutStrategy::Half` takes every byte buffered. + carried.push(Carried { + id: SendId(Repr::Transfer(t.transfer_number)), + completes, + }); + if completes { + // The transfer's End is packed; nothing further will + // be emitted for it, so its slot is free. + self.allocator.release(t.transfer_number); + } + completes + } + }; + if consumed { + self.dequeue(at); + } + } + if options.flush { + let mut from = 0; + while let Some((at, len)) = self.flush_source(from, buf.len(), pdu_size) { + let t = self.pending[at] + .as_transfer_mut() + .expect("flush_source names transfers"); + self.queued_bytes -= t.cut(len, &mut buf); + // A flushed segment never ends its transfer, and a transfer + // packed above was left no room or no bytes, so this is + // its only entry. + carried.push(Carried { + id: SendId(Repr::Transfer(t.transfer_number)), + completes: false, + }); + from = at + 1; + } + } + pad_pdu(&mut buf, padded_len); + Some(buf.freeze()) + } + + /// The queue position of the entry that supplies the next message of a + /// PDU holding `used` bytes, or `None` to end the PDU. + /// + /// The first entry in queue order that can supply its next message + /// (see [`QueueEntry::supplies`]) does so. A transfer that cannot, + /// because its next segment waits on bytes not yet pushed or does not + /// fit the room left, is passed over, interleaving transfers as Section + /// 4.1 permits; any other entry that cannot ends the PDU, so bundles + /// are not reordered to fill it. Neither condition can clear while a + /// PDU is packed, so the entries chosen for one PDU advance through the + /// queue. Emission order cannot break Section 5, because the allocator + /// keeps every outstanding number within the window of the newest + /// allocated. + /// + /// Costs a scan of the queued transfers ahead of the entry chosen, at + /// most the window size. This is the choice a priority scheduler + /// would make. + fn next_source(&self, used: usize, pdu_size: usize) -> Option { + let room = pdu_size.saturating_sub(used); + for (at, entry) in self.pending.iter().enumerate() { + if entry.supplies(room, used == 0) { + return Some(at); + } + // Only a transfer is passed over. + entry.as_transfer()?; + } + None + } + + /// The queue position, at or after `from`, of the next transfer a flush + /// cuts into a PDU holding `used` bytes, and the data bytes it cuts + /// (see [`QueuedTransfer::flush_chunk`]), or `None` to end the PDU. + /// Like [`Self::next_source`], it stops at the first entry that is not a + /// transfer. + fn flush_source(&self, from: usize, used: usize, pdu_size: usize) -> Option<(usize, usize)> { + let room = pdu_size.saturating_sub(used); + for (at, entry) in self.pending.iter().enumerate().skip(from) { + let t = entry.as_transfer()?; + if let Some(len) = t.flush_chunk(room, pdu_size) { + return Some((at, len)); + } + } + None + } + + /// The encoded size of the messages the next PDU will pack, so a + /// [`LinkFraming::Variable`] buffer is allocated once at the right size. + /// It mirrors [`Self::next_source`] but only sizes the buffer: were the + /// two to disagree, the buffer would grow or carry spare capacity, and + /// the PDU would be the same. + fn planned_len(&self, pdu_size: usize) -> usize { + let mut total = 0; + for entry in &self.pending { + let len = match entry { + QueueEntry::Transfer(t) => { + // A transfer that cannot supply is passed over. + total += t.planned_len(pdu_size - total); + continue; + } + QueueEntry::Message { message, .. } => encoded_message_len(message), + QueueEntry::Cancel(transfer_number) => cancel_message_len(*transfer_number), + QueueEntry::BareBundle { .. } => return total, + }; + if total + len > pdu_size { + return total; + } + total += len; + } + total + } + + /// Returns `true` if there are messages pending for transmission. + pub fn has_pending(&self) -> bool { + !self.pending.is_empty() + } + + /// Whether the sender still has bytes of the bundle `id` names to + /// emit: what [`Self::cancel`] would act on. A transfer is outstanding, + /// holding its window slot, until its End is packed or it is cancelled; + /// a Bundle Message or bare frame until it is packed or cancelled. + /// + /// Costs a scan of the outstanding transfer numbers (at most the window + /// size) for a transfer, and of the send queue otherwise. + pub fn is_outstanding(&self, id: SendId) -> bool { + match id.0 { + Repr::Transfer(transfer_number) => self.allocator.is_outstanding(transfer_number), + Repr::Message(_) | Repr::Bare(_) => self.find_unsegmented(id).is_some(), + } + } + + /// Where the unsegmented bundle `id` is held, if it is. + fn find_unsegmented(&self, id: SendId) -> Option { + if let Some(at) = self + .pending + .iter() + .position(|e| e.unsegmented_id() == Some(id)) + { + return Some(Unsegmented::Queued(at)); + } + self.assembling + .iter() + .position(|a| a.id == id) + .map(Unsegmented::Assembling) + } + + /// Wrap a Bundle Message for the queue, checking that it fits an empty + /// PDU (see [`Self::pack`]). + fn message_entry(&self, id: SendId, message: Message) -> QueueEntry { + debug_assert!( + encoded_message_len(&message) <= self.pdu_size.get(), + "message larger than the PDU" + ); + QueueEntry::Message { id, message } + } + + /// Take the next [`SendKind::Message`] or + /// [`SendKind::Bare`] counter value. + fn take_bundle_id(&mut self) -> u32 { + let id = self.next_bundle_id; + self.next_bundle_id = id.wrapping_add(1); + id + } + + /// Set the next unsegmented bundle ID, so a test can reach the wrap + /// without queueing 2³² bundles. + #[cfg(test)] + fn set_next_bundle_id(&mut self, id: u32) { + self.next_bundle_id = id; + } + + /// The length every PDU is padded up to. + fn padded_len(&self) -> usize { + let pdu_size = self.pdu_size.get(); + match self.link_framing { + LinkFraming::FixedSize => pdu_size, + LinkFraming::Variable { min_pdu_len, .. } => min_pdu_len.min(pdu_size), + } + } + + /// Whether fitting bundles may be emitted as bare bundle frames. + fn bare_bundles(&self) -> bool { + matches!( + self.link_framing, + LinkFraming::Variable { + bundle_framing: BundleFraming::Bare, + .. + } + ) + } +} + +/// How [`Sender::plan`] frames a bundle. +enum Planned { + /// A Bundle Message or bare bundle frame, by the ID's variant. + Whole { id: SendId, hints: Vec }, + /// A segmented bundle, its window slot taken. + Transfer(QueuedTransfer), +} + +/// The pushed chunks of a whole bundle as one buffer, `len` bytes long. +fn gather(mut chunks: SmallVec<[Bytes; 1]>, len: usize) -> Bytes { + if chunks.len() == 1 { + return chunks.pop().expect("one chunk"); + } + let mut data = BytesMut::with_capacity(len); + for chunk in &chunks { + data.extend_from_slice(chunk); + } + data.freeze() +} + +/// Append a queued message to a PDU being packed. +fn encode_queued(message: &Message, buf: &mut BytesMut) { + encode_message(message, buf).expect("queued messages are validated at enqueue"); +} + +#[cfg(test)] +mod tests { + use alloc::vec; + + use super::*; + use crate::codec::{decode_pdu, hint::HintValue}; + + #[test] + fn unsegmented_ids_wrap_across_message_and_bare() { + let mut s = Sender::new( + SenderConfig { + link_framing: LinkFraming::variable(BundleFraming::Bare), + ..SenderConfig::default() + }, + 0, + ); + s.set_next_bundle_id(u32::MAX); + // 0x9F, a bundle's CBOR array head, lets a hint-free bundle go out + // as a bare frame; the hinted one must be a Bundle Message. + let bare = Bytes::from_static(&[0x9F, 0xFF]); + let hinted = SendOptions { + hints: Hints::from_iter([HintItem::Unknown { + hint_type: HintType::new(0x40).unwrap(), + value: HintValue::new(Bytes::from_static(&[1])).unwrap(), + }]), + }; + assert_eq!( + s.enqueue(bare.clone(), SendOptions::default()), + Ok(SendId(Repr::Bare(u32::MAX))) + ); + assert_eq!( + s.enqueue(bare.clone(), hinted), + Ok(SendId(Repr::Message(0))) + ); + assert_eq!( + s.enqueue(bare, SendOptions::default()), + Ok(SendId(Repr::Bare(1))) + ); + } + + #[test] + fn oversized_queued_message_goes_out_alone_and_the_queue_moves_on() { + const PDU: usize = 64; + let mut s = Sender::new( + SenderConfig { + pdu_size: PduSize::new(PDU).unwrap(), + link_framing: LinkFraming::variable(BundleFraming::Message), + ..SenderConfig::default() + }, + 0, + ); + // Break the invariant `enqueue` keeps, bypassing `message_entry`. + let oversized = Message::Bundle { + hints: Vec::new(), + data: Bytes::from(vec![0; 2 * PDU]), + }; + let oversized_len = encoded_message_len(&oversized); + s.pending.push_back(QueueEntry::Message { + id: SendId(Repr::Message(100)), + message: oversized, + }); + s.queued_bytes += 2 * PDU; + let small = s + .enqueue(Bytes::from_static(&[1, 2, 3]), SendOptions::default()) + .unwrap(); + + let pdu = s.next_pdu().unwrap(); + assert_eq!(pdu.data.len(), oversized_len); + assert_eq!( + &pdu.carried[..], + &[Carried { + id: SendId(Repr::Message(100)), + completes: true, + }] + ); + let pdu = s.next_pdu().unwrap(); + assert_eq!( + &pdu.carried[..], + &[Carried { + id: small, + completes: true, + }] + ); + assert_eq!(s.next_pdu(), None); + } + + // A transfer of `total_len` patterned bytes, of which those from + // `offset` on are pushed, its cursor at `offset` and segment `index`. + fn transfer_at(total_len: usize, offset: usize, index: u32) -> QueuedTransfer { + let mut t = QueuedTransfer::new(7, total_len, Vec::new(), 40, 40, SegmentCutStrategy::Full); + t.push((offset..total_len).map(|i| i as u8).collect()); + t.offset = offset; + t.next_index = index; + t + } + + #[test] + fn a_last_segment_takes_any_tail_it_fits_and_a_cut_keeps_its_length() { + // Five bytes remain, well under half a segment, and still fit. + let mut t = transfer_at(100, 95, 2); + assert_eq!(t.chunk_for(95, 2, SEGMENT_FRAMING + 5), Some(5)); + assert_eq!(t.chunk_for(95, 2, SEGMENT_FRAMING + 4), None); + + // A segment with copies left goes out at its first copy's length or + // not at all, whatever the room. + t = transfer_at(100, 50, 2); + t.cut = Some(Cut { + len: 25, + copies_left: 1, + }); + assert_eq!(t.next_chunk(SEGMENT_FRAMING + 40), Some(25)); + assert_eq!(t.next_chunk(SEGMENT_FRAMING + 24), None); + let mut buf = BytesMut::new(); + assert_eq!(t.cut(25, &mut buf), 25); + let Some(Ok(Message::TransferSegment(m))) = decode_pdu(buf.freeze()).next() else { + panic!("expected a segment"); + }; + assert_eq!(m.segment_index, 2); + assert_eq!(m.data, (50..75).map(|i| i as u8).collect::>()); + assert_eq!((t.offset, t.next_index, t.cut.is_none()), (75, 3, true)); + assert_eq!(t.buffered, 25); + } + + #[test] + fn a_segment_waits_for_all_of_its_bytes() { + let mut t = QueuedTransfer::new(7, 100, Vec::new(), 40, 40, SegmentCutStrategy::Full); + assert_eq!(t.next_chunk(SEGMENT_FRAMING + 40), None); + // Enough for a half-PDU tail segment, but the segment would be cut + // to what has arrived rather than to the room. + t.push(Bytes::from(vec![0; 30])); + assert_eq!(t.next_chunk(SEGMENT_FRAMING + 40), None); + assert_eq!(t.next_chunk(SEGMENT_FRAMING + 30), Some(30)); + // A segment spanning two pushed chunks is copied from both, and + // only the bytes it carries are released. + t.push(Bytes::from(vec![1; 30])); + let mut buf = BytesMut::new(); + assert_eq!(t.cut(40, &mut buf), 40); + assert_eq!(t.buffered, 20); + assert_eq!(t.chunks.len(), 1); + let Some(Ok(Message::TransferSegment(m))) = decode_pdu(buf.freeze()).next() else { + panic!("expected a segment"); + }; + assert_eq!(&m.data[..], [[0; 30].as_slice(), &[1; 10]].concat()); + } + + #[test] + fn a_flush_is_refused_if_the_rest_could_need_index_u32_max() { + // Ten of 100 bytes pushed: a flush cuts all ten, leaving 90 bytes, + // which could take five more segments of at least half of 40. + let mut t = QueuedTransfer::new(7, 100, Vec::new(), 40, 40, SegmentCutStrategy::Full); + t.push(Bytes::from(vec![0; 10])); + t.next_index = u32::MAX - 6; + assert_eq!( + t.flush_chunk(SEGMENT_FRAMING + 40, SEGMENT_FRAMING + 40), + Some(10) + ); + // Room for fewer bytes than are pushed cuts what fits. + assert_eq!( + t.flush_chunk(SEGMENT_FRAMING + 4, SEGMENT_FRAMING + 40), + Some(4) + ); + assert_eq!(t.flush_chunk(SEGMENT_FRAMING, SEGMENT_FRAMING + 40), None); + // One index later the last segment could be u32::MAX. + t.next_index = u32::MAX - 5; + assert_eq!( + t.flush_chunk(SEGMENT_FRAMING + 40, SEGMENT_FRAMING + 40), + None + ); + } +} diff --git a/btpu/src/service.rs b/btpu/src/service.rs new file mode 100644 index 000000000..266317dec --- /dev/null +++ b/btpu/src/service.rs @@ -0,0 +1,77 @@ +//! Tower [`Service`] and [`Stream`] implementations for [`Sender`] and +//! [`Receiver`], enabled by the `tower` feature: thin wrappers over the +//! synchronous core, whose futures are [`core::future::Ready`]. The +//! backpressure and sharing contract is documented on [`Sender`] under +//! Concurrency; [`Receiver`]'s service is always ready and never fails. + +use alloc::vec::Vec; +use core::{ + convert::Infallible, + future::{Ready, ready}, + pin::Pin, + task::{Context, Poll}, +}; + +use bytes::Bytes; +use futures_core::Stream; +use tower::Service; + +use crate::{ + receiver::{Receiver, ReceiverEvent}, + // Aliased: distinguishes it from the receiver's and codec's `Error`s. + sender::{Error as SenderError, Pdu, SendId, SendRequest, Sender}, +}; + +impl Service for Receiver { + type Response = Vec; + type Error = Infallible; + type Future = Ready>; + + fn poll_ready(&mut self, _: &mut Context<'_>) -> Poll> { + Poll::Ready(Ok(())) + } + + fn call(&mut self, pdu: Bytes) -> Self::Future { + ready(Ok(self.receive_pdu(pdu))) + } +} + +impl Service for Sender { + type Response = SendId; + type Error = SenderError; + type Future = Ready>; + + fn poll_ready(&mut self, cx: &mut Context<'_>) -> Poll> { + // Two admission gates: transfer-window capacity (segmented bundles + // allocate a number in `call`) and send-queue capacity. The queue + // gate is what bounds unsegmented bundles, which never take a + // window slot. + if self.is_window_available() && !self.is_send_queue_full() { + Poll::Ready(Ok(())) + } else { + self.register_enqueue_waker(cx.waker()); + Poll::Pending + } + } + + fn call(&mut self, request: SendRequest) -> Self::Future { + ready(self.enqueue(request.data, request.options)) + } +} + +impl Stream for Sender { + type Item = Pdu; + + fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll> { + match self.next_pdu() { + Some(pdu) => Poll::Ready(Some(pdu)), + None => { + // The sender is a perpetual source: yielding Ready(None) + // would mean "stream finished forever," which it isn't. + // Park until enqueue or push supplies more. + self.register_drain_waker(cx.waker().clone()); + Poll::Pending + } + } + } +} diff --git a/btpu/src/transfer.rs b/btpu/src/transfer.rs new file mode 100644 index 000000000..25e23fac8 --- /dev/null +++ b/btpu/src/transfer.rs @@ -0,0 +1,675 @@ +//! Transfer numbers and the Section 5 transfer window: the receiver's +//! acceptance test and the sender's number allocation. + +use alloc::collections::VecDeque; +use core::{fmt, num::NonZeroU16}; + +/// Shorthand for results whose error is [`enum@Error`] unless stated. +pub type Result = core::result::Result; + +/// Errors from transfer number allocation. +#[derive(Debug, Clone, PartialEq, Eq, thiserror::Error)] +pub enum Error { + /// The sender's transfer window is full: the next transfer number would + /// push the oldest outstanding transfer out of the window. + #[error("Transfer window full (size {window_size})")] + WindowFull { + /// The configured window size. + window_size: WindowSize, + }, +} + +/// A validated transfer window size (Section 5: 4..=4095). +/// +/// Construct via [`WindowSize::new`] or [`TryFrom`], which enforce the +/// range invariant at the edge; every consumer of a `WindowSize` can then +/// rely on it. +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] +#[repr(transparent)] +#[cfg_attr( + feature = "serde", + derive(serde::Serialize, serde::Deserialize), + serde(try_from = "u16", into = "u16") +)] +pub struct WindowSize(NonZeroU16); + +impl WindowSize { + /// What the value configures, as error messages name it. + const NAME: &str = "window size"; + + /// Minimum allowed transfer window size (Section 5). + pub const MIN: Self = Self(NonZeroU16::new(4).unwrap()); + + /// Maximum allowed transfer window size (Section 5: less than 2^12). + pub const MAX: Self = Self(NonZeroU16::new(4095).unwrap()); + + /// The RECOMMENDED window size (Section 5), 16. The draft marks the + /// value as provisional ("needs discussing by the WG"), so it may + /// change in a later revision. + pub const DEFAULT: Self = Self(NonZeroU16::new(16).unwrap()); + + /// Returns the window size for `transfers`, or `None` if it is outside + /// [`MIN`](Self::MIN)..=[`MAX`](Self::MAX). + pub const fn new(transfers: u16) -> Option { + match NonZeroU16::new(transfers) { + Some(n) if transfers >= Self::MIN.get() && transfers <= Self::MAX.get() => { + Some(Self(n)) + } + _ => None, + } + } + + /// Returns the window size as a plain integer. + pub const fn get(self) -> u16 { + self.0.get() + } +} + +impl Default for WindowSize { + fn default() -> Self { + Self::DEFAULT + } +} + +config_newtype!(WindowSize: u16); + +/// A transfer admitted by a receive window, as events name it. +/// +/// Transfer numbers wrap at 2³², so a number alone can name two transfers +/// a receiver saw at different times. An id extends the number to a +/// 64-bit serial that keeps counting where the `u32` wraps and across +/// [`Receiver::reset`](crate::receiver::Receiver::reset), so within one +/// [`Receiver`](crate::receiver::Receiver) an id names exactly one +/// transfer for the receiver's lifetime: a kept id never names a later +/// transfer that reuses its number. Ids order transfers oldest first. +/// +/// Only the receiver's window makes ids, and only for numbers inside it; +/// there is no public constructor. An id is scoped to the receiver that +/// made it: ids +/// from two receivers are not told apart, so a caller holding several +/// keys them by receiver first. +/// +/// The serial's low 32 bits are the transfer number itself, so an id is +/// 8 bytes, which matters on a constrained receiver holding up to the +/// window size of them across its in-progress and closed transfers. +#[derive(Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] +pub struct TransferId { + serial: u64, +} + +const _: () = assert!(size_of::() == 8); + +impl TransferId { + /// The transfer number as it appears on the wire. + pub fn transfer_number(self) -> u32 { + // Truncation is the point: the low 32 bits are the wire number. + self.serial as u32 + } +} + +/// Shows the transfer number, which is what a reader can match against +/// the wire; the serial's high bits only order ids within one window. +impl fmt::Debug for TransferId { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.debug_tuple("TransferId") + .field(&self.transfer_number()) + .finish() + } +} + +/// The outcome of [`TransferWindow::admit`]. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) enum Admission { + /// Ahead of everything seen; the window advanced to it. + New(TransferId), + /// Inside the window. + InProgress(TransferId), + /// Outside the window; to be ignored. + OutsideWindow, +} + +/// Receiver-side sliding transfer window. +/// +/// Implements the algorithm from Section 5 (Figure 2) of draft-ietf-dtn-btpu. +/// +/// What the acceptance test makes of a restarted sender is documented on +/// [`Receiver`](crate::receiver::Receiver) under Sender restarts. +#[derive(Debug, Clone)] +pub(crate) struct TransferWindow { + greatest: Option, + window_size: WindowSize, + /// The serial's high 32 bits for the first number admitted while + /// `greatest` is `None`. [`Self::reset`] moves it past every serial + /// issued, so ids stay unique across resets. + epoch: u64, +} + +impl TransferWindow { + /// Create a new transfer window. + pub fn new(window_size: WindowSize) -> Self { + Self { + greatest: None, + window_size, + // Not zero, so that stepping back a window from any first + // number stays positive. + epoch: 1, + } + } + + /// Forget the greatest transfer number seen: the next transfer number + /// received is accepted as new whatever its value. Ids keep counting + /// rather than restarting, so no id issued before the reset names a + /// transfer admitted after it. + pub fn reset(&mut self) { + if let Some(g) = self.greatest.take() { + // Two epochs on, not one: the new first number's window + // reaches up to 4095 serials below it, which must stay above + // `g`. An epoch per reset (plus one per 2^32 transfers) + // cannot exhaust 2^32 epochs in practice. + self.epoch = (g.serial >> 32) + 2; + } + } + + /// Classify a received transfer number, advancing the window if it is + /// new, and return its id unless it is outside the window. After a + /// [`Admission::New`] the caller expires the transfers now outside the + /// window (see [`Self::is_expired`]). + /// + /// The window's classification decides which way a number is extended + /// to its serial: forward for a new number, which Section 5 accepts up + /// to 2³¹ + W/2 ahead, beyond the half-space where serial-number + /// comparison alone is defined; backward, by less than W, for one in + /// progress. + pub fn admit(&mut self, t: u32) -> Admission { + if self.is_new_transfer(t) { + let serial = match self.greatest { + Some(g) => g.serial + u64::from(t.wrapping_sub(g.transfer_number())), + None => (self.epoch << 32) + u64::from(t), + }; + let id = TransferId { serial }; + self.greatest = Some(id); + Admission::New(id) + } else if let Some(id) = self.id(t) { + Admission::InProgress(id) + } else { + Admission::OutsideWindow + } + } + + /// The id of `t` if it is an in-progress transfer number: within the + /// window below the greatest seen, roll-over included. Section 5 + /// defines the set of in-progress transfers by this range alone, + /// whether or not any message for `t` has arrived. Never advances the + /// window. + /// + /// From the Section 5 Figure 2 pseudocode: + /// ```text + /// RETURN ((GREATEST - T + 2^32) MOD 2^32) < WINDOW_SIZE + /// ``` + pub fn id(&self, t: u32) -> Option { + let g = self.greatest?; + let behind = g.transfer_number().wrapping_sub(t); + (behind < u32::from(self.window_size.get())).then(|| TransferId { + serial: g.serial - u64::from(behind), + }) + } + + /// Whether the window has moved past `id`. Ids order oldest first, + /// so the expired ids of a map are the leading run for which this + /// holds. + pub fn is_expired(&self, id: TransferId) -> bool { + self.greatest + .is_some_and(|g| id.serial + u64::from(self.window_size.get()) <= g.serial) + } + + /// Whether `id` is not ahead of the greatest number seen: true of + /// every id the window has handed out, unless it has been reset + /// since and admitted nothing. + pub fn is_behind_greatest(&self, id: TransferId) -> bool { + self.greatest.is_some_and(|g| id <= g) + } + + /// Returns the greatest transfer number seen so far, if any. + #[cfg(test)] + pub fn greatest(&self) -> Option { + self.greatest.map(TransferId::transfer_number) + } + + /// Returns the configured window size. + pub fn window_size(&self) -> WindowSize { + self.window_size + } + + /// Check if `t` is a "new" transfer (greater than anything seen). + /// + /// From the Section 5 Figure 2 pseudocode: + /// ```text + /// IF T = GREATEST THEN RETURN FALSE + /// RETURN ((T - GREATEST + 2^32) MOD 2^32) < (2^32 / 2) + (WINDOW_SIZE / 2) + /// ``` + /// The first line is the `diff != 0` guard: a repeated message for the + /// greatest transfer is in progress, not new, so it must not re-trigger + /// window expiry. `WINDOW_SIZE / 2` is read as integer division, so an + /// odd window size rounds the margin down; the draft does not say. + fn is_new_transfer(&self, t: u32) -> bool { + match self.greatest { + None => true, + Some(g) => { + let diff = t.wrapping_sub(g.transfer_number()); + let half_space = u32::MAX / 2 + 1; // 2^31 + let half_window = u32::from(self.window_size.get()) / 2; + diff != 0 && diff < half_space + half_window + } + } + } +} + +/// Allocates monotonically increasing transfer numbers for the sender. +/// +/// Enforces the sender half of the Section 5 window rule: no emitted message +/// may carry a transfer number less than or equal to the greatest emitted +/// minus the window size. Since numbers are allocated sequentially, this is +/// a bound on the *span* of outstanding numbers, not their count: the next +/// number is refused while it would push the oldest outstanding transfer out +/// of the window, even if slots have been released out of order. Keeping +/// every outstanding transfer in-window is what lets a late or reordered +/// message for it still land inside the receiver's window. +#[derive(Debug, Clone)] +pub(crate) struct TransferNumberAllocator { + next: u32, + window_size: WindowSize, + /// Outstanding transfer numbers in allocation order; the front is the + /// oldest and anchors the window. Kept as a sequence rather than an + /// ordered set because allocation order, not numeric order, is what + /// survives the modulo 2^32 roll-over. + active: VecDeque, +} + +impl TransferNumberAllocator { + /// Create a new allocator that will allocate `initial_transfer_number` + /// first, then increment from there. + /// + /// The BTP-U spec recommends choosing this value unpredictably (typically + /// from a random source) to reduce the likelihood of a receiver mistaking + /// the new sender for an old one after a restart. + pub fn new(window_size: WindowSize, initial_transfer_number: u32) -> Self { + Self { + next: initial_transfer_number, + window_size, + active: VecDeque::new(), + } + } + + /// Whether [`Self::allocate`] would currently succeed. + /// + /// The next number is allocatable only if every outstanding transfer + /// stays within the window once it becomes the greatest, i.e. while the + /// oldest outstanding number is fewer than `window_size` numbers behind + /// it (modulo 2^32). + pub fn can_allocate(&self) -> bool { + match self.active.front() { + None => true, + Some(&oldest) => self.next.wrapping_sub(oldest) < u32::from(self.window_size.get()), + } + } + + /// Allocate the next transfer number. + /// + /// Returns [`Error::WindowFull`] if allocating it would push the oldest + /// outstanding transfer out of the window (see [`Self::can_allocate`]). + pub fn allocate(&mut self) -> Result { + if !self.can_allocate() { + return Err(Error::WindowFull { + window_size: self.window_size, + }); + } + let t = self.next; + self.next = self.next.wrapping_add(1); + self.active.push_back(t); + Ok(t) + } + + /// Release a completed or cancelled transfer. + /// + /// Returns `true` if `transfer_number` was outstanding and has been + /// released, `false` if it was never allocated or was already released; + /// a `false` release changes nothing. Only releasing the oldest + /// outstanding transfer lets the window advance. + /// + /// Costs a scan of the outstanding transfer numbers, at most the window + /// size. + pub fn release(&mut self, transfer_number: u32) -> bool { + match self.active.iter().position(|&t| t == transfer_number) { + Some(i) => { + self.active.remove(i); + true + } + None => false, + } + } + + /// Whether `transfer_number` is outstanding: allocated and not yet + /// released. Costs a scan of the outstanding transfer numbers, at most + /// the window size. + pub fn is_outstanding(&self, transfer_number: u32) -> bool { + self.active.contains(&transfer_number) + } + + /// Returns the number of transfers currently in progress. + pub fn in_progress(&self) -> usize { + self.active.len() + } + + /// Returns the configured window size. + pub fn window_size(&self) -> WindowSize { + self.window_size + } +} + +#[cfg(test)] +mod tests { + use alloc::{vec, vec::Vec}; + + use super::*; + + fn ws(transfers: u16) -> WindowSize { + WindowSize::new(transfers).unwrap() + } + + fn window(transfers: u16) -> TransferWindow { + TransferWindow::new(ws(transfers)) + } + + /// An [`Admission`] without its id, for tests of the classification. + #[derive(Debug, PartialEq)] + enum Seen { + New, + InProgress, + Outside, + } + + fn seen(w: &mut TransferWindow, t: u32) -> Seen { + match w.admit(t) { + Admission::New(_) => Seen::New, + Admission::InProgress(_) => Seen::InProgress, + Admission::OutsideWindow => Seen::Outside, + } + } + + #[test] + fn window_and_allocator_report_their_size() { + assert_eq!(window(7).window_size(), ws(7)); + assert_eq!(TransferNumberAllocator::new(ws(7), 0).window_size(), ws(7)); + } + + #[test] + fn first_transfer_is_new() { + let mut w = window(16); + assert_eq!(seen(&mut w, 100), Seen::New); + assert_eq!(w.greatest(), Some(100)); + } + + #[test] + fn same_transfer_is_in_progress() { + let mut w = window(16); + assert_eq!(seen(&mut w, 100), Seen::New); + assert_eq!(seen(&mut w, 100), Seen::InProgress); + } + + #[test] + fn sequential_transfers_advance() { + let mut w = window(4); + for i in 0..10u32 { + assert_eq!(seen(&mut w, i), Seen::New); + } + assert_eq!(w.greatest(), Some(9)); + } + + #[test] + fn old_transfer_outside_window() { + let mut w = window(4); + for i in 0..10u32 { + w.admit(i); + } + // greatest = 9, window = 4: valid numbers are 6..=9. + assert_eq!(seen(&mut w, 0), Seen::Outside); + assert_eq!(seen(&mut w, 6), Seen::InProgress); + assert_eq!(seen(&mut w, 5), Seen::Outside); + } + + #[test] + fn new_transfer_boundary_is_half_space_plus_half_window() { + // Figure 2: T is new iff (T - GREATEST) mod 2^32 < 2^31 + WINDOW_SIZE/2. + let mut w = window(16); + assert_eq!(seen(&mut w, 0), Seen::New); + let boundary = (1u32 << 31) + 8; + assert_eq!(seen(&mut w, boundary), Seen::Outside); + assert_eq!(w.greatest(), Some(0)); + assert_eq!(seen(&mut w, boundary - 1), Seen::New); + assert_eq!(w.greatest(), Some(boundary - 1)); + } + + #[test] + fn odd_window_size_rounds_the_margin_down() { + // WINDOW_SIZE / 2 is integer division: for 5 the margin is 2. + let mut w = window(5); + w.admit(0); + assert_eq!(seen(&mut w, (1u32 << 31) + 2), Seen::Outside); + assert_eq!(seen(&mut w, (1u32 << 31) + 1), Seen::New); + } + + #[test] + fn wraparound() { + let mut w = window(16); + let start = u32::MAX - 5; + for i in 0..20u32 { + let t = start.wrapping_add(i); + assert_eq!(seen(&mut w, t), Seen::New, "transfer {t}"); + } + assert_eq!(w.greatest(), Some(start.wrapping_add(19))); + } + + #[test] + fn expired_ids_are_those_a_full_window_behind() { + let mut w = window(4); + let mut ids = Vec::new(); + for t in 0..10 { + let Admission::New(id) = w.admit(t) else { + panic!("{t} is ahead of everything before it"); + }; + ids.push(id); + } + // Greatest = 9, window = 4. Valid: 6, 7, 8, 9 + let expired: Vec = ids + .into_iter() + .filter(|&id| w.is_expired(id)) + .map(TransferId::transfer_number) + .collect(); + assert_eq!(expired, vec![0, 1, 2, 3, 4, 5]); + } + + #[test] + fn reset_forgets_the_greatest() { + let mut w = window(4); + w.admit(1000); + assert_eq!(seen(&mut w, 3), Seen::Outside); + w.reset(); + assert_eq!(w.greatest(), None); + assert_eq!(seen(&mut w, 3), Seen::New); + } + + #[test] + fn allocate_sequential() { + let mut a = TransferNumberAllocator::new(ws(16), 100); + assert_eq!(a.allocate(), Ok(100)); + assert_eq!(a.allocate(), Ok(101)); + assert_eq!(a.allocate(), Ok(102)); + assert_eq!(a.in_progress(), 3); + assert!(a.is_outstanding(101)); + assert!(!a.is_outstanding(103)); + } + + #[test] + fn release_of_oldest_frees_slot() { + let mut a = TransferNumberAllocator::new(ws(4), 0); + for _ in 0..4 { + a.allocate().unwrap(); + } + assert!(!a.can_allocate()); + assert_eq!(a.allocate(), Err(Error::WindowFull { window_size: ws(4) })); + assert!(a.release(0)); + assert!(a.can_allocate()); + assert_eq!(a.allocate(), Ok(4)); + } + + #[test] + fn window_gates_on_span_not_count() { + // Section 5: the sender MUST NOT emit a transfer number <= greatest + // - window_size. Releasing the newest transfer frees a *count* slot + // but the span 0..=4 would still exceed the window while 0 is + // outstanding. + let mut a = TransferNumberAllocator::new(ws(4), 0); + for _ in 0..4 { + a.allocate().unwrap(); + } + assert!(a.release(3)); + assert_eq!(a.in_progress(), 3); + assert!(!a.can_allocate()); + assert_eq!(a.allocate(), Err(Error::WindowFull { window_size: ws(4) })); + + // Releasing the oldest advances the window base to 1: 4 - 1 < 4. + assert!(a.release(0)); + assert!(a.can_allocate()); + assert_eq!(a.allocate(), Ok(4)); + // Now 1 anchors the window: 5 - 1 == 4, refused again. + assert!(!a.can_allocate()); + } + + #[test] + fn span_gate_survives_wraparound() { + let start = u32::MAX - 1; + let mut a = TransferNumberAllocator::new(ws(4), start); + // Allocates MAX-1, MAX, 0, 1. + for _ in 0..4 { + a.allocate().unwrap(); + } + // Numerically 1 is the smallest outstanding number, but MAX-1 is the + // oldest and must anchor the window. + assert!(a.release(1)); + assert!(!a.can_allocate()); + assert!(a.release(start)); + assert_eq!(a.allocate(), Ok(2)); + } + + #[test] + fn release_of_unknown_number_is_ignored() { + let mut a = TransferNumberAllocator::new(ws(4), 0); + for _ in 0..4 { + a.allocate().unwrap(); + } + assert!(!a.release(999)); + assert_eq!(a.in_progress(), 4); + assert!(!a.can_allocate()); + // A repeated release of an already-released number frees nothing. + assert!(a.release(0)); + assert!(!a.release(0)); + assert_eq!(a.in_progress(), 3); + } + + #[test] + fn allocator_wraps() { + let mut a = TransferNumberAllocator::new(ws(4), u32::MAX - 1); + assert_eq!(a.allocate(), Ok(u32::MAX - 1)); + assert_eq!(a.allocate(), Ok(u32::MAX)); + assert_eq!(a.allocate(), Ok(0)); + assert_eq!(a.allocate(), Ok(1)); + } + + #[test] + fn ids_cover_exactly_the_window_behind_a_greatest_of_u32_max() { + let mut w = window(4); + assert_eq!(w.id(u32::MAX), None); + let Admission::New(g) = w.admit(u32::MAX) else { + panic!("the first number is new"); + }; + assert_eq!(w.id(u32::MAX), Some(g)); + let oldest = w.id(u32::MAX - 3).unwrap(); + assert_eq!(oldest.transfer_number(), u32::MAX - 3); + assert!(oldest < g); + assert_eq!(w.id(u32::MAX - 4), None); + assert_eq!(w.id(0), None); + } + + #[test] + fn an_id_expires_once_the_window_is_a_full_width_past_it() { + let mut w = window(4); + w.admit(u32::MAX); + let oldest = w.id(u32::MAX - 3).unwrap(); + let next = w.id(u32::MAX - 2).unwrap(); + assert!(!w.is_expired(oldest)); + + let Admission::New(g) = w.admit(0) else { + panic!("0 is one ahead of u32::MAX"); + }; + assert!(next < g); + assert!(w.is_expired(oldest)); + assert!(!w.is_expired(next)); + assert_eq!(w.id(u32::MAX - 2), Some(next)); + } + + #[test] + fn expiry_agrees_with_validity_across_the_wrap() { + let mut w = window(4); + let mut ids = Vec::new(); + for t in (u32::MAX - 8..=u32::MAX).chain(0..8) { + let (Admission::New(id) | Admission::InProgress(id)) = w.admit(t) else { + panic!("{t} is inside the window"); + }; + ids.push(id); + for &id in &ids { + assert_eq!( + w.is_expired(id), + w.id(id.transfer_number()).is_none(), + "{id:?}" + ); + } + assert!(ids.is_sorted()); + } + } + + #[test] + fn ids_after_a_reset_are_above_every_id_before_it() { + // The greatest id before the reset has the highest low bits, and the + // first number after it is 0, whose window reaches the furthest + // back: every id in the new window is still above it. + let mut w = window(WindowSize::MAX.get()); + w.admit(u32::MAX); + let before = w.id(u32::MAX).unwrap(); + w.reset(); + assert_eq!(w.id(u32::MAX), None); + let Admission::New(first) = w.admit(0) else { + panic!("the first number after a reset is new"); + }; + let oldest = w + .id(0u32.wrapping_sub(u32::from(WindowSize::MAX.get()) - 1)) + .unwrap(); + assert!(before < oldest); + assert_eq!(first.transfer_number(), 0); + + // The same number again after another reset is a different id. + w.reset(); + let Admission::New(again) = w.admit(0) else { + panic!("the first number after a reset is new"); + }; + assert_ne!(again, first); + assert!(first < again); + } + + #[test] + fn a_reset_before_any_transfer_changes_nothing() { + let mut fresh = window(4); + let mut reset = window(4); + reset.reset(); + assert_eq!(fresh.admit(9), reset.admit(9)); + } +} diff --git a/btpu/tests/budget.rs b/btpu/tests/budget.rs new file mode 100644 index 000000000..979dcd9d9 --- /dev/null +++ b/btpu/tests/budget.rs @@ -0,0 +1,173 @@ +//! A retention budget shared between receivers and the CLA, through the +//! public `budget` and `receiver` APIs. + +#![cfg(target_has_atomic = "ptr")] + +mod common; + +use std::{sync::Arc, thread}; + +use hardy_btpu::{ + budget::RetentionBudget, + receiver::{ + Delivery, MaxRetainedBytes, Receiver, ReceiverConfig, RejectReason, SEGMENT_OVERHEAD, + }, +}; + +use self::common::{cancel, none, receiver, receiver_config, rejected, segment}; + +const DATA: &[u8] = b"0123456789"; + +// What a receiver is charged for holding one segment of `DATA` given +// outside a PDU. +const CHARGE: usize = DATA.len() + SEGMENT_OVERHEAD; + +fn budget(limit: usize) -> Arc { + Arc::new(RetentionBudget::new( + MaxRetainedBytes::try_from(limit).unwrap(), + )) +} + +fn joined(budget: &Arc) -> Receiver { + receiver(16, 1024).with_budget(Arc::clone(budget)) +} + +#[test] +fn a_charge_is_released_when_dropped() { + let budget = budget(100); + let first = budget.try_charge(60).unwrap(); + assert!(budget.try_charge(41).is_none()); + let second = budget.try_charge(40).unwrap(); + assert_eq!((first.bytes(), second.bytes()), (60, 40)); + assert_eq!(budget.used(), 100); + drop(first); + assert_eq!(budget.used(), 40); + drop(second); + assert_eq!(budget.used(), 0); +} + +#[test] +fn concurrent_charges_never_exceed_the_limit() { + const LIMIT: usize = 1500; + let budget = budget(LIMIT); + let charges: Vec<_> = thread::scope(|s| { + let workers: Vec<_> = (0..4) + .map(|_| { + s.spawn(|| { + (0..1000) + .filter_map(|_| budget.try_charge(1)) + .collect::>() + }) + }) + .collect(); + workers + .into_iter() + .flat_map(|w| w.join().unwrap()) + .collect() + }); + assert_eq!(charges.len(), LIMIT); + assert_eq!(budget.used(), LIMIT); + drop(charges); + assert_eq!(budget.used(), 0); +} + +#[test] +fn receivers_sharing_a_budget_are_bounded_together() { + let budget = budget(2 * CHARGE); + let mut first = joined(&budget); + let mut second = joined(&budget); + assert_eq!(first.process_message(segment(0, 0, DATA)), none()); + assert_eq!(second.process_message(segment(0, 0, DATA)), none()); + assert_eq!(budget.used(), 2 * CHARGE); + + // Within its own limit, but not the budget's. + assert_eq!( + second.process_message(segment(1, 0, DATA)), + vec![rejected(1, RejectReason::BudgetFull)] + ); + assert_eq!(second.retained_bytes(), CHARGE); + assert_eq!(budget.used(), 2 * CHARGE); + + // Dropping a receiver returns its share. + drop(first); + assert_eq!(budget.used(), CHARGE); + assert_eq!(second.process_message(segment(2, 0, DATA)), none()); + assert_eq!(budget.used(), 2 * CHARGE); +} + +#[test] +fn receiver_full_is_reported_before_budget_full() { + let budget = budget(CHARGE); + let mut r = Receiver::new(ReceiverConfig { + max_retained_bytes: Some(MaxRetainedBytes::try_from(CHARGE).unwrap()), + ..receiver_config(16, 1024) + }) + .with_budget(Arc::clone(&budget)); + assert_eq!(r.process_message(segment(0, 0, DATA)), none()); + assert_eq!( + r.process_message(segment(1, 0, DATA)), + vec![rejected(1, RejectReason::ReceiverFull)] + ); + assert_eq!(budget.used(), CHARGE); +} + +#[test] +fn a_cla_charge_leaves_less_for_receivers() { + let budget = budget(2 * CHARGE); + let mut r = joined(&budget); + let charge = budget.try_charge(CHARGE + 1).unwrap(); + assert_eq!( + r.process_message(segment(0, 0, DATA)), + vec![rejected(0, RejectReason::BudgetFull)] + ); + drop(charge); + assert_eq!(r.process_message(segment(1, 0, DATA)), none()); +} + +#[test] +fn joining_a_budget_charges_what_is_already_held() { + let earlier = budget(1024); + let mut r = joined(&earlier); + r.process_message(segment(0, 0, DATA)); + assert_eq!(earlier.used(), CHARGE); + + // Charged whatever the limit, and moved off the earlier budget. + let small = budget(1); + let mut r = r.with_budget(Arc::clone(&small)); + assert_eq!((earlier.used(), small.used()), (0, CHARGE)); + assert_eq!( + r.process_message(segment(1, 0, DATA)), + vec![rejected(1, RejectReason::BudgetFull)] + ); + + // What was held over the limit is released as it leaves. + r.process_message(cancel(0)); + assert_eq!(small.used(), 0); +} + +#[test] +fn a_reset_releases_the_receivers_share() { + let budget = budget(1024); + let mut r = joined(&budget); + r.process_message(segment(0, 0, DATA)); + r.process_message(segment(1, 0, DATA)); + assert_eq!(budget.used(), 2 * CHARGE); + r.reset(); + assert_eq!(budget.used(), 0); +} + +#[test] +fn released_segments_are_not_charged() { + let budget = budget(1024); + let mut r = Receiver::new(ReceiverConfig { + delivery: Delivery::Streamed, + ..receiver_config(16, 1024) + }) + .with_budget(Arc::clone(&budget)); + r.process_message(segment(0, 0, DATA)); + assert_eq!(budget.used(), 0); + r.process_message(segment(0, 2, DATA)); + assert_eq!(budget.used(), CHARGE); + r.process_message(segment(0, 1, DATA)); + assert_eq!(budget.used(), 0); +} diff --git a/btpu/tests/codec.rs b/btpu/tests/codec.rs new file mode 100644 index 000000000..02639b317 --- /dev/null +++ b/btpu/tests/codec.rs @@ -0,0 +1,782 @@ +//! Wire-format round trips and PDU framing through the public `codec` API. + +mod common; + +use bytes::{BufMut, Bytes, BytesMut}; +use hardy_btpu::{ + codec::{ + DecodeOptions, Error, Result, decode_pdu, decode_pdu_with, encode_message, + encoded_message_len, + header::{HEADER_SIZE, MAX_CONTENT_LENGTH}, + hint::{HintItem, HintType}, + message::{Message, MessageFlags}, + pad_pdu, + }, + fec::{ExplicitFecMessage, PreAgreedFecMessage}, +}; + +use self::common::{ + bundle_msg, bundle_with, cancel, encode, end_with, segment, segment_with, unknown_hint, +}; + +const FEC: DecodeOptions<'static> = DecodeOptions { + fec: true, + bundle_extent: None, +}; + +// Collect the lazy decoder for tests that assert on a whole PDU. +fn decode_all(pdu: Bytes) -> Result> { + decode_pdu(pdu).collect() +} + +fn decode_all_with(pdu: Bytes, options: DecodeOptions<'_>) -> Result> { + decode_pdu_with(pdu, options).collect() +} + +fn fec_messages() -> [Message; 4] { + let payload = Bytes::from_static(b"\x01\x02fssi-or-id-plus-data"); + [ + Message::PreAgreedFecSource(PreAgreedFecMessage { + transfer_number: 7, + fec_instance_id: 3, + hints: vec![], + payload: payload.clone(), + }), + Message::ExplicitFecSource(ExplicitFecMessage { + transfer_number: 7, + fec_encoding_id: 3, + hints: vec![HintItem::BundleLength(9)], + payload: payload.clone(), + }), + Message::PreAgreedFecRepair(PreAgreedFecMessage { + transfer_number: 7, + fec_instance_id: 3, + hints: vec![], + payload: payload.clone(), + }), + Message::ExplicitFecRepair(ExplicitFecMessage { + transfer_number: 7, + fec_encoding_id: 3, + hints: vec![], + payload, + }), + ] +} + +#[test] +fn round_trip_fec_messages_with_fec_decoding_on() { + // The payload is opaque: whatever FSSI/payload-ID/data bytes a scheme + // packed into it must survive encode -> decode untouched. + for msg in &fec_messages() { + let wire = encode(msg); + assert_eq!(wire.len(), encoded_message_len(msg)); + assert_eq!(decode_all_with(wire, FEC).unwrap(), vec![msg.clone()]); + } +} + +#[test] +fn fec_types_relay_as_unknown_by_default() { + // 0x70..=0x73 are Private Use (Section 12.1): a decoder that was not + // told to expect the FEC extension must treat them like any other + // unknown type, byte-exact, so a peer's private types are untouched. + for msg in &fec_messages() { + let wire = encode(msg); + let decoded = decode_all(wire.clone()).unwrap(); + let [ + Message::Unknown { + message_type, + flags, + data, + }, + ] = decoded.as_slice() + else { + panic!("expected one Unknown, got {decoded:?}"); + }; + assert!((0x70..=0x73).contains(message_type)); + assert_eq!( + flags.hint, + !matches!( + msg, + Message::PreAgreedFecSource(_) + | Message::PreAgreedFecRepair(_) + | Message::ExplicitFecRepair(_) + ) + ); + assert_eq!(data.len(), wire.len() - HEADER_SIZE); + assert_eq!(encode(&decoded[0]), wire); + } +} + +#[test] +fn round_trip_core_messages() { + let messages = [ + bundle_msg(b"hello bundle"), + bundle_with( + vec![HintItem::BundleLength(42)], + Bytes::from_static(b"data"), + ), + segment(0x12345678, 0, b"seg0"), + end_with( + 99, + 3, + vec![HintItem::BundleLength(1000)], + Bytes::from_static(b"final"), + ), + end_with( + 1, + 1, + vec![unknown_hint(3, b"zz")], + Bytes::from_static(b"end"), + ), + cancel(42), + Message::DefinitePadding { len: 10 }, + Message::Unknown { + message_type: 0x50, + flags: MessageFlags::default(), + data: Bytes::from_static(b"opaque"), + }, + ]; + for msg in &messages { + let wire = encode(msg); + assert_eq!(wire.len(), encoded_message_len(msg), "{msg:?}"); + assert_eq!(decode_all(wire).unwrap(), vec![msg.clone()]); + } +} + +#[test] +fn indefinite_padding_skipped() { + let bundle = bundle_msg(b"x"); + let mut buf = BytesMut::new(); + buf.put_bytes(0, 3); + buf.put_slice(&encode(&bundle)); + buf.put_bytes(0, 2); + assert_eq!(decode_all(buf.freeze()).unwrap(), vec![bundle]); +} + +#[test] +fn all_zeros_pdu() { + assert_eq!(decode_all(Bytes::from(vec![0u8; 64])).unwrap(), vec![]); +} + +#[test] +fn multiple_messages_in_pdu() { + let msgs = vec![ + bundle_msg(b"a"), + cancel(1), + Message::DefinitePadding { len: 2 }, + ]; + let mut buf = BytesMut::new(); + for m in &msgs { + buf.put_slice(&encode(m)); + } + assert_eq!(decode_all(buf.freeze()).unwrap(), msgs); +} + +#[test] +fn pad_pdu_fills_to_target() { + let msg = bundle_msg(b"hi"); + let mut buf = BytesMut::from(encode(&msg).as_ref()); + let pre_pad_len = buf.len(); + pad_pdu(&mut buf, 64); + assert_eq!(buf.len(), 64); + assert_eq!( + decode_all(buf.clone().freeze()).unwrap(), + vec![ + msg, + Message::DefinitePadding { + len: 64 - pre_pad_len - HEADER_SIZE + } + ] + ); + + // Padding already sufficient: a no-op. + pad_pdu(&mut buf, pre_pad_len); + assert_eq!(buf.len(), 64); +} + +#[test] +fn pad_pdu_small_remainder() { + let mut buf = BytesMut::new(); + // Fill so that only 2 bytes remain (less than HEADER_SIZE). + buf.put_bytes(0xFF, 62); + pad_pdu(&mut buf, 64); + assert_eq!(&buf[62..], &[0, 0]); +} + +#[test] +fn pad_pdu_beyond_max_content_length_chains_messages() { + // The largest single Definite Padding message. + const MAX_MESSAGE: usize = HEADER_SIZE + MAX_CONTENT_LENGTH; + + // Exactly one maximum-size message fits. + let mut buf = BytesMut::new(); + pad_pdu(&mut buf, MAX_MESSAGE); + assert_eq!(buf.len(), MAX_MESSAGE); + assert_eq!( + decode_all(buf.freeze()).unwrap(), + vec![Message::DefinitePadding { + len: MAX_CONTENT_LENGTH + }] + ); + + // One byte past a single message's reach: the 20-bit length field + // cannot declare it, so an indefinite padding byte follows; the header + // must stay truthful rather than truncating. + let mut buf = BytesMut::new(); + pad_pdu(&mut buf, MAX_MESSAGE + 1); + assert_eq!(buf.len(), MAX_MESSAGE + 1); + assert_eq!( + decode_all(buf.freeze()).unwrap(), + vec![Message::DefinitePadding { + len: MAX_CONTENT_LENGTH + }] + ); + + // Enough space past the first message for a whole second header: + // a chain of two Definite Padding messages. + let mut buf = BytesMut::new(); + pad_pdu(&mut buf, MAX_MESSAGE + HEADER_SIZE + 1); + assert_eq!(buf.len(), MAX_MESSAGE + HEADER_SIZE + 1); + assert_eq!( + decode_all(buf.freeze()).unwrap(), + vec![ + Message::DefinitePadding { + len: MAX_CONTENT_LENGTH + }, + Message::DefinitePadding { len: 1 }, + ] + ); +} + +#[test] +fn bare_bpv6_bundle_decoded_as_bundle_message() { + // Without an extent hook, a frame starting with the BPv6 reserved byte + // is a bare bundle running to the end of the frame. + let frame = Bytes::from_static(&[0x06, 0xDE, 0xAD, 0xBE, 0xEF]); + assert_eq!( + decode_all(frame.clone()).unwrap(), + vec![bundle_with(vec![], frame)] + ); +} + +#[test] +fn bare_bpv7_bundle_decoded_as_bundle_message() { + // Any first byte in 0x80..=0x9F (CBOR array headers, how BPv7 bundles + // start) is treated as a bare bundle. + for t in 0x80u8..=0x9F { + let frame = Bytes::copy_from_slice(&[t, 0xCA, 0xFE, 0xBA, 0xBE]); + assert_eq!( + decode_all(frame.clone()).unwrap(), + vec![bundle_with(vec![], frame)], + "byte {t:#04x}" + ); + } +} + +#[test] +fn bare_bundle_after_indefinite_padding_is_the_rest_of_the_frame() { + // Leading zeros are Indefinite Padding; with no message yet framed the + // hookless rule still applies and the bundle runs to the end. + let frame = Bytes::from_static(&[0, 0, 0x9F, 1, 2]); + assert_eq!( + decode_all(frame.clone()).unwrap(), + vec![bundle_with(vec![], frame.slice(2..))] + ); +} + +#[test] +fn bare_frame_zero_fill_is_delivered_as_bundle_bytes_without_hook() { + // The documented pitfall: a 4-byte bundle zero-filled to Ethernet's + // 46-byte minimum is delivered as 46 bytes when nothing can delimit it. + let mut frame = vec![0x9F, 1, 2, 0xFF]; + frame.resize(46, 0); + let frame = Bytes::from(frame); + assert_eq!( + decode_all(frame.clone()).unwrap(), + vec![bundle_with(vec![], frame)] + ); +} + +// A stand-in for a bundle parser: a 0x9F "bundle" here is four bytes long, +// or unparseable if fewer than four remain. +fn four_byte_bundles(bytes: &[u8]) -> Option { + (bytes.first() == Some(&0x9F) && bytes.len() >= 4).then_some(4) +} + +#[test] +fn extent_hook_trims_bare_frame_padding() { + let mut frame = vec![0x9F, 1, 2, 0xFF]; + frame.resize(46, 0); + let frame = Bytes::from(frame); + let options = DecodeOptions { + fec: false, + bundle_extent: Some(&four_byte_bundles), + }; + // The bundle is exactly its four bytes; the zero-fill behind it decodes + // as Indefinite Padding. + assert_eq!( + decode_all_with(frame.clone(), options).unwrap(), + vec![bundle_with(vec![], frame.slice(..4))] + ); +} + +#[test] +fn extent_hook_delivers_mid_pdu_bundle_and_iteration_continues() { + // Section 7.3: a receiver that can delimit an encapsulated bundle + // handles it as a Bundle Message and continues with what follows. + let mut pdu = BytesMut::new(); + pdu.put_slice(&encode(&cancel(1))); + pdu.put_slice(&[0x9F, 1, 2, 3]); + pdu.put_slice(&encode(&cancel(2))); + let pdu = pdu.freeze(); + let options = DecodeOptions { + fec: false, + bundle_extent: Some(&four_byte_bundles), + }; + assert_eq!( + decode_all_with(pdu.clone(), options).unwrap(), + vec![cancel(1), bundle_with(vec![], pdu.slice(8..12)), cancel(2),] + ); +} + +#[test] +fn mid_pdu_encapsulated_bundle_without_hook_is_terminal() { + // With nothing to delimit it, the bundle's extent is unknowable and + // Section 7.3 forbids processing the remainder: iteration stops, and + // every message already parsed is kept. The error locates the bundle + // in a clone of the PDU kept by the caller. + let mut pdu = BytesMut::new(); + pdu.put_slice(&encode(&cancel(1))); + let bundle_offset = pdu.len(); + pdu.put_slice(&[0x06, 0, 0, 0]); + let pdu = pdu.freeze(); + let mut iter = decode_pdu(pdu.clone()); + assert_eq!(iter.next(), Some(Ok(cancel(1)))); + let Some(Err(Error::EncapsulatedBundle { first_byte, offset })) = iter.next() else { + panic!("expected a terminal EncapsulatedBundle"); + }; + assert_eq!((first_byte, offset), (0x06, bundle_offset)); + assert_eq!(pdu[offset..], [0x06, 0, 0, 0]); + assert!(iter.is_exhausted()); + assert_eq!(iter.next(), None); +} + +#[test] +fn extent_hook_declining_is_terminal() { + // The hook says it cannot delimit the bytes: same outcome as no hook, + // and no guess is made even at the start of the PDU. + let frame = Bytes::from_static(&[0x9F, 1]); + let options = DecodeOptions { + fec: false, + bundle_extent: Some(&four_byte_bundles), + }; + let mut iter = decode_pdu_with(frame, options); + assert_eq!( + iter.next(), + Some(Err(Error::EncapsulatedBundle { + first_byte: 0x9F, + offset: 0 + })) + ); + assert!(iter.is_exhausted()); + assert_eq!(iter.next(), None); +} + +#[test] +fn extent_hook_claiming_zero_bytes_is_terminal() { + // A zero extent would deliver an empty bundle and leave the decoder at + // the same offset; it is treated as the hook declining. + let frame = Bytes::from_static(&[0x9F, 1, 2, 3]); + let zero = |_: &[u8]| Some(0); + let options = DecodeOptions { + fec: false, + bundle_extent: Some(&zero), + }; + let mut iter = decode_pdu_with(frame, options); + assert_eq!( + iter.next(), + Some(Err(Error::EncapsulatedBundle { + first_byte: 0x9F, + offset: 0 + })) + ); + assert!(iter.is_exhausted()); + assert_eq!(iter.next(), None); +} + +#[test] +fn extent_hook_overrunning_the_pdu_is_terminal() { + let frame = Bytes::from_static(&[0x9F, 1, 2]); + let overrun = |_: &[u8]| Some(100); + let options = DecodeOptions { + fec: false, + bundle_extent: Some(&overrun), + }; + let mut iter = decode_pdu_with(frame, options); + assert_eq!( + iter.next(), + Some(Err(Error::InsufficientData { + needed: 100, + available: 3, + })) + ); + assert!(iter.is_exhausted()); +} + +#[test] +fn malformed_interior_skips_only_that_message() { + // A known-type message with a bounded extent but a malformed interior + // (a hint header promising a 255-byte value with nothing behind it) + // yields an Err, and iteration resumes at the next message boundary + // given by the Section 7 header length (Section 7.3 skip-and-continue). + let ok = bundle_msg(b"ok"); + let mut pdu = BytesMut::new(); + pdu.put_u8(0x02); // Bundle + pdu.put_u8(0x80); // H flag set, length high nibble 0 + pdu.put_u16(2); + pdu.put_slice(b"\x1F\xFF"); // malformed hint chain + pdu.put_slice(&encode(&ok)); + + let mut iter = decode_pdu(pdu.freeze()); + assert_eq!( + iter.next(), + Some(Err(Error::InsufficientData { + needed: 257, + available: 2, + })) + ); + assert!(!iter.is_exhausted()); + assert_eq!(iter.next(), Some(Ok(ok))); + assert_eq!(iter.next(), None); +} + +#[test] +fn length_past_buffer_is_terminal() { + // A header promising more content than the PDU holds: the next + // message boundary is unknowable, so iteration stops permanently + // and the remainder is discarded. + let mut pdu = BytesMut::new(); + pdu.put_u8(0x02); // Bundle + pdu.put_u8(0x00); + pdu.put_u16(100); // length 100, but no content follows + let mut iter = decode_pdu(pdu.freeze()); + assert_eq!( + iter.next(), + Some(Err(Error::InsufficientData { + needed: 104, + available: 4, + })) + ); + assert!(iter.is_exhausted()); + assert_eq!(iter.next(), None); +} + +#[test] +fn short_bodies_are_contained_to_their_message() { + // Each core message type with a body shorter than its fixed fields: + // the header still bounds the message, so the fault is recoverable + // and the Cancel behind it decodes. The error reports the field that + // ran short: here always a 4-byte number with 3 bytes left for it. + let cases: [(u8, usize); 3] = [ + (0x05, 3), // Cancel: transfer number + (0x03, 7), // Segment: transfer number + segment index + (0x04, 7), // End: transfer number + segment index + ]; + for (message_type, body_len) in cases { + let mut pdu = BytesMut::new(); + pdu.put_u8(message_type); + pdu.put_u8(0); + pdu.put_u16(body_len as u16); + pdu.put_bytes(0xAA, body_len); + pdu.put_slice(&encode(&cancel(9))); + let mut iter = decode_pdu(pdu.freeze()); + assert_eq!( + iter.next(), + Some(Err(Error::InsufficientData { + needed: 4, + available: 3, + })), + "type {message_type:#04x}" + ); + assert!(!iter.is_exhausted()); + assert_eq!(iter.next(), Some(Ok(cancel(9)))); + } + + // FEC messages: transfer number + one ID byte, with none left for the + // ID. + for message_type in 0x70u8..=0x73 { + let mut pdu = BytesMut::new(); + pdu.put_u8(message_type); + pdu.put_u8(0); + pdu.put_u16(4); + pdu.put_bytes(0xAA, 4); + pdu.put_slice(&encode(&cancel(9))); + let mut iter = decode_pdu_with(pdu.freeze(), FEC); + assert_eq!( + iter.next(), + Some(Err(Error::InsufficientData { + needed: 1, + available: 0, + })), + "type {message_type:#04x}" + ); + assert!(!iter.is_exhausted()); + assert_eq!(iter.next(), Some(Ok(cancel(9)))); + } +} + +#[test] +fn unknown_type_preserved() { + let msg = Message::Unknown { + message_type: 0x50, + flags: MessageFlags::default(), + data: Bytes::from_static(b"\x01\x02\x03"), + }; + assert_eq!(decode_all(encode(&msg)).unwrap(), vec![msg]); +} + +#[test] +fn unknown_message_with_hints_relays_intact() { + // An unknown message with the H flag set must round-trip with the + // flag AND the raw hint bytes preserved, or a relayed copy would be + // misparsed downstream (hint bytes read as message body). + let mut original = BytesMut::new(); + original.put_u8(0x50); + original.put_u8(0x80); // flags nibble H=1, top 4 bits of length = 0 + original.put_u16(5); + original.put_slice(b"\x00\x01\x2A"); // a valid hint chain + original.put_slice(b"xy"); // opaque body + let original = original.freeze(); + + let decoded = decode_all(original.clone()).unwrap(); + assert_eq!( + decoded, + vec![Message::Unknown { + message_type: 0x50, + flags: MessageFlags { hint: true, rfu: 0 }, + data: original.slice(HEADER_SIZE..), + }] + ); + assert_eq!(encode(&decoded[0]), original); +} + +#[test] +fn unknown_message_rfu_flag_bits_relay_intact() { + // Flags nibble 0xD: H plus two of the unassigned bits. The flags + // registry is Standards Action (Section 12.3), so a future sender + // may validly set them; a relayed unknown message must keep the + // nibble bit-exact. + let mut original = BytesMut::new(); + original.put_u8(0x50); + original.put_u8(0xD0); + original.put_u16(2); + original.put_slice(b"xy"); + let original = original.freeze(); + + let decoded = decode_all(original.clone()).unwrap(); + assert_eq!( + decoded, + vec![Message::Unknown { + message_type: 0x50, + flags: MessageFlags { + hint: true, + rfu: 0x5 + }, + data: original.slice(HEADER_SIZE..), + }] + ); + assert_eq!(encode(&decoded[0]), original); +} + +#[test] +fn rfu_flag_bits_on_known_type_are_ignored() { + // Section 7.1: a receiver MUST ignore the unassigned flag bits, so a + // Bundle message with them set decodes as a plain Bundle. + let mut pdu = BytesMut::new(); + pdu.put_u8(0x02); + pdu.put_u8(0x50); // rfu bits 0b101, H clear + pdu.put_u16(2); + pdu.put_slice(b"hi"); + assert_eq!(decode_all(pdu.freeze()).unwrap(), vec![bundle_msg(b"hi")]); +} + +#[test] +fn malformed_hints_in_unknown_message_do_not_poison_pdu() { + // Unknown messages are skipped via the Section 7 header length field + // (Section 7.3). Hint bytes that would not parse inside an unknown + // message must not error the PDU; the following Bundle still decodes. + let ok = bundle_msg(b"ok"); + let mut pdu = BytesMut::new(); + pdu.put_u8(0x50); + pdu.put_u8(0x80); + pdu.put_u16(2); + pdu.put_slice(b"\x1F\xFF"); + pdu.put_slice(&encode(&ok)); + + assert_eq!( + decode_all(pdu.freeze()).unwrap(), + vec![ + Message::Unknown { + message_type: 0x50, + flags: MessageFlags { hint: true, rfu: 0 }, + data: Bytes::from_static(b"\x1F\xFF"), + }, + ok, + ] + ); +} + +#[test] +fn malformed_bundle_length_hint_keeps_the_segment() { + // A Bundle Length hint with a 3-byte value breaks Section 9.1, but the + // segment is fully framed; it decodes with the hint carried as unknown + // and its data intact. + let mut pdu = BytesMut::new(); + pdu.put_u8(0x03); // Segment + pdu.put_u8(0x80); // H + pdu.put_u16(2 + 3 + 8 + 3); + pdu.put_slice(&[HintType::BUNDLE_LENGTH.get() << 1, 3, 1, 2, 3]); + pdu.put_u32(7); + pdu.put_u32(0); + pdu.put_slice(b"abc"); + assert_eq!( + decode_all(pdu.freeze()).unwrap(), + vec![segment_with( + 7, + 0, + vec![unknown_hint(HintType::BUNDLE_LENGTH.get(), &[1, 2, 3])], + Bytes::from_static(b"abc"), + )] + ); +} + +#[test] +fn unknown_cannot_carry_defined_or_reserved_type() { + // Encoding an Unknown under a base-protocol or bundle-reserved type + // would produce a message the decoder reads as something else. + for t in [0x00u8, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x80, 0x9F] { + let msg = Message::Unknown { + message_type: t, + flags: MessageFlags::default(), + data: Bytes::from_static(b"x"), + }; + let mut buf = BytesMut::new(); + assert_eq!( + encode_message(&msg, &mut buf), + Err(Error::NotAnUnknownType(t)), + "type {t:#04x}" + ); + assert!(buf.is_empty()); + } + // The provisional FEC values are Private Use and relay fine. + for t in 0x70u8..=0x73 { + let msg = Message::Unknown { + message_type: t, + flags: MessageFlags::default(), + data: Bytes::from_static(b"x"), + }; + assert_eq!(decode_all(encode(&msg)).unwrap(), vec![msg]); + } +} + +#[test] +fn unknown_encodes_exactly_the_types_the_decoder_reads_as_unknown() { + // The encoder's relayable set and the decoder's unknown set are defined + // separately; across every type value they must agree. The decoder + // runs with FEC off, which is how a relay that knows no extension sees + // the types. + let mut relayable = 0; + for t in 0..=u8::MAX { + let msg = Message::Unknown { + message_type: t, + flags: MessageFlags::default(), + data: Bytes::from_static(b"x"), + }; + let encodes = encode_message(&msg, &mut BytesMut::new()).is_ok(); + let decoded = decode_pdu(Bytes::copy_from_slice(&[t, 0x00, 0x00, 0x01, b'x'])).next(); + assert_eq!( + encodes, + decoded == Some(Ok(msg)), + "type {t:#04x} decodes as {decoded:?}" + ); + relayable += usize::from(encodes); + } + // Every value but the six base-protocol types, BPv6's 0x06, and BPv7's + // 0x80..=0x9F. + assert_eq!(relayable, 256 - 6 - 1 - 32); +} + +#[test] +fn encode_errors_leave_the_buffer_untouched() { + let mut buf = BytesMut::new(); + let too_long = bundle_with(vec![], Bytes::from(vec![0u8; MAX_CONTENT_LENGTH + 1])); + assert_eq!( + encode_message(&too_long, &mut buf), + Err(Error::LengthOverflow { + length: MAX_CONTENT_LENGTH + 1, + max: MAX_CONTENT_LENGTH, + }) + ); + assert!(buf.is_empty()); + + let too_long_unknown = Message::Unknown { + message_type: 0x70, + flags: MessageFlags::default(), + data: Bytes::from(vec![0u8; MAX_CONTENT_LENGTH + 1]), + }; + assert_eq!( + encode_message(&too_long_unknown, &mut buf), + Err(Error::LengthOverflow { + length: MAX_CONTENT_LENGTH + 1, + max: MAX_CONTENT_LENGTH, + }) + ); + assert!(buf.is_empty()); +} + +#[test] +fn truncation_at_every_offset_keeps_the_whole_messages_before_it() { + let messages = [ + cancel(1), + segment_with( + 2, + 0, + vec![HintItem::BundleLength(9)], + Bytes::from_static(b"segment"), + ), + bundle_msg(b"bundle"), + ]; + let mut pdu = BytesMut::new(); + let mut ends = Vec::new(); + for m in &messages { + pdu.put_slice(&encode(m)); + ends.push(pdu.len()); + } + let pdu = pdu.freeze(); + + for cut in 0..=pdu.len() { + // Every message wholly inside the cut decodes; a partial one is a + // single InsufficientData naming the bytes its header (or, for a + // partial header, the header itself) needs. + let whole = ends.iter().take_while(|&&end| end <= cut).count(); + let mut expected: Vec> = + messages[..whole].iter().cloned().map(Ok).collect(); + let start = whole.checked_sub(1).map_or(0, |i| ends[i]); + if cut > start { + let needed = if cut - start < HEADER_SIZE { + start + HEADER_SIZE + } else { + ends[whole] + }; + expected.push(Err(Error::InsufficientData { + needed, + available: cut, + })); + } + assert_eq!( + decode_pdu(pdu.slice(..cut)).collect::>(), + expected, + "cut at {cut}" + ); + } +} diff --git a/btpu/tests/codec_header.rs b/btpu/tests/codec_header.rs new file mode 100644 index 000000000..44e5c41e7 --- /dev/null +++ b/btpu/tests/codec_header.rs @@ -0,0 +1,101 @@ +//! Message header encode/decode through the public `codec::header` API. + +use hardy_btpu::codec::{ + Error, + header::{ + ContentLength, HEADER_SIZE, MAX_CONTENT_LENGTH, MessageHeader, decode_header, encode_header, + }, + message::MessageFlags, +}; + +fn content_length(len: usize) -> ContentLength { + ContentLength::new(len).unwrap() +} + +fn assert_round_trips(message_type: u8, flags: MessageFlags, length: usize) { + let hdr = MessageHeader { + message_type, + flags, + length: content_length(length), + }; + let buf = encode_header(&hdr); + assert_eq!(decode_header(&buf), Ok(hdr)); +} + +#[test] +fn round_trip_basic() { + assert_round_trips(3, MessageFlags::default(), 256); +} + +#[test] +fn round_trip_with_hint_flag() { + assert_round_trips(2, MessageFlags { hint: true, rfu: 0 }, 42); +} + +#[test] +fn round_trip_max_length() { + assert_round_trips(1, MessageFlags::default(), MAX_CONTENT_LENGTH); +} + +#[test] +fn length_above_20_bits_is_refused() { + // The field cannot hold it; truncating would emit a header claiming + // a different length. + assert_eq!(ContentLength::new(MAX_CONTENT_LENGTH + 1), None); + assert_eq!( + ContentLength::try_from(MAX_CONTENT_LENGTH + 1), + Err(Error::LengthOverflow { + length: MAX_CONTENT_LENGTH + 1, + max: MAX_CONTENT_LENGTH, + }) + ); + assert_eq!(ContentLength::MAX.get(), MAX_CONTENT_LENGTH); +} + +#[test] +fn round_trip_zero_length() { + assert_round_trips(5, MessageFlags::default(), 0); +} + +#[test] +fn decode_insufficient_data() { + assert_eq!( + decode_header(&[0, 0]), + Err(Error::InsufficientData { + needed: HEADER_SIZE, + available: 2, + }) + ); + assert_eq!( + decode_header(&[]), + Err(Error::InsufficientData { + needed: HEADER_SIZE, + available: 0, + }) + ); +} + +#[test] +fn wire_format_layout() { + // Type=3, Flags=0x8 (hint), Length=0x12345 + let hdr = MessageHeader { + message_type: 3, + flags: MessageFlags { hint: true, rfu: 0 }, + length: content_length(0x1_2345), + }; + let buf = encode_header(&hdr); + assert_eq!(buf, [3, 0x81, 0x23, 0x45]); +} + +#[test] +fn all_message_types_round_trip() { + for t in [0u8, 1, 2, 3, 4, 5, 0x70, 0x71, 0x72, 0x73, 0xFF] { + let hdr = MessageHeader { + message_type: t, + flags: MessageFlags::default(), + length: content_length(100), + }; + let buf = encode_header(&hdr); + assert_eq!(decode_header(&buf).unwrap().message_type, t); + } +} diff --git a/btpu/tests/codec_hint.rs b/btpu/tests/codec_hint.rs new file mode 100644 index 000000000..2ecf252cd --- /dev/null +++ b/btpu/tests/codec_hint.rs @@ -0,0 +1,276 @@ +//! Hint item encode/decode through the public `codec::hint` API. + +mod common; + +use bytes::{BufMut, Bytes, BytesMut}; +use hardy_btpu::codec::{ + Error, + hint::{ + HINT_HEADER_SIZE, HintItem, HintType, HintValue, Hints, MAX_HINT_VALUE_LEN, decode_hints, + encode_hints, encoded_hints_len, + }, +}; + +use self::common::unknown_hint; + +fn round_trip(hints: Vec) { + let mut buf = BytesMut::new(); + encode_hints(&hints, &mut buf); + let bytes = buf.freeze(); + let (decoded, consumed) = decode_hints(&bytes).unwrap(); + assert_eq!(consumed, bytes.len()); + assert_eq!(decoded, hints); +} + +#[test] +fn hint_value_holds_at_most_what_the_length_field_declares() { + // 256 bytes cannot be declared by the 8-bit length field. + assert_eq!( + HintValue::new(Bytes::from(vec![0u8; MAX_HINT_VALUE_LEN + 1])), + None + ); + + // Exactly 255 bytes is fine and round-trips. + let value = HintValue::new(Bytes::from(vec![0u8; MAX_HINT_VALUE_LEN])).unwrap(); + round_trip(vec![HintItem::Unknown { + hint_type: HintType::new(0x2A).unwrap(), + value, + }]); +} + +#[test] +fn hint_type_holds_at_most_seven_bits() { + // Types above 0x7F would lose their top bit to the << 1 shift. + assert_eq!(HintType::new(0x80), None); + assert_eq!(HintType::new(0x7F), Some(HintType::MAX)); + + // Exactly 0x7F is fine and round-trips. + round_trip(vec![unknown_hint(HintType::MAX.get(), b"x")]); +} + +#[test] +fn hint_type_reports_the_wire_type() { + assert_eq!( + HintItem::BundleLength(42).hint_type(), + HintType::BUNDLE_LENGTH + ); + assert_eq!( + unknown_hint(0x41, b"x").hint_type(), + HintType::new(0x41).unwrap() + ); + // A malformed Bundle Length is carried as unknown under its own type. + assert_eq!(unknown_hint(0, b"abc").hint_type(), HintType::BUNDLE_LENGTH); +} + +#[test] +fn round_trip_bundle_length_every_width() { + round_trip(vec![HintItem::BundleLength(200)]); + round_trip(vec![HintItem::BundleLength(2000)]); + round_trip(vec![HintItem::BundleLength(100_000)]); + round_trip(vec![HintItem::BundleLength(u64::MAX)]); +} + +#[test] +fn bundle_length_uses_the_shortest_width_either_side_of_each_boundary() { + for (len, width) in [ + (0, 1), + (0xFF, 1), + (0x100, 2), + (0xFFFF, 2), + (0x1_0000, 4), + (0xFFFF_FFFF, 4), + (0x1_0000_0000, 8), + (u64::MAX, 8), + ] { + let mut buf = BytesMut::new(); + encode_hints(&[HintItem::BundleLength(len)], &mut buf); + let mut expected = vec![HintType::BUNDLE_LENGTH.get() << 1, width]; + expected.extend_from_slice(&len.to_be_bytes()[8 - usize::from(width)..]); + assert_eq!(buf[..], expected[..], "Bundle Length {len:#x}"); + } +} + +#[test] +fn round_trip_chained_hints() { + let hints = vec![HintItem::BundleLength(42), unknown_hint(5, b"\x01\x02\x03")]; + let mut buf = BytesMut::new(); + encode_hints(&hints, &mut buf); + + // H is set on every item but the last. BundleLength(42) takes a + // one-byte value. + assert_eq!(buf[0] & 1, 1); + assert_eq!(buf[HINT_HEADER_SIZE + 1] & 1, 0); + + round_trip(hints); +} + +#[test] +fn encoded_len_matches_actual() { + let hints = vec![HintItem::BundleLength(2000), unknown_hint(10, b"test")]; + let expected = encoded_hints_len(&hints); + let mut buf = BytesMut::new(); + encode_hints(&hints, &mut buf); + assert_eq!(buf.len(), expected); +} + +#[test] +fn malformed_bundle_length_size_is_carried_as_unknown() { + // Section 9.1 requires a value length of 1, 2, 4, or 8. A 3-byte + // value is a sender fault, but the item is fully framed, so it is + // carried opaquely rather than failing the message. + let bytes = Bytes::from_static(&[ + 0, // type=0 (Bundle Length), H=0 + 3, // length=3 (invalid) + 0x01, 0x02, 0x03, + ]); + let (decoded, consumed) = decode_hints(&bytes).unwrap(); + assert_eq!(consumed, bytes.len()); + assert_eq!( + decoded, + vec![unknown_hint(HintType::BUNDLE_LENGTH.get(), &[1, 2, 3])] + ); +} + +#[test] +fn truncated_chain_errors() { + // Header promising a 5-byte value with 2 bytes behind it. + let bytes = Bytes::from_static(&[0x0A, 5, 0xAA, 0xBB]); + assert_eq!( + decode_hints(&bytes), + Err(Error::InsufficientData { + needed: 7, + available: 4, + }) + ); + // Header cut short. + let bytes = Bytes::from_static(&[0x0A]); + assert_eq!( + decode_hints(&bytes), + Err(Error::InsufficientData { + needed: 2, + available: 1, + }) + ); +} + +#[test] +fn repeated_types_fold_latest_wins_in_first_appearance_order() { + // Chain: type 5 = "a", type 0 = 42, type 5 = "b". Type 5 keeps its + // first position but takes the later value. + let mut buf = BytesMut::new(); + buf.put_slice(&[(5 << 1) | 1, 1, b'a']); + buf.put_slice(&[(HintType::BUNDLE_LENGTH.get() << 1) | 1, 1, 42]); + buf.put_slice(&[5 << 1, 1, b'b']); + let bytes = buf.freeze(); + let (decoded, consumed) = decode_hints(&bytes).unwrap(); + assert_eq!(consumed, bytes.len()); + assert_eq!( + decoded, + vec![unknown_hint(5, b"b"), HintItem::BundleLength(42),] + ); +} + +#[test] +fn long_chain_of_repeats_folds_to_one_item() { + // Ten thousand repeats of one type decode to a single item: the + // returned Vec is bounded by the type space, not the chain length. + let mut buf = BytesMut::new(); + for _ in 0..9_999 { + buf.put_slice(&[(7 << 1) | 1, 0]); + } + buf.put_slice(&[7 << 1, 1, 0xEE]); + let bytes = buf.freeze(); + let (decoded, consumed) = decode_hints(&bytes).unwrap(); + assert_eq!(consumed, bytes.len()); + assert_eq!(decoded, vec![unknown_hint(7, &[0xEE])]); +} + +#[test] +fn unknown_hint_preserved() { + round_trip(vec![unknown_hint(0x7F, b"\xDE\xAD")]); +} + +#[test] +fn hints_keep_one_item_per_type_latest_wins_in_type_order() { + let hints = Hints::from(vec![ + unknown_hint(0x41, b"old"), + unknown_hint(0x07, b"x"), + HintItem::BundleLength(10), + unknown_hint(0x41, b"new"), + ]); + assert_eq!(hints.len(), 3); + assert_eq!( + hints.iter().collect::>(), + vec![ + HintItem::BundleLength(10), + unknown_hint(0x07, b"x"), + unknown_hint(0x41, b"new"), + ] + ); + assert_eq!(hints.clone().into_vec(), hints.iter().collect::>()); + assert_eq!(hints.into_iter().count(), 3); +} + +#[test] +fn hints_compare_by_items_not_insertion_order() { + let a = Hints::from(vec![unknown_hint(0x07, b"x"), unknown_hint(0x41, b"y")]); + let b = Hints::from(vec![unknown_hint(0x41, b"y"), unknown_hint(0x07, b"x")]); + assert_eq!(a, b); +} + +#[test] +fn malformed_bundle_length_and_bundle_length_replace_each_other() { + // A type-0 item of a length Section 9.1 does not allow is carried + // opaquely, and it replaces a well-formed Bundle Length... + let hints = Hints::from(vec![HintItem::BundleLength(10), unknown_hint(0, b"abc")]); + assert_eq!(hints.bundle_length(), None); + assert_eq!( + hints.get(HintType::BUNDLE_LENGTH), + Some(unknown_hint(0, b"abc")) + ); + assert_eq!(hints.len(), 1); + + // ...as a well-formed one replaces it. + let hints = Hints::from(vec![unknown_hint(0, b"abc"), HintItem::BundleLength(10)]); + assert_eq!(hints.bundle_length(), Some(10)); + assert_eq!( + hints.get(HintType::BUNDLE_LENGTH), + Some(HintItem::BundleLength(10)) + ); + assert_eq!(hints.len(), 1); +} + +#[test] +fn hints_get_and_remove_by_type() { + let mut hints = Hints::from(vec![HintItem::BundleLength(10), unknown_hint(0x41, b"c")]); + let correlator = HintType::new(0x41).unwrap(); + let absent = HintType::new(0x42).unwrap(); + assert_eq!(hints.get(correlator), Some(unknown_hint(0x41, b"c"))); + assert_eq!(hints.get(absent), None); + + assert_eq!(hints.remove(absent), None); + assert_eq!( + hints.remove(HintType::BUNDLE_LENGTH), + Some(HintItem::BundleLength(10)) + ); + assert_eq!(hints.remove(correlator), Some(unknown_hint(0x41, b"c"))); + assert!(hints.is_empty()); + assert_eq!(hints, Hints::new()); +} + +#[test] +fn hints_encoded_len_matches_the_encoder() { + for hints in [ + Hints::new(), + Hints::from(vec![HintItem::BundleLength(0x1_0000)]), + Hints::from(vec![ + HintItem::BundleLength(u64::MAX), + unknown_hint(0x41, b"corr"), + unknown_hint(0x7F, b""), + ]), + ] { + let mut buf = BytesMut::new(); + encode_hints(&hints.clone().into_vec(), &mut buf); + assert_eq!(hints.encoded_len(), buf.len()); + } +} diff --git a/btpu/tests/codec_message.rs b/btpu/tests/codec_message.rs new file mode 100644 index 000000000..a10c82704 --- /dev/null +++ b/btpu/tests/codec_message.rs @@ -0,0 +1,124 @@ +//! Message type, flag, and frame classification through the public +//! `codec::message` API. + +use hardy_btpu::codec::message::{ + FrameKind, MessageFlags, MessageType, frame_kind, is_reserved_bpv6, is_reserved_bpv7, +}; + +#[test] +fn message_flags_nibble_round_trips_all_bits() { + // The registry governs the whole nibble (Section 12.3, Standards + // Action); every value must survive decode -> encode so relayed + // unknown messages keep future-assigned bits. + for nibble in 0..=0xF { + let flags = MessageFlags::from_nibble(nibble); + assert_eq!(flags.to_nibble(), nibble, "nibble {nibble:#x}"); + assert_eq!(flags.hint, nibble & 0x8 != 0, "nibble {nibble:#x}"); + assert_eq!(flags.rfu, nibble & 0x7, "nibble {nibble:#x}"); + } +} + +#[test] +fn from_byte_accepts_known_types() { + let cases = [ + (0x00, MessageType::IndefinitePadding), + (0x01, MessageType::DefinitePadding), + (0x02, MessageType::Bundle), + (0x03, MessageType::TransferSegment), + (0x04, MessageType::TransferEnd), + (0x05, MessageType::TransferCancel), + (0x70, MessageType::PreAgreedFecSource), + (0x71, MessageType::ExplicitFecSource), + (0x72, MessageType::PreAgreedFecRepair), + (0x73, MessageType::ExplicitFecRepair), + ]; + for (byte, expected) in cases { + assert_eq!(MessageType::from_byte(byte), Some(expected)); + assert_eq!(u8::from(expected), byte); + } +} + +#[test] +fn from_byte_rejects_reserved_and_unassigned_values() { + assert_eq!(MessageType::from_byte(0x06), None); + for b in 0x80u8..=0x9F { + assert_eq!(MessageType::from_byte(b), None, "byte {b:#04x}"); + } + for b in [0x07u8, 0x10, 0x50, 0x6F, 0x74, 0xA0, 0xFF] { + assert_eq!(MessageType::from_byte(b), None, "byte {b:#04x}"); + } +} + +#[test] +fn is_fec_covers_exactly_the_four_extension_types() { + for t in [ + MessageType::PreAgreedFecSource, + MessageType::ExplicitFecSource, + MessageType::PreAgreedFecRepair, + MessageType::ExplicitFecRepair, + ] { + assert!(t.is_fec(), "{t:?}"); + } + for t in [ + MessageType::IndefinitePadding, + MessageType::DefinitePadding, + MessageType::Bundle, + MessageType::TransferSegment, + MessageType::TransferEnd, + MessageType::TransferCancel, + ] { + assert!(!t.is_fec(), "{t:?}"); + } +} + +#[test] +fn is_reserved_bpv6_covers_value() { + assert!(is_reserved_bpv6(0x06)); + assert!(!is_reserved_bpv6(0x05)); + assert!(!is_reserved_bpv6(0x07)); + assert!(!is_reserved_bpv6(0x00)); +} + +#[test] +fn is_reserved_bpv7_covers_range() { + assert!(!is_reserved_bpv7(0x7F)); + for b in 0x80u8..=0x9F { + assert!(is_reserved_bpv7(b)); + } + assert!(!is_reserved_bpv7(0xA0)); +} + +#[test] +fn frame_kind_empty_is_btpu() { + assert_eq!(frame_kind(&[]), FrameKind::BtpuPdu); +} + +#[test] +fn frame_kind_bpv6() { + assert_eq!(frame_kind(&[0x06]), FrameKind::Bpv6Bundle); + assert_eq!(frame_kind(&[0x06, 0x01, 0x02]), FrameKind::Bpv6Bundle); +} + +#[test] +fn frame_kind_bpv7_full_range() { + for b in 0x80u8..=0x9F { + assert_eq!(frame_kind(&[b]), FrameKind::Bpv7Bundle, "byte {b:#04x}"); + } +} + +#[test] +fn frame_kind_known_btpu_types_classify_as_pdu() { + for b in [0x00u8, 0x01, 0x02, 0x03, 0x04, 0x05, 0x70, 0x71, 0x72, 0x73] { + assert_eq!(frame_kind(&[b]), FrameKind::BtpuPdu, "byte {b:#04x}"); + } +} + +#[test] +fn frame_kind_unallocated_btpu_space_classifies_as_pdu() { + // Bytes outside the reserved ranges and not yet assigned a BTP-U + // message type still belong to BTP-U; decoders parse them as + // Message::Unknown. + for b in [0x07u8, 0x6F, 0x74, 0x7F, 0xA0, 0xFF] { + assert_eq!(frame_kind(&[b]), FrameKind::BtpuPdu, "byte {b:#04x}"); + } +} diff --git a/btpu/tests/common/mod.rs b/btpu/tests/common/mod.rs new file mode 100644 index 000000000..81050281b --- /dev/null +++ b/btpu/tests/common/mod.rs @@ -0,0 +1,548 @@ +//! Fixtures shared between the integration-test binaries. + +// Each test binary uses its own subset of these helpers, so per-binary +// unused ones are expected. +#![allow(dead_code)] + +#[cfg(feature = "rand")] +use core::convert::Infallible; + +use bytes::{Bytes, BytesMut}; +use hardy_btpu::{ + codec::{ + Error as CodecError, decode_pdu, encode_message, + header::HEADER_SIZE, + hint::{HintItem, HintType, HintValue, Hints, encoded_hints_len}, + message::{Message, TransferSegmentMessage}, + }, + receiver::{ + DropReason, MaxSegments, MaxTransferSize, Receiver, ReceiverConfig, ReceiverEvent, + RejectReason, + }, + sender::{ + BundleFraming, Carried, Error, LinkFraming, Pdu, PduSize, SendId, SendKind, SendOptions, + SendQueueBytes, Sender, SenderConfig, + }, + transfer::{TransferId, WindowSize}, +}; +#[cfg(feature = "rand")] +use rand::rand_core::TryRng; + +// An unknown hint item of type `hint_type` carrying `value`. +pub fn unknown_hint(hint_type: u8, value: &'static [u8]) -> HintItem { + HintItem::Unknown { + hint_type: HintType::new(hint_type).unwrap(), + value: HintValue::new(Bytes::from_static(value)).unwrap(), + } +} + +pub fn window_size(v: u16) -> WindowSize { + WindowSize::try_from(v).unwrap() +} + +pub fn max_transfer_size(v: usize) -> MaxTransferSize { + MaxTransferSize::try_from(v).unwrap() +} + +// A sender with the given PDU and window sizes, default queue depth, +// fixed-size framing, and transfer numbers starting at zero. +pub fn sender(pdu_size: usize, window: u16) -> Sender { + Sender::new(sender_config(pdu_size, window), 0) +} + +pub fn sender_config(pdu_size: usize, window: u16) -> SenderConfig { + SenderConfig { + pdu_size: PduSize::try_from(pdu_size).unwrap(), + window_size: window_size(window), + ..SenderConfig::default() + } +} + +// As `sender` with a 16-transfer window, and the configuration then +// changed by `f`. +pub fn sender_with(pdu_size: usize, f: impl FnOnce(&mut SenderConfig)) -> Sender { + let mut config = sender_config(pdu_size, 16); + f(&mut config); + Sender::new(config, 0) +} + +// As `sender` with a 16-transfer window and the given link framing, +// admitting until `bytes` bundle bytes are queued. +pub fn sender_with_queue_bytes(pdu_size: usize, bytes: usize, link_framing: LinkFraming) -> Sender { + sender_with(pdu_size, |c| { + c.send_queue_bytes = SendQueueBytes::try_from(bytes).unwrap(); + c.link_framing = link_framing; + }) +} + +// As `sender` with a 16-transfer window on a variable-length link with no +// padding floor. +pub fn variable_sender(pdu_size: usize, bundle_framing: BundleFraming) -> Sender { + floored_sender(pdu_size, bundle_framing, 0) +} + +// As `variable_sender`, padding every PDU up to `min_pdu_len`. +pub fn floored_sender( + pdu_size: usize, + bundle_framing: BundleFraming, + min_pdu_len: usize, +) -> Sender { + sender_with(pdu_size, |c| { + c.link_framing = LinkFraming::Variable { + bundle_framing, + min_pdu_len, + }; + }) +} + +// The transfer number and segment index every segment message carries. +pub const SEGMENT_FIELDS: usize = 8; + +// The data bytes segment 0 of a `len`-byte bundle carries in a full PDU. +pub fn first_capacity(pdu_size: usize, len: usize) -> usize { + pdu_size + - HEADER_SIZE + - SEGMENT_FIELDS + - encoded_hints_len(&[HintItem::BundleLength(len as u64)]) +} + +// `id`, checked to name a segmented bundle. A test knows its transfer +// number from the order of allocation: the `sender*` helpers number +// transfers from zero. +pub fn segmented(id: SendId) -> SendId { + assert_eq!(id.kind(), SendKind::Transfer, "{id:?}"); + id +} + +// The transfer numbers whose segments `pdus` carry, in order of first +// appearance. +pub fn wire_transfer_numbers(pdus: &[Bytes]) -> Vec { + let mut numbers = Vec::new(); + for message in pdus + .iter() + .flat_map(|pdu| decode_pdu(pdu.clone()).map(Result::unwrap)) + { + if let Message::TransferSegment(m) | Message::TransferEnd(m) = message + && !numbers.contains(&m.transfer_number) + { + numbers.push(m.transfer_number); + } + } + numbers +} + +// Enqueue `data` with no caller hints. +pub fn enqueue(s: &mut Sender, data: Bytes) -> Result { + s.enqueue(data, SendOptions::default()) +} + +// Every PDU `s` has pending, in order, with the bundles each carries. +pub fn drain_pdus(s: &mut Sender) -> Vec { + let mut pdus = Vec::new(); + while let Some(pdu) = s.next_pdu() { + pdus.push(pdu); + } + pdus +} + +// Every PDU `s` has pending, in order. +pub fn drain(s: &mut Sender) -> Vec { + drain_pdus(s).into_iter().map(|pdu| pdu.data).collect() +} + +// Every message in `pdu`, its padding included. +pub fn decode_all(pdu: Bytes) -> Vec { + decode_pdu(pdu).collect::>().unwrap() +} + +pub fn carried(id: SendId, completes: bool) -> Carried { + Carried { id, completes } +} + +// The data `pdus` carry for transfer `t`, in order. +pub fn transfer_data(pdus: &[Bytes], t: u32) -> Vec { + pdus.iter() + .flat_map(|pdu| decode_pdu(pdu.clone()).map(Result::unwrap)) + .filter_map(|m| match m { + Message::TransferSegment(m) | Message::TransferEnd(m) if m.transfer_number == t => { + Some(m.data) + } + _ => None, + }) + .flatten() + .collect() +} + +// A receiver with the given window size and bundle-size cap and every +// other setting at its default (FEC decoding off, no segment or retention +// limit). +pub fn receiver(window: u16, max_transfer_size: usize) -> Receiver { + Receiver::new(receiver_config(window, max_transfer_size)) +} + +pub fn receiver_config(window: u16, cap: usize) -> ReceiverConfig { + ReceiverConfig { + window_size: window_size(window), + max_transfer_size: max_transfer_size(cap), + ..ReceiverConfig::default() + } +} + +// As `receiver`, with a per-transfer segment limit. +pub fn receiver_with_max_segments(window: u16, cap: usize, max_segments: MaxSegments) -> Receiver { + Receiver::new(ReceiverConfig { + max_segments_per_transfer: Some(max_segments), + ..receiver_config(window, cap) + }) +} + +/// A [`ReceiverEvent`] with each `TransferId` replaced by its wire +/// transfer number, so that a test can spell the events it expects: a +/// `TransferId` has no public constructor. A `ReceiverEvent` compares +/// equal to the `Event` it maps to. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum Event { + Received { + data: Bytes, + hints: Hints, + }, + TransferStarted { + transfer_number: u32, + hints: Hints, + }, + TransferData { + transfer_number: u32, + data: Bytes, + hints: Option, + }, + TransferFinished { + transfer_number: u32, + data: Bytes, + hints: Option, + }, + TransferCancelled { + transfer_number: u32, + }, + TransferExpired { + transfer_number: u32, + }, + MessageDropped { + transfer_number: u32, + reason: DropReason, + }, + TransferRejected { + transfer_number: u32, + reason: RejectReason, + }, + BundleRejected { + len: usize, + }, + MalformedMessage { + error: CodecError, + }, + MalformedPdu { + error: CodecError, + }, +} + +impl From for Event { + fn from(event: ReceiverEvent) -> Self { + match event { + ReceiverEvent::Received { data, hints } => Self::Received { data, hints }, + ReceiverEvent::TransferStarted { id, hints } => Self::TransferStarted { + transfer_number: id.transfer_number(), + hints, + }, + ReceiverEvent::TransferData { id, data, hints } => Self::TransferData { + transfer_number: id.transfer_number(), + data, + hints, + }, + ReceiverEvent::TransferFinished { id, data, hints } => Self::TransferFinished { + transfer_number: id.transfer_number(), + data, + hints, + }, + ReceiverEvent::TransferCancelled { id } => Self::TransferCancelled { + transfer_number: id.transfer_number(), + }, + ReceiverEvent::TransferExpired { id } => Self::TransferExpired { + transfer_number: id.transfer_number(), + }, + ReceiverEvent::MessageDropped { + transfer_number, + id, + reason, + } => { + assert!(drop_id_consistent(transfer_number, id, reason)); + Self::MessageDropped { + transfer_number, + reason, + } + } + ReceiverEvent::TransferRejected { id, reason } => Self::TransferRejected { + transfer_number: id.transfer_number(), + reason, + }, + ReceiverEvent::BundleRejected { len } => Self::BundleRejected { len }, + ReceiverEvent::MalformedMessage { error } => Self::MalformedMessage { error }, + ReceiverEvent::MalformedPdu { error } => Self::MalformedPdu { error }, + } + } +} + +impl PartialEq for ReceiverEvent { + fn eq(&self, other: &Event) -> bool { + match (self, other) { + (ReceiverEvent::Received { data, hints }, Event::Received { data: d, hints: h }) => { + data == d && hints == h + } + ( + ReceiverEvent::TransferStarted { id, hints }, + Event::TransferStarted { + transfer_number: n, + hints: h, + }, + ) => id.transfer_number() == *n && hints == h, + ( + ReceiverEvent::TransferData { id, data, hints }, + Event::TransferData { + transfer_number: n, + data: d, + hints: h, + }, + ) + | ( + ReceiverEvent::TransferFinished { id, data, hints }, + Event::TransferFinished { + transfer_number: n, + data: d, + hints: h, + }, + ) => id.transfer_number() == *n && data == d && hints == h, + ( + ReceiverEvent::TransferCancelled { id }, + Event::TransferCancelled { transfer_number: n }, + ) + | ( + ReceiverEvent::TransferExpired { id }, + Event::TransferExpired { transfer_number: n }, + ) => id.transfer_number() == *n, + ( + ReceiverEvent::MessageDropped { + transfer_number, + id, + reason, + }, + Event::MessageDropped { + transfer_number: n, + reason: r, + }, + ) => { + transfer_number == n + && reason == r + && drop_id_consistent(*transfer_number, *id, *reason) + } + ( + ReceiverEvent::TransferRejected { id, reason }, + Event::TransferRejected { + transfer_number: n, + reason: r, + }, + ) => id.transfer_number() == *n && reason == r, + (ReceiverEvent::BundleRejected { len }, Event::BundleRejected { len: l }) => len == l, + (ReceiverEvent::MalformedMessage { error }, Event::MalformedMessage { error: e }) + | (ReceiverEvent::MalformedPdu { error }, Event::MalformedPdu { error: e }) => { + error == e + } + _ => false, + } + } +} + +// What `ReceiverEvent::MessageDropped` documents of its `id`: none outside +// the receive window, and otherwise one naming the dropped number. +fn drop_id_consistent(transfer_number: u32, id: Option, reason: DropReason) -> bool { + let outside = matches!( + reason, + DropReason::OutsideWindow | DropReason::UnknownTransfer + ); + match id { + None => outside, + Some(id) => !outside && id.transfer_number() == transfer_number, + } +} + +// No events, for comparing with what a call returned: a bare `vec![]` +// could be a `Vec` of either event type. +pub fn none() -> Vec { + Vec::new() +} + +pub fn received(data: &'static [u8]) -> Event { + received_with(Bytes::from_static(data), vec![]) +} + +pub fn received_with(data: Bytes, hints: Vec) -> Event { + Event::Received { + data, + hints: Hints::from(hints), + } +} + +pub fn dropped(transfer_number: u32, reason: DropReason) -> Event { + Event::MessageDropped { + transfer_number, + reason, + } +} + +pub fn rejected(transfer_number: u32, reason: RejectReason) -> Event { + Event::TransferRejected { + transfer_number, + reason, + } +} + +pub fn expired(transfer_number: u32) -> Event { + Event::TransferExpired { transfer_number } +} + +pub fn cancelled(transfer_number: u32) -> Event { + Event::TransferCancelled { transfer_number } +} + +pub fn bundle_msg(data: &'static [u8]) -> Message { + bundle_with(vec![], Bytes::from_static(data)) +} + +pub fn bundle_with(hints: Vec, data: Bytes) -> Message { + Message::Bundle { hints, data } +} + +pub fn segment(transfer_number: u32, segment_index: u32, data: &'static [u8]) -> Message { + segment_with( + transfer_number, + segment_index, + vec![], + Bytes::from_static(data), + ) +} + +pub fn segment_with( + transfer_number: u32, + segment_index: u32, + hints: Vec, + data: Bytes, +) -> Message { + Message::TransferSegment(TransferSegmentMessage { + transfer_number, + segment_index, + hints, + data, + }) +} + +pub fn end(transfer_number: u32, segment_index: u32, data: &'static [u8]) -> Message { + end_with( + transfer_number, + segment_index, + vec![], + Bytes::from_static(data), + ) +} + +pub fn end_with( + transfer_number: u32, + segment_index: u32, + hints: Vec, + data: Bytes, +) -> Message { + Message::TransferEnd(TransferSegmentMessage { + transfer_number, + segment_index, + hints, + data, + }) +} + +pub fn cancel(transfer_number: u32) -> Message { + Message::TransferCancel { transfer_number } +} + +// A bundle of `len` filler bytes. +pub fn bundle(len: usize) -> Bytes { + Bytes::from(vec![0x42; len]) +} + +// A bundle whose bytes differ, so a misplaced or reordered byte would show. +pub fn patterned(len: usize) -> Bytes { + (0..len).map(|i| i as u8).collect() +} + +// A payload `frame_kind` classifies as a BPv7 bundle: the CBOR +// indefinite-array header (0x9F) followed by filler. +pub fn bpv7_like(len: usize) -> Bytes { + let mut v = vec![0x9F]; + v.resize(len, 0xAB); + Bytes::from(v) +} + +// Whether `inner` is a view into the allocation `outer` was decoded from. +pub fn is_within(inner: &Bytes, outer: &Bytes) -> bool { + let start = outer.as_ptr() as usize; + let p = inner.as_ptr() as usize; + p >= start && p + inner.len() <= start + outer.len() +} + +// One message on the wire, as a PDU of its own or a piece of one. +pub fn encode(msg: &Message) -> Bytes { + let mut buf = BytesMut::new(); + encode_message(msg, &mut buf).unwrap(); + buf.freeze() +} + +// A deterministic RNG for the `from_rng` constructors. Implemented +// against the `rand` crate's `rand_core` re-export, proving this crate's +// `rand_core` version lines up with the `rand` in use. +#[cfg(feature = "rand")] +pub struct FixedRng(pub u32); + +#[cfg(feature = "rand")] +impl TryRng for FixedRng { + type Error = Infallible; + fn try_next_u32(&mut self) -> Result { + Ok(self.0) + } + fn try_next_u64(&mut self) -> Result { + Ok(self.0 as u64) + } + fn try_fill_bytes(&mut self, dst: &mut [u8]) -> Result<(), Self::Error> { + dst.fill(0); + Ok(()) + } +} + +// An RNG that always fails, for the error path of the `try_from_rng` +// constructors. +#[cfg(feature = "rand")] +pub struct FailingRng; + +#[cfg(feature = "rand")] +#[derive(Debug, PartialEq, Eq, thiserror::Error)] +#[error("RNG failure")] +pub struct RngFailure; + +#[cfg(feature = "rand")] +impl TryRng for FailingRng { + type Error = RngFailure; + fn try_next_u32(&mut self) -> Result { + Err(RngFailure) + } + fn try_next_u64(&mut self) -> Result { + Err(RngFailure) + } + fn try_fill_bytes(&mut self, _dst: &mut [u8]) -> Result<(), Self::Error> { + Err(RngFailure) + } +} diff --git a/btpu/tests/config.rs b/btpu/tests/config.rs new file mode 100644 index 000000000..1abe1af61 --- /dev/null +++ b/btpu/tests/config.rs @@ -0,0 +1,146 @@ +//! Serde round trips and range validation of the configuration types. + +#![cfg(feature = "serde")] + +use hardy_btpu::{ + receiver::{Delivery, MaxRetainedBytes, MaxSegments, MaxTransferSize, ReceiverConfig}, + sender::{ + BundleFraming, LinkFraming, PduSize, SegmentCutStrategy, SendQueueBytes, SenderConfig, + }, + transfer::WindowSize, +}; +use serde_json::{from_str, json, to_value}; + +#[test] +fn sender_config_round_trips_as_kebab_case_integers() { + let config = SenderConfig { + pdu_size: PduSize::try_from(1200).unwrap(), + window_size: WindowSize::try_from(32).unwrap(), + send_queue_bytes: SendQueueBytes::try_from(8).unwrap(), + link_framing: LinkFraming::Variable { + bundle_framing: BundleFraming::Bare, + min_pdu_len: 46, + }, + segment_cut_strategy: SegmentCutStrategy::Half, + }; + let value = to_value(config).unwrap(); + assert_eq!( + value, + json!({ + "pdu-size": 1200, + "window-size": 32, + "send-queue-bytes": 8, + "link-framing": { "variable": { "bundle-framing": "bare", "min-pdu-len": 46 } }, + "segment-cut-strategy": "half", + }) + ); + assert_eq!( + from_str::(&value.to_string()).unwrap(), + config + ); +} + +#[test] +fn receiver_config_round_trips_as_kebab_case_integers() { + let config = ReceiverConfig { + window_size: WindowSize::try_from(32).unwrap(), + max_transfer_size: MaxTransferSize::try_from(4096).unwrap(), + max_segments_per_transfer: Some(MaxSegments::try_from(1500).unwrap()), + max_retained_bytes: Some(MaxRetainedBytes::try_from(65_536).unwrap()), + fec: true, + delivery: Delivery::Streamed, + }; + let value = to_value(config).unwrap(); + assert_eq!( + value, + json!({ + "window-size": 32, + "max-transfer-size": 4096, + "max-segments-per-transfer": 1500, + "max-retained-bytes": 65536, + "fec": true, + "delivery": "streamed", + }) + ); + assert_eq!( + from_str::(&value.to_string()).unwrap(), + config + ); +} + +#[test] +fn missing_fields_take_their_defaults() { + assert_eq!( + from_str::("{}").unwrap(), + SenderConfig::default() + ); + assert_eq!( + from_str::("{}").unwrap(), + ReceiverConfig::default() + ); + assert_eq!(ReceiverConfig::default().max_segments_per_transfer, None); + assert_eq!(ReceiverConfig::default().max_retained_bytes, None); + assert_eq!( + from_str::(r#"{"max-segments-per-transfer": null}"#).unwrap(), + ReceiverConfig::default() + ); + assert_eq!( + from_str::(r#"{"link-framing": "fixed-size"}"#).unwrap(), + SenderConfig::default() + ); + // The struct variant's own field defaults too. + assert_eq!( + from_str::(r#"{"link-framing": {"variable": {}}}"#).unwrap(), + SenderConfig { + link_framing: LinkFraming::Variable { + bundle_framing: BundleFraming::Message, + min_pdu_len: 0, + }, + ..SenderConfig::default() + } + ); +} + +#[test] +fn out_of_range_values_are_rejected_on_deserialize() { + // The newtypes re-validate, so a configuration file cannot smuggle in + // a value TryFrom would refuse. serde_json wraps the newtype's own + // error, so the message is the observable: it must be the TryFrom + // error's text naming the range, classified as a data error. + let cases = [ + ( + from_str::(r#"{"pdu-size": 3}"#).unwrap_err(), + "Invalid PDU size 3 (must be 4..=1048579)", + ), + ( + from_str::(r#"{"window-size": 4096}"#).unwrap_err(), + "Invalid window size 4096 (must be 4..=4095)", + ), + ( + from_str::(r#"{"send-queue-bytes": 0}"#).unwrap_err(), + "Invalid send queue bytes 0 (must be at least 1)", + ), + ( + from_str::(r#"{"max-transfer-size": 0}"#).unwrap_err(), + "Invalid max transfer size 0 (must be at least 1)", + ), + ( + from_str::(r#"{"max-segments-per-transfer": 0}"#).unwrap_err(), + "Invalid max segments per transfer 0 (must be at least 1)", + ), + ( + from_str::(r#"{"max-retained-bytes": 0}"#).unwrap_err(), + "Invalid max retained bytes 0 (must be at least 1)", + ), + ( + from_str::(r#"{"window-size": 3}"#).unwrap_err(), + "Invalid window size 3 (must be 4..=4095)", + ), + ]; + for (err, expected) in cases { + assert!(err.is_data(), "{err}"); + let text = err.to_string(); + let (message, _location) = text.split_once(" at line ").unwrap_or((&text, "")); + assert_eq!(message, expected); + } +} diff --git a/btpu/tests/receiver.rs b/btpu/tests/receiver.rs new file mode 100644 index 000000000..b66434c0f --- /dev/null +++ b/btpu/tests/receiver.rs @@ -0,0 +1,2064 @@ +//! Reassembly, window bookkeeping, dispositions, and PDU fault containment +//! through the public `receiver` API. + +mod common; + +use std::{ + num::{NonZeroU32, NonZeroUsize}, + panic::{AssertUnwindSafe, catch_unwind}, +}; + +use bytes::{BufMut, Bytes, BytesMut}; +use hardy_btpu::{ + OutOfRange, ParseError, + codec::{ + Error as CodecError, encode_message, + header::HEADER_SIZE, + hint::{HintItem, HintType, HintValue}, + message::Message, + pad_pdu, + }, + fec::{ExplicitFecMessage, PreAgreedFecMessage}, + receiver::{ + Delivery, DropReason, HINT_OVERHEAD, MIN_OVERHEAD_BUDGET, MaxRetainedBytes, MaxSegments, + MaxTransferSize, Receiver, ReceiverConfig, ReceiverEvent, RejectReason, SEGMENT_OVERHEAD, + }, + sender::Sender, +}; + +use self::common::{ + Event, bpv7_like, bundle, bundle_msg, bundle_with, cancel, cancelled, drain, dropped, encode, + end, end_with, enqueue, expired, is_within, max_transfer_size, none, received, received_with, + receiver, receiver_config, receiver_with_max_segments, rejected, segment, segment_with, sender, + unknown_hint, +}; + +fn default_receiver() -> Receiver { + Receiver::new(ReceiverConfig::default()) +} + +#[test] +fn config_defaults() { + let c = ReceiverConfig::default(); + assert_eq!(c.window_size.get(), 16); + assert_eq!(c.max_transfer_size.get(), 0x4000_0000); + assert_eq!(c.max_segments_per_transfer, None); + assert!(!c.fec); + assert_eq!(c.delivery, Delivery::Whole); +} + +#[test] +fn max_transfer_size_zero_rejected() { + assert_eq!(MaxTransferSize::new(0), None); + assert_eq!( + MaxTransferSize::try_from(0), + Err(OutOfRange { + name: "max transfer size", + value: 0, + min: 1, + max: None, + }) + ); + assert_eq!(MaxTransferSize::try_from(1), Ok(MaxTransferSize::MIN)); + assert_eq!( + MaxTransferSize::try_from(usize::MAX), + Ok(MaxTransferSize::MAX) + ); +} + +#[test] +fn max_transfer_size_converts_to_and_from_non_zero() { + let n = NonZeroUsize::new(4096).unwrap(); + let cap = MaxTransferSize::from(n); + assert_eq!(cap.get(), 4096); + assert_eq!(NonZeroUsize::from(cap), n); + assert_eq!(usize::from(cap), 4096); + assert_eq!(cap.to_string(), "4096"); + assert_eq!(MaxTransferSize::default(), MaxTransferSize::DEFAULT); +} + +#[test] +fn max_transfer_size_parses_and_formats_as_its_integer() { + assert_eq!("1073741824".parse(), Ok(MaxTransferSize::DEFAULT)); + assert_eq!( + "0".parse::(), + Err(ParseError::OutOfRange( + MaxTransferSize::try_from(0).unwrap_err() + )) + ); + assert_eq!( + "1 GiB".parse::(), + Err(ParseError::Syntax { + name: "max transfer size", + source: "1 GiB".parse::().unwrap_err(), + }) + ); + let m = MaxTransferSize::DEFAULT; + assert_eq!(format!("{m:#x} {m:#o}"), "0x40000000 0o10000000000"); + let m = max_transfer_size(0xBEEF); + assert_eq!(format!("{m:x} {m:X}"), "beef BEEF"); +} + +#[test] +fn bundle_message_immediate() { + let mut r = default_receiver(); + let events = r.process_message(bundle_msg(b"hello")); + assert_eq!(events, vec![received(b"hello")]); +} + +#[test] +fn two_segment_transfer() { + let mut r = default_receiver(); + assert_eq!(r.process_message(segment(0, 0, b"hel")), none()); + assert_eq!( + r.process_message(end(0, 1, b"lo")), + vec![received(b"hello")] + ); +} + +#[test] +fn out_of_order_completes_on_end_recheck() { + let mut r = default_receiver(); + assert_eq!(r.process_message(segment(0, 1, b"ld")), none()); + assert_eq!(r.process_message(segment(0, 0, b"wor")), none()); + assert_eq!( + r.process_message(end(0, 2, b"!")), + vec![received(b"world!")] + ); +} + +#[test] +fn end_before_late_segment_completes_on_the_segment() { + let mut r = default_receiver(); + assert_eq!(r.process_message(end(0, 1, b"ld")), none()); + // The late segment 0 fills the final gap; completion fires here, on + // the segment insert, even though the End arrived earlier. + assert_eq!( + r.process_message(segment(0, 0, b"wor")), + vec![received(b"world")] + ); +} + +#[test] +fn conflicting_end_dropped_and_transfer_still_completes() { + let mut r = default_receiver(); + r.process_message(segment(0, 0, b"a")); + r.process_message(end(0, 2, b"c")); + + // A second End disagreeing with the established final index is + // dropped; accepting it would wedge completion forever. + assert_eq!( + r.process_message(end(0, 1, b"y")), + vec![dropped(0, DropReason::SegmentIndexConflict)] + ); + + // The established index (and not the bogus End's data) still stands: + // the missing middle segment completes the transfer. + assert_eq!( + r.process_message(segment(0, 1, b"b")), + vec![received(b"abc")] + ); +} + +#[test] +fn segment_beyond_final_index_dropped() { + let mut r = default_receiver(); + r.process_message(segment(0, 0, b"a")); + r.process_message(end(0, 2, b"c")); + + // A stray segment above the established final index is dropped; + // storing it would keep the map's highest key above N forever. + assert_eq!( + r.process_message(segment(0, 7, b"x")), + vec![dropped(0, DropReason::SegmentIndexConflict)] + ); + assert_eq!( + r.process_message(segment(0, 1, b"b")), + vec![received(b"abc")] + ); +} + +#[test] +fn end_below_seen_segment_dropped() { + let mut r = default_receiver(); + r.process_message(segment(0, 0, b"a")); + r.process_message(segment(0, 1, b"b")); + r.process_message(segment(0, 2, b"c")); + + // An End claiming a final index below a segment already seen is + // bogus (segments beyond the final index cannot exist); it must not + // record a final index the map can never satisfy. + assert_eq!( + r.process_message(end(0, 1, b"y")), + vec![dropped(0, DropReason::SegmentIndexConflict)] + ); + // The genuine End (a repeat of segment 2's data) completes the transfer. + assert_eq!(r.process_message(end(0, 2, b"c")), vec![received(b"abc")]); +} + +#[test] +fn duplicate_segment_dropped_and_first_copy_kept() { + let mut r = default_receiver(); + r.process_message(segment(0, 0, b"abc")); + // Section 6 permits duplicates; the first copy wins. + assert_eq!( + r.process_message(segment(0, 0, b"SHOULD BE IGNORED")), + vec![dropped(0, DropReason::Duplicate)] + ); + assert_eq!( + r.process_message(end(0, 1, b"def")), + vec![received(b"abcdef")] + ); +} + +#[test] +fn duplicate_end_dropped() { + let mut r = default_receiver(); + r.process_message(end(0, 1, b"def")); + assert_eq!( + r.process_message(end(0, 1, b"def")), + vec![dropped(0, DropReason::Duplicate)] + ); + // A Segment at the final index repeats the End too. + assert_eq!( + r.process_message(segment(0, 1, b"def")), + vec![dropped(0, DropReason::Duplicate)] + ); + assert_eq!( + r.process_message(segment(0, 0, b"abc")), + vec![received(b"abcdef")] + ); +} + +#[test] +fn duplicate_hints_are_not_applied() { + let mut r = default_receiver(); + let first = unknown_hint(0x41, b"\x07"); + r.process_message(segment_with( + 0, + 0, + vec![first.clone()], + Bytes::from_static(b"abc"), + )); + assert_eq!( + r.process_message(segment_with( + 0, + 0, + vec![unknown_hint(0x41, b"\x09")], + Bytes::from_static(b"abc"), + )), + vec![dropped(0, DropReason::Duplicate)] + ); + assert_eq!( + r.process_message(end(0, 1, b"def")), + vec![received_with(Bytes::from_static(b"abcdef"), vec![first])] + ); +} + +#[test] +fn empty_end_completes_the_transfer() { + // Section 4: the transfer is complete once segments 0..=N are present. + // A streaming sender that only learns the input ended after emitting a + // full segment finishes with an empty End at N (Section 8.3 says it + // SHOULD NOT, not that a receiver may refuse it). + let mut r = default_receiver(); + r.process_message(segment(0, 0, b"ab")); + assert_eq!(r.process_message(end(0, 1, b"")), vec![received(b"ab")]); +} + +#[test] +fn empty_middle_segment_counts_toward_completion() { + let mut r = default_receiver(); + r.process_message(segment(0, 0, b"a")); + assert_eq!(r.process_message(segment(0, 1, b"")), none()); + assert_eq!(r.process_message(end(0, 2, b"c")), vec![received(b"ac")]); +} + +#[test] +fn repeated_cancel_is_idempotent() { + let mut r = default_receiver(); + r.process_message(segment(5, 0, b"data")); + assert_eq!(r.process_message(cancel(5)), vec![cancelled(5)]); + // A repeated Cancel is reported as a drop, not a second cancellation. + assert_eq!( + r.process_message(cancel(5)), + vec![dropped(5, DropReason::Cancelled)] + ); + // And a late End must not deliver a bundle. + assert_eq!( + r.process_message(end(5, 1, b"tail")), + vec![dropped(5, DropReason::Cancelled)] + ); +} + +#[test] +fn cancel_of_unknown_transfer_ignored() { + let mut r = default_receiver(); + r.process_message(segment(0, 0, b"hel")); + + // Section 8.4: Cancel for a never-seen transfer number is ignored: + // no TransferCancelled, and no window advance (a large number here + // would otherwise expire transfer 0). + assert_eq!( + r.process_message(cancel(1000)), + vec![dropped(1000, DropReason::UnknownTransfer)] + ); + assert_eq!( + r.process_message(end(0, 1, b"lo")), + vec![received(b"hello")] + ); +} + +#[test] +fn cancel_of_an_in_window_transfer_before_its_segments_is_remembered() { + // Section 8.4: "prior or later received Segments ... MUST be + // discarded", and Section 5 makes every number in the window an + // in-progress transfer. A Cancel reordered ahead of the segments it + // cancels (or repeated after them) must therefore stick. + let mut r = default_receiver(); + r.process_message(segment(5, 0, b"other")); + assert_eq!(r.process_message(cancel(3)), vec![cancelled(3)]); + assert_eq!( + r.process_message(segment(3, 0, b"hel")), + vec![dropped(3, DropReason::Cancelled)] + ); + assert_eq!( + r.process_message(end(3, 1, b"lo")), + vec![dropped(3, DropReason::Cancelled)] + ); +} + +#[test] +fn final_segment_index_of_u32_max_leaves_the_transfer_open() { + // 2^32 segments can never all be present, so the completeness check + // must neither overflow nor complete: the transfer stays held. + let mut r = default_receiver(); + assert_eq!(r.process_message(end(0, u32::MAX, b"end")), none()); + assert_eq!(r.process_message(segment(0, 0, b"x")), none()); + assert_eq!( + r.retained_bytes(), + b"end".len() + b"x".len() + 2 * SEGMENT_OVERHEAD + ); +} + +#[test] +fn repeated_messages_of_a_delivered_transfer_do_not_redeliver() { + // Section 6 lets a sender repeat any message. A repeat of a completed + // transfer must neither deliver the bundle twice nor leave a phantom + // transfer behind to expire later. + let mut r = receiver(4, usize::MAX); + let mut pdu = BytesMut::new(); + put_completing_transfer(&mut pdu); + let pdu = pdu.freeze(); + assert_eq!(r.receive_pdu(pdu.clone()), vec![received(b"hello")]); + assert_eq!( + r.receive_pdu(pdu), + vec![ + dropped(0, DropReason::Delivered), + dropped(0, DropReason::Delivered) + ] + ); + // A lone repeated segment opens nothing: when the window moves past + // transfer 0 there is nothing to expire. + assert_eq!( + r.process_message(segment(0, 0, b"hel")), + vec![dropped(0, DropReason::Delivered)] + ); + for t in 1..=4u32 { + assert_eq!(r.process_message(segment(t, 0, b"x")), none()); + } +} + +#[test] +fn outside_window_drop_reported() { + let mut r = receiver(4, usize::MAX); + for t in 0..8u32 { + r.process_message(segment(t, 0, b"x")); + } + // greatest = 7, window = 4: transfer 0 is well outside. + assert_eq!( + r.process_message(segment(0, 1, b"y")), + vec![dropped(0, DropReason::OutsideWindow)] + ); +} + +#[test] +fn window_wraparound_with_live_transfers() { + let mut r = receiver(4, usize::MAX); + let first = u32::MAX - 1; + for t in [first, u32::MAX, 0, 1] { + assert_eq!(r.process_message(segment(t, 0, b"x")), none()); + } + // greatest = 1; MAX-1 is 3 behind it modulo 2^32, still in window. + assert_eq!( + r.process_message(end(first, 1, b"y")), + vec![received(b"xy")] + ); + // 2 pushes nothing out (MAX is 3 behind); 3 expires MAX exactly. + assert_eq!(r.process_message(segment(2, 0, b"x")), none()); + assert_eq!( + r.process_message(segment(3, 0, b"x")), + vec![expired(u32::MAX)] + ); +} + +#[test] +fn transfers_straddling_the_wrap_expire_oldest_first() { + let mut r = receiver(4, usize::MAX); + for t in [u32::MAX - 1, u32::MAX, 0] { + assert_eq!(r.process_message(segment(t, 0, b"x")), none()); + } + // Numeric order would report 0 first; window order puts it last. + assert_eq!( + r.process_message(segment(5, 0, b"x")), + vec![expired(u32::MAX - 1), expired(u32::MAX), expired(0),] + ); +} + +#[test] +fn number_behind_a_just_wrapped_greatest_expires_before_it() { + let mut r = receiver(4, usize::MAX); + assert_eq!(r.process_message(segment(0, 0, b"x")), none()); + // Both are behind 0, not far ahead of it. + assert_eq!(r.process_message(segment(u32::MAX - 2, 0, b"x")), none()); + assert_eq!(r.process_message(segment(u32::MAX, 0, b"x")), none()); + + assert_eq!( + r.process_message(segment(1, 0, b"x")), + vec![expired(u32::MAX - 2)] + ); + assert_eq!( + r.process_message(segment(4, 0, b"x")), + vec![expired(u32::MAX), expired(0),] + ); +} + +#[test] +fn number_more_than_half_the_space_ahead_advances_the_window() { + // Section 5 treats up to 2^31 + W/2 - 1 ahead as new, past the half of + // the space where serial-number comparison would call it behind. + let mut r = receiver(4, usize::MAX); + assert_eq!(r.process_message(segment(0, 0, b"x")), none()); + let far = (1 << 31) + 1; + assert_eq!(r.process_message(segment(far, 0, b"x")), vec![expired(0)]); + // The window now trails `far`: a number just behind it is open, and the + // next new one expires them oldest first. + assert_eq!(r.process_message(segment(far - 3, 0, b"x")), none()); + assert_eq!( + r.process_message(segment(far + 1, 0, b"x")), + vec![expired(far - 3)] + ); + assert_eq!( + r.process_message(segment(far + 5, 0, b"x")), + vec![expired(far), expired(far + 1),] + ); +} + +#[test] +fn reset_forgets_delivered_transfers() { + let mut r = default_receiver(); + assert_eq!(r.process_message(end(3, 0, b"x")), vec![received(b"x")]); + assert_eq!( + r.process_message(end(3, 0, b"x")), + vec![dropped(3, DropReason::Delivered)] + ); + + // After a reset the same number is a new transfer from a new sender. + r.reset(); + assert_eq!(r.process_message(end(3, 0, b"x")), vec![received(b"x")]); +} + +#[test] +fn reset_accepts_a_restarted_sender() { + let mut r = receiver(4, usize::MAX); + for t in 1000..1004u32 { + r.process_message(segment(t, 0, b"x")); + } + // A peer that restarted from a low number is outside the window... + assert_eq!( + r.process_message(segment(3, 0, b"he")), + vec![dropped(3, DropReason::OutsideWindow)] + ); + // ...until the CLA, told of the restart, resets the receiver. + r.reset(); + assert_eq!(r.process_message(segment(3, 0, b"he")), none()); + assert_eq!(r.process_message(end(3, 1, b"y")), vec![received(b"hey")]); +} + +// A pre-agreed FEC Source message for `transfer_number`. +fn pre_agreed_fec(transfer_number: u32, fec_instance_id: u8) -> Message { + Message::PreAgreedFecSource(PreAgreedFecMessage { + transfer_number, + fec_instance_id, + hints: vec![], + payload: Bytes::from_static(b"fec"), + }) +} + +// An explicit FEC Repair message for `transfer_number`. +fn explicit_fec(transfer_number: u32, fec_encoding_id: u8) -> Message { + Message::ExplicitFecRepair(ExplicitFecMessage { + transfer_number, + fec_encoding_id, + hints: vec![], + payload: Bytes::from_static(b"fec"), + }) +} + +fn fec_receiver() -> Receiver { + Receiver::new(ReceiverConfig { + fec: true, + ..ReceiverConfig::default() + }) +} + +#[test] +fn fec_transfer_expires_like_a_core_one() { + let mut r = fec_receiver(); + assert_eq!(r.process_message(pre_agreed_fec(0, 1)), none()); + assert_eq!(r.process_message(segment(16, 0, b"x")), vec![expired(0)]); +} + +#[test] +fn fec_message_on_core_transfer_rejects_it() { + // FEC Section 3.2: mixing MUST cancel the transfer, so the core + // transfer can no longer complete. + let mut r = default_receiver(); + r.process_message(segment(0, 0, b"hel")); + assert_eq!( + r.process_message(pre_agreed_fec(0, 1)), + vec![rejected(0, RejectReason::FecCoreMixing)] + ); + assert_eq!( + r.process_message(end(0, 1, b"lo")), + vec![dropped( + 0, + DropReason::Rejected(RejectReason::FecCoreMixing) + )] + ); +} + +#[test] +fn core_message_on_fec_transfer_rejects_it() { + let mut r = default_receiver(); + assert_eq!(r.process_message(pre_agreed_fec(0, 1)), none()); + assert_eq!( + r.process_message(segment(0, 0, b"x")), + vec![rejected(0, RejectReason::FecCoreMixing)] + ); + assert_eq!( + r.process_message(pre_agreed_fec(0, 1)), + vec![dropped( + 0, + DropReason::Rejected(RejectReason::FecCoreMixing) + )] + ); +} + +#[test] +fn core_fec_mixing_rejects_through_receive_pdu() { + // Both directions through the decoder, with FEC decoding enabled. + let mut r = fec_receiver(); + assert_eq!(r.receive_pdu(encode(&segment(0, 0, b"abc"))), none()); + assert_eq!( + r.receive_pdu(encode(&pre_agreed_fec(0, 1))), + vec![rejected(0, RejectReason::FecCoreMixing)] + ); + assert_eq!( + r.receive_pdu(encode(&end(0, 1, b"def"))), + vec![dropped( + 0, + DropReason::Rejected(RejectReason::FecCoreMixing) + )] + ); + + assert_eq!(r.receive_pdu(encode(&explicit_fec(1, 6))), none()); + assert_eq!( + r.receive_pdu(encode(&end(1, 0, b"x"))), + vec![rejected(1, RejectReason::FecCoreMixing)] + ); + assert_eq!( + r.receive_pdu(encode(&explicit_fec(1, 6))), + vec![dropped( + 1, + DropReason::Rejected(RejectReason::FecCoreMixing) + )] + ); +} + +#[test] +fn cancel_closes_an_fec_transfer() { + let mut r = fec_receiver(); + assert_eq!(r.process_message(pre_agreed_fec(0, 1)), none()); + assert_eq!(r.process_message(cancel(0)), vec![cancelled(0)]); + assert_eq!( + r.process_message(pre_agreed_fec(0, 1)), + vec![dropped(0, DropReason::Cancelled)] + ); +} + +#[test] +fn fec_hint_bytes_count_against_retention() { + // The payload is not stored, so the retained hint is the whole charge. + let value = b"corr"; + let hint_charge = HINT_OVERHEAD + value.len(); + let message = || { + Message::PreAgreedFecSource(PreAgreedFecMessage { + transfer_number: 0, + fec_instance_id: 1, + hints: vec![unknown_hint(0x41, value)], + payload: Bytes::from_static(b"fec"), + }) + }; + let mut r = fec_receiver(); + assert_eq!(r.process_message(message()), none()); + assert_eq!(r.retained_bytes(), hint_charge); + + // Under a limit one byte short, the same message is refused. + let mut r = Receiver::new(ReceiverConfig { + fec: true, + max_retained_bytes: Some(MaxRetainedBytes::try_from(hint_charge - 1).unwrap()), + ..ReceiverConfig::default() + }); + assert_eq!( + r.process_message(message()), + vec![rejected(0, RejectReason::ReceiverFull)] + ); +} + +#[test] +fn fec_messages_with_one_configuration_keep_the_transfer_open() { + let mut r = fec_receiver(); + let repair = Message::PreAgreedFecRepair(PreAgreedFecMessage { + transfer_number: 0, + fec_instance_id: 1, + hints: vec![], + payload: Bytes::from_static(b"repair"), + }); + assert_eq!(r.receive_pdu(encode(&pre_agreed_fec(0, 1))), none()); + assert_eq!(r.receive_pdu(encode(&repair)), none()); + assert_eq!(r.receive_pdu(encode(&pre_agreed_fec(0, 1))), none()); +} + +#[test] +fn changed_fec_instance_id_rejects_the_transfer() { + // FEC Section 3.1: a change of FEC Instance ID MUST cancel the + // transfer, and the original ID cannot revive it. + let mut r = fec_receiver(); + assert_eq!(r.receive_pdu(encode(&pre_agreed_fec(0, 1))), none()); + assert_eq!( + r.receive_pdu(encode(&pre_agreed_fec(0, 2))), + vec![rejected(0, RejectReason::FecConfigurationChanged)] + ); + assert_eq!( + r.receive_pdu(encode(&pre_agreed_fec(0, 1))), + vec![dropped( + 0, + DropReason::Rejected(RejectReason::FecConfigurationChanged) + )] + ); +} + +#[test] +fn changed_fec_encoding_id_rejects_the_transfer() { + // FEC Section 3: the FEC Framework Configuration Information MUST NOT + // change mid-transfer; the Encoding ID is the part visible without a + // scheme. + let mut r = fec_receiver(); + assert_eq!(r.receive_pdu(encode(&explicit_fec(0, 6))), none()); + assert_eq!( + r.receive_pdu(encode(&explicit_fec(0, 5))), + vec![rejected(0, RejectReason::FecConfigurationChanged)] + ); +} + +#[test] +fn switching_between_pre_agreed_and_explicit_fec_rejects_the_transfer() { + // An Instance ID and an Encoding ID name different things, so the + // switch is a change even when the two values are equal. + let mut r = fec_receiver(); + assert_eq!(r.receive_pdu(encode(&pre_agreed_fec(0, 1))), none()); + assert_eq!( + r.receive_pdu(encode(&explicit_fec(0, 1))), + vec![rejected(0, RejectReason::FecConfigurationChanged)] + ); + + assert_eq!(r.receive_pdu(encode(&explicit_fec(1, 1))), none()); + assert_eq!( + r.receive_pdu(encode(&pre_agreed_fec(1, 1))), + vec![rejected(1, RejectReason::FecConfigurationChanged)] + ); +} + +#[test] +fn fec_types_do_not_touch_the_window_unless_enabled() { + // 0x70..=0x73 are Private Use. With FEC decoding off, a peer's + // private message of type 0x70 naming a far-off transfer number must + // not open an FEC transfer and expire the live core transfer. + let mut r = receiver(4, usize::MAX); + r.process_message(segment(0, 0, b"hel")); + assert_eq!(r.receive_pdu(encode(&pre_agreed_fec(42, 1))), none()); + assert_eq!( + r.process_message(end(0, 1, b"lo")), + vec![received(b"hello")] + ); + + // With FEC enabled, the same message for transfer 1000 opens an FEC + // transfer and expires 0. + let mut r = fec_receiver(); + r.process_message(segment(0, 0, b"hel")); + assert_eq!( + r.receive_pdu(encode(&pre_agreed_fec(1000, 1))), + vec![expired(0)] + ); +} + +#[test] +fn fec_transfer_promising_an_oversized_bundle_is_rejected() { + let mut r = Receiver::new(ReceiverConfig { + fec: true, + max_transfer_size: max_transfer_size(5), + ..ReceiverConfig::default() + }); + let pdu = encode(&Message::PreAgreedFecSource(PreAgreedFecMessage { + transfer_number: 9, + fec_instance_id: 1, + hints: vec![HintItem::BundleLength(1 << 40)], + payload: Bytes::from_static(b"fec"), + })); + assert_eq!( + r.receive_pdu(pdu), + vec![rejected(9, RejectReason::TooLarge)] + ); +} + +// Encode a two-segment transfer of b"hello" into `pdu`. +fn put_completing_transfer(pdu: &mut BytesMut) { + pdu.put_slice(&encode(&segment(0, 0, b"hel"))); + pdu.put_slice(&encode(&end(0, 1, b"lo"))); +} + +#[test] +fn malformed_message_mid_pdu_keeps_prior_events_and_continues() { + let mut r = default_receiver(); + let mut pdu = BytesMut::new(); + put_completing_transfer(&mut pdu); + // Bundle message, H flag set, content = malformed hint chain (a hint + // header promising a 255-byte value with no bytes behind it). + pdu.put_slice(&[0x02, 0x80, 0x00, 0x02, 0x1F, 0xFF]); + pdu.put_slice(&encode(&bundle_msg(b"ok"))); + + // The completed transfer's bundle survives the later fault, the bad + // message is reported and skipped, and the trailing Bundle is still + // processed via the next header-length boundary. + assert_eq!( + r.receive_pdu(pdu.freeze()), + vec![ + received(b"hello"), + Event::MalformedMessage { + error: CodecError::InsufficientData { + needed: 257, + available: 2, + }, + }, + received(b"ok"), + ] + ); +} + +#[test] +fn malformed_pdu_keeps_prior_events() { + let mut r = default_receiver(); + // A completing transfer followed by a truncated header: the trailing + // bytes are undecodable (no message boundary), but the bundle + // completed earlier in the PDU must still be delivered alongside the + // MalformedPdu report. + let mut pdu = BytesMut::new(); + put_completing_transfer(&mut pdu); + let header_at = pdu.len(); + pdu.put_slice(&[0x02, 0x30]); // 2 bytes: too short for a 4-byte header + + // Counted from the start of the PDU. + assert_eq!( + r.receive_pdu(pdu.freeze()), + vec![ + received(b"hello"), + Event::MalformedPdu { + error: CodecError::InsufficientData { + needed: header_at + HEADER_SIZE, + available: header_at + 2, + }, + }, + ] + ); +} + +#[test] +fn transfer_is_rejected_as_its_data_passes_the_cap() { + let mut r = receiver(16, 5); + assert_eq!(r.process_message(segment(0, 0, b"abc")), none()); + // Six bytes held: rejected on arrival, not at the End. + assert_eq!( + r.process_message(segment(0, 1, b"def")), + vec![rejected(0, RejectReason::TooLarge)] + ); + assert_eq!(r.retained_bytes(), 0); + assert_eq!( + r.process_message(segment(0, 2, b"x")), + vec![dropped(0, DropReason::Rejected(RejectReason::TooLarge))] + ); +} + +#[test] +fn oversized_bundle_message_rejected_with_event() { + let mut r = receiver(16, 5); + let data = Bytes::from_static(b"too long bundle"); + let len = data.len(); + assert_eq!( + r.process_message(bundle_with(vec![], data)), + vec![Event::BundleRejected { len }] + ); + // Exactly at the limit is accepted. + assert_eq!( + r.process_message(bundle_msg(b"12345")), + vec![received(b"12345")] + ); +} + +#[test] +fn empty_bundle_message_rejected() { + // Section 8.1: the content MUST be a valid bundle, which zero bytes + // cannot be. + let mut r = default_receiver(); + assert_eq!( + r.process_message(bundle_with(vec![], Bytes::new())), + vec![Event::BundleRejected { len: 0 }] + ); +} + +#[test] +fn transfer_with_no_data_is_rejected() { + // The Section 8.1 policy above applies to a reassembled bundle: every + // segment may be empty individually, but not all of them. + let mut r = default_receiver(); + assert_eq!(r.process_message(segment(0, 0, b"")), none()); + assert_eq!( + r.process_message(end(0, 1, b"")), + vec![rejected(0, RejectReason::Empty)] + ); + assert_eq!( + r.process_message(end(0, 1, b"")), + vec![dropped(0, DropReason::Rejected(RejectReason::Empty))] + ); +} + +#[test] +fn empty_single_end_is_rejected_through_receive_pdu() { + let mut r = default_receiver(); + assert_eq!( + r.receive_pdu(encode(&end(0, 0, b""))), + vec![rejected(0, RejectReason::Empty)] + ); +} + +#[test] +fn bundle_length_hint_on_bundle_message_is_ignored() { + // Section 9.1: receivers SHOULD ignore the hint outside Segment/End + // messages; other hints on the Bundle message still surface. + let mut r = default_receiver(); + let other = unknown_hint(0x41, b"z"); + assert_eq!( + r.process_message(bundle_with( + vec![HintItem::BundleLength(2), other.clone()], + Bytes::from_static(b"hi") + )), + vec![received_with(Bytes::from_static(b"hi"), vec![other])] + ); +} + +#[test] +fn transfer_at_exactly_the_cap_is_accepted_and_one_byte_over_rejected() { + let exact = 6; + let mut r = receiver(16, exact); + r.process_message(segment_with( + 0, + 0, + vec![HintItem::BundleLength(6)], + Bytes::from_static(b"abc"), + )); + assert_eq!( + r.process_message(end(0, 1, b"def")), + vec![received_with( + Bytes::from_static(b"abcdef"), + vec![HintItem::BundleLength(6)] + )] + ); + + let mut r = receiver(16, exact - 1); + r.process_message(segment(0, 0, b"abc")); + assert_eq!( + r.process_message(end(0, 1, b"def")), + vec![rejected(0, RejectReason::TooLarge)] + ); +} + +#[test] +fn bundle_of_exactly_the_cap_is_delivered_in_one_byte_segments() { + // The cap is a policy on the bundle's length: per-segment bookkeeping + // is budgeted separately, and six segments are well within the + // bookkeeping floor, so a cap-sized bundle in one-byte segments is not + // pushed over. + let mut r = receiver(16, 6); + for i in 0..5u32 { + assert_eq!(r.process_message(segment(0, i, b"x")), none()); + } + assert_eq!( + r.process_message(end(0, 5, b"x")), + vec![received(b"xxxxxx")] + ); + + // One byte more, and the transfer is rejected the moment the cap is + // exceeded, before its End. + let mut r = receiver(16, 6); + for i in 0..6u32 { + assert_eq!(r.process_message(segment(1, i, b"x")), none()); + } + assert_eq!( + r.process_message(segment(1, 6, b"x")), + vec![rejected(1, RejectReason::TooLarge)] + ); +} + +#[test] +fn tiny_segment_flood_is_rejected_as_too_fragmented() { + // Empty segments carry no bytes the cap could see, so only the + // bookkeeping budget bounds them: with a cap below the floor, the + // floor's worth of segments and not one more. + let mut r = receiver(16, 1); + let budget = MIN_OVERHEAD_BUDGET / SEGMENT_OVERHEAD; + for i in 0..budget { + assert_eq!(r.process_message(segment(0, i as u32, b"")), none()); + } + assert_eq!( + r.process_message(segment(0, budget as u32, b"")), + vec![rejected(0, RejectReason::TooFragmented)] + ); + // And it stays rejected. + assert_eq!( + r.process_message(end(0, budget as u32 + 1, b"")), + vec![dropped( + 0, + DropReason::Rejected(RejectReason::TooFragmented) + )] + ); +} + +#[test] +fn retained_hint_bytes_count_against_the_bookkeeping_budget() { + // Sixteen distinct 255-byte hints, with one segment's overhead, + // exceed the 4 KiB floor. + let hint_charge = 16 * (HINT_OVERHEAD + 255); + assert!(hint_charge + SEGMENT_OVERHEAD > MIN_OVERHEAD_BUDGET); + let mut r = receiver(16, 1); + let hints: Vec = (0x40..0x50u8) + .map(|hint_type| HintItem::Unknown { + hint_type: HintType::new(hint_type).unwrap(), + value: HintValue::new(Bytes::from(vec![hint_type; 255])).unwrap(), + }) + .collect(); + assert_eq!( + r.process_message(segment_with(0, 0, hints.clone(), Bytes::from_static(b"a"))), + vec![rejected(0, RejectReason::TooFragmented)] + ); + + // The same hints on a transfer with a cap to match are fine. + let mut r = receiver(16, hint_charge + SEGMENT_OVERHEAD); + assert_eq!( + r.process_message(end_with(0, 0, hints.clone(), Bytes::from_static(b"a"))), + vec![received_with(Bytes::from_static(b"a"), hints)] + ); +} + +#[test] +fn bundle_length_hint_rejects_before_accumulation() { + let mut r = receiver(16, 5); + assert_eq!( + r.process_message(segment_with( + 0, + 0, + vec![HintItem::BundleLength(100)], + Bytes::from_static(b"a") + )), + vec![rejected(0, RejectReason::TooLarge)] + ); + // A further segment must not re-create the rejected transfer. + assert_eq!( + r.process_message(segment(0, 1, b"x")), + vec![dropped(0, DropReason::Rejected(RejectReason::TooLarge))] + ); +} + +#[test] +fn bundle_length_hint_smaller_than_the_data_still_delivers() { + // The hint is advisory and only ever used to reject early; a sender + // that under-reports is not second-guessed. + let mut r = default_receiver(); + r.process_message(segment_with( + 0, + 0, + vec![HintItem::BundleLength(2)], + Bytes::from_static(b"abc"), + )); + assert_eq!( + r.process_message(end(0, 1, b"def")), + vec![received_with( + Bytes::from_static(b"abcdef"), + vec![HintItem::BundleLength(2)] + )] + ); +} + +#[test] +fn malformed_bundle_length_hint_does_not_discard_the_segment() { + // A type-0 item with a 3-byte value, which Section 9.1 does not allow. + let malformed = unknown_hint(HintType::BUNDLE_LENGTH.get(), &[1, 2, 3]); + let mut r = default_receiver(); + assert_eq!( + r.receive_pdu(encode(&segment_with( + 0, + 0, + vec![malformed.clone()], + Bytes::from_static(b"abc") + ))), + none() + ); + assert_eq!( + r.process_message(end(0, 1, b"d")), + vec![received_with(Bytes::from_static(b"abcd"), vec![malformed])] + ); +} + +#[test] +fn single_segment_transfer_shares_the_segment_bytes() { + let mut r = default_receiver(); + let payload = Bytes::from_static(b"solo bundle"); + let events = r.process_message(end_with(0, 0, vec![], payload.clone())); + // Delivered without copying: the event's Bytes views the same + // allocation as the received segment (a refcount bump, not a + // rebuild). + assert_eq!(events, vec![received(b"solo bundle")]); + let [ReceiverEvent::Received { data, .. }] = events.as_slice() else { + panic!("expected one Received, got {events:?}"); + }; + assert_eq!(data.as_ptr(), payload.as_ptr()); +} + +#[test] +fn segments_shorter_than_half_the_pdu_are_copied_out_of_it() { + // A 4-byte segment in a 64-byte PDU would pin all 64 bytes for the + // life of the transfer; it is copied instead. + let mut r = default_receiver(); + let mut pdu = BytesMut::from(encode(&end(0, 0, b"solo")).as_ref()); + pdu.resize(64, 0); + let pdu = pdu.freeze(); + let events = r.receive_pdu(pdu.clone()); + let [ReceiverEvent::Received { data, .. }] = events.as_slice() else { + panic!("expected one Received, got {events:?}"); + }; + assert_eq!(data.as_ref(), b"solo"); + assert!(!is_within(data, &pdu)); + + // A segment of at least half the PDU stays a view into it. + let mut r = default_receiver(); + let pdu = encode(&end(0, 0, b"twelve bytes")); // 12 + 12 = 24-byte PDU + let events = r.receive_pdu(pdu.clone()); + let [ReceiverEvent::Received { data, .. }] = events.as_slice() else { + panic!("expected one Received, got {events:?}"); + }; + assert!(is_within(data, &pdu)); +} + +#[test] +fn retained_hint_values_are_copied_out_of_the_pdu() { + let mut r = default_receiver(); + let mut pdu = BytesMut::from( + encode(&segment_with( + 0, + 0, + vec![unknown_hint(0x41, b"corr")], + Bytes::from_static(b"hel"), + )) + .as_ref(), + ); + pdu.resize(64, 0); + let pdu = pdu.freeze(); + assert_eq!(r.receive_pdu(pdu.clone()), none()); + let events = r.process_message(end(0, 1, b"lo")); + let [ReceiverEvent::Received { hints, .. }] = events.as_slice() else { + panic!("expected one Received, got {events:?}"); + }; + let hints: Vec = hints.iter().collect(); + let [HintItem::Unknown { value, .. }] = hints.as_slice() else { + panic!("expected the correlator hint, got {hints:?}"); + }; + assert_eq!(value.as_ref(), b"corr"); + assert!(!is_within(value, &pdu)); +} + +#[test] +fn bundle_received_surfaces_transfer_hints() { + let mut r = default_receiver(); + let correlator_v1 = unknown_hint(0x41, b"\x07"); + let correlator_v2 = unknown_hint(0x41, b"\x09"); + r.process_message(segment_with( + 0, + 0, + vec![HintItem::BundleLength(5), correlator_v1], + Bytes::from_static(b"hel"), + )); + // A later message repeating the hint type supersedes the value. + assert_eq!( + r.process_message(end_with( + 0, + 1, + vec![correlator_v2.clone()], + Bytes::from_static(b"lo") + )), + vec![received_with( + Bytes::from_static(b"hello"), + vec![HintItem::BundleLength(5), correlator_v2] + )] + ); +} + +#[test] +fn bundle_message_hints_deduped_latest_wins() { + // The Bundle message path honours the same Received contract as + // the transfer path: one item per hint type, latest wins, ordered by + // type. + let mut r = default_receiver(); + let stale = unknown_hint(0x41, b"old"); + let fresh = unknown_hint(0x41, b"new"); + let other = unknown_hint(0x02, b"x"); + assert_eq!( + // Deliberately out of type order, with the repeat last. + r.process_message(bundle_with( + vec![stale, other.clone(), fresh.clone()], + Bytes::from_static(b"hi"), + )), + vec![received_with(Bytes::from_static(b"hi"), vec![other, fresh])] + ); +} + +#[test] +fn bare_frame_through_receive_pdu() { + let mut r = default_receiver(); + let frame = bpv7_like(20); + assert_eq!( + r.receive_pdu(frame.clone()), + vec![received_with(frame, vec![])] + ); +} + +fn cancels(transfer_numbers: impl IntoIterator) -> Bytes { + let mut pdu = BytesMut::new(); + for transfer_number in transfer_numbers { + pdu.put(encode(&cancel(transfer_number))); + } + pdu.freeze() +} + +#[test] +fn every_message_in_a_pdu_can_produce_an_event() { + let mut r = default_receiver(); + assert_eq!( + r.receive_pdu(cancels(1000..1004)), + (1000..1004) + .map(|t| dropped(t, DropReason::UnknownTransfer)) + .collect::>() + ); +} + +#[test] +fn receive_pdu_into_replaces_the_callers_list() { + let mut r = default_receiver(); + let mut events = r.process_message(bundle_msg(b"stale")); + r.receive_pdu_into(cancels([7, 8]), &mut events); + assert_eq!( + events, + vec![ + dropped(7, DropReason::UnknownTransfer), + dropped(8, DropReason::UnknownTransfer), + ] + ); + + r.receive_pdu_into(Bytes::new(), &mut events); + assert_eq!(events, none()); +} + +#[test] +fn receive_pdu_into_reuses_the_callers_allocation() { + let mut r = default_receiver(); + let mut events = Vec::with_capacity(4); + let allocation = events.as_ptr(); + r.receive_pdu_into(cancels(1000..1004), &mut events); + assert_eq!(events.len(), 4); + r.receive_pdu_into(cancels(2000..2003), &mut events); + assert_eq!( + events, + (2000..2003) + .map(|t| dropped(t, DropReason::UnknownTransfer)) + .collect::>() + ); + assert_eq!(events.as_ptr(), allocation); +} + +#[test] +fn bundle_extent_hook_trims_padding_and_steps_over_mid_pdu_bundles() { + // The caller's "peek": a 0x9F bundle is 20 bytes long here. + let mut r = default_receiver() + .with_bundle_extent(|b: &[u8]| (b.first() == Some(&0x9F) && b.len() >= 20).then_some(20)); + + // A bare frame zero-filled to 46 bytes delivers exactly the bundle. + let mut frame = BytesMut::from(bpv7_like(20).as_ref()); + frame.resize(46, 0); + assert_eq!( + r.receive_pdu(frame.freeze()), + vec![received_with(bpv7_like(20), vec![])] + ); + + // An encapsulated bundle between two transfer messages is delivered + // and the End behind it still completes the transfer (Section 7.3). + let mut pdu = BytesMut::new(); + pdu.put_slice(&encode(&segment(0, 0, b"hel"))); + pdu.put_slice(&bpv7_like(20)); + pdu.put_slice(&encode(&end(0, 1, b"lo"))); + assert_eq!( + r.receive_pdu(pdu.freeze()), + vec![received_with(bpv7_like(20), vec![]), received(b"hello"),] + ); +} + +#[test] +fn bundle_extent_hook_survives_its_own_panic() { + // A hook that panics must not leave the receiver silently hookless: the + // next bare frame must reach the same hook again rather than being + // taken whole as a bundle. + let mut r = default_receiver().with_bundle_extent(|_: &[u8]| -> Option { + panic!("hook failed"); + }); + let frame = bpv7_like(8); + for _ in 0..2 { + let payload = catch_unwind(AssertUnwindSafe(|| r.receive_pdu(frame.clone()))) + .expect_err("the hook was not consulted"); + assert_eq!(payload.downcast_ref::<&str>(), Some(&"hook failed")); + } +} + +#[test] +fn mid_pdu_bundle_without_hook_ends_the_pdu() { + let mut r = default_receiver(); + let mut pdu = BytesMut::new(); + pdu.put_slice(&encode(&segment(0, 0, b"hel"))); + let bundle_offset = pdu.len(); + pdu.put_slice(&bpv7_like(20)); + pdu.put_slice(&encode(&end(0, 1, b"lo"))); + let pdu = pdu.freeze(); + assert_eq!( + r.receive_pdu(pdu.clone()), + vec![Event::MalformedPdu { + error: CodecError::EncapsulatedBundle { + first_byte: 0x9F, + offset: bundle_offset + } + }] + ); + // The offset locates the discarded bytes in the caller's clone. + assert!(pdu[bundle_offset..].starts_with(&bpv7_like(20))); +} + +#[test] +fn sender_receiver_round_trip() { + let mut s = sender(64, 16); + let mut r = default_receiver(); + let original = bundle(200); + enqueue(&mut s, original.clone()).unwrap(); + assert_eq!( + drain_into(&mut s, &mut r), + vec![received_with( + original.clone(), + vec![HintItem::BundleLength(200)] + )] + ); +} + +#[test] +fn sender_receiver_round_trip_small() { + let mut s = sender(256, 16); + let mut r = default_receiver(); + enqueue(&mut s, Bytes::from_static(b"tiny")).unwrap(); + assert_eq!( + r.receive_pdu(s.next_pdu().unwrap().data), + vec![received(b"tiny")] + ); +} + +// Drain `s` into `r` through the PDU entry point, collecting every event. +fn drain_into(s: &mut Sender, r: &mut Receiver) -> Vec { + drain(s) + .into_iter() + .flat_map(|pdu| r.receive_pdu(pdu)) + .collect() +} + +fn for_link(link_pdu_size: usize, cap: usize) -> u32 { + MaxSegments::for_link_pdu_size(link_pdu_size, max_transfer_size(cap)).get() +} + +#[test] +fn max_segments_zero_rejected() { + assert_eq!(MaxSegments::new(0), None); + assert_eq!( + MaxSegments::try_from(0), + Err(OutOfRange { + name: "max segments per transfer", + value: 0, + min: 1, + max: None, + }) + ); + assert_eq!(MaxSegments::try_from(1), Ok(MaxSegments::MIN)); + assert_eq!(MaxSegments::try_from(u32::MAX), Ok(MaxSegments::MAX)); +} + +#[test] +fn max_segments_converts_to_and_from_non_zero() { + let n = NonZeroU32::new(1500).unwrap(); + let limit = MaxSegments::from(n); + assert_eq!(limit.get(), 1500); + assert_eq!(NonZeroU32::from(limit), n); + assert_eq!(u32::from(limit), 1500); + assert_eq!(limit.to_string(), "1500"); +} + +#[test] +fn max_segments_parses_and_formats_as_its_integer() { + assert_eq!("1500".parse(), Ok(MaxSegments::try_from(1500).unwrap())); + assert_eq!( + "0".parse::(), + Err(ParseError::OutOfRange( + MaxSegments::try_from(0).unwrap_err() + )) + ); + assert_eq!( + "".parse::(), + Err(ParseError::Syntax { + name: "max segments per transfer", + source: "".parse::().unwrap_err(), + }) + ); + let m = MaxSegments::MAX; + assert_eq!(format!("{m:x} {m:X}"), "ffffffff FFFFFFFF"); +} + +#[test] +fn link_derived_limit_is_four_times_the_reference_count() { + // 1036-byte PDUs carry 1024 data bytes per segment: 1 MiB is 1024 + // segments, so the limit is 4096. + assert_eq!(for_link(1036, 1 << 20), 4096); + // One byte more needs a 1025th segment. + assert_eq!(for_link(1036, (1 << 20) + 1), 4100); +} + +#[test] +fn link_derived_limit_never_falls_below_64() { + assert_eq!(for_link(1036, 100), 64); + assert_eq!(for_link(1036, 1), 64); +} + +#[test] +fn link_derived_limit_of_a_pdu_no_larger_than_the_framing_assumes_one_byte_segments() { + // A PDU of 12 bytes or fewer has no room for segment data; the + // reference capacity is one byte rather than a division by zero. + for pdu in [0, 1, 11, 12, 13] { + assert_eq!(for_link(pdu, 1000), 4000, "PDU size {pdu}"); + } +} + +#[test] +fn link_derived_limit_caps_segment_data_at_the_content_length_ceiling() { + // A PDU beyond one message's ceiling still carries at most 1048567 + // data bytes per segment (20-bit content length less 8 bytes of + // transfer number and index). + assert_eq!(for_link(usize::MAX, 1_048_567 * 100), 400); +} + +#[test] +fn link_derived_limit_saturates_at_u32_max() { + assert_eq!(for_link(1, usize::MAX), u32::MAX); + // 2^30 one-byte segments, times four, is 2^32: one past u32::MAX. + assert_eq!(for_link(13, 1 << 30), u32::MAX); +} + +#[test] +fn small_pdu_link_delivers_a_bundle_the_per_segment_charge_rejects() { + let link_48 = MaxSegments::for_link_pdu_size(48, max_transfer_size(10_000)); + // 48-byte PDUs carry 36 data bytes per segment (26 in the first, which + // carries the Bundle Length hint), so a 9000-byte bundle is 251 + // segments. Charged 64 bytes each, they exceed a 10000-byte budget; + // limited by count instead, they are well within 4 × 278. + let data = bundle(9000); + let expected = vec![received_with( + data.clone(), + vec![HintItem::BundleLength(9000)], + )]; + + let mut s = sender(48, 16); + enqueue(&mut s, data.clone()).unwrap(); + let mut r = receiver_with_max_segments(16, 10_000, link_48); + assert_eq!(drain_into(&mut s, &mut r), expected); + + let mut s = sender(48, 16); + enqueue(&mut s, data).unwrap(); + let mut r = receiver(16, 10_000); + let events = drain_into(&mut s, &mut r); + assert_eq!(events[0], rejected(0, RejectReason::TooFragmented)); + assert!( + events[1..] + .iter() + .all(|e| *e == dropped(0, DropReason::Rejected(RejectReason::TooFragmented))), + "{events:?}" + ); +} + +#[test] +fn bundle_of_exactly_the_cap_is_delivered_with_a_link_derived_limit() { + let link_48 = MaxSegments::for_link_pdu_size(48, max_transfer_size(10_000)); + let data = bundle(10_000); + let mut s = sender(48, 16); + enqueue(&mut s, data.clone()).unwrap(); + let mut r = receiver_with_max_segments(16, 10_000, link_48); + assert_eq!( + drain_into(&mut s, &mut r), + vec![received_with(data, vec![HintItem::BundleLength(10_000)])] + ); +} + +// Empty segments count toward the limit without counting toward the +// 100-byte cap. +const LIMIT: u32 = 400; + +fn receiver_limited_to(limit: u32) -> Receiver { + receiver_with_max_segments(16, 100, MaxSegments::try_from(limit).unwrap()) +} + +fn at_the_limit() -> Receiver { + let mut r = receiver_limited_to(LIMIT); + for i in 0..LIMIT { + assert_eq!(r.process_message(segment(0, i, b"")), none(), "segment {i}"); + } + r +} + +#[test] +fn transfer_of_exactly_the_segment_limit_completes() { + let mut r = receiver_limited_to(LIMIT); + // The End first: its segment counts, and completion still works when + // the earlier segments follow. + assert_eq!(r.process_message(end(0, LIMIT - 1, b"x")), none()); + for i in 0..LIMIT - 2 { + assert_eq!(r.process_message(segment(0, i, b"")), none()); + } + assert_eq!( + r.process_message(segment(0, LIMIT - 2, b"")), + vec![received(b"x")] + ); +} + +#[test] +fn segment_beyond_the_limit_rejects_the_transfer_as_too_fragmented() { + let mut r = at_the_limit(); + assert_eq!( + r.process_message(segment(0, LIMIT, b"")), + vec![rejected(0, RejectReason::TooFragmented)] + ); + // Later messages do not re-create it. + assert_eq!( + r.process_message(segment(0, 0, b"")), + vec![dropped( + 0, + DropReason::Rejected(RejectReason::TooFragmented) + )] + ); + assert_eq!( + r.process_message(end(0, LIMIT + 1, b"")), + vec![dropped( + 0, + DropReason::Rejected(RejectReason::TooFragmented) + )] + ); +} + +#[test] +fn repeated_segments_do_not_consume_the_allowance() { + let mut r = at_the_limit(); + for i in [0, LIMIT / 2, LIMIT - 1] { + assert_eq!( + r.process_message(segment(0, i, b"")), + vec![dropped(0, DropReason::Duplicate)] + ); + } + // Repeating the final index as an End is not a new segment either: it + // completes the transfer rather than rejecting it as fragmented. With + // every segment empty there is no bundle to deliver. + assert_eq!( + r.process_message(end(0, LIMIT - 1, b"")), + vec![rejected(0, RejectReason::Empty)] + ); +} + +#[test] +fn transfer_end_cannot_bypass_the_segment_limit() { + let mut r = at_the_limit(); + assert_eq!( + r.process_message(end(0, LIMIT, b"")), + vec![rejected(0, RejectReason::TooFragmented)] + ); +} + +#[test] +fn too_large_takes_precedence_over_too_fragmented() { + // The segment over the limit also carries the cap past its bytes. + let mut r = at_the_limit(); + assert_eq!( + r.process_message(segment_with(0, LIMIT, vec![], Bytes::from(vec![0; 101]))), + vec![rejected(0, RejectReason::TooLarge)] + ); +} + +#[test] +fn bundle_length_hint_does_not_raise_the_segment_limit() { + let mut r = receiver_limited_to(LIMIT); + r.process_message(segment_with( + 0, + 0, + vec![HintItem::BundleLength(100)], + Bytes::new(), + )); + for i in 1..LIMIT { + assert_eq!(r.process_message(segment(0, i, b"")), none()); + } + assert_eq!( + r.process_message(segment(0, LIMIT, b"")), + vec![rejected(0, RejectReason::TooFragmented)] + ); +} + +#[test] +fn large_pdus_do_not_raise_the_segment_limit() { + // The same limit through receive_pdu, with every segment in a 4 KiB + // PDU. + let mut r = receiver_limited_to(LIMIT); + let padded = |msg: Message| { + let mut pdu = BytesMut::from(&encode(&msg)[..]); + pdu.resize(4096, 0); + pdu.freeze() + }; + for i in 0..LIMIT { + assert_eq!(r.receive_pdu(padded(segment(0, i, b""))), none()); + } + assert_eq!( + r.receive_pdu(padded(segment(0, LIMIT, b""))), + vec![rejected(0, RejectReason::TooFragmented)] + ); +} + +#[test] +fn without_a_segment_limit_the_per_segment_charge_still_applies() { + // The same empty segments, charged 64 bytes each against the 4 KiB + // floor, are refused at the 65th. + let mut r = receiver(16, 100); + let floor = (MIN_OVERHEAD_BUDGET / SEGMENT_OVERHEAD) as u32; + for i in 0..floor { + assert_eq!(r.process_message(segment(0, i, b"")), none()); + } + assert_eq!( + r.process_message(segment(0, floor, b"")), + vec![rejected(0, RejectReason::TooFragmented)] + ); +} + +#[test] +fn debug_summarises_held_and_closed_transfers() { + let mut r = receiver(4, 1024); + r.process_message(end(7, 0, b"done")); + r.process_message(segment(8, 0, b"open")); + let limit = 2 * 1024 + MIN_OVERHEAD_BUDGET; + let retained = b"open".len() + SEGMENT_OVERHEAD; + assert_eq!( + format!("{r:?}"), + format!( + "Receiver {{ max_transfer_size: MaxTransferSize(1024), segment_limit: None, \ + max_retained_bytes: MaxRetainedBytes({limit}), retained: {retained}, fec: false, \ + delivery: Whole, budget: None, bundle_extent: false, window: TransferWindow {{ \ + greatest: Some(TransferId(8)), window_size: WindowSize(4), epoch: 1 }}, \ + transfers: 1, closed: 1 }}" + ) + ); +} + +#[test] +fn max_retained_bytes_zero_rejected() { + assert_eq!(MaxRetainedBytes::new(0), None); + assert_eq!( + MaxRetainedBytes::try_from(0), + Err(OutOfRange { + name: "max retained bytes", + value: 0, + min: 1, + max: None, + }) + ); + assert_eq!(MaxRetainedBytes::try_from(1), Ok(MaxRetainedBytes::MIN)); + assert_eq!( + MaxRetainedBytes::try_from(usize::MAX), + Ok(MaxRetainedBytes::MAX) + ); +} + +#[test] +fn max_retained_bytes_converts_parses_and_formats_as_its_integer() { + let n = NonZeroUsize::new(65_536).unwrap(); + let limit = MaxRetainedBytes::from(n); + assert_eq!(limit.get(), 65_536); + assert_eq!(NonZeroUsize::from(limit), n); + assert_eq!(usize::from(limit), 65_536); + assert_eq!(limit.to_string(), "65536"); + assert_eq!(format!("{limit:x}"), "10000"); + let lettered = MaxRetainedBytes::try_from(0xBEEF).unwrap(); + assert_eq!(format!("{lettered:x} {lettered:X}"), "beef BEEF"); + assert_eq!("65536".parse(), Ok(limit)); + assert_eq!( + "0".parse::(), + Err(ParseError::OutOfRange( + MaxRetainedBytes::try_from(0).unwrap_err() + )) + ); + assert_eq!( + "".parse::(), + Err(ParseError::Syntax { + name: "max retained bytes", + source: "".parse::().unwrap_err(), + }) + ); +} + +#[test] +fn retention_for_transfers_is_the_finest_segmentation_charge() { + let one = NonZeroUsize::MIN; + // Twice the cap of PDUs kept alive plus a bookkeeping budget of the + // cap... + assert_eq!( + MaxRetainedBytes::for_transfers(one, MaxTransferSize::DEFAULT, None).get(), + 3 * MaxTransferSize::DEFAULT.get() + ); + // ...or of MIN_OVERHEAD_BUDGET under a small cap. + assert_eq!( + MaxRetainedBytes::for_transfers(one, max_transfer_size(100), None).get(), + 2 * 100 + MIN_OVERHEAD_BUDGET + ); + assert_eq!( + MaxRetainedBytes::for_transfers(one, MaxTransferSize::MAX, None), + MaxRetainedBytes::MAX + ); + // With a segment limit: every allowed segment's overhead, plus one + // maximal hint of each of the 128 types... + let limit = MaxSegments::try_from(1000).unwrap(); + assert_eq!( + MaxRetainedBytes::for_transfers(one, MaxTransferSize::DEFAULT, Some(limit)).get(), + 2 * MaxTransferSize::DEFAULT.get() + 1000 * SEGMENT_OVERHEAD + MAX_HINT_CHARGE + ); + // ...or the smaller bookkeeping budget, which hints cannot exceed. + assert_eq!( + MaxRetainedBytes::for_transfers(one, max_transfer_size(100), Some(limit)).get(), + 2 * 100 + 1000 * SEGMENT_OVERHEAD + MIN_OVERHEAD_BUDGET + ); + // Each further transfer adds one transfer's charge, saturating. + let three = NonZeroUsize::new(3).unwrap(); + assert_eq!( + MaxRetainedBytes::for_transfers(three, max_transfer_size(100), Some(limit)).get(), + 3 * (2 * 100 + 1000 * SEGMENT_OVERHEAD + MIN_OVERHEAD_BUDGET) + ); + assert_eq!( + MaxRetainedBytes::for_transfers(NonZeroUsize::MAX, max_transfer_size(100), None), + MaxRetainedBytes::MAX + ); +} + +#[test] +fn max_retained_bytes_reports_the_limit_in_effect() { + // The default, derived from the other limits... + let max_segments = MaxSegments::try_from(10).unwrap(); + let r = receiver_with_max_segments(16, 100, max_segments); + assert_eq!( + r.max_retained_bytes(), + MaxRetainedBytes::for_transfers( + NonZeroUsize::MIN, + max_transfer_size(100), + Some(max_segments) + ) + ); + // ...or the configured value, as given. + let limit = MaxRetainedBytes::try_from(12_345).unwrap(); + let r = receiver_with_retention(16, 100, Some(limit)); + assert_eq!(r.max_retained_bytes(), limit); +} + +#[test] +fn smallest_retention_limit_admits_only_what_is_charged_nothing() { + let mut r = Receiver::new(ReceiverConfig { + fec: true, + max_retained_bytes: Some(MaxRetainedBytes::MIN), + ..ReceiverConfig::default() + }); + // Every stored segment is charged its overhead, so every segmented + // transfer is refused... + assert_eq!( + r.process_message(segment(0, 0, b"")), + vec![rejected(0, RejectReason::ReceiverFull)] + ); + // ...while an FEC transfer without hints stores nothing and is held. + assert_eq!(r.process_message(pre_agreed_fec(1, 1)), none()); + assert_eq!(r.retained_bytes(), 0); +} + +// The most one bundle from this crate's sender, filling PDUs of +// `link_pdu_size` bytes, is charged: the ReceiverConfig "Filled PDUs" +// formula. +fn filled_pdu_charge(cap: usize, link_pdu_size: usize) -> usize { + // A segment's framing: the header, transfer number and segment index. + let data_per_segment = link_pdu_size - (HEADER_SIZE + 8); + // The first segment's Bundle Length hint takes at most 10 bytes. + (cap + 10).div_ceil(data_per_segment) * (link_pdu_size + SEGMENT_OVERHEAD) +} + +// The figures in the ReceiverConfig sizing table, for the default cap. +// `filled_pdus_charge_at_most_the_documented_figure` checks the formula +// behind the third column against the real sender. +#[cfg(target_pointer_width = "64")] +#[test] +fn receiver_config_sizing_table_figures() { + const GIB: f64 = (1u64 << 30) as f64; + let cap = MaxTransferSize::DEFAULT; + let gib = |bytes: usize| format!("{:.2}", bytes as f64 / GIB); + let rows: Vec<_> = [1500, 64] + .into_iter() + .map(|link_pdu_size| { + let segments = MaxSegments::for_link_pdu_size(link_pdu_size, cap); + let finest = MaxRetainedBytes::for_transfers(NonZeroUsize::MIN, cap, Some(segments)); + ( + segments.get(), + gib(filled_pdu_charge(cap.get(), link_pdu_size)), + gib(finest.get()), + ) + }) + .collect(); + assert_eq!( + rows, + [ + (2_886_404, "1.05".to_string(), "2.17".to_string()), + (82_595_528, "2.46".to_string(), "6.92".to_string()), + ] + ); +} + +#[test] +fn filled_pdus_charge_at_most_the_documented_figure() { + // A cap-sized bundle from the real sender, under a retention limit of + // exactly the formula's figure, is delivered. The highest charge seen + // between PDUs, which is before the PDU that completes the bundle, is + // within that one segment's charge of the figure, so the formula is + // tight as well as sufficient. + let cap = 64 * 1024; + for link_pdu_size in [1500, 64] { + let figure = filled_pdu_charge(cap, link_pdu_size); + let mut r = Receiver::new(ReceiverConfig { + max_segments_per_transfer: Some(MaxSegments::for_link_pdu_size( + link_pdu_size, + max_transfer_size(cap), + )), + max_retained_bytes: Some(MaxRetainedBytes::try_from(figure).unwrap()), + ..receiver_config(16, cap) + }); + let mut s = sender(link_pdu_size, 16); + enqueue(&mut s, bundle(cap)).unwrap(); + let mut events = Vec::new(); + let mut peak = 0; + for pdu in drain(&mut s) { + events.extend(r.receive_pdu(pdu)); + peak = peak.max(r.retained_bytes()); + } + assert_eq!( + events, + vec![received_with( + bundle(cap), + vec![HintItem::BundleLength(cap as u64)] + )], + "{link_pdu_size}-byte PDUs" + ); + assert!( + figure - peak <= link_pdu_size + SEGMENT_OVERHEAD, + "{link_pdu_size}-byte PDUs: peak {peak}, figure {figure}" + ); + } +} + +#[test] +fn one_transfer_can_be_charged_exactly_for_transfers() { + // Without a segment limit: 64 segments of 64 bytes, each alone in a + // padded 128-byte PDU so it is kept as a view and charged the PDU, + // hold twice the 4 KiB cap and fill the 4 KiB bookkeeping budget. + let mut r = receiver(16, CAP); + for segment_index in 0..64 { + let mut pdu = BytesMut::new(); + encode_message(&zeros(0, segment_index, 64), &mut pdu).unwrap(); + pad_pdu(&mut pdu, 128); + assert_eq!(r.receive_pdu(pdu.freeze()), none()); + } + let limit = MaxRetainedBytes::for_transfers(NonZeroUsize::MIN, max_transfer_size(CAP), None); + assert_eq!(r.retained_bytes(), limit.get()); + + // With one: the same 64 segments and an empty 65th carrying one + // longest hint of every type, under a cap large enough that the + // bookkeeping budget admits them all. + let cap = 64 * 1024; + let max_segments = MaxSegments::try_from(65).unwrap(); + let mut r = receiver_with_max_segments(16, cap, max_segments); + let hints: Vec = (0..=HintType::MAX.get()) + .map(|hint_type| HintItem::Unknown { + hint_type: HintType::new(hint_type).unwrap(), + value: HintValue::new(Bytes::from(vec![0; 255])).unwrap(), + }) + .collect(); + assert_eq!( + r.process_message(segment_with(0, 64, hints, Bytes::new())), + none() + ); + for segment_index in 0..64 { + let mut pdu = BytesMut::new(); + encode_message(&zeros(0, segment_index, 1024), &mut pdu).unwrap(); + pad_pdu(&mut pdu, 2048); + assert_eq!(r.receive_pdu(pdu.freeze()), none()); + } + let limit = MaxRetainedBytes::for_transfers( + NonZeroUsize::MIN, + max_transfer_size(cap), + Some(max_segments), + ); + assert_eq!(r.retained_bytes(), limit.get()); +} + +// The most hint charge one transfer can hold: one longest value of every +// hint type. +const MAX_HINT_CHARGE: usize = 128 * (HINT_OVERHEAD + 255); + +fn receiver_with_retention( + window: u16, + cap: usize, + max_retained_bytes: Option, +) -> Receiver { + Receiver::new(ReceiverConfig { + max_retained_bytes, + ..receiver_config(window, cap) + }) +} + +// A segment of `len` zero bytes, charged `len + SEGMENT_OVERHEAD`. +fn zeros(transfer_number: u32, segment_index: u32, len: usize) -> Message { + segment_with( + transfer_number, + segment_index, + vec![], + Bytes::from(vec![0; len]), + ) +} + +// An End carrying `len` zero bytes. +fn zeros_end(transfer_number: u32, segment_index: u32, len: usize) -> Message { + end_with( + transfer_number, + segment_index, + vec![], + Bytes::from(vec![0; len]), + ) +} + +// The retention limit a 4 KiB cap gets by default: 8 KiB of PDUs kept +// alive plus the 4 KiB MIN_OVERHEAD_BUDGET. +const CAP: usize = 4096; +const DEFAULT_RETENTION: usize = 3 * CAP; +const _: () = assert!(MIN_OVERHEAD_BUDGET == CAP); + +#[test] +fn transfer_that_would_exceed_the_retention_limit_is_rejected() { + let mut r = receiver_with_retention(16, CAP, None); + let held = 4000 + SEGMENT_OVERHEAD; + for transfer_number in 0..3 { + assert_eq!(r.process_message(zeros(transfer_number, 0, 4000)), none()); + } + + // A fourth transfer, within its own limits, would take the total over. + assert!(3 * held + 100 + SEGMENT_OVERHEAD > DEFAULT_RETENTION); + assert_eq!( + r.process_message(zeros(3, 0, 100)), + vec![rejected(3, RejectReason::ReceiverFull)] + ); + assert_eq!( + r.process_message(zeros(3, 1, 1)), + vec![dropped(3, DropReason::Rejected(RejectReason::ReceiverFull))] + ); + + // The transfers already held are kept and complete. + let mut bundle = vec![0; 4000]; + bundle.push(b'x'); + assert_eq!( + r.process_message(end(0, 1, b"x")), + vec![received_with(Bytes::from(bundle), vec![])] + ); + + // Delivery released transfer 0's charge. + assert_eq!(r.process_message(zeros(4, 0, 4000)), none()); +} + +#[test] +fn retention_is_released_by_cancel_expiry_and_rejection() { + // Room for two of these transfers and no more. + let limit = MaxRetainedBytes::try_from(2 * (3900 + SEGMENT_OVERHEAD)).unwrap(); + let mut r = receiver_with_retention(4, CAP, Some(limit)); + r.process_message(zeros(0, 0, 3900)); + r.process_message(zeros(1, 0, 3900)); + + assert_eq!(r.process_message(cancel(1)), vec![cancelled(1)]); + assert_eq!(r.process_message(zeros(2, 0, 3900)), none()); + + // Transfer 4 expires transfer 0, and fits in the space it leaves. + assert_eq!(r.process_message(zeros(4, 0, 3900)), vec![expired(0)]); + + // Rejecting transfer 4 as over its cap leaves room for transfer 3. + assert_eq!( + r.process_message(zeros(4, 1, 300)), + vec![rejected(4, RejectReason::TooLarge)] + ); + assert_eq!(r.process_message(zeros(3, 0, 3900)), none()); +} + +#[test] +fn retention_limit_below_one_transfers_allowance_is_enforced() { + // A total equal to the cap leaves no room for the cap-sized bundle's + // segment bookkeeping, so the bundle is refused though nothing else is + // held and it is within its own limits. + let limit = MaxRetainedBytes::try_from(CAP).unwrap(); + let mut r = receiver_with_retention(16, CAP, Some(limit)); + assert_eq!(r.process_message(zeros(0, 0, CAP - 96)), none()); + assert_eq!( + r.process_message(zeros_end(0, 1, 96)), + vec![rejected(0, RejectReason::ReceiverFull)] + ); + assert_eq!(r.retained_bytes(), 0); + + // A bundle whose charge fits is delivered. + assert_eq!( + r.process_message(zeros(1, 0, CAP - 2 * SEGMENT_OVERHEAD)), + none() + ); + assert_eq!( + r.process_message(zeros_end(1, 1, 0)), + vec![received_with( + Bytes::from(vec![0; CAP - 2 * SEGMENT_OVERHEAD]), + vec![] + )] + ); +} + +#[test] +fn retained_bytes_reports_the_charged_state() { + let mut r = receiver_with_retention(16, CAP, None); + assert_eq!(r.retained_bytes(), 0); + + // Segments are charged their data and SEGMENT_OVERHEAD each. + r.process_message(zeros(0, 0, 1000)); + r.process_message(zeros(0, 1, 1000)); + r.process_message(zeros(1, 0, 500)); + assert_eq!(r.retained_bytes(), 2500 + 3 * SEGMENT_OVERHEAD); + + // A repeated segment is not charged again. + r.process_message(zeros(1, 0, 500)); + assert_eq!(r.retained_bytes(), 2500 + 3 * SEGMENT_OVERHEAD); + + // Delivery and cancellation release a transfer's charge. + r.process_message(zeros_end(0, 2, 0)); + assert_eq!(r.retained_bytes(), 500 + SEGMENT_OVERHEAD); + r.process_message(cancel(1)); + assert_eq!(r.retained_bytes(), 0); + + // As does expiry: transfer 18 moves the window of 16 past transfer 2. + r.process_message(zeros(2, 0, 100)); + assert_eq!(r.process_message(zeros(18, 0, 200)), vec![expired(2)]); + assert_eq!(r.retained_bytes(), 200 + SEGMENT_OVERHEAD); + + // And a reset. + r.reset(); + assert_eq!(r.retained_bytes(), 0); +} + +#[test] +fn retention_limit_above_one_transfers_allowance_is_enforced() { + let limit = MaxRetainedBytes::try_from(3 * (3900 + SEGMENT_OVERHEAD)).unwrap(); + let mut r = receiver_with_retention(16, CAP, Some(limit)); + for transfer_number in 0..3 { + assert_eq!(r.process_message(zeros(transfer_number, 0, 3900)), none()); + } + assert_eq!( + r.process_message(zeros(3, 0, 1)), + vec![rejected(3, RejectReason::ReceiverFull)] + ); +} + +#[test] +fn empty_segments_count_against_the_retention_limit_under_a_segment_limit() { + // The default limit: twice the 100-byte cap, 10 segments' overhead, + // and a 4 KiB hint allowance. Each transfer of ten empty segments is + // charged 640 bytes. + let limit = MaxSegments::try_from(10).unwrap(); + let default = + MaxRetainedBytes::for_transfers(NonZeroUsize::MIN, max_transfer_size(100), Some(limit)) + .get(); + assert_eq!( + default, + 2 * 100 + 10 * SEGMENT_OVERHEAD + MIN_OVERHEAD_BUDGET + ); + let mut r = receiver_with_max_segments(16, 100, limit); + // The first segment over the limit is the one after the last that fits. + let first_over = u32::try_from(default / SEGMENT_OVERHEAD).unwrap(); + let (full_transfer, full_segment) = (first_over / 10, first_over % 10); + let mut charged = 0; + for transfer_number in 0.. { + for segment_index in 0..10 { + let events = r.process_message(segment(transfer_number, segment_index, b"")); + charged += SEGMENT_OVERHEAD; + if charged > default { + let events: Vec = events.into_iter().map(Event::from).collect(); + assert_eq!( + (transfer_number, segment_index, events), + ( + full_transfer, + full_segment, + vec![rejected(full_transfer, RejectReason::ReceiverFull)] + ) + ); + return; + } + assert_eq!(events, none()); + } + } +} diff --git a/btpu/tests/send_handle.rs b/btpu/tests/send_handle.rs new file mode 100644 index 000000000..8a63d0994 --- /dev/null +++ b/btpu/tests/send_handle.rs @@ -0,0 +1,591 @@ +//! Bundles pushed in chunks through `Sender::begin`, `push`, and `finish`. + +mod common; + +use bytes::Bytes; +use hardy_btpu::{ + codec::{ + header::HEADER_SIZE, + hint::{HintItem, encoded_hints_len}, + message::Message, + }, + sender::{ + BundleFraming, CarriedList, Error, LinkFraming, NextPduOptions, SegmentCutStrategy, + SendHandle, SendKind, SendOptions, Sender, + }, + transfer::Error as TransferError, +}; + +use self::common::{ + SEGMENT_FIELDS, bpv7_like, bundle, bundle_msg, cancel, carried, decode_all, drain, enqueue, + first_capacity, patterned, sender, sender_with, sender_with_queue_bytes, transfer_data, + variable_sender, window_size, +}; + +fn begin(s: &mut Sender, total_len: usize) -> SendHandle { + s.begin(total_len, SendOptions::default()).unwrap() +} + +// Push `data` through `handle` in chunks of `chunk` bytes. +fn push_in_chunks(s: &mut Sender, handle: &mut SendHandle, data: &Bytes, chunk: usize) { + for start in (0..data.len()).step_by(chunk) { + let end = (start + chunk).min(data.len()); + s.push(handle, data.slice(start..end)).unwrap(); + } +} + +#[test] +fn a_pushed_bundle_goes_out_as_the_same_bundle_enqueued() { + for len in [10, 200] { + let data = patterned(len); + let mut pushed = sender(64, 16); + let mut handle = begin(&mut pushed, len); + push_in_chunks(&mut pushed, &mut handle, &data, 7); + let id = pushed.finish(handle).unwrap(); + + let mut enqueued = sender(64, 16); + assert_eq!(enqueue(&mut enqueued, data), Ok(id)); + assert_eq!(drain(&mut pushed), drain(&mut enqueued)); + } +} + +#[test] +fn a_fitting_bundle_is_queued_when_its_last_byte_is_pushed() { + let mut s = sender(64, 16); + let mut handle = begin(&mut s, 10); + let id = handle.id(); + assert_eq!(id.kind(), SendKind::Message); + s.push(&mut handle, Bytes::from_static(b"0123")).unwrap(); + assert_eq!(s.queued_bytes(), 4); + assert_eq!(s.next_pdu(), None); + + s.push(&mut handle, Bytes::from_static(b"456789")).unwrap(); + assert_eq!(s.finish(handle), Ok(id)); + let pdu = s.next_pdu().unwrap(); + assert_eq!(&pdu.carried[..], &[carried(id, true)]); + assert_eq!(decode_all(pdu.data)[0], bundle_msg(b"0123456789")); + assert_eq!(s.queued_bytes(), 0); +} + +#[test] +fn a_segment_goes_out_once_its_bytes_are_pushed() { + let data = patterned(200); + let mut s = sender(64, 16); + let mut handle = begin(&mut s, data.len()); + assert!(s.has_pending()); + assert_eq!(s.next_pdu(), None); + + // Segment 0 carries the Bundle Length hint, so fewer than 64 bytes + // fill it. + s.push(&mut handle, data.slice(..64)).unwrap(); + let first = s.next_pdu().unwrap(); + assert_eq!(&first.carried[..], &[carried(handle.id(), false)]); + assert_eq!(s.next_pdu(), None); + assert!(s.queued_bytes() < 64); + + s.push(&mut handle, data.slice(64..)).unwrap(); + s.finish(handle).unwrap(); + let mut pdus = vec![first.data]; + pdus.extend(drain(&mut s)); + let mut enqueued = sender(64, 16); + enqueue(&mut enqueued, data).unwrap(); + assert_eq!(pdus, drain(&mut enqueued)); +} + +#[test] +fn an_overrun_is_refused_and_leaves_the_bundle_as_it_was() { + let mut s = sender(64, 16); + let mut handle = begin(&mut s, 10); + let id = handle.id(); + s.push(&mut handle, Bytes::from_static(b"01234")).unwrap(); + assert_eq!( + s.push(&mut handle, Bytes::from_static(b"567890")), + Err(Error::Overrun { + total_len: 10, + pushed: 5, + chunk: 6, + }) + ); + assert_eq!((handle.pushed(), s.queued_bytes()), (5, 5)); + s.push(&mut handle, Bytes::from_static(b"56789")).unwrap(); + assert_eq!(s.finish(handle), Ok(id)); + assert_eq!(id.kind(), SendKind::Message); + assert_eq!( + decode_all(s.next_pdu().unwrap().data)[0], + bundle_msg(b"0123456789") + ); +} + +#[test] +fn an_empty_chunk_does_nothing() { + let mut s = sender(64, 16); + let mut handle = begin(&mut s, 3); + let id = handle.id(); + s.push(&mut handle, Bytes::new()).unwrap(); + assert_eq!((handle.pushed(), s.queued_bytes()), (0, 0)); + s.push(&mut handle, Bytes::from_static(b"abc")).unwrap(); + // Even once the bundle is complete. + s.push(&mut handle, Bytes::new()).unwrap(); + assert_eq!(s.finish(handle), Ok(id)); + assert_eq!(id.kind(), SendKind::Message); +} + +#[test] +fn an_underrun_at_finish_cancels_a_started_transfer() { + let data = patterned(200); + let mut s = sender(64, 16); + let mut handle = begin(&mut s, data.len()); + let id = handle.id(); + s.push(&mut handle, data.slice(..100)).unwrap(); + s.next_pdu().unwrap(); + + assert_eq!( + s.finish(handle), + Err(Error::Underrun { + total_len: 200, + pushed: 100, + }) + ); + assert!(!s.is_outstanding(id)); + assert_eq!(s.queued_bytes(), 0); + // The sender's first transfer number. + assert_eq!(decode_all(s.next_pdu().unwrap().data)[0], cancel(0)); + assert_eq!(s.next_pdu(), None); +} + +#[test] +fn an_underrun_at_finish_drops_a_partly_pushed_fitting_bundle() { + let mut s = sender(64, 16); + let mut handle = begin(&mut s, 10); + s.push(&mut handle, Bytes::from_static(b"0123")).unwrap(); + assert_eq!( + s.finish(handle), + Err(Error::Underrun { + total_len: 10, + pushed: 4, + }) + ); + assert_eq!(s.queued_bytes(), 0); + assert_eq!(s.next_pdu(), None); +} + +#[test] +fn cancelling_a_handle_before_any_push_sends_nothing() { + let mut s = sender(64, 16); + let fitting = begin(&mut s, 10); + let segmented = begin(&mut s, 200); + let id = segmented.id(); + assert!(s.cancel(fitting)); + assert!(s.cancel(segmented)); + assert!(!s.is_outstanding(id)); + assert!(!s.has_pending()); + assert_eq!(s.next_pdu(), None); +} + +#[test] +fn cancelling_a_handle_drops_its_pushed_bytes() { + let data = patterned(200); + let mut s = sender(64, 16); + let mut fitting = begin(&mut s, 10); + let mut segmented = begin(&mut s, data.len()); + s.push(&mut fitting, Bytes::from_static(b"01234")).unwrap(); + s.push(&mut segmented, data.slice(..30)).unwrap(); + assert_eq!(s.queued_bytes(), 35); + assert!(s.cancel(fitting)); + assert_eq!(s.queued_bytes(), 30); + assert!(s.cancel(segmented)); + assert_eq!(s.queued_bytes(), 0); + // Nothing of the transfer had been emitted, so no Cancel goes out. + assert_eq!(s.next_pdu(), None); +} + +#[test] +fn a_push_after_cancelling_by_id_is_not_in_progress() { + let mut s = sender(64, 16); + let mut fitting = begin(&mut s, 10); + let mut segmented = begin(&mut s, 200); + assert!(s.cancel(fitting.id())); + assert!(s.cancel(segmented.id())); + for handle in [&mut fitting, &mut segmented] { + assert_eq!( + s.push(handle, Bytes::from_static(b"x")), + Err(Error::NotInProgress) + ); + assert_eq!(handle.pushed(), 0); + } + assert_eq!(s.queued_bytes(), 0); +} + +#[test] +fn a_bare_bundle_must_start_with_a_bundle_reserved_byte() { + let mut s = variable_sender(64, BundleFraming::Bare); + let mut handle = begin(&mut s, 20); + assert_eq!(handle.id().kind(), SendKind::Bare); + assert_eq!( + s.push(&mut handle, Bytes::from_static(&[0x01; 20])), + Err(Error::NotABundle) + ); + assert_eq!((handle.pushed(), s.queued_bytes()), (0, 0)); + + // A bundle split anywhere after its first byte goes out bare. + let data = bpv7_like(20); + push_in_chunks(&mut s, &mut handle, &data, 1); + s.finish(handle).unwrap(); + assert_eq!(drain(&mut s), vec![data]); +} + +#[test] +fn begin_refuses_what_enqueue_refuses() { + let mut s = sender(64, 4); + assert_eq!(s.begin(0, SendOptions::default()), Err(Error::Empty)); + let handles: Vec<_> = (0..4).map(|_| begin(&mut s, 200)).collect(); + assert_eq!( + s.begin(200, SendOptions::default()), + Err(Error::Window(TransferError::WindowFull { + window_size: window_size(4) + })) + ); + // A fitting bundle takes no window slot. + let fitting = begin(&mut s, 10); + assert!(s.cancel(fitting)); + for handle in handles { + assert!(s.cancel(handle)); + } +} + +#[test] +fn queued_bytes_bound_reports_full_on_pushed_bytes() { + let mut s = sender_with_queue_bytes(64, 50, LinkFraming::FixedSize); + let data = patterned(200); + let mut handle = begin(&mut s, data.len()); + // Beginning takes no bytes; pushing does, and is never refused for it. + assert!(!s.is_send_queue_full()); + s.push(&mut handle, data.slice(..100)).unwrap(); + assert!(s.is_send_queue_full()); + s.push(&mut handle, data.slice(100..)).unwrap(); + s.finish(handle).unwrap(); + drain(&mut s); + assert!(!s.is_send_queue_full()); +} + +#[test] +fn a_transfer_waiting_on_its_producer_is_passed_over() { + const PDU: usize = 64; + let data = patterned(200); + let mut s = sender(PDU, 16); + let mut waiting = begin(&mut s, data.len()); + // Segment 0 goes out, and the rest waits on bytes not yet pushed. + s.push(&mut waiting, data.slice(..PDU)).unwrap(); + let mut waiting_pdus = vec![s.next_pdu().unwrap().data]; + assert_eq!(s.next_pdu(), None); + + let small = bundle(10); + let small_id = enqueue(&mut s, small).unwrap(); + let other = patterned(100); + let other_id = enqueue(&mut s, other.clone()).unwrap(); + + // Both go ahead of the waiting transfer, the segmented one filling the + // tail behind the Bundle Message. + let first = s.next_pdu().unwrap(); + assert_eq!( + &first.carried[..], + &[carried(small_id, true), carried(other_id, false)] + ); + let mut other_pdus = vec![first.data]; + while let Some(pdu) = s.next_pdu() { + assert!(pdu.carried.iter().all(|c| c.id == other_id)); + other_pdus.push(pdu.data); + } + // The second transfer begun, after the waiting one. + assert_eq!(transfer_data(&other_pdus, 1), other); + assert!(s.has_pending()); + + s.push(&mut waiting, data.slice(PDU..)).unwrap(); + s.finish(waiting).unwrap(); + waiting_pdus.extend(drain(&mut s)); + // The sender's first transfer number. + assert_eq!(transfer_data(&waiting_pdus, 0), data); + assert!(!s.has_pending()); +} + +#[test] +fn every_outstanding_transfer_can_end_in_one_pdu_within_the_list_bound() { + const PDU: usize = 64; + const WINDOW: u16 = 4; + // Two full segments one byte short of the bundle, so each End carries + // one byte, 13 bytes as a message. + let hints_len = encoded_hints_len(&[HintItem::BundleLength(2 * PDU as u64)]); + let len = + (PDU - HEADER_SIZE - SEGMENT_FIELDS - hints_len) + (PDU - HEADER_SIZE - SEGMENT_FIELDS) + 1; + assert_eq!( + first_capacity(PDU, len), + PDU - HEADER_SIZE - SEGMENT_FIELDS - hints_len + ); + let mut s = sender(PDU, WINDOW); + let mut handles: Vec<_> = (0..WINDOW).map(|_| begin(&mut s, len)).collect(); + for handle in &mut handles { + s.push(handle, patterned(len - 1)).unwrap(); + } + assert_eq!(drain(&mut s).len(), 2 * usize::from(WINDOW)); + + // Four Ends and a Bundle Message fill the PDU exactly. + let fitting = enqueue(&mut s, bundle(PDU - 4 * 13 - HEADER_SIZE)).unwrap(); + let mut expected = Vec::new(); + for mut handle in handles { + s.push(&mut handle, Bytes::from_static(&[0])).unwrap(); + expected.push(carried(s.finish(handle).unwrap(), true)); + } + expected.push(carried(fitting, true)); + + let mut list = CarriedList::with_capacity(PDU / 32 + usize::from(WINDOW)); + let capacity = list.capacity(); + let pdu = s.next_pdu_into(&mut list).unwrap(); + assert_eq!(&list[..], &expected[..]); + assert_eq!(list.capacity(), capacity); + // Nothing but the five messages, so no padding. + assert_eq!(decode_all(pdu).len(), 5); + assert!(!s.has_pending()); +} + +fn half_cut_sender(pdu_size: usize) -> Sender { + sender_with(pdu_size, |c| { + c.segment_cut_strategy = SegmentCutStrategy::Half + }) +} + +#[test] +fn a_half_cut_sends_half_a_segment_without_waiting_for_the_rest() { + const PDU: usize = 64; + let capacity = PDU - HEADER_SIZE - SEGMENT_FIELDS; + let data = patterned(200); + let first = first_capacity(PDU, data.len()); + // A chunk of segment 0 and exactly half of segment 1. + let chunk = first + capacity.div_ceil(2); + + let mut full = sender(PDU, 16); + let mut handle = begin(&mut full, data.len()); + full.push(&mut handle, data.slice(..chunk)).unwrap(); + assert_eq!(drain(&mut full).len(), 1); + + let mut half = half_cut_sender(PDU); + let mut handle = begin(&mut half, data.len()); + // The sender's first transfer number. + let t = 0; + half.push(&mut handle, data.slice(..chunk)).unwrap(); + let mut pdus = drain(&mut half); + assert_eq!(pdus.len(), 2); + assert_eq!(transfer_data(&pdus[1..], t), data[first..chunk]); + assert_eq!(half.queued_bytes(), 0); + + // One byte short of half a segment waits. + let short = chunk + capacity.div_ceil(2) - 1; + half.push(&mut handle, data.slice(chunk..short)).unwrap(); + assert_eq!(half.next_pdu(), None); + + half.push(&mut handle, data.slice(short..)).unwrap(); + half.finish(handle).unwrap(); + pdus.extend(drain(&mut half)); + assert_eq!(transfer_data(&pdus, t), data); +} + +#[cfg(target_pointer_width = "64")] +#[test] +fn begin_refuses_a_bundle_whose_last_index_could_reach_u32_max() { + const PDU: usize = 64; + // Segments cut short carry at least half a segment, so the last index + // is bounded by one more than the bundle over half a segment. + let half = (PDU - HEADER_SIZE - SEGMENT_FIELDS).div_ceil(2); + let refused = half * (u32::MAX as usize - 1); + let mut s = sender(PDU, 16); + assert_eq!( + s.begin(refused, SendOptions::default()), + Err(Error::TooManySegments { + len: refused, + pdu_size: PDU + }) + ); + assert_eq!( + s.begin(refused - 1, SendOptions::default()) + .map(|h| h.id().kind()), + Ok(SendKind::Transfer) + ); +} + +#[test] +fn a_producer_gated_on_push_readiness_never_waits_on_itself() { + const PDU: usize = 64; + // A bound under one segment: gating each push on a full queue would + // stop at the first push, with no segment complete to drain. + const BOUND: usize = 10; + const CHUNK: usize = 7; + let data = patterned(500); + let mut s = sender_with_queue_bytes(PDU, BOUND, LinkFraming::FixedSize); + let mut handle = begin(&mut s, data.len()); + // The sender's first transfer number. + let t = 0; + + let mut pdus = Vec::new(); + for start in (0..data.len()).step_by(CHUNK) { + while !s.is_push_ready(&handle) { + pdus.push(s.next_pdu().expect("a segment is complete").data); + } + let end = (start + CHUNK).min(data.len()); + s.push(&mut handle, data.slice(start..end)).unwrap(); + assert!(s.queued_bytes() < BOUND + PDU + CHUNK); + } + s.finish(handle).unwrap(); + pdus.extend(drain(&mut s)); + assert_eq!(transfer_data(&pdus, t), data); +} + +#[test] +fn push_readiness_past_the_bound_follows_whether_the_bundle_waits() { + const PDU: usize = 64; + let data = patterned(200); + let mut s = sender_with_queue_bytes(PDU, 50, LinkFraming::FixedSize); + let mut handle = begin(&mut s, data.len()); + s.push(&mut handle, data.slice(..30)).unwrap(); + assert!(s.is_push_ready(&handle)); + + // Past the bound, still short of segment 0: the bundle waits on its + // producer, so its push is admitted. + let mut other = begin(&mut s, data.len()); + s.push(&mut other, data.slice(..30)).unwrap(); + assert!(s.is_send_queue_full()); + assert!(s.is_push_ready(&handle)); + + // Once segment 0 is complete, it can go out, so the producer drains. + s.push(&mut handle, data.slice(30..60)).unwrap(); + assert!(!s.is_push_ready(&handle)); + assert!(s.is_push_ready(&other)); + s.next_pdu().unwrap(); + assert!(!s.is_send_queue_full()); + assert!(s.is_push_ready(&handle)); +} + +#[test] +fn push_readiness_admits_a_fitting_bundle_and_one_not_in_progress() { + let mut s = sender_with_queue_bytes(64, 4, LinkFraming::FixedSize); + let mut fitting = begin(&mut s, 10); + s.push(&mut fitting, Bytes::from_static(b"01234")).unwrap(); + assert!(s.is_send_queue_full()); + // A fitting bundle goes out only once whole. + assert!(s.is_push_ready(&fitting)); + + let mut cancelled = begin(&mut s, 200); + assert!(s.cancel(cancelled.id())); + assert!(s.is_push_ready(&cancelled)); + assert_eq!( + s.push(&mut cancelled, Bytes::from_static(b"x")), + Err(Error::NotInProgress) + ); +} + +const FLUSH: NextPduOptions = NextPduOptions { flush: true }; + +#[test] +fn a_flush_sends_what_a_waiting_transfer_has_buffered() { + const PDU: usize = 64; + let data = patterned(200); + let mut s = sender(PDU, 16); + let mut fitting = begin(&mut s, 10); + s.push(&mut fitting, Bytes::from_static(b"0123")).unwrap(); + // A bundle that fits one PDU is not queued until it is whole. + assert_eq!(s.next_pdu_with(FLUSH), None); + + let mut handle = begin(&mut s, data.len()); + // The sender's first transfer number: the fitting bundle takes none. + let t = 0; + s.push(&mut handle, data.slice(..10)).unwrap(); + assert_eq!(s.next_pdu(), None); + + let pdu = s.next_pdu_with(FLUSH).unwrap(); + assert_eq!(&pdu.carried[..], &[carried(handle.id(), false)]); + let Message::TransferSegment(m) = &decode_all(pdu.data.clone())[0] else { + panic!("expected a segment"); + }; + assert_eq!((m.segment_index, &m.data[..]), (0, &data[..10])); + assert_eq!(s.queued_bytes(), 4); + // Nothing left buffered to flush. + assert_eq!(s.next_pdu_with(FLUSH), None); + + let mut pdus = vec![pdu.data]; + s.push(&mut handle, data.slice(10..)).unwrap(); + s.finish(handle).unwrap(); + pdus.extend(drain(&mut s)); + assert_eq!(transfer_data(&pdus, t), data); +} + +#[test] +fn a_flush_fills_the_room_left_with_waiting_transfers_in_queue_order() { + const PDU: usize = 100; + let data = patterned(200); + let first_framing = PDU - first_capacity(PDU, data.len()); + let mut s = sender(PDU, 16); + let mut a = begin(&mut s, data.len()); + let mut b = begin(&mut s, data.len()); + s.push(&mut a, data.slice(..10)).unwrap(); + s.push(&mut b, data.slice(..40)).unwrap(); + let fitting = enqueue(&mut s, bundle(20)).unwrap(); + + // The ready Bundle Message first, then each waiting transfer's bytes, + // b's cut to the room left. + let b_fits = PDU - (HEADER_SIZE + 20) - (first_framing + 10) - first_framing; + assert!(b_fits < 40); + let pdu = s.next_pdu_with(FLUSH).unwrap(); + assert_eq!( + &pdu.carried[..], + &[ + carried(fitting, true), + carried(a.id(), false), + carried(b.id(), false) + ] + ); + let messages = decode_all(pdu.data.clone()); + assert_eq!(messages.len(), 3, "the PDU is full, so it has no padding"); + let mut pdus = vec![pdu.data]; + // a and b are the sender's first two transfer numbers. + assert_eq!(transfer_data(&pdus, 0), data[..10]); + assert_eq!(transfer_data(&pdus, 1), data[..b_fits]); + + // What b could not fit goes in the next flush, as segment 1. + let pdu = s.next_pdu_with(FLUSH).unwrap(); + assert_eq!(&pdu.carried[..], &[carried(b.id(), false)]); + let Message::TransferSegment(m) = &decode_all(pdu.data.clone())[0] else { + panic!("expected a segment"); + }; + assert_eq!((m.segment_index, &m.data[..]), (1, &data[b_fits..40])); + pdus.push(pdu.data); + + for (mut handle, from) in [(a, 10), (b, 40)] { + s.push(&mut handle, data.slice(from..)).unwrap(); + s.finish(handle).unwrap(); + } + pdus.extend(drain(&mut s)); + for t in [0, 1] { + assert_eq!(transfer_data(&pdus, t), data); + } +} + +#[test] +fn a_flush_does_not_reorder_past_a_message_that_does_not_fit() { + const PDU: usize = 64; + let data = patterned(200); + let mut s = sender(PDU, 16); + let mut ahead = begin(&mut s, data.len()); + s.push(&mut ahead, data.slice(..10)).unwrap(); + let small = enqueue(&mut s, bundle(10)).unwrap(); + let large = enqueue(&mut s, bundle(PDU - HEADER_SIZE)).unwrap(); + let mut behind = begin(&mut s, data.len()); + s.push(&mut behind, data.slice(..10)).unwrap(); + + let pdu = s.next_pdu_with(FLUSH).unwrap(); + assert_eq!( + &pdu.carried[..], + &[carried(small, true), carried(ahead.id(), false)] + ); + let pdu = s.next_pdu_with(FLUSH).unwrap(); + assert_eq!(&pdu.carried[..], &[carried(large, true)]); + let pdu = s.next_pdu_with(FLUSH).unwrap(); + assert_eq!(&pdu.carried[..], &[carried(behind.id(), false)]); +} diff --git a/btpu/tests/sender.rs b/btpu/tests/sender.rs new file mode 100644 index 000000000..305231780 --- /dev/null +++ b/btpu/tests/sender.rs @@ -0,0 +1,1407 @@ +//! Segmentation, PDU packing, window release, and link framing through the +//! public `sender` API. + +mod common; + +use std::{ + hash::{DefaultHasher, Hash, Hasher}, + num::NonZeroUsize, + slice::from_ref, +}; + +use bytes::Bytes; +use hardy_btpu::{ + OutOfRange, ParseError, + codec::{ + decode_pdu, + header::{HEADER_SIZE, MAX_CONTENT_LENGTH}, + hint::{HintItem, HintType, HintValue, encoded_hints_len}, + message::{FrameKind, Message, TransferSegmentMessage, frame_kind}, + }, + sender::{ + BundleFraming, Carried, CarriedList, Error, LinkFraming, PduSize, SendId, SendKind, + SendOptions, SendQueueBytes, Sender, SenderConfig, + }, + transfer::Error as TransferError, +}; + +use self::common::{ + SEGMENT_FIELDS, bpv7_like, bundle, bundle_with, cancel, carried, decode_all, drain, drain_pdus, + enqueue, first_capacity, floored_sender, patterned, segment_with, segmented, sender, + sender_config, sender_with_queue_bytes, transfer_data, unknown_hint, variable_sender, + window_size, wire_transfer_numbers, +}; + +#[test] +fn config_defaults() { + let c = SenderConfig::default(); + assert_eq!(c.pdu_size.get(), 1500); + assert_eq!(c.window_size.get(), 16); + assert_eq!(c.send_queue_bytes.get(), 1 << 20); + assert_eq!(c.link_framing, LinkFraming::FixedSize); +} + +#[test] +fn pdu_size_boundaries() { + assert_eq!(PduSize::MIN.get(), HEADER_SIZE); + assert_eq!(PduSize::MAX.get(), HEADER_SIZE + MAX_CONTENT_LENGTH); + assert_eq!(PduSize::try_from(HEADER_SIZE), Ok(PduSize::MIN)); + assert_eq!( + PduSize::try_from(HEADER_SIZE + MAX_CONTENT_LENGTH), + Ok(PduSize::MAX) + ); + let out_of_range = |value: usize| OutOfRange { + name: "PDU size", + value: value as u64, + min: HEADER_SIZE as u64, + max: Some((HEADER_SIZE + MAX_CONTENT_LENGTH) as u64), + }; + for value in [0, HEADER_SIZE - 1, HEADER_SIZE + MAX_CONTENT_LENGTH + 1] { + assert_eq!(PduSize::new(value), None); + assert_eq!(PduSize::try_from(value), Err(out_of_range(value))); + } +} + +#[test] +fn pdu_size_default_is_the_const() { + assert_eq!(PduSize::default(), PduSize::DEFAULT); + assert_eq!(PduSize::DEFAULT.get(), 1500); + assert_eq!(PduSize::new(1500), Some(PduSize::DEFAULT)); + assert_eq!(PduSize::DEFAULT.to_string(), "1500"); +} + +#[test] +fn pdu_size_parses_and_formats_as_its_integer() { + assert_eq!("1500".parse(), Ok(PduSize::DEFAULT)); + assert_eq!( + "3".parse::(), + Err(ParseError::OutOfRange(PduSize::try_from(3).unwrap_err())) + ); + assert_eq!( + "3".parse::().unwrap_err().to_string(), + "Invalid PDU size 3 (must be 4..=1048579)" + ); + let source = "1.5k".parse::().unwrap_err(); + assert_eq!( + "1.5k".parse::(), + Err(ParseError::Syntax { + name: "PDU size", + source: source.clone(), + }) + ); + assert_eq!( + "1.5k".parse::().unwrap_err().to_string(), + format!("Invalid PDU size: {source}") + ); + let p = PduSize::DEFAULT; + assert_eq!( + format!("{p} {p:b} {p:o} {p:x} {p:#X} {p:>6}"), + "1500 10111011100 2734 5dc 0x5DC 1500" + ); +} + +#[test] +fn send_queue_bytes_zero_rejected() { + assert_eq!(SendQueueBytes::new(0), None); + assert_eq!( + SendQueueBytes::try_from(0), + Err(OutOfRange { + name: "send queue bytes", + value: 0, + min: 1, + max: None, + }) + ); + assert_eq!( + SendQueueBytes::try_from(0).unwrap_err().to_string(), + "Invalid send queue bytes 0 (must be at least 1)" + ); + assert_eq!(SendQueueBytes::try_from(1), Ok(SendQueueBytes::MIN)); + assert_eq!( + SendQueueBytes::try_from(usize::MAX), + Ok(SendQueueBytes::MAX) + ); +} + +#[test] +fn send_queue_bytes_converts_to_and_from_non_zero() { + let n = NonZeroUsize::new(7).unwrap(); + let bound = SendQueueBytes::from(n); + assert_eq!(bound.get(), 7); + assert_eq!(NonZeroUsize::from(bound), n); + assert_eq!(usize::from(bound), 7); + assert_eq!(bound.to_string(), "7"); + assert_eq!(SendQueueBytes::default(), SendQueueBytes::DEFAULT); + assert_eq!(SendQueueBytes::DEFAULT.get(), 1 << 20); +} + +#[test] +fn send_queue_bytes_parses_and_formats_as_its_integer() { + assert_eq!("1048576".parse(), Ok(SendQueueBytes::DEFAULT)); + assert_eq!( + "0".parse::(), + Err(ParseError::OutOfRange( + SendQueueBytes::try_from(0).unwrap_err() + )) + ); + assert_eq!( + "-1".parse::(), + Err(ParseError::Syntax { + name: "send queue bytes", + source: "-1".parse::().unwrap_err(), + }) + ); + let b = SendQueueBytes::DEFAULT; + assert_eq!( + format!("{b:b} {b:o} {b:x} {b:X}"), + "100000000000000000000 4000000 100000 100000" + ); +} + +#[test] +fn empty_bundle_rejected_at_enqueue() { + let mut s = sender(256, 16); + assert_eq!(enqueue(&mut s, Bytes::new()), Err(Error::Empty)); + assert!(!s.has_pending()); + assert_eq!(s.next_pdu(), None); +} + +#[test] +fn minimum_pdu_size_cannot_queue_an_undrainable_message() { + // At PduSize::MIN only a zero-content Bundle message would fit, and + // empty bundles are rejected; a one-byte bundle must take the + // segmentation path and fail cleanly rather than queue a message + // larger than any PDU (which would make the drain loop spin). + let mut s = sender(PduSize::MIN.get(), 16); + assert_eq!( + enqueue(&mut s, Bytes::from_static(b"x")), + Err(Error::PduTooSmall { + required: HEADER_SIZE + + SEGMENT_FIELDS + + encoded_hints_len(&[HintItem::BundleLength(1)]) + + 1, + pdu_size: PduSize::MIN.get(), + }) + ); + assert!(!s.has_pending()); + assert_eq!(s.next_pdu(), None); +} + +#[test] +fn segmenting_floor_grows_with_the_bundle_length_hint() { + // The derived Bundle Length hint on the first segment takes 1, 2, or 4 + // value bytes as the bundle grows (8 past 4 GiB, too large to queue + // here), and one data byte must fit beside it. + for (len, floor) in [(255, 16), (256, 17), (65_535, 17), (65_536, 19)] { + assert_eq!( + enqueue(&mut sender(floor - 1, 4), bundle(len)), + Err(Error::PduTooSmall { + required: floor, + pdu_size: floor - 1, + }), + "{len}-byte bundle" + ); + assert_eq!( + enqueue(&mut sender(floor, 4), bundle(len)).map(SendId::kind), + Ok(SendKind::Transfer) + ); + } +} + +#[test] +fn pdu_too_small_to_segment_leaves_window_untouched() { + // A hint chain that leaves no room for segment data fails before a + // transfer number is taken, so the next bundle still gets number 0. + let mut s = sender(64, 4); + let wide = HintItem::Unknown { + hint_type: HintType::new(0x41).unwrap(), + value: HintValue::new(Bytes::from(vec![0u8; 60])).unwrap(), + }; + assert!(matches!( + s.enqueue( + bundle(200), + SendOptions { + hints: vec![wide].into() + } + ), + Err(Error::PduTooSmall { pdu_size: 64, .. }) + )); + assert!(s.is_window_available()); + assert!(!s.has_pending()); + segmented(enqueue(&mut s, bundle(200)).unwrap()); + assert_eq!(wire_transfer_numbers(&drain(&mut s)), [0]); +} + +#[test] +fn caller_hints_ride_first_segment_with_derived_bundle_length() { + let mut s = sender(64, 4); + let correlator = unknown_hint(0x41, b"\x01\x02"); + // The caller-supplied BundleLength is discarded; the sender derives + // the truthful one and puts it first. + let options = SendOptions { + hints: vec![HintItem::BundleLength(999), correlator.clone()].into(), + }; + s.enqueue(bundle(200), options).unwrap(); + + let first = decode_pdu(s.next_pdu().unwrap().data) + .next() + .unwrap() + .unwrap(); + let Message::TransferSegment(m) = first else { + panic!("expected a leading Transfer Segment, got {first:?}") + }; + assert_eq!(m.hints, vec![HintItem::BundleLength(200), correlator]); + // The first segment's data capacity shrinks by exactly the merged + // hint chain's encoded size. + assert_eq!( + m.data.len(), + 64 - HEADER_SIZE - SEGMENT_FIELDS - encoded_hints_len(&m.hints) + ); +} + +#[test] +fn repeated_caller_hint_types_go_out_once_latest_wins() { + let mut s = sender(64, 4); + let options = SendOptions { + hints: vec![unknown_hint(0x41, b"old"), unknown_hint(0x41, b"new")].into(), + }; + s.enqueue(bundle(200), options).unwrap(); + + let first = decode_pdu(s.next_pdu().unwrap().data) + .next() + .unwrap() + .unwrap(); + let Message::TransferSegment(m) = first else { + panic!("expected a leading Transfer Segment, got {first:?}") + }; + // The decoder folds repeats too, so the data length is what shows the + // repeat never reached the wire: segment 0 spends only one item's bytes. + let sent = [HintItem::BundleLength(200), unknown_hint(0x41, b"new")]; + assert_eq!(m.hints, sent); + assert_eq!( + m.data.len(), + 64 - HEADER_SIZE - SEGMENT_FIELDS - encoded_hints_len(&sent) + ); +} + +#[test] +fn first_segment_capacity_reduced_by_exactly_the_hint_bytes() { + let pdu_size = 32; + let mut s = sender(pdu_size, 16); + enqueue(&mut s, Bytes::from(vec![0xAB; 100])).unwrap(); + + let messages: Vec = drain(&mut s).into_iter().flat_map(decode_all).collect(); + + // Every segment carries a fixed overhead of the 4-byte message header + // plus the 8-byte transfer number + segment index prefix. + let full_capacity = pdu_size - HEADER_SIZE - SEGMENT_FIELDS; + + let mut segments: Vec<&TransferSegmentMessage> = messages + .iter() + .filter_map(|m| match m { + Message::TransferSegment(seg) => Some(seg), + _ => None, + }) + .collect(); + segments.sort_by_key(|seg| seg.segment_index); + let (first, middle) = segments.split_first().unwrap(); + assert_eq!(first.segment_index, 0); + + // The first segment cedes exactly the encoded hint bytes to the + // Bundle Length hint... + let hint_len = encoded_hints_len(&first.hints); + assert!(hint_len > 0); + assert_eq!(first.data.len(), full_capacity - hint_len); + + // ...while every hintless middle segment fills its PDU exactly. + assert!(!middle.is_empty()); + for seg in middle { + assert!(seg.hints.is_empty()); + assert_eq!( + seg.data.len(), + full_capacity, + "segment {}", + seg.segment_index + ); + } +} + +#[test] +fn caller_hints_ride_unsegmented_bundle_message() { + let mut s = sender(64, 4); + let hint = unknown_hint(0x41, b"z"); + let data = Bytes::from_static(b"tiny"); + assert_eq!( + s.enqueue( + data.clone(), + SendOptions { + hints: vec![hint.clone()].into(), + }, + ) + .map(SendId::kind), + Ok(SendKind::Message) + ); + + let msg = decode_pdu(s.next_pdu().unwrap().data) + .next() + .unwrap() + .unwrap(); + assert_eq!(msg, bundle_with(vec![hint], data)); +} + +#[test] +fn max_pdu_size_bundle_encodes_without_panic() { + // Regression: with pdu_size at the limit, both the largest possible + // Bundle message and the segmentation path must stay within the + // 20-bit content length; next_pdu must never hit its expect(). + let mut s = sender(PduSize::MAX.get(), 16); + + // Largest bundle that fits unsegmented: content == MAX_CONTENT_LENGTH. + assert_eq!( + enqueue(&mut s, bundle(MAX_CONTENT_LENGTH)).map(SendId::kind), + Ok(SendKind::Message) + ); + // One byte more: must segment, and every segment must encode. + assert_eq!( + enqueue(&mut s, bundle(MAX_CONTENT_LENGTH + 1)).map(SendId::kind), + Ok(SendKind::Transfer) + ); + + let pdus = drain(&mut s); + // The unsegmented bundle fills one PDU, and the other two segments. + assert_eq!(pdus.len(), 3); + for pdu in &pdus { + assert_eq!(pdu.len(), PduSize::MAX.get()); + } +} + +#[test] +fn small_bundle_no_segmentation() { + let mut s = sender(256, 16); + let data = Bytes::from_static(b"hello"); + let len = data.len(); + assert_eq!( + enqueue(&mut s, data.clone()).map(SendId::kind), + Ok(SendKind::Message) + ); + + let pdu = s.next_pdu().unwrap().data; + assert_eq!(pdu.len(), 256); + assert_eq!( + decode_all(pdu), + vec![ + bundle_with(vec![], data), + Message::DefinitePadding { + len: 256 - HEADER_SIZE - len - HEADER_SIZE + }, + ] + ); +} + +#[test] +fn large_bundle_segmented() { + let pdu_size = 32; + let mut s = sender(pdu_size, 16); + let data = Bytes::from(vec![0xAB; 100]); + assert_eq!( + enqueue(&mut s, data.clone()).map(SendId::kind), + Ok(SendKind::Transfer) + ); + + let mut all_messages = Vec::new(); + for pdu in drain(&mut s) { + assert_eq!(pdu.len(), pdu_size); + all_messages.extend(decode_all(pdu)); + } + + // Segments in index order, one End, and the data reassembles. + let mut indexed: Vec<(u32, Bytes)> = Vec::new(); + let mut ends = 0; + for msg in &all_messages { + match msg { + Message::TransferSegment(m) => indexed.push((m.segment_index, m.data.clone())), + Message::TransferEnd(m) => { + ends += 1; + indexed.push((m.segment_index, m.data.clone())); + } + Message::DefinitePadding { .. } => {} + other => panic!("unexpected {other:?}"), + } + } + assert_eq!(ends, 1); + let indices: Vec = indexed.iter().map(|(i, _)| *i).collect(); + assert_eq!(indices, (0..indices.len() as u32).collect::>()); + let combined: Vec = indexed.into_iter().flat_map(|(_, d)| d.to_vec()).collect(); + assert_eq!(combined, data.to_vec()); +} + +#[cfg(feature = "rand")] +#[test] +fn from_rng_seeds_initial_transfer_number() { + let mut s = Sender::from_rng(sender_config(64, 16), &mut common::FixedRng(0xDEAD_BEEF)); + segmented(enqueue(&mut s, bundle(200)).unwrap()); + assert_eq!(wire_transfer_numbers(&drain(&mut s)), [0xDEAD_BEEF]); +} + +#[cfg(feature = "rand")] +#[test] +fn try_from_rng_seeds_initial_transfer_number() { + let mut s = + Sender::try_from_rng(sender_config(64, 16), &mut common::FixedRng(0xDEAD_BEEF)).unwrap(); + segmented(enqueue(&mut s, bundle(200)).unwrap()); + assert_eq!(wire_transfer_numbers(&drain(&mut s)), [0xDEAD_BEEF]); +} + +#[cfg(feature = "rand")] +#[test] +fn try_from_rng_returns_the_rng_error() { + assert_eq!( + Sender::try_from_rng(sender_config(64, 16), &mut common::FailingRng).err(), + Some(common::RngFailure) + ); +} + +#[test] +fn window_slot_is_released_when_the_transfer_end_is_packed() { + let mut s = sender(64, 4); + let ids: Vec = (0..4) + .map(|_| segmented(enqueue(&mut s, bundle(200)).unwrap())) + .collect(); + assert!(!s.is_window_available()); + assert_eq!( + enqueue(&mut s, bundle(200)), + Err(Error::Window(TransferError::WindowFull { + window_size: window_size(4) + })) + ); + assert!(s.is_outstanding(ids[0])); + + // Drain PDU by PDU: the window opens exactly when transfer 0's End + // leaves the queue, with no call from the caller. + let mut opened_on_end = false; + while !s.is_window_available() { + let pdu = s + .next_pdu() + .expect("window cannot open with nothing pending") + .data; + let packed_end_of_0 = decode_all(pdu) + .iter() + .any(|m| matches!(m, Message::TransferEnd(e) if e.transfer_number == 0)); + if s.is_window_available() { + opened_on_end = packed_end_of_0; + } else { + assert!( + !packed_end_of_0, + "End of transfer 0 packed but window still closed" + ); + } + } + assert!( + opened_on_end, + "window opened before transfer 0's End was packed" + ); + assert!(!s.is_outstanding(ids[0])); + assert!(s.is_outstanding(ids[1])); + segmented(enqueue(&mut s, bundle(200)).unwrap()); + assert_eq!(wire_transfer_numbers(&drain(&mut s)), [1, 2, 3, 4]); +} + +#[test] +fn cancelling_the_newest_transfer_keeps_the_window_span() { + // Section 5: the sender MUST NOT emit a message whose transfer number + // is <= greatest - window_size. Freeing the newest transfer while the + // oldest is outstanding must not admit a new number, or a later + // message for the oldest would fall outside the receiver's window. + let mut s = sender(64, 4); + let ids: Vec = (0..4) + .map(|_| segmented(enqueue(&mut s, bundle(200)).unwrap())) + .collect(); + assert!(s.cancel(ids[3])); + assert!(!s.is_window_available()); + assert!(matches!( + enqueue(&mut s, bundle(200)), + Err(Error::Window(TransferError::WindowFull { .. })) + )); + + // Releasing the oldest advances the window base; 4 is now admissible + // and every still-active number (1, 2) stays within 4 - 4 + 1..=4. + assert!(s.cancel(ids[0])); + assert!(s.is_window_available()); + segmented(enqueue(&mut s, bundle(200)).unwrap()); + // Nothing of 0 or 3 had been sent, so no Cancel for either. + assert_eq!(wire_transfer_numbers(&drain(&mut s)), [1, 2, 4]); +} + +#[test] +fn cancel_before_any_emission_queues_no_cancel_message() { + // Nothing of the transfer reached the link, so the receiver never + // learned of it and a Cancel would only be ignored (Section 8.4). + let mut s = sender(32, 4); + let id = segmented(enqueue(&mut s, bundle(200)).unwrap()); + assert!(s.cancel(id)); + assert!(!s.is_outstanding(id)); + assert!(s.is_window_available()); + assert!(!s.has_pending()); + assert_eq!(s.next_pdu(), None); +} + +#[test] +fn cancel_after_partial_emission_discards_the_rest_and_queues_a_cancel() { + let mut s = sender(32, 4); + let id = segmented(enqueue(&mut s, bundle(200)).unwrap()); + // Emit the first PDU: segment 0 has left. + let first = decode_all(s.next_pdu().unwrap().data); + assert!(matches!( + first.as_slice(), + [Message::TransferSegment(m), ..] if m.segment_index == 0 + )); + + assert!(s.cancel(id)); + assert!(!s.is_outstanding(id)); + // The remaining segments are gone; only the Cancel (plus padding) is + // emitted. + let pdus = drain(&mut s); + assert_eq!(pdus.len(), 1); + assert_eq!( + decode_all(pdus[0].clone()), + vec![ + // The sender's first transfer number. + cancel(0), + Message::DefinitePadding { + len: 32 - (HEADER_SIZE + 4) - HEADER_SIZE + }, + ] + ); +} + +#[test] +fn bogus_cancel_is_a_noop() { + let mut s = sender(64, 4); + let mut untouched = sender(64, 4); + let id = segmented(enqueue(&mut s, bundle(200)).unwrap()); + enqueue(&mut untouched, bundle(200)).unwrap(); + // A transfer ID this sender never issued, from a sender numbering + // from 999. + let foreign = + segmented(enqueue(&mut Sender::new(sender_config(64, 4), 999), bundle(200)).unwrap()); + assert_ne!(foreign, id); + + // Nothing queued, nothing released, so the drain is exactly that of a + // sender never asked to cancel. + assert!(!s.cancel(foreign)); + assert!(s.is_outstanding(id)); + assert_eq!(drain(&mut s), drain(&mut untouched)); + + // Fully emitted, so no longer outstanding: cancelling it now is a + // no-op too. + assert!(!s.is_outstanding(id)); + assert!(!s.cancel(id)); + assert_eq!(s.next_pdu(), None); +} + +#[test] +fn first_segment_has_bundle_length_hint() { + let mut s = sender(32, 16); + enqueue(&mut s, Bytes::from(vec![0xCC; 80])).unwrap(); + let msgs = decode_all(s.next_pdu().unwrap().data); + let Some(Message::TransferSegment(seg)) = msgs.first() else { + panic!("expected a leading Transfer Segment, got {msgs:?}"); + }; + assert_eq!(seg.segment_index, 0); + assert_eq!(seg.hints, vec![HintItem::BundleLength(80)]); +} + +#[test] +fn variable_link_pdus_are_not_padded() { + let mut s = variable_sender(256, BundleFraming::Message); + let data = Bytes::from_static(b"hello"); + assert_eq!( + enqueue(&mut s, data.clone()).map(SendId::kind), + Ok(SendKind::Message) + ); + + let pdu = s.next_pdu().unwrap().data; + assert_eq!(pdu.len(), HEADER_SIZE + data.len()); + assert_eq!(decode_all(pdu), vec![bundle_with(vec![], data)]); +} + +#[test] +fn variable_link_segmented_pdus_fill_the_pdu_except_the_last() { + let mut s = variable_sender(64, BundleFraming::Message); + enqueue(&mut s, bundle(150)).unwrap(); + let pdus = drain(&mut s); + let (last, full) = pdus.split_last().unwrap(); + assert!(full.iter().all(|pdu| pdu.len() == 64)); + + // The last PDU is its End and nothing else: no padding. + let [Message::TransferEnd(end)] = &decode_all(last.clone())[..] else { + panic!("expected a lone Transfer End"); + }; + assert_eq!(last.len(), HEADER_SIZE + SEGMENT_FIELDS + end.data.len()); + let data: usize = pdus + .iter() + .flat_map(|pdu| decode_all(pdu.clone())) + .map(|m| match m { + Message::TransferSegment(m) | Message::TransferEnd(m) => m.data.len(), + other => panic!("unexpected {other:?}"), + }) + .sum(); + assert_eq!(data, 150); +} + +// The Ethernet minimum frame payload. +const ETHERNET_FLOOR: usize = 46; + +#[test] +fn a_short_pdu_is_padded_up_to_the_floor() { + let mut s = floored_sender(256, BundleFraming::Message, ETHERNET_FLOOR); + let data = Bytes::from_static(b"hello"); + enqueue(&mut s, data.clone()).unwrap(); + + let pdu = s.next_pdu().unwrap().data; + assert_eq!(pdu.len(), ETHERNET_FLOOR); + let gap = ETHERNET_FLOOR - (HEADER_SIZE + data.len()); + assert_eq!( + decode_all(pdu), + vec![ + bundle_with(vec![], data), + Message::DefinitePadding { + len: gap - HEADER_SIZE + }, + ] + ); +} + +#[test] +fn a_gap_too_small_for_a_padding_header_is_zero_filled() { + let data = Bytes::from_static(b"hello"); + let message_len = HEADER_SIZE + data.len(); + let mut s = floored_sender(256, BundleFraming::Message, message_len + 3); + enqueue(&mut s, data.clone()).unwrap(); + + let pdu = s.next_pdu().unwrap().data; + assert_eq!(pdu.len(), message_len + 3); + assert_eq!(pdu[message_len..], [0, 0, 0]); + // Indefinite Padding decodes to nothing. + assert_eq!(decode_all(pdu), vec![bundle_with(vec![], data)]); +} + +#[test] +fn a_pdu_at_or_over_the_floor_is_not_padded() { + let mut s = floored_sender(256, BundleFraming::Message, ETHERNET_FLOOR); + let at = bundle(ETHERNET_FLOOR - HEADER_SIZE); + let over = bundle(ETHERNET_FLOOR); + enqueue(&mut s, at.clone()).unwrap(); + assert_eq!(s.next_pdu().unwrap().data.len(), ETHERNET_FLOOR); + enqueue(&mut s, over.clone()).unwrap(); + let pdu = s.next_pdu().unwrap().data; + assert_eq!(pdu.len(), HEADER_SIZE + ETHERNET_FLOOR); + assert_eq!(decode_all(pdu), vec![bundle_with(vec![], over)]); +} + +#[test] +fn a_floor_above_the_pdu_size_pads_to_the_pdu_size() { + let mut s = floored_sender(64, BundleFraming::Message, 1000); + enqueue(&mut s, Bytes::from_static(b"hello")).unwrap(); + assert_eq!(s.next_pdu().unwrap().data.len(), 64); +} + +#[test] +fn a_bundle_shorter_than_the_floor_is_a_padded_message_not_a_bare_frame() { + let mut s = floored_sender(256, BundleFraming::Bare, ETHERNET_FLOOR); + let bundle = bpv7_like(ETHERNET_FLOOR - 1); + assert_eq!( + enqueue(&mut s, bundle.clone()).map(SendId::kind), + Ok(SendKind::Message) + ); + + let pdu = s.next_pdu().unwrap().data; + assert_eq!(pdu.len(), HEADER_SIZE + bundle.len()); + assert_eq!(decode_all(pdu), vec![bundle_with(vec![], bundle)]); +} + +#[test] +fn a_bare_frame_at_the_floor_is_sent_unpadded() { + let mut s = floored_sender(256, BundleFraming::Bare, ETHERNET_FLOOR); + let bundle = bpv7_like(ETHERNET_FLOOR); + assert_eq!( + enqueue(&mut s, bundle.clone()).map(SendId::kind), + Ok(SendKind::Bare) + ); + assert_eq!(s.next_pdu().unwrap().data, bundle); +} + +#[test] +fn a_floor_above_the_pdu_size_still_allows_a_full_size_bare_frame() { + let mut s = floored_sender(64, BundleFraming::Bare, 1000); + let bundle = bpv7_like(64); + assert_eq!( + enqueue(&mut s, bundle.clone()).map(SendId::kind), + Ok(SendKind::Bare) + ); + assert_eq!(s.next_pdu().unwrap().data, bundle); +} + +#[test] +fn bare_framing_emits_the_bundle_bytes_alone_using_the_whole_pdu() { + let pdu_size = 64; + let mut s = variable_sender(pdu_size, BundleFraming::Bare); + // Exactly pdu_size bytes: too big for a Bundle Message (which needs + // a header) but it fits as a bare frame, so it is not segmented. + let bundle = bpv7_like(pdu_size); + assert_eq!( + enqueue(&mut s, bundle.clone()).map(SendId::kind), + Ok(SendKind::Bare) + ); + + let pdu = s.next_pdu().unwrap().data; + assert_eq!(pdu, bundle); + // Shared with the enqueued buffer, not copied. + assert_eq!(pdu.as_ptr(), bundle.as_ptr()); + assert!(!s.has_pending()); + // The receive side sees it as a bundle. + assert_eq!(decode_all(pdu), vec![bundle_with(vec![], bundle)]); +} + +#[test] +fn bare_frames_keep_queue_order_and_never_share_a_pdu() { + let mut s = variable_sender(64, BundleFraming::Bare); + // A segmented transfer, then a bare frame, then a hinted (so framed) + // bundle: the bare frame must neither jump the transfer nor be + // packed with the Bundle Message behind it. + segmented(enqueue(&mut s, bpv7_like(100)).unwrap()); + let bare = bpv7_like(10); + assert_eq!( + enqueue(&mut s, bare.clone()).map(SendId::kind), + Ok(SendKind::Bare) + ); + let hint = unknown_hint(0x41, b"z"); + assert_eq!( + s.enqueue( + bpv7_like(10), + SendOptions { + hints: vec![hint.clone()].into() + } + ) + .map(SendId::kind), + Ok(SendKind::Message) + ); + + let pdus = drain(&mut s); + let bare_at = pdus.iter().position(|p| *p == bare).unwrap(); + assert!(bare_at >= 1, "bare frame overtook the transfer's segments"); + assert_eq!(pdus.len(), bare_at + 2); + // Everything before the bare frame is the transfer, and decodes + // cleanly (the bare bytes appended to any of them would be a + // framing fault). + let mut ends = 0; + for pdu in &pdus[..bare_at] { + for msg in decode_all(pdu.clone()) { + match msg { + Message::TransferSegment(_) => {} + Message::TransferEnd(_) => ends += 1, + other => panic!("unexpected message before the bare frame: {other:?}"), + } + } + } + assert_eq!(ends, 1); + // The framed bundle follows in a PDU of its own, unpadded. + let last = &pdus[bare_at + 1]; + assert_eq!( + last.len(), + HEADER_SIZE + encoded_hints_len(from_ref(&hint)) + 10 + ); + assert_eq!( + decode_all(last.clone()), + vec![bundle_with(vec![hint], bpv7_like(10))] + ); +} + +#[test] +fn bare_framing_keeps_hinted_bundles_framed() { + let mut s = variable_sender(64, BundleFraming::Bare); + let hint = unknown_hint(0x41, b"z"); + assert_eq!( + s.enqueue( + bpv7_like(10), + SendOptions { + hints: vec![hint.clone()].into(), + }, + ) + .map(SendId::kind), + Ok(SendKind::Message) + ); + assert_eq!( + decode_all(s.next_pdu().unwrap().data), + vec![bundle_with(vec![hint], bpv7_like(10))] + ); +} + +#[test] +fn bare_framing_requires_a_bundle_reserved_first_byte() { + let mut s = variable_sender(64, BundleFraming::Bare); + // 0x00 is a BTP-U message-type byte, not a bundle: a receiver could + // not tell the bare frame from a PDU, so the bundle is framed. + let data = Bytes::from_static(b"\x00not-a-bundle"); + assert_eq!( + enqueue(&mut s, data.clone()).map(SendId::kind), + Ok(SendKind::Message) + ); + + let pdu = s.next_pdu().unwrap().data; + assert_eq!(frame_kind(&pdu), FrameKind::BtpuPdu); + assert_eq!(decode_all(pdu), vec![bundle_with(vec![], data)]); +} + +#[test] +fn bare_bundles_count_against_send_queue_bytes() { + let mut s = sender_with_queue_bytes(64, 10, LinkFraming::variable(BundleFraming::Bare)); + assert!(!s.is_send_queue_full()); + assert_eq!( + enqueue(&mut s, bpv7_like(10)).map(SendId::kind), + Ok(SendKind::Bare) + ); + assert!(s.is_send_queue_full()); + s.next_pdu().unwrap(); + assert!(!s.is_send_queue_full()); +} + +#[test] +fn caller_hint_of_the_bundle_length_type_is_discarded_whatever_its_shape() { + // The sender owns the Bundle Length hint. A caller item of type 0 that + // arrived as `Unknown` (a value length Section 9.1 does not allow, or + // one built by hand) would otherwise ride behind the derived hint and + // supersede it at the receiver. + let mut s = sender(64, 16); + s.enqueue( + bundle(200), + SendOptions { + hints: vec![unknown_hint(0, &[0, 0, 0, 1])].into(), + }, + ) + .unwrap(); + let first = decode_all(s.next_pdu().unwrap().data); + let Message::TransferSegment(m) = &first[0] else { + panic!("expected a segment"); + }; + assert_eq!(m.hints, vec![HintItem::BundleLength(200)]); +} + +#[test] +fn a_segmented_transfer_frees_queue_bytes_as_its_segments_are_packed() { + let mut s = sender_with_queue_bytes(64, 200, LinkFraming::FixedSize); + assert!(!s.is_send_queue_full()); + segmented(enqueue(&mut s, bundle(200)).unwrap()); + assert!(s.is_send_queue_full()); + let mut queued = s.queued_bytes(); + assert_eq!(queued, 200); + while let Some(pdu) = s.next_pdu() { + let carried: usize = decode_all(pdu.data) + .iter() + .map(|m| match m { + Message::TransferSegment(m) | Message::TransferEnd(m) => m.data.len(), + _ => 0, + }) + .sum(); + queued -= carried; + assert_eq!(s.queued_bytes(), queued); + assert!(!s.is_send_queue_full()); + } + assert_eq!(queued, 0); +} + +#[test] +fn a_segment_is_cut_short_to_fill_the_tail_of_a_pdu() { + const PDU: usize = 64; + let mut s = sender(PDU, 16); + let small = bundle(10); + let big = patterned(150); + enqueue(&mut s, small.clone()).unwrap(); + let id = segmented(enqueue(&mut s, big.clone()).unwrap()); + // The sender's first transfer number. + let t = 0; + + // As Appendix A.1 shows: segment 0 fills what the Bundle Message + // leaves, and the PDU needs no padding. + let pdu = s.next_pdu().unwrap(); + let taken = PDU - (HEADER_SIZE + small.len()) - (PDU - first_capacity(PDU, big.len())); + assert_eq!( + decode_all(pdu.data.clone()), + vec![ + bundle_with(vec![], small), + segment_with(t, 0, vec![HintItem::BundleLength(150)], big.slice(..taken)), + ] + ); + assert_eq!(pdu.carried[1..], [carried(id, false)]); + + let mut pdus = vec![pdu.data]; + pdus.extend(drain(&mut s)); + assert_eq!(transfer_data(&pdus, t), big); +} + +#[test] +fn a_tail_under_half_a_segment_is_left_to_padding() { + const PDU: usize = 64; + let big = patterned(150); + let capacity = first_capacity(PDU, big.len()); + let framing = PDU - capacity; + // The Bundle Message length that leaves room for exactly half of + // segment 0's capacity behind it. + let half = capacity.div_ceil(2); + let leaves_half = PDU - framing - half - HEADER_SIZE; + + // At half, the tail is filled. + let mut s = sender(PDU, 16); + enqueue(&mut s, bundle(leaves_half)).unwrap(); + let id = segmented(enqueue(&mut s, big.clone()).unwrap()); + let pdu = s.next_pdu().unwrap(); + assert_eq!(pdu.carried[1..], [carried(id, false)]); + // The sender's first transfer number. + assert_eq!(transfer_data(from_ref(&pdu.data), 0), big[..half]); + + // One byte short of half, the PDU is padded and segment 0 goes out + // whole in the next. + let mut s = sender(PDU, 16); + let small = bundle(leaves_half + 1); + enqueue(&mut s, small.clone()).unwrap(); + segmented(enqueue(&mut s, big.clone()).unwrap()); + let pdu = s.next_pdu().unwrap(); + assert_eq!(pdu.carried.len(), 1); + assert_eq!( + decode_all(pdu.data), + vec![ + bundle_with(vec![], small), + Message::DefinitePadding { + len: half - 1 + framing - HEADER_SIZE + }, + ] + ); + let next = s.next_pdu().unwrap().data; + assert_eq!(transfer_data(from_ref(&next), 0), big[..capacity]); +} + +#[test] +fn a_transfer_without_room_in_the_tail_is_passed_over_but_a_message_is_not() { + const PDU: usize = 64; + let big = patterned(150); + let capacity = first_capacity(PDU, big.len()); + let framing = PDU - capacity; + // Leaves room for one byte less than half of segment 0. + let first = bundle(PDU - framing - capacity.div_ceil(2) - HEADER_SIZE + 1); + let small = bundle(4); + let room = PDU - (HEADER_SIZE + first.len()) - (HEADER_SIZE + small.len()); + // One byte too long for what `small` leaves, with one that would fit + // behind it. + let too_long = bundle(room - HEADER_SIZE + 1); + + let mut s = sender(PDU, 16); + let first_id = enqueue(&mut s, first.clone()).unwrap(); + segmented(enqueue(&mut s, big.clone()).unwrap()); + let small_id = enqueue(&mut s, small.clone()).unwrap(); + enqueue(&mut s, too_long).unwrap(); + enqueue(&mut s, bundle(1)).unwrap(); + + // The transfer cannot fill the tail, so `small` goes ahead of it; the + // next message does not fit, and ends the PDU. + let pdu = s.next_pdu().unwrap(); + assert_eq!( + &pdu.carried[..], + &[carried(first_id, true), carried(small_id, true)] + ); + assert_eq!( + decode_all(pdu.data), + vec![ + bundle_with(vec![], first), + bundle_with(vec![], small), + Message::DefinitePadding { + len: room - HEADER_SIZE + }, + ] + ); + let next = s.next_pdu().unwrap().data; + // The sender's first transfer number. + assert_eq!(transfer_data(from_ref(&next), 0), big[..capacity]); +} + +#[test] +fn cancel_leaves_queued_bare_bundles_intact() { + let mut s = variable_sender(32, BundleFraming::Bare); + let id = segmented(enqueue(&mut s, bpv7_like(100)).unwrap()); + let bare = bpv7_like(10); + enqueue(&mut s, bare.clone()).unwrap(); + // Emit segment 0 so the cancel has to send a Cancel message. + s.next_pdu().unwrap(); + assert!(s.cancel(id)); + + // The queue is now [Cancel, bare frame], and the Cancel (of the + // sender's first transfer number) is packed without the frame. + assert_eq!(decode_all(s.next_pdu().unwrap().data), vec![cancel(0)]); + assert_eq!(s.next_pdu().unwrap().data, bare); + assert!(!s.has_pending()); +} + +// Drain `s`, keeping only the bundles each PDU carries. +fn drain_carried(s: &mut Sender) -> Vec> { + drain_pdus(s) + .into_iter() + .map(|pdu| pdu.carried.to_vec()) + .collect() +} + +#[test] +fn pdu_lists_every_bundle_it_carries_and_flags_those_it_completes() { + let mut s = sender(64, 4); + // Size A so its End carries 20 bytes: 32 bytes of framing and data, + // leaving room for B and D (14 bytes each) in the same PDU. + let hints_len = encoded_hints_len(&[HintItem::BundleLength(0x40)]); + let a_len = 64 - HEADER_SIZE - SEGMENT_FIELDS - hints_len + 20; + assert_eq!( + encoded_hints_len(&[HintItem::BundleLength(a_len as u64)]), + hints_len + ); + let a = enqueue(&mut s, bundle(a_len)).unwrap(); + let b = enqueue(&mut s, bundle(10)).unwrap(); + let d = enqueue(&mut s, bundle(10)).unwrap(); + // A transfer's first segment fills its PDU, so C starts a new one. + let c = enqueue(&mut s, bundle(100)).unwrap(); + assert_eq!( + [a, b, d, c].map(SendId::kind), + [ + SendKind::Transfer, + SendKind::Message, + SendKind::Message, + SendKind::Transfer, + ] + ); + + assert_eq!( + drain_carried(&mut s), + vec![ + vec![carried(a, false)], + vec![carried(a, true), carried(b, true), carried(d, true)], + vec![carried(c, false)], + vec![carried(c, true)], + ] + ); +} + +#[test] +fn middle_segments_list_their_transfer_as_incomplete() { + let mut s = sender(64, 4); + let x = enqueue(&mut s, bundle(150)).unwrap(); + assert_eq!( + drain_carried(&mut s), + vec![ + vec![carried(x, false)], + vec![carried(x, false)], + vec![carried(x, true)], + ] + ); +} + +#[test] +fn bare_frame_pdu_lists_its_bundle_as_complete() { + let mut s = variable_sender(64, BundleFraming::Bare); + let id = enqueue(&mut s, bpv7_like(10)).unwrap(); + assert_eq!(id.kind(), SendKind::Bare); + assert_eq!(drain_carried(&mut s), vec![vec![carried(id, true)]]); +} + +#[test] +fn cancelled_transfer_is_never_listed_as_complete() { + // Unpadded, so the Cancel's PDU decodes to the Cancel alone. + let mut s = variable_sender(64, BundleFraming::Message); + let x = enqueue(&mut s, bundle(150)).unwrap(); + assert_eq!(s.next_pdu().unwrap().carried[..], [carried(x, false)]); + assert!(s.cancel(x)); + // The PDU holding only the Transfer Cancel carries no bundle. + let pdu = s.next_pdu().unwrap(); + assert_eq!(decode_all(pdu.data), vec![cancel(0)]); + assert!(pdu.carried.is_empty()); + assert_eq!(s.next_pdu(), None); +} + +#[test] +fn next_pdu_into_replaces_the_callers_list() { + let mut s = sender(64, 4); + let x = enqueue(&mut s, bundle(100)).unwrap(); + let mut bundles = CarriedList::new(); + + assert_eq!(s.next_pdu_into(&mut bundles).map(|pdu| pdu.len()), Some(64)); + assert_eq!(bundles[..], [carried(x, false)]); + assert_eq!(s.next_pdu_into(&mut bundles).map(|pdu| pdu.len()), Some(64)); + assert_eq!(bundles[..], [carried(x, true)]); + + assert_eq!(s.next_pdu_into(&mut bundles), None); + assert!(bundles.is_empty()); +} + +// Enqueue `n` one-byte bundles, which share a 64-byte PDU. +fn enqueue_tiny(s: &mut Sender, n: usize) -> Vec { + (0..n).map(|_| enqueue(s, bundle(1)).unwrap()).collect() +} + +#[test] +fn next_pdu_list_moves_to_the_heap_only_past_the_inline_entries() { + let mut s = sender(64, 4); + enqueue_tiny(&mut s, CarriedList::INLINE); + let pdu = s.next_pdu().unwrap(); + assert_eq!(pdu.carried.len(), CarriedList::INLINE); + assert_eq!(pdu.carried.capacity(), CarriedList::INLINE); + + let ids = enqueue_tiny(&mut s, CarriedList::INLINE + 1); + let pdu = s.next_pdu().unwrap(); + let expected: Vec<_> = ids.into_iter().map(|id| carried(id, true)).collect(); + assert_eq!(pdu.carried[..], expected[..]); + assert!(pdu.carried.capacity() > CarriedList::INLINE); +} + +#[test] +fn reused_list_keeps_its_heap_buffer_for_a_pdu_that_would_fit_inline() { + let mut s = sender(64, 4); + let mut bundles = CarriedList::new(); + enqueue_tiny(&mut s, CarriedList::INLINE + 1); + s.next_pdu_into(&mut bundles).unwrap(); + let grown = bundles.capacity(); + assert!(grown > CarriedList::INLINE); + + let x = enqueue(&mut s, bundle(10)).unwrap(); + s.next_pdu_into(&mut bundles).unwrap(); + assert_eq!(bundles[..], [carried(x, true)]); + assert_eq!(bundles.capacity(), grown); + // Cleared by an empty queue, the buffer still stays. + assert_eq!(s.next_pdu_into(&mut bundles), None); + assert_eq!(bundles.capacity(), grown); +} + +#[test] +fn with_capacity_below_the_inline_entries_stays_inline() { + assert_eq!( + CarriedList::with_capacity(0).capacity(), + CarriedList::INLINE + ); + assert_eq!( + CarriedList::with_capacity(CarriedList::INLINE).capacity(), + CarriedList::INLINE + ); + assert!(CarriedList::with_capacity(CarriedList::INLINE + 1).capacity() > CarriedList::INLINE); +} + +#[test] +fn list_sized_by_the_bundle_bound_holds_the_fullest_pdu_of_valid_bundles() { + // The smallest valid BPv7 bundle, 32 bytes as a Bundle Message. + const MIN_BUNDLE: usize = 28; + // A 13-byte Transfer End and six such messages fill this PDU exactly. + const PDU: usize = 13 + 6 * (HEADER_SIZE + MIN_BUNDLE); + const WINDOW: u16 = 4; + let bound = PDU / 32 + usize::from(WINDOW); + let mut s = sender(PDU, WINDOW); + + // Size A so its End, after two full segments, carries one byte. (A + // two-segment End carries at least 12, since only a bundle too long + // for a Bundle Message is segmented.) + let hints_len = encoded_hints_len(&[HintItem::BundleLength(2 * PDU as u64)]); + let a_len = + (PDU - HEADER_SIZE - SEGMENT_FIELDS - hints_len) + (PDU - HEADER_SIZE - SEGMENT_FIELDS) + 1; + assert_eq!( + encoded_hints_len(&[HintItem::BundleLength(a_len as u64)]), + hints_len + ); + enqueue(&mut s, bundle(a_len)).unwrap(); + for _ in 0..2 * bound { + enqueue(&mut s, bundle(MIN_BUNDLE)).unwrap(); + } + + let mut bundles = CarriedList::with_capacity(bound); + let capacity = bundles.capacity(); + let mut lens = Vec::new(); + while s.next_pdu_into(&mut bundles).is_some() { + lens.push(bundles.len()); + } + // The End's PDU names one transfer and `PDU / 32` messages; one of + // messages alone holds `PDU / 32`. + assert_eq!(lens[..4], [1, 1, PDU / 32 + 1, PDU / 32]); + assert_eq!(bundles.capacity(), capacity); +} + +#[test] +fn lists_compare_by_entries_not_storage() { + let (mut a, mut b) = (sender(64, 4), sender(64, 4)); + let x = enqueue(&mut a, bundle(10)).unwrap(); + assert_eq!(enqueue(&mut b, bundle(10)).unwrap(), x); + + let mut heap = CarriedList::with_capacity(2 * CarriedList::INLINE); + a.next_pdu_into(&mut heap).unwrap(); + let inline = b.next_pdu().unwrap().carried; + assert!(heap.capacity() > inline.capacity()); + + assert_eq!(heap, inline); + let hash = |list: &CarriedList| { + let mut h = DefaultHasher::new(); + list.hash(&mut h); + h.finish() + }; + assert_eq!(hash(&heap), hash(&inline)); + let expected = format!("{:?}", [carried(x, true)]); + assert_eq!(format!("{heap:?}"), expected); + assert_eq!(format!("{inline:?}"), expected); +} + +#[test] +fn cancel_goes_ahead_of_the_queued_backlog() { + // Unpadded, so each PDU decodes to its messages alone. + let mut s = variable_sender(64, BundleFraming::Message); + let x = enqueue(&mut s, bundle(150)).unwrap(); + let y = enqueue(&mut s, bundle(150)).unwrap(); + let b = enqueue(&mut s, bundle(10)).unwrap(); + assert_eq!(s.next_pdu().unwrap().carried[..], [carried(x, false)]); + + assert!(s.cancel(x)); + // The Cancel leads the next PDU rather than waiting behind Y and B, + // and Y's first segment, cut short, fills the rest of it. + let pdu = s.next_pdu().unwrap(); + assert_eq!(pdu.data.len(), 64); + assert_eq!(pdu.carried[..], [carried(y, false)]); + assert_eq!(decode_all(pdu.data)[0], cancel(0)); + assert_eq!( + drain_carried(&mut s), + vec![ + vec![carried(y, false)], + vec![carried(y, false)], + vec![carried(y, true), carried(b, true)], + ] + ); +} + +#[test] +fn cancel_removes_a_queued_bundle_message_and_sends_nothing_for_it() { + let mut s = variable_sender(64, BundleFraming::Message); + let a = enqueue(&mut s, bundle(10)).unwrap(); + let b = enqueue(&mut s, bundle(20)).unwrap(); + assert!(s.cancel(a)); + // Gone from the queue, so a second cancel finds nothing. + assert!(!s.cancel(a)); + + let pdu = s.next_pdu().unwrap(); + assert_eq!(decode_all(pdu.data), vec![bundle_with(vec![], bundle(20))]); + assert_eq!(pdu.carried[..], [carried(b, true)]); + assert_eq!(s.next_pdu(), None); +} + +#[test] +fn cancel_removes_a_queued_bare_frame() { + let mut s = variable_sender(64, BundleFraming::Bare); + let a = enqueue(&mut s, bpv7_like(10)).unwrap(); + let b = bpv7_like(20); + enqueue(&mut s, b.clone()).unwrap(); + assert!(s.cancel(a)); + assert_eq!(drain(&mut s), vec![b]); +} + +#[test] +fn cancel_after_the_last_bytes_are_packed_changes_nothing() { + let mut s = variable_sender(64, BundleFraming::Bare); + let message = enqueue(&mut s, bundle(10)).unwrap(); + s.next_pdu().unwrap(); + let bare = enqueue(&mut s, bpv7_like(10)).unwrap(); + s.next_pdu().unwrap(); + let transfer = enqueue(&mut s, bundle(100)).unwrap(); + drain(&mut s); + + for id in [message, bare, transfer] { + assert!(!s.cancel(id), "{id:?}"); + } + assert!(!s.has_pending()); +} + +#[test] +fn cancel_matches_the_id_variant_as_well_as_the_number() { + // A Bundle Message, a bare frame, and a transfer are queued, numbered + // 0, 1, and 0. A second sender, enqueueing in another order, issues + // the other kind for each of those numbers: a bare frame 0, a Bundle + // Message 1, and a transfer 1 behind a transfer 0. None names a + // bundle of the first sender. + let mut s = variable_sender(64, BundleFraming::Bare); + let message = enqueue(&mut s, bundle(10)).unwrap(); + let bare = enqueue(&mut s, bpv7_like(10)).unwrap(); + let transfer = enqueue(&mut s, bundle(100)).unwrap(); + assert_eq!( + [message, bare, transfer].map(SendId::kind), + [SendKind::Message, SendKind::Bare, SendKind::Transfer] + ); + let mut other = variable_sender(64, BundleFraming::Bare); + let other_bare = enqueue(&mut other, bpv7_like(10)).unwrap(); + let other_message = enqueue(&mut other, bundle(10)).unwrap(); + enqueue(&mut other, bundle(100)).unwrap(); + let other_transfer = enqueue(&mut other, bundle(100)).unwrap(); + assert_eq!( + [other_bare, other_message, other_transfer].map(SendId::kind), + [SendKind::Bare, SendKind::Message, SendKind::Transfer] + ); + for id in [other_bare, other_message, other_transfer] { + assert!(!s.cancel(id), "{id:?}"); + } + assert_eq!( + drain_carried(&mut s), + vec![ + vec![carried(message, true)], + vec![carried(bare, true)], + vec![carried(transfer, false)], + vec![carried(transfer, true)], + ] + ); +} + +#[test] +fn cancel_by_one_variant_leaves_the_others_with_that_number() { + // Message(0) and Transfer(0) share a number; cancelling the first must + // not touch the second. + let mut s = variable_sender(64, BundleFraming::Bare); + let message = enqueue(&mut s, bundle(10)).unwrap(); + let bare = enqueue(&mut s, bpv7_like(10)).unwrap(); + let transfer = enqueue(&mut s, bundle(100)).unwrap(); + assert_eq!( + [message, transfer].map(SendId::kind), + [SendKind::Message, SendKind::Transfer] + ); + + assert!(s.cancel(message)); + assert!(s.is_outstanding(transfer)); + assert_eq!( + drain_carried(&mut s), + vec![ + vec![carried(bare, true)], + vec![carried(transfer, false)], + vec![carried(transfer, true)], + ] + ); +} + +#[test] +fn cancelling_a_queued_bundle_frees_send_queue_bytes() { + let mut s = sender_with_queue_bytes(64, 10, LinkFraming::FixedSize); + let id = enqueue(&mut s, bundle(10)).unwrap(); + assert!(s.is_send_queue_full()); + assert!(s.cancel(id)); + assert!(!s.is_send_queue_full()); + assert_eq!(s.queued_bytes(), 0); +} + +#[test] +fn debug_summarises_the_queue_instead_of_printing_it() { + let mut s = sender(64, 4); + enqueue(&mut s, Bytes::from(vec![0xAB; 1000])).unwrap(); + enqueue(&mut s, Bytes::from_static(b"small")).unwrap(); + let bound = SendQueueBytes::DEFAULT; + let summary = format!( + "Sender {{ pdu_size: PduSize(64), send_queue_bytes: SendQueueBytes({bound}), \ + link_framing: FixedSize, segment_cut_strategy: Full, window_size: WindowSize(4), \ + transfers_outstanding: 1, window_available: true, queued: 2, \ + queued_bytes: 1005, assembling: 0, next_bundle_id: 1" + ); + #[cfg(not(feature = "tower"))] + let expected = format!("{summary} }}"); + #[cfg(feature = "tower")] + let expected = format!("{summary}, enqueue_wakers: 0, drain_waker: false }}"); + assert_eq!(format!("{s:?}"), expected); +} diff --git a/btpu/tests/streaming.rs b/btpu/tests/streaming.rs new file mode 100644 index 000000000..ff1c3452b --- /dev/null +++ b/btpu/tests/streaming.rs @@ -0,0 +1,415 @@ +//! Streamed delivery and `Receiver::refuse` through the public `receiver` +//! API. + +mod common; + +use bytes::Bytes; +use hardy_btpu::{ + codec::{ + hint::{HintItem, Hints}, + message::Message, + }, + fec::PreAgreedFecMessage, + receiver::{ + Delivery, DropReason, MaxSegments, Receiver, ReceiverConfig, ReceiverEvent, RejectReason, + }, + transfer::TransferId, +}; + +use self::common::{ + Event, bundle_msg, cancel, cancelled, dropped, encode, end, expired, is_within, none, received, + receiver_config, rejected, segment, segment_with, unknown_hint, +}; + +fn streamed_config(window: u16, cap: usize) -> ReceiverConfig { + ReceiverConfig { + delivery: Delivery::Streamed, + ..receiver_config(window, cap) + } +} + +fn streamed(window: u16, cap: usize) -> Receiver { + Receiver::new(streamed_config(window, cap)) +} + +fn started(transfer_number: u32, hints: Vec) -> Event { + Event::TransferStarted { + transfer_number, + hints: Hints::from(hints), + } +} + +fn data(transfer_number: u32, data: &'static [u8]) -> Event { + data_with(transfer_number, data, None) +} + +fn data_with(transfer_number: u32, data: &'static [u8], hints: Option>) -> Event { + Event::TransferData { + transfer_number, + data: Bytes::from_static(data), + hints: hints.map(Hints::from), + } +} + +fn finished(transfer_number: u32, data: &'static [u8]) -> Event { + Event::TransferFinished { + transfer_number, + data: Bytes::from_static(data), + hints: None, + } +} + +// The id `events` started a transfer with. +fn started_id(events: &[ReceiverEvent]) -> TransferId { + events + .iter() + .find_map(|e| match e { + ReceiverEvent::TransferStarted { id, .. } => Some(*id), + _ => None, + }) + .expect("a transfer started") +} + +#[test] +fn in_order_segments_are_released_as_views_of_their_pdus() { + let mut r = streamed(16, 1024); + // A segment this short is copied out of its PDU when held (see + // `segments_shorter_than_half_the_pdu_are_copied_out_of_it` in + // receiver.rs); released at once, it is not held. + let pdu = encode(&segment(0, 0, b"abc")); + let events = r.receive_pdu(pdu.clone()); + assert_eq!(events, vec![started(0, vec![]), data(0, b"abc")]); + let ReceiverEvent::TransferData { data, .. } = &events[1] else { + panic!("the second event carries the data"); + }; + assert!(is_within(data, &pdu)); + assert_eq!(r.retained_bytes(), 0); + + let pdu = encode(&end(0, 1, b"def")); + let events = r.receive_pdu(pdu.clone()); + assert_eq!(events, vec![finished(0, b"def")]); + let ReceiverEvent::TransferFinished { data, .. } = &events[0] else { + panic!("the event carries the data"); + }; + assert!(is_within(data, &pdu)); +} + +#[test] +fn a_filled_gap_releases_the_run_behind_it() { + let mut r = streamed(16, 1024); + assert_eq!(r.process_message(segment(0, 1, b"b")), none()); + assert_eq!(r.process_message(segment(0, 2, b"c")), none()); + assert!(r.retained_bytes() > 0); + assert_eq!( + r.process_message(segment(0, 0, b"a")), + vec![ + started(0, vec![]), + data(0, b"a"), + data(0, b"b"), + data(0, b"c") + ] + ); + assert_eq!(r.retained_bytes(), 0); + assert_eq!(r.process_message(end(0, 3, b"d")), vec![finished(0, b"d")]); +} + +#[test] +fn an_end_arriving_before_the_gap_fills_finishes_with_the_run() { + let mut r = streamed(16, 1024); + assert_eq!(r.process_message(end(0, 1, b"b")), none()); + assert_eq!( + r.process_message(segment(0, 0, b"a")), + vec![started(0, vec![]), data(0, b"a"), finished(0, b"b")] + ); + assert_eq!( + r.process_message(end(0, 1, b"b")), + vec![dropped(0, DropReason::Delivered)] + ); +} + +#[test] +fn a_lone_end_starts_and_finishes() { + let mut r = streamed(16, 1024); + assert_eq!( + r.process_message(end(0, 0, b"whole")), + vec![started(0, vec![]), finished(0, b"whole")] + ); +} + +#[test] +fn an_end_naming_a_released_segment_finishes_with_no_data() { + let mut r = streamed(16, 1024); + assert_eq!( + r.process_message(segment(0, 0, b"a")), + vec![started(0, vec![]), data(0, b"a")] + ); + // The released Segment keeps its bytes; the End only records N. + assert_eq!(r.process_message(end(0, 0, b"a")), vec![finished(0, b"")]); +} + +#[test] +fn empty_segments_are_not_reported() { + let mut r = streamed(16, 1024); + assert_eq!(r.process_message(segment(0, 0, b"")), none()); + assert_eq!( + r.process_message(segment(0, 1, b"a")), + vec![started(0, vec![]), data(0, b"a")] + ); + assert_eq!(r.process_message(segment(0, 2, b"")), none()); + assert_eq!(r.process_message(end(0, 3, b"")), vec![finished(0, b"")]); +} + +#[test] +fn a_transfer_with_no_data_is_rejected_without_starting() { + let mut r = streamed(16, 1024); + assert_eq!(r.process_message(segment(0, 0, b"")), none()); + assert_eq!( + r.process_message(end(0, 1, b"")), + vec![rejected(0, RejectReason::Empty)] + ); +} + +#[test] +fn a_started_transfer_can_be_cancelled() { + let mut r = streamed(16, 1024); + r.process_message(segment(0, 0, b"a")); + assert_eq!(r.process_message(cancel(0)), vec![cancelled(0)]); + assert_eq!( + r.process_message(segment(0, 1, b"b")), + vec![dropped(0, DropReason::Cancelled)] + ); +} + +#[test] +fn a_started_transfer_can_expire() { + let mut r = streamed(4, 1024); + r.process_message(segment(0, 0, b"a")); + assert_eq!( + r.process_message(segment(4, 0, b"x")), + vec![expired(0), started(4, vec![]), data(4, b"x")] + ); +} + +#[test] +fn released_bytes_count_toward_the_transfer_size_cap() { + let mut r = streamed(16, 4); + assert_eq!( + r.process_message(segment(0, 0, b"abc")), + vec![started(0, vec![]), data(0, b"abc")] + ); + assert_eq!( + r.process_message(segment(0, 1, b"de")), + vec![rejected(0, RejectReason::TooLarge)] + ); +} + +#[test] +fn a_repeat_of_a_released_segment_is_a_duplicate() { + let mut r = streamed(16, 1024); + r.process_message(segment(0, 0, b"a")); + assert_eq!( + r.process_message(segment(0, 0, b"a")), + vec![dropped(0, DropReason::Duplicate)] + ); +} + +#[test] +fn an_end_below_a_released_segment_conflicts() { + let mut r = streamed(16, 1024); + r.process_message(segment(0, 0, b"a")); + r.process_message(segment(0, 1, b"b")); + assert_eq!( + r.process_message(end(0, 0, b"a")), + vec![dropped(0, DropReason::SegmentIndexConflict)] + ); +} + +#[test] +fn both_deliveries_reject_at_the_same_segment() { + // Released segments count toward the segment limit and the + // bookkeeping budget, so a transfer fails at the same message whether + // its segments were released or held. + let configs = |max_segments| { + [Delivery::Whole, Delivery::Streamed].map(|delivery| ReceiverConfig { + delivery, + max_segments_per_transfer: max_segments, + ..receiver_config(16, 64) + }) + }; + for max_segments in [Some(MaxSegments::try_from(3).unwrap()), None] { + let rejected_at = configs(max_segments).map(|config| { + let mut r = Receiver::new(config); + (0..) + .find(|&i| { + r.process_message(segment(0, i, b"")) + .iter() + .any(|e| matches!(e, ReceiverEvent::TransferRejected { .. })) + }) + .unwrap() + }); + assert_eq!(rejected_at[0], rejected_at[1], "{max_segments:?}"); + } +} + +#[test] +fn hints_are_reported_when_they_change() { + let a = unknown_hint(0x41, b"a"); + let b = unknown_hint(0x42, b"b"); + let c = unknown_hint(0x43, b"c"); + let mut r = streamed(16, 1024); + assert_eq!( + r.process_message(segment_with( + 0, + 0, + vec![a.clone()], + Bytes::from_static(b"0") + )), + vec![started(0, vec![a.clone()]), data(0, b"0")] + ); + // A repeat of a value already reported is no change. + assert_eq!( + r.process_message(segment_with( + 0, + 1, + vec![a.clone()], + Bytes::from_static(b"1") + )), + vec![data(0, b"1")] + ); + assert_eq!( + r.process_message(segment_with( + 0, + 2, + vec![b.clone()], + Bytes::from_static(b"2") + )), + vec![data_with(0, b"2", Some(vec![a.clone(), b.clone()]))] + ); + // A change on a message that releases nothing goes out on the next + // event. + assert_eq!( + r.process_message(segment_with( + 0, + 4, + vec![c.clone()], + Bytes::from_static(b"4") + )), + none() + ); + assert_eq!( + r.process_message(segment(0, 3, b"3")), + vec![data_with(0, b"3", Some(vec![a, b, c])), data(0, b"4")] + ); +} + +#[test] +fn hints_before_the_first_data_go_out_on_started() { + let a = unknown_hint(0x41, b"a"); + let mut r = streamed(16, 1024); + assert_eq!( + r.process_message(segment_with(0, 0, vec![a.clone()], Bytes::new())), + none() + ); + assert_eq!( + r.process_message(segment(0, 1, b"x")), + vec![started(0, vec![a]), data(0, b"x")] + ); +} + +#[test] +fn fec_transfers_release_nothing() { + let mut r = Receiver::new(ReceiverConfig { + fec: true, + ..streamed_config(16, 1024) + }); + let fec = Message::PreAgreedFecSource(PreAgreedFecMessage { + transfer_number: 0, + fec_instance_id: 1, + hints: vec![], + payload: Bytes::from_static(b"fec"), + }); + assert_eq!(r.process_message(fec), none()); +} + +#[test] +fn bundle_messages_are_received_whole() { + let mut r = streamed(16, 1024); + assert_eq!( + r.process_message(bundle_msg(b"hello")), + vec![received(b"hello")] + ); +} + +#[test] +fn refuse_closes_a_held_transfer() { + let mut r = streamed(16, 1024); + let id = started_id(&r.process_message(segment(0, 0, b"a"))); + r.process_message(segment(0, 2, b"c")); + assert!(r.retained_bytes() > 0); + + assert!(r.refuse(id)); + assert_eq!(r.retained_bytes(), 0); + assert_eq!( + r.process_message(segment(0, 1, b"b")), + vec![dropped(0, DropReason::Refused)] + ); + assert!(!r.refuse(id)); +} + +#[test] +fn refuse_ignores_a_finished_transfer() { + let mut r = streamed(16, 1024); + let id = started_id(&r.process_message(end(0, 0, b"a"))); + assert!(!r.refuse(id)); + assert_eq!( + r.process_message(end(0, 0, b"a")), + vec![dropped(0, DropReason::Delivered)] + ); +} + +#[test] +fn refuse_ignores_an_expired_transfer() { + let mut r = streamed(4, 1024); + let id = started_id(&r.process_message(segment(0, 0, b"a"))); + r.process_message(segment(4, 0, b"x")); + assert!(!r.refuse(id)); +} + +#[test] +fn refuse_ignores_an_id_from_before_a_reset() { + let mut r = streamed(16, 1024); + let before = started_id(&r.process_message(segment(0, 0, b"a"))); + r.reset(); + let after = started_id(&r.process_message(segment(0, 0, b"a"))); + assert_ne!(before, after); + assert_eq!(before.transfer_number(), after.transfer_number()); + + assert!(!r.refuse(before)); + assert_eq!(r.process_message(end(0, 1, b"b")), vec![finished(0, b"b")]); +} + +#[test] +fn a_dropped_message_names_its_transfer_by_id_inside_the_window() { + let mut r = streamed(4, 1024); + let id = started_id(&r.process_message(segment(0, 0, b"a"))); + let drop = |transfer_number, id, reason| ReceiverEvent::MessageDropped { + transfer_number, + id, + reason, + }; + + assert_eq!( + r.process_message(segment(0, 0, b"a")), + vec![drop(0, Some(id), DropReason::Duplicate)] + ); + assert_eq!(r.process_message(end(0, 1, b"b")), vec![finished(0, b"b")]); + assert_eq!( + r.process_message(end(0, 1, b"b")), + vec![drop(0, Some(id), DropReason::Delivered)] + ); + // No id is assigned to a number outside the window. + assert_eq!( + r.process_message(cancel(100)), + vec![drop(100, None, DropReason::UnknownTransfer)] + ); +} diff --git a/btpu/tests/tower.rs b/btpu/tests/tower.rs new file mode 100644 index 000000000..a9636e8c7 --- /dev/null +++ b/btpu/tests/tower.rs @@ -0,0 +1,492 @@ +//! Integration tests for the `tower` feature. + +#![cfg(feature = "tower")] + +mod common; + +use std::{ + pin::Pin, + sync::{ + Arc, Mutex, + atomic::{AtomicBool, Ordering}, + }, + task::{Context, Poll, Wake, Waker}, +}; + +use bytes::Bytes; +use futures::{executor::block_on, task::noop_waker_ref}; +use futures_core::Stream; +use hardy_btpu::{ + codec::hint::HintItem, + receiver::{Receiver, ReceiverConfig, ReceiverEvent}, + sender::{ + BundleFraming, LinkFraming, Pdu, SendId, SendKind, SendOptions, SendRequest, Sender, + SenderConfig, + }, +}; +use tower::{Service, ServiceBuilder, ServiceExt}; + +use self::common::{ + bpv7_like, bundle, carried, received, received_with, receiver, sender, sender_with_queue_bytes, + unknown_hint, +}; + +fn poll_stream_until_idle(sender: &mut Sender) -> Vec { + let mut pdus = Vec::new(); + let mut cx = Context::from_waker(noop_waker_ref()); + loop { + match poll_next(sender, &mut cx) { + Poll::Ready(Some(pdu)) => pdus.push(pdu.data), + // The sender is documented as a perpetual source; treating None + // as "idle" here would silently mask that regression. + Poll::Ready(None) => panic!("sender stream must never finish"), + Poll::Pending => break, + } + } + pdus +} + +fn poll_next(sender: &mut Sender, cx: &mut Context<'_>) -> Poll> { + Pin::new(sender).poll_next(cx) +} + +fn poll_ready(sender: &mut Sender, cx: &mut Context<'_>) -> Poll<()> { + Service::poll_ready(sender, cx).map(|r| r.unwrap()) +} + +fn call(sender: &mut Sender, data: Bytes) -> SendId { + block_on(Service::call(sender, SendRequest::from(data))).unwrap() +} + +fn receive_all(receiver: &mut Receiver, pdus: Vec) -> Vec { + pdus.into_iter() + .flat_map(|pdu| block_on(Service::call(&mut *receiver, pdu)).unwrap()) + .collect() +} + +// A sender whose window of 4 is filled by segmented bundles, and the id of +// the oldest. +fn saturated_sender() -> (Sender, SendId) { + let mut sender = sender(32, 4); + let oldest = call(&mut sender, bundle(200)); + for _ in 1..4 { + call(&mut sender, bundle(200)); + } + (sender, oldest) +} + +// A tiny waker that raises a flag when woken. `std::task::Wake` on an +// `Arc` supplies the vtable, so no unsafe is needed. +struct Flag(AtomicBool); + +impl Flag { + fn waker() -> (Arc, Waker) { + let flag = Arc::new(Self(AtomicBool::new(false))); + let waker = Waker::from(flag.clone()); + (flag, waker) + } + + fn raised(&self) -> bool { + self.0.load(Ordering::SeqCst) + } +} + +impl Wake for Flag { + fn wake(self: Arc) { + self.wake_by_ref(); + } + + fn wake_by_ref(self: &Arc) { + self.0.store(true, Ordering::SeqCst); + } +} + +#[test] +fn receiver_service_round_trip() { + let mut receiver = Receiver::new(ReceiverConfig::default()); + let mut sender = sender(256, 16); + call(&mut sender, Bytes::from_static(b"hello")); + let pdu = poll_stream_until_idle(&mut sender).pop().unwrap(); + + let events = block_on(Service::call(&mut receiver, pdu)).unwrap(); + assert_eq!(events, vec![received(b"hello")]); +} + +#[test] +fn sender_service_enqueue_then_stream_drain() { + let mut sender = sender(64, 16); + let mut receiver = Receiver::new(ReceiverConfig::default()); + let original = bundle(200); + + assert_eq!( + call(&mut sender, original.clone()).kind(), + SendKind::Transfer, + "200-byte bundle in 64-byte PDU must segment" + ); + + let pdus = poll_stream_until_idle(&mut sender); + assert_eq!( + receive_all(&mut receiver, pdus), + vec![received_with(original, vec![HintItem::BundleLength(200)])] + ); +} + +#[test] +fn sender_service_with_layer() { + // Compile-time + runtime check that Sender slots into ServiceBuilder. + let sender = Sender::new(SenderConfig::default(), 0); + let mut svc = ServiceBuilder::new().concurrency_limit(4).service(sender); + + let res = block_on(async { + svc.ready() + .await + .unwrap() + .call(Bytes::from_static(b"small").into()) + .await + }) + .unwrap(); + // Small bundle in default 1500-byte PDU goes as a single Bundle message, + // so no transfer number is allocated. + assert_eq!(res.kind(), SendKind::Message); +} + +#[test] +fn sender_service_poll_ready_blocks_until_the_oldest_end_drains() { + let (mut sender, oldest) = saturated_sender(); + let mut cx = Context::from_waker(noop_waker_ref()); + assert_eq!(poll_ready(&mut sender, &mut cx), Poll::Pending); + + // Draining PDU by PDU: poll_ready turns Ready exactly when the oldest + // transfer's End has been packed, freeing its slot. + loop { + assert!(matches!( + poll_next(&mut sender, &mut cx), + Poll::Ready(Some(_)) + )); + let ready = poll_ready(&mut sender, &mut cx).is_ready(); + assert_eq!(ready, !sender.is_outstanding(oldest)); + if ready { + break; + } + } +} + +#[test] +fn sender_stream_pending_when_idle() { + let mut sender = Sender::new(SenderConfig::default(), 0); + let mut cx = Context::from_waker(noop_waker_ref()); + + // No pending: poll_next must be Pending (not Ready(None); the sender + // is a perpetual source until dropped). + assert_eq!(poll_next(&mut sender, &mut cx), Poll::Pending); + + call(&mut sender, Bytes::from_static(b"hello")); + assert!(matches!( + poll_next(&mut sender, &mut cx), + Poll::Ready(Some(_)) + )); + + // Drained: back to Pending. + assert_eq!(poll_next(&mut sender, &mut cx), Poll::Pending); +} + +#[test] +fn sender_stream_yields_the_bundles_each_pdu_carries() { + let mut sender = sender(64, 4); + let id = call(&mut sender, bundle(100)); + let mut cx = Context::from_waker(noop_waker_ref()); + let mut pdus: Vec = Vec::new(); + while let Poll::Ready(Some(pdu)) = poll_next(&mut sender, &mut cx) { + pdus.push(pdu); + } + + // 100 bytes in 64-byte PDUs spans two: the head, then the End. + let bundles: Vec<_> = pdus.into_iter().map(|p| p.carried.to_vec()).collect(); + assert_eq!( + bundles, + vec![vec![carried(id, false)], vec![carried(id, true)]] + ); +} + +#[test] +fn sender_drain_wakes_pending_enqueue_task_when_window_full() { + let (mut sender, oldest) = saturated_sender(); + + let (woke, waker) = Flag::waker(); + let mut cx = Context::from_waker(&waker); + assert_eq!(poll_ready(&mut sender, &mut cx), Poll::Pending); + assert!(!woke.raised()); + + // Draining the End of the oldest transfer frees a window slot, so it + // wakes the parked task, which then finds the service ready. No + // earlier PDU frees anything, so none of them wakes it. + drain_until_window_opens(&mut sender, oldest, &[&woke]); + assert!(woke.raised(), "draining should wake the enqueue waker"); + assert_eq!(poll_ready(&mut sender, &mut cx), Poll::Ready(())); +} + +// Drain `sender` one PDU at a time until the PDU carrying the End of +// `oldest`, asserting that no flag is raised before it. +fn drain_until_window_opens(sender: &mut Sender, oldest: SendId, flags: &[&Flag]) { + let mut noop_cx = Context::from_waker(noop_waker_ref()); + loop { + for flag in flags { + assert!(!flag.raised(), "woken before capacity freed"); + } + let Poll::Ready(Some(pdu)) = poll_next(sender, &mut noop_cx) else { + panic!("the queue ran dry before the oldest transfer ended"); + }; + if pdu.carried.contains(&carried(oldest, true)) { + return; + } + } +} + +#[test] +fn sender_cancel_wakes_pending_enqueue_task() { + let (mut sender, oldest) = saturated_sender(); + + let (woke, waker) = Flag::waker(); + let mut cx = Context::from_waker(&waker); + assert_eq!(poll_ready(&mut sender, &mut cx), Poll::Pending); + assert!(!woke.raised()); + + // Cancelling the oldest transfer frees the window and must wake the + // parked task, which then finds the service ready. + assert!(sender.cancel(oldest)); + assert!(woke.raised(), "cancel() should wake the enqueue waker"); + assert_eq!(poll_ready(&mut sender, &mut cx), Poll::Ready(())); +} + +#[test] +fn every_parked_producer_is_woken() { + // Two producers sharing a Sender each park in poll_ready; when + // capacity frees both must wake, or the one not woken sleeps forever + // once the other has nothing more to send. + let (mut sender, oldest) = saturated_sender(); + + let (a, waker_a) = Flag::waker(); + let (b, waker_b) = Flag::waker(); + assert_eq!( + poll_ready(&mut sender, &mut Context::from_waker(&waker_a)), + Poll::Pending + ); + assert_eq!( + poll_ready(&mut sender, &mut Context::from_waker(&waker_b)), + Poll::Pending + ); + // Re-polling with a waker already registered does not displace the + // other one. + assert_eq!( + poll_ready(&mut sender, &mut Context::from_waker(&waker_a)), + Poll::Pending + ); + + drain_until_window_opens(&mut sender, oldest, &[&a, &b]); + assert!(a.raised(), "producer A was not woken"); + assert!(b.raised(), "producer B was not woken"); +} + +// The documented fan-in pattern: one lock acquisition spans `poll_ready` +// and `call`, and the guard is dropped before parking, so the admission a +// producer was told about cannot be taken by another producer in between +// and a `Pending` producer never holds the drain out. +fn admit(shared: &Mutex, cx: &mut Context<'_>, data: Bytes) -> Poll { + let mut sender = shared.lock().unwrap(); + match poll_ready(&mut sender, cx) { + Poll::Ready(()) => Poll::Ready(call(&mut sender, data)), + Poll::Pending => Poll::Pending, + } +} + +#[test] +fn producers_admitted_under_one_lock_hold_never_see_window_full() { + // Window of 4, all four slots taken. Two producers compete for the + // next slot; each is either admitted or parked, never told Ready and + // then refused with WindowFull by `call`. + let shared = Mutex::new(sender(32, 4)); + let big = Bytes::from(vec![0u8; 200]); + let mut noop_cx = Context::from_waker(noop_waker_ref()); + for _ in 0..4 { + assert_eq!( + admit(&shared, &mut noop_cx, big.clone()).map(SendId::kind), + Poll::Ready(SendKind::Transfer) + ); + } + + let (a, waker_a) = Flag::waker(); + let (b, waker_b) = Flag::waker(); + assert_eq!( + admit(&shared, &mut Context::from_waker(&waker_a), big.clone()), + Poll::Pending + ); + assert_eq!( + admit(&shared, &mut Context::from_waker(&waker_b), big.clone()), + Poll::Pending + ); + + // Drain until exactly one slot frees (the first transfer's End is + // packed). The drain task takes the lock per PDU, which the parked + // producers are not holding. + while !shared.lock().unwrap().is_window_available() { + let mut sender = shared.lock().unwrap(); + assert!(matches!( + poll_next(&mut sender, &mut noop_cx), + Poll::Ready(Some(_)) + )); + } + assert!(a.raised() && b.raised()); + + // Both race for the one slot: the first is admitted, the second parks + // again rather than failing. + assert_eq!( + admit(&shared, &mut Context::from_waker(&waker_a), big.clone()).map(SendId::kind), + Poll::Ready(SendKind::Transfer) + ); + assert_eq!( + admit(&shared, &mut Context::from_waker(&waker_b), big.clone()), + Poll::Pending + ); + while !shared.lock().unwrap().is_window_available() { + let mut sender = shared.lock().unwrap(); + assert!(matches!( + poll_next(&mut sender, &mut noop_cx), + Poll::Ready(Some(_)) + )); + } + assert_eq!( + admit(&shared, &mut Context::from_waker(&waker_b), big).map(SendId::kind), + Poll::Ready(SendKind::Transfer) + ); +} + +#[test] +fn cancelling_a_queued_bundle_wakes_a_producer_parked_on_queue_bytes() { + let mut sender = sender_with_queue_bytes(256, 4, LinkFraming::FixedSize); + let id = call(&mut sender, Bytes::from_static(b"tiny")); + + let (woke, waker) = Flag::waker(); + let mut cx = Context::from_waker(&waker); + assert_eq!(poll_ready(&mut sender, &mut cx), Poll::Pending); + + assert!(sender.cancel(id)); + assert!(woke.raised(), "cancel() should wake the enqueue waker"); + assert_eq!(poll_ready(&mut sender, &mut cx), Poll::Ready(())); +} + +#[test] +fn draining_a_bare_frame_wakes_a_producer_parked_on_queue_bytes() { + let mut sender = sender_with_queue_bytes(256, 20, LinkFraming::variable(BundleFraming::Bare)); + assert_eq!(call(&mut sender, bpv7_like(20)).kind(), SendKind::Bare); + + let (woke, waker) = Flag::waker(); + let mut cx = Context::from_waker(&waker); + assert_eq!(poll_ready(&mut sender, &mut cx), Poll::Pending); + + assert_eq!(poll_stream_until_idle(&mut sender), vec![bpv7_like(20)]); + assert!( + woke.raised(), + "popping a bare frame should wake the enqueue waker" + ); + assert_eq!(poll_ready(&mut sender, &mut cx), Poll::Ready(())); +} + +#[test] +fn stream_stays_pending_after_cancel_empties_the_queue() { + let mut sender = sender(256, 16); + let id = call(&mut sender, Bytes::from_static(b"tiny")); + assert!(sender.cancel(id)); + + let mut cx = Context::from_waker(noop_waker_ref()); + assert_eq!(poll_next(&mut sender, &mut cx), Poll::Pending); +} + +#[test] +fn sender_service_poll_ready_blocks_when_send_queue_full() { + // Small bundles take the unsegmented path and never allocate a window + // slot, so only the send-queue bound can bound them. + let mut sender = sender_with_queue_bytes(256, 8, LinkFraming::FixedSize); + for _ in 0..2 { + call(&mut sender, Bytes::from_static(b"tiny")); + } + + // The window is untouched, yet the sender must exert backpressure: the + // queue is at its configured bound. + let (woke, waker) = Flag::waker(); + let mut cx = Context::from_waker(&waker); + assert_eq!(poll_ready(&mut sender, &mut cx), Poll::Pending); + assert!(!woke.raised()); + + // Draining a PDU frees queue capacity and wakes the parked task. + let mut noop_cx = Context::from_waker(noop_waker_ref()); + assert!(matches!( + poll_next(&mut sender, &mut noop_cx), + Poll::Ready(Some(_)) + )); + assert!( + woke.raised(), + "draining next_pdu should wake the enqueue waker" + ); + assert_eq!(poll_ready(&mut sender, &mut noop_cx), Poll::Ready(())); +} + +#[test] +fn a_push_that_readies_a_segment_wakes_the_pending_drain_task() { + let mut sender = sender(64, 16); + let mut handle = sender.begin(200, SendOptions::default()).unwrap(); + + // Begun but with nothing pushed, the transfer supplies nothing: the + // stream parks. + let (woke, waker) = Flag::waker(); + let mut cx = Context::from_waker(&waker); + assert_eq!(poll_next(&mut sender, &mut cx), Poll::Pending); + assert!(!woke.raised()); + + sender.push(&mut handle, bundle(200)).unwrap(); + assert!(woke.raised(), "push should wake the drain waker"); + sender.finish(handle).unwrap(); + assert!(!poll_stream_until_idle(&mut sender).is_empty()); +} + +#[test] +fn sender_enqueue_wakes_pending_drain_task() { + let mut sender = Sender::new(SenderConfig::default(), 0); + + // Nothing pending: the stream parks and registers our waker. + let (woke, waker) = Flag::waker(); + let mut cx = Context::from_waker(&waker); + assert_eq!(poll_next(&mut sender, &mut cx), Poll::Pending); + assert!(!woke.raised()); + + call(&mut sender, Bytes::from_static(b"hello")); + assert!(woke.raised(), "enqueue should wake the drain waker"); + let mut noop_cx = Context::from_waker(noop_waker_ref()); + assert!(matches!( + poll_next(&mut sender, &mut noop_cx), + Poll::Ready(Some(_)) + )); +} + +#[test] +fn sender_service_send_request_carries_hints() { + let mut sender = sender(64, 16); + let mut receiver = receiver(16, usize::MAX); + + let correlator = unknown_hint(0x41, b"\x2A"); + let request = SendRequest { + data: bundle(200), + options: SendOptions { + hints: vec![correlator.clone()].into(), + }, + }; + block_on(Service::call(&mut sender, request)).unwrap(); + + let pdus = poll_stream_until_idle(&mut sender); + assert_eq!( + receive_all(&mut receiver, pdus), + vec![received_with( + bundle(200), + vec![HintItem::BundleLength(200), correlator] + )] + ); +} diff --git a/btpu/tests/transfer.rs b/btpu/tests/transfer.rs new file mode 100644 index 000000000..da5a62b56 --- /dev/null +++ b/btpu/tests/transfer.rs @@ -0,0 +1,70 @@ +//! The public `transfer` API: window sizes and the window-full error. + +mod common; + +use hardy_btpu::{ + OutOfRange, ParseError, + transfer::{Error, WindowSize}, +}; + +use self::common::window_size as ws; + +#[test] +fn window_size_boundaries() { + // Section 5: 4..=4095 (less than 2^12). + assert_eq!(WindowSize::MIN.get(), 4); + assert_eq!(WindowSize::MAX.get(), 4095); + assert_eq!(WindowSize::try_from(4), Ok(WindowSize::MIN)); + assert_eq!(WindowSize::try_from(4095), Ok(WindowSize::MAX)); + for value in [0, 3, 4096] { + assert_eq!(WindowSize::new(value), None); + assert_eq!( + WindowSize::try_from(value), + Err(OutOfRange { + name: "window size", + value: u64::from(value), + min: 4, + max: Some(4095), + }) + ); + } +} + +#[test] +fn window_size_default_is_recommended() { + assert_eq!(WindowSize::default(), WindowSize::DEFAULT); + assert_eq!(WindowSize::DEFAULT.get(), 16); + assert_eq!(WindowSize::DEFAULT.to_string(), "16"); +} + +#[test] +fn window_size_parses_and_formats_as_its_integer() { + assert_eq!("16".parse(), Ok(WindowSize::DEFAULT)); + assert_eq!( + "4096".parse::(), + Err(ParseError::OutOfRange( + WindowSize::try_from(4096).unwrap_err() + )) + ); + // Too wide for the u16 it parses into: a syntax error, not a range one. + assert_eq!( + "65536".parse::(), + Err(ParseError::Syntax { + name: "window size", + source: "65536".parse::().unwrap_err(), + }) + ); + let w = WindowSize::MAX; + assert_eq!( + format!("{w:b} {w:o} {w:x} {w:X}"), + "111111111111 7777 fff FFF" + ); +} + +#[test] +fn window_full_names_the_window_size() { + assert_eq!( + Error::WindowFull { window_size: ws(4) }.to_string(), + "Transfer window full (size 4)" + ); +} diff --git a/btpu/tests/tunnel.rs b/btpu/tests/tunnel.rs new file mode 100644 index 000000000..a759ad0ff --- /dev/null +++ b/btpu/tests/tunnel.rs @@ -0,0 +1,235 @@ +//! A packet tunnel over a datagram link: the crate carrying payloads that +//! are not bundles, as an IP-over-UDP tunnel would. Inner packets range +//! from small to jumbo, so some share a datagram and some are segmented +//! across several. + +mod common; + +use std::{ + collections::BTreeSet, + net::{Ipv4Addr, UdpSocket}, + num::NonZeroUsize, + time::Duration, +}; + +use bytes::Bytes; +use hardy_btpu::{ + receiver::{MaxRetainedBytes, Receiver, ReceiverConfig, ReceiverEvent}, + sender::{BundleFraming, LinkFraming, Pdu, SendId, SendKind, SendOptions, Sender}, +}; + +use common::{max_transfer_size, receiver_config, sender_config}; + +/// The UDP payload of a 1500-byte IPv4 path. +const DATAGRAM: usize = 1472; + +const WINDOW: u16 = 16; + +/// The largest inner packet. +const MAX_PACKET: usize = 9000; + +/// Inner packet sizes, cycled: an ACK-sized packet, the IPv4 minimum +/// reassembly size, a full Ethernet MTU, a jumbo frame, a tiny packet, and +/// the IPv6 minimum MTU. The MTU and jumbo packets do not fit a datagram. +const SIZES: [usize; 6] = [40, 576, 1500, MAX_PACKET, 64, 1280]; + +/// Packets enqueued before each drain, so small ones share datagrams. +const BURST: usize = 6; + +fn tunnel_sender() -> Sender { + let mut config = sender_config(DATAGRAM, WINDOW); + config.link_framing = LinkFraming::variable(BundleFraming::Message); + Sender::new(config, 0) +} + +/// A receiver that can hold a full window of damaged transfers. The +/// default retention limit holds one transfer's worth, so on a lossy link +/// a second damaged jumbo packet would be refused as `ReceiverFull` until +/// expiry freed the first; with a small cap, budgeting for the window costs +/// little (432 KB here). +fn tunnel_receiver() -> Receiver { + let transfers = NonZeroUsize::new(usize::from(WINDOW)).unwrap(); + Receiver::new(ReceiverConfig { + max_retained_bytes: Some(MaxRetainedBytes::for_transfers( + transfers, + max_transfer_size(MAX_PACKET), + None, + )), + ..receiver_config(WINDOW, MAX_PACKET) + }) +} + +/// Packet `i`, `len` bytes, its content distinct from every other packet's. +fn packet(i: usize, len: usize) -> Bytes { + (0..len) + .map(|j| (i.wrapping_mul(31).wrapping_add(j)) as u8) + .collect::>() + .into() +} + +fn packets(n: usize) -> Vec { + (0..n).map(|i| packet(i, SIZES[i % SIZES.len()])).collect() +} + +/// Carries `packets` through `sender` in bursts, handing every PDU to +/// `link`, and returns the PDUs with the packet index each one carries +/// bytes of. +fn send_all( + sender: &mut Sender, + packets: &[Bytes], + mut link: impl FnMut(&Pdu), +) -> Vec<(Pdu, Vec)> { + let mut sent = Vec::new(); + for (burst, chunk) in packets.chunks(BURST).enumerate() { + let ids: Vec = chunk + .iter() + .map(|p| sender.enqueue(p.clone(), SendOptions::default()).unwrap()) + .collect(); + while let Some(pdu) = sender.next_pdu() { + assert!(pdu.data.len() <= DATAGRAM); + link(&pdu); + let carried = pdu + .carried + .iter() + .map(|c| burst * BURST + ids.iter().position(|id| *id == c.id).unwrap()) + .collect(); + sent.push((pdu, carried)); + } + } + sent +} + +/// What the receiver reported: the delivered payloads in order, and the +/// numbers of the transfers it expired. Any other event fails the test. +#[derive(Debug, Default)] +struct Outcome { + delivered: Vec, + expired: BTreeSet, +} + +impl Outcome { + fn record(&mut self, events: Vec) { + for event in events { + match event { + ReceiverEvent::Received { data, .. } => self.delivered.push(data), + ReceiverEvent::TransferExpired { id } => { + assert!(self.expired.insert(id.transfer_number())); + } + other => panic!("unexpected event {other:?}"), + } + } + } +} + +#[test] +fn lossless_link_delivers_every_packet_in_order() { + let packets = packets(60); + let mut sender = tunnel_sender(); + let mut receiver = tunnel_receiver(); + let mut outcome = Outcome::default(); + + let sent = send_all(&mut sender, &packets, |pdu| { + outcome.record(receiver.receive_pdu(pdu.data.clone())) + }); + + assert_eq!(outcome.delivered, packets); + assert!(outcome.expired.is_empty()); + // Small packets shared datagrams and large ones spanned several. + assert!(sent.iter().any(|(_, carried)| carried.len() > 1)); + assert!(sent.len() > packets.len()); +} + +#[test] +fn lossy_link_delivers_exactly_the_packets_it_did_not_damage() { + // Drop every seventh datagram. + const DROP_EVERY: usize = 7; + + let packets = packets(60); + let mut sender = tunnel_sender(); + let mut receiver = tunnel_receiver(); + let mut outcome = Outcome::default(); + + let mut n = 0; + let sent = send_all(&mut sender, &packets, |pdu| { + n += 1; + if n % DROP_EVERY != 0 { + outcome.record(receiver.receive_pdu(pdu.data.clone())); + } + }); + + // A packet is damaged if any datagram carrying its bytes was dropped. + let mut damaged = BTreeSet::new(); + let mut arrived = BTreeSet::new(); + for (i, (_, carried)) in sent.iter().enumerate() { + let target = if (i + 1) % DROP_EVERY == 0 { + &mut damaged + } else { + &mut arrived + }; + target.extend(carried.iter().copied()); + } + let expected: Vec = (0..packets.len()) + .filter(|i| !damaged.contains(i)) + .map(|i| packets[i].clone()) + .collect(); + assert!(!damaged.is_empty()); + assert_eq!(outcome.delivered, expected); + + // A segmented packet that lost some datagrams but not all is held until + // a window's worth of newer transfers expires it; an unsegmented one + // lost whole leaves no trace. Flush with a window of whole transfers. + let flush: Vec = (0..usize::from(WINDOW)) + .map(|i| packet(packets.len() + i, 3000)) + .collect(); + send_all(&mut sender, &flush, |pdu| { + outcome.record(receiver.receive_pdu(pdu.data.clone())) + }); + + // Segmented packets take transfer numbers in the order they were + // enqueued, from the sender's first, 0. + let segmented: BTreeSet = sent + .iter() + .flat_map(|(pdu, carried)| pdu.carried.iter().zip(carried)) + .filter(|(c, _)| c.id.kind() == SendKind::Transfer) + .map(|(_, &i)| i) + .collect(); + let transfer_number = |i: usize| { + segmented + .contains(&i) + .then(|| u32::try_from(segmented.range(..i).count()).unwrap()) + }; + let expected_expired: BTreeSet = damaged + .intersection(&arrived) + .filter_map(|&i| transfer_number(i)) + .collect(); + assert!(!expected_expired.is_empty()); + assert_eq!(outcome.expired, expected_expired); + assert_eq!(&outcome.delivered[expected.len()..], &flush[..]); +} + +#[test] +fn udp_loopback_carries_every_packet() { + let packets = packets(60); + let mut sender = tunnel_sender(); + let mut receiver = tunnel_receiver(); + let mut outcome = Outcome::default(); + + let rx = UdpSocket::bind((Ipv4Addr::LOCALHOST, 0)).unwrap(); + let tx = UdpSocket::bind((Ipv4Addr::LOCALHOST, 0)).unwrap(); + tx.connect(rx.local_addr().unwrap()).unwrap(); + // The timeout only bounds a regression. + rx.set_read_timeout(Some(Duration::from_secs(30))).unwrap(); + + // Lockstep, one datagram in flight, so the socket buffer cannot + // overflow and drop one. + let mut buf = [0; DATAGRAM]; + send_all(&mut sender, &packets, |pdu| { + assert_eq!(tx.send(&pdu.data).unwrap(), pdu.data.len()); + let (len, from) = rx.recv_from(&mut buf).unwrap(); + assert_eq!(from, tx.local_addr().unwrap()); + outcome.record(receiver.receive_pdu(Bytes::copy_from_slice(&buf[..len]))); + }); + + assert_eq!(outcome.delivered, packets); + assert!(outcome.expired.is_empty()); +} diff --git a/docs/test_coverage_report.md b/docs/test_coverage_report.md index 89ade950c..ce3d5372b 100644 --- a/docs/test_coverage_report.md +++ b/docs/test_coverage_report.md @@ -22,7 +22,7 @@ This report summarizes the test planning and execution status for the Hardy proj ## 2. Coverage Summary -The full test plan inventory (32 plans across Unit, Component, Integration, Fuzz, and System levels) is maintained in the [Test Strategy](test_strategy.md) §2. All plans are Complete. +The full test plan inventory (34 plans across Unit, Component, Integration, Fuzz, and System levels) is maintained in the [Test Strategy](test_strategy.md) §2. All plans are Complete. Current line and fuzz coverage figures are generated by `scripts/run_lcov.sh` into [`coverage_summary.md`](coverage_summary.md) — the single source of truth — and published by CI/CFLite. They are no longer hand-maintained here; the table below links the per-crate reports and tracks plan-scenario coverage. @@ -37,6 +37,7 @@ Current line and fuzz coverage figures are generated by `scripts/run_lcov.sh` in | **proto** | [`test_coverage_report.md`](../proto/docs/test_coverage_report.md) | 31/31 (100%) | Generic monomorphisation depresses the headline line figure | | **otel** | [`test_coverage_report.md`](../otel/docs/test_coverage_report.md) | 26/26 (100%) | — | | **tcpclv4** | [`test_coverage_report.md`](../tcpclv4/docs/test_coverage_report.md) | 10/10 (100%) | Low unit coverage; exercised by fuzz + interop | +| **btpu** | [`test_coverage_report.md`](../btpu/docs/test_coverage_report.md) | 80/80 (100%) | — | | **tvr** | [`test_coverage_report.md`](../tvr/docs/test_coverage_report.md) | 137/137 (100%) | — | | **async** | [`test_coverage_report.md`](../async/docs/test_coverage_report.md) | Not yet measured | — | | **ipn-legacy-filter** | [`test_coverage_report.md`](../ipn-legacy-filter/docs/test_coverage_report.md) | 7/7 (100%) | — | @@ -56,8 +57,8 @@ Current line and fuzz coverage figures are generated by `scripts/run_lcov.sh` in | :--- | :--- | | Workspace crates | 33 | | `#[test]` functions | ~315 | -| Fuzz targets | 8 (cbor: 1, bpv7: 3, eid-patterns: 1, bpa: 1, tcpclv4: 2) | -| Test plan documents | 32 (all present) | +| Fuzz targets | 10 (cbor: 1, bpv7: 3, eid-patterns: 1, bpa: 1, tcpclv4: 2, btpu: 2) | +| Test plan documents | 34 (all present) | | PICS items mapped to tests | 49 (16 fully tested, 14 planned, 15 N/A or not implemented) | | Interop peers | 7 passing (dtn7-rs, HDTN, DTNME, ud3tn, ION, ESA-BP, cFS) | diff --git a/docs/test_strategy.md b/docs/test_strategy.md index d02b192a4..2f3219d0e 100644 --- a/docs/test_strategy.md +++ b/docs/test_strategy.md @@ -35,6 +35,8 @@ This Strategy is the parent document. Verification is executed according to the | **OpenTelemetry** | Component | [`COMP-OTEL-01`](../otel/docs/component_test_plan.md) | OTLP export verification (traces, metrics, logs). | | **TCPCLv4** | Component | [`PLAN-TCPCL-01`](../tcpclv4/docs/component_test_plan.md) | Session state machine via `duplex` harness. | | **TCPCLv4** | Fuzz | [`FUZZ-TCPCL-01`](../tcpclv4/docs/fuzz_test_plan.md) | Protocol stream parsing and state machine robustness. | +| **BTP-U** | Unit | [`UTP-BTPU-01`](../btpu/docs/unit_test_plan.md) | Draft-04 framing, segmentation, reassembly, window, and receiver limits. | +| **BTP-U** | Fuzz | [`FUZZ-BTPU-01`](../btpu/docs/fuzz_test_plan.md) | PDU decoding and receiver state robustness. | | **TCPCLv4 Server** | System | [`PLAN-TCPCL-SERVER-01`](../tcpclv4-server/docs/test_plan.md) | Application lifecycle, config, packaging. | | **CLA Trait** | Integration | [`PLAN-CLA-01`](../bpa/docs/cla_integration_test_plan.md) | Generic Convergence Layer Trait verification. | | **Service Trait** | Integration | [`PLAN-SVC-01`](../bpa/docs/service_integration_test_plan.md) | Generic Application Service Trait verification. | @@ -79,7 +81,7 @@ This Strategy is the parent document. Verification is executed according to the * **Scope:** Parsers (CBOR, Bundle, EID string/CBOR, EID patterns), protocol streams (TCPCLv4 passive/active), and the BPA async pipeline. * **Goal:** Identify panics, memory safety issues, and deadlocks from adversarial input. * **Methodology:** Dedicated fuzz plans per target using `cargo fuzz` (libFuzzer). Executed continuously in CI via **ClusterFuzzLite** (the OSS-Fuzz engine hosted in GitHub Actions): per-PR fuzzing of changed code and a nightly batch run, both reporting minimised, replayable crash reproducers. Coverage measured separately from unit tests. -* **Targets:** 8 fuzz binaries across 5 crates (cbor, bpv7, eid-patterns, bpa, tcpclv4). +* **Targets:** 10 fuzz binaries across 6 crates (cbor, bpv7, eid-patterns, bpa, tcpclv4, btpu). ### 3.4 System & Interoperability Testing @@ -132,7 +134,7 @@ Each peer implementation runs in its own Docker container alongside a Hardy node | Risk | Impact | Mitigation | | ----- | ----- | ----- | | **Protocol Non-Compliance** | Interop failure with other BPv7 implementations. | Interoperability verified against 7 implementations ([`PLAN-INTEROP-01`](../tests/interop/docs/test_plan.md)). | -| **Parser Panics** | DoS vulnerability in production. | Continuous CI fuzz testing on all public-facing parsers (8 targets across 5 crates) via ClusterFuzzLite. | +| **Parser Panics** | DoS vulnerability in production. | Continuous CI fuzz testing on all public-facing parsers (10 targets across 6 crates) via ClusterFuzzLite. | | **Key Wrapping Failures** | Data loss or security breach. | Unit tests for RFC 9173 Key Wrapping (AES-KW, HMAC-SHA2). | | **Async Deadlocks** | Router hangs under load. | BPA pipeline fuzz target exercises concurrent message processing. | | **Storage Corruption** | Data loss after crash or restart. | Storage harness tests recovery and restart across all backends. | diff --git a/scripts/run_lcov.sh b/scripts/run_lcov.sh index 1efdeef29..ff07215a1 100755 --- a/scripts/run_lcov.sh +++ b/scripts/run_lcov.sh @@ -105,6 +105,7 @@ UNIT_CRATES=( hardy-bpa hardy-proto hardy-tcpclv4 + hardy-btpu hardy-otel hardy-async hardy-ipn-legacy-filter