[MarcusKainth]

What this changes, and why

Part of #1390.

This adds an allocation snapshot for the Kafka sink's encode path, so a change to what an encode allocates in steady state fails CI with the case and both figures. Each case is counted after one warm pass, so an allocation an encoder makes only on its first record is outside the snapshot.

The first commit moves the rig and encode_run out of encode_gungraun.rs into benches/support/encode_rig.rs, unchanged, so the counted tier and benches/encode_alloc.rs drive one region. The second adds the target and its files. The four accepting cases allocate nothing in steady state, and stamped_over_limit pins the error each rejected record builds. The x86_64 and aarch64 files are identical apart from the arch line.

Implemented by a Claude Opus coding agent under maintainer direction, and reviewed by Claude Opus agents.

Changelog: none

Invariants

None.

Semver

• Additive — nothing existing changes

Checks

• cargo xtask ci
• Title is area: description for one area, no AI attribution trailers
• A line reading Changelog: none in this body

Anything else

The files were written with cargo xtask bench alloc --bless in Linux containers on a darwin host, arm64 natively and amd64 under emulation, and a second run against them exits 0. The CI legs compare against them.

cargo xtask mutants --since origin/main generates no mutant: the diff is bench targets and committed files.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[spate-review]

Review of fa2734f2f87a87020463653c449544d39ea00964, round 1.

Must-fix: 0. Notes: 3. Outside this change: 1.

Notes

• crates/spate-kafka/benches/encode_alloc.rs:28 (intent): an encoder that reserves an extra 1 MiB on its first record leaves every figure in the snapshot unchanged, so the body's "any change to what an encode allocates fails CI" does not cover an allocation made once per encoder. Each shard clones its own encoder with fresh scratch (crates/spate-kafka/src/sink/encoder.rs:195-205), so a running sink pays such an allocation once per shard. Trace: crates/spate-kafka/benches/encode_alloc.rs:28 → test-support/src/alloc.rs:179 (Report::case) → test-support/src/alloc.rs:138 (uncounted pass) → crates/spate-kafka/benches/support/encode_rig.rs:47 (encode_run reuses one KafkaEncoder) → crates/spate-kafka/src/sink/encoder.rs:74 (KafkaMessage::set_payload). Proof: with the mutant applied, cargo bench -p spate-kafka --bench encode_alloc --locked prints rows identical to crates/spate-kafka/benches/alloc/encode_alloc.default.aarch64.tsv (the comparison exits 1, "snapshot unchanged"). The warm pass is recorded as a choice at test-support/src/alloc.rs:130-131 and crates/spate-kafka/benches/encode_alloc.rs:8-9; the note is against the body's wording.
• crates/spate-kafka/benches/support/encode_rig.rs:70 (docs): the moved rustdoc on records says the drop happens outside the collected region because "the rig is returned rather than dropped". That reason holds for the counted tier only. In encode_alloc.rs each rig is a local that the case closure borrows (crates/spate-kafka/benches/encode_alloc.rs:28-29), and it drops when the main_with closure ends. The conclusion holds for both includers. Proof: contradiction between crates/spate-kafka/benches/support/encode_rig.rs:70 ("the rig is returned rather than dropped") and crates/spate-kafka/benches/encode_alloc.rs:28 (let mut r = bytes_keyless();, borrowed by a closure that returns ()).
• crates/spate-kafka/benches/support/messages.rs:25 (callers): the dead_code rationale says "The bench and the fixture test each compile this module separately", and this change adds a third includer at crates/spate-kafka/benches/encode_alloc.rs:15. The module header at crates/spate-kafka/benches/support/messages.rs:1-2 ("for the message-encode bench") has the same single-bench framing. Proof: git grep -n 'support/messages.rs' fa2734f2 -- crates/spate-kafka exits 0 and lists three #[path] includers (encode_alloc.rs:15, encode_gungraun.rs:72, tests/bench_fixtures.rs:25). crates/spate-json/benches/support/decode_rig.rs:20 words the same rationale as "Each includer compiles this module separately", which stays true as includers are added.

Evidence for the notes

The intent proof ran on darwin aarch64, where the bench at the head prints rows byte-identical to encode_alloc.default.aarch64.tsv. The comparison is the bench's stdout against the committed file with its four header lines removed: exit 1 when they match, 0 when a figure moves.

One-time reservation. Mutant:

--- a/crates/spate-kafka/src/sink/encoder.rs
+++ b/crates/spate-kafka/src/sink/encoder.rs
@@ -73,6 +73,9 @@ impl KafkaMessage {
     /// Set the payload by copying `payload` in.
     pub fn set_payload(&mut self, payload: &[u8]) {
         self.tombstone = false;
+        if self.payload.capacity() == 0 {
+            self.payload.reserve(1 << 20);
+        }
         self.payload.clear();
         self.payload.extend_from_slice(payload);
     }

With it applied the bench prints the committed figures:

bytes_keyed	0	0	0
bytes_keyless	0	0	0
json_typed	0	0	0
stamped_headers	0	0	0
stamped_over_limit	10000	0	3380000

A per-record key.to_vec() in set_key moves bytes_keyed to 10000 0 160000, so a per-record allocation is caught. No remedy was tested. Rewording the body's claim to "any change to what an encode allocates in steady state" would match the recorded warm pass.

records rustdoc. One wording that holds for both includers: "the counted tier returns the rig rather than dropping it, and the snapshot holds it outside its window, so the drop happens at teardown".

messages.rs includers. Rewording line 25 to "Each includer compiles this module separately and uses a different subset of it" matches the spate-json rig. The module-wide allow it defends is still the right shape, since encode_alloc.rs uses only RECORDS directly.

Outside this change

• DEVELOPING.md:296 (docs). Proposed as an issue: docs: name the allocation snapshot file after the bench stem. The Allocation snapshots section binds <name> by "benches/<name>_alloc.rs" at DEVELOPING.md:293 and names the committed file benches/alloc/<name>.<arm>.<arch>.tsv, which gives encode.default.x86_64.tsv for this target. The driver builds the path from the bench file stem with its _alloc suffix (xtask/src/checks/alloc_snapshot.rs:75, bench: stem.to_owned() at :91), so it reads and writes encode_alloc.default.x86_64.tsv, as the committed decode_alloc files in spate-json also show. Proof: test -f crates/spate-kafka/benches/alloc/encode.default.x86_64.tsv exits 1 at the head. --bless writes the right name, so only someone reading or creating a file by hand is misled. The page predates this pull request.

Dimensions

Ran: performance, tests, docs, record, build, intent, callers.

Skipped:

• api: the diff adds no pub item reachable from a crate root, no config key, builder, feature or trait; the new items are pub(crate) inside bench targets.
• delivery: no source, feed, sink runtime, tracker, ack, commit or shutdown code changes.
• liveness: no threads, channels, locks, .await, poll loops or timers change.
• failure: no error type, classification, retry or timeout changes.
• observability: no metric or log line is added, renamed or changed.
• backends: no code talks to an external system, no client library bump, no container pin or request shape change.
• security: no parser over outside bytes, no config or credential type, no unsafe, TLS, auth, listener, Cargo.lock or workflow change.
• website: no site component, style, config, mdx component or blog change.

[MarcusKainth]

Thanks for the review. All three notes are fixed in 5060273.

• First-record allocations: the warm pass is deliberate, since each case pins steady state. I've corrected the body, which now says a change to what an encode allocates in steady state fails CI, and that an allocation an encoder makes only on its first record is outside the snapshot.
• records rustdoc in encode_rig.rs: it now says the rig outlives the measured closure, which holds for both encode_gungraun.rs and encode_alloc.rs.
• messages.rs: the dead_code rationale now opens "Each includer compiles this module separately", matching the spate-json rig, and the header says "benches". I'd keep the module-wide allow.

The outside item, the snapshot file name in DEVELOPING.md, is fixed on #1426.