Skip to content

perf(aead-frame): write the storage profile three times faster and read it a third faster, with every blob byte-for-byte the same - #559

Merged
coderdan merged 25 commits into
mainfrom
perf/storage-writer-key-check
Nov 2, 2026
Merged

coderdan merged 25 commits into
mainfrom
perf/storage-writer-key-check

Conversation

@coderdan

@coderdan coderdan commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Closes #563. Stacked on #534.

Summary

Seven changes to where the storage and transport profiles spend their time, each measured on its own before it was kept. Together they make a 1,000-record storage write 3.1x faster and read 1.6x faster. The storage profile's bytes and version 1 rules are untouched: every blob written before and after is the same.

 packages/protected/src/
+├── buffer.rs            # Buffer: a growable byte buffer that wipes as it grows and when dropped, read through risky_ref
 └── locked/mod.rs        # wipe_raw stores 16 bytes at a time; a crate-private wipe_vec wraps it for Buffer
 packages/aead/src/
 ├── frame.rs             # FrameWriter writes into a Buffer; SeenKeys finds a repeated key with no hash set
 └── leaf.rs              # write() is generic over the sealed ByteSink (Vec<u8> or Buffer), kept out of line by measurement
 packages/aead-frame/src/
 ├── storage.rs           # refuses a repeated key as it writes; returns a Buffer; SeenKeys in validate
 └── serde.rs             # returns a Buffer; no allocation per field; classifies a node by its tag

Where the time went, and what each commit does about it:

  1. A storage write read its blob back to find a repeated key, 55% of a 1,000-record write. The writer now compares an object's field names on the stack as it writes them, and checks a map's keys where they lie in the blob when the map ends.
  2. The growth wipe ran a byte at a time. wipe_raw, the locked regions' Kani-proved wipe, stores sixteen bytes at a time; frames and blobs wipe their old allocation through it as they grow.
  3. The drop wipe also ran a byte at a time, through zeroize's Vec<u8> impl. Buffer owns the bytes a frame or blob is built in and wipes them word-wise on drop. to_frame, decrypt_to_frame, serde::to_vec and storage::to_vec return it.
  4. Buffer is a Controlled type, not a Deref to [u8], so its plaintext is read through risky_ref like Locked and Protected. Measured at no cost.
  5. The reader allocated a String per field to hold an error path it almost never placed. A Copy placeholder becomes a path segment only when an error lands on it. The transport writer had the same allocation.
  6. The reader decoded a text value three times: once to validate, once to tell its kind, once to read it. The kind now comes from the tag byte.
  7. Both validators allocated a hash set per object. SeenKeys compares up to sixteen keys inline and spills to a set past that, so a host object with thousands of keys stays linear.

Two inlining experiments (#[inline] on leaf::max_len and on the FrameWriter's small methods) were measured and reverted: the first changed nothing and the second made the storage writer 11 to 21% slower. leaf::write carries #[inline(never)] with its measurement in the doc.

Evidence

All numbers are from cargo bench -p vitaminc-aead-frame --features serde --bench storage on the machine in #534 (Apple M4, macOS 26.5, rustc 1.98.1), on mains power. Each step was measured as an A/B/A/B of two bench binaries built side by side, the commit before it against the commit, three alternating rounds, median of three. Same-binary spread on this machine is 1 to 3%.

Before is #534's tip, after is this branch. Both are mains runs; they are from different sessions, so compare them to within a few percent.

Time to write:

Value Storage before Storage after Transport before Transport after MessagePack
One record of 7 fields 0.50 µs 0.22 µs 0.57 µs 0.34 µs 0.17 µs
1,000 records 473 µs 151 µs 382 µs 226 µs 70.6 µs
Map of 1,000 text keys 59.6 µs 28.0 µs 42.2 µs 38.2 µs 6.9 µs
64 KiB of text 17.7 µs 1.9 µs 16.8 µs 1.9 µs 1.1 µs
64 KiB of bytes 16.7 µs 1.9 µs 17.2 µs 1.9 µs 1.1 µs

Time to read:

Value Storage before Storage after Transport before Transport after MessagePack
One record of 7 fields 0.79 µs 0.48 µs 0.80 µs 0.48 µs 0.19 µs
1,000 records 815 µs 525 µs 834 µs 497 µs 205 µs
Map of 1,000 text keys 137 µs 137 µs 144 µs 144 µs 92.5 µs
64 KiB of text 4.35 µs 3.29 µs 4.78 µs 3.31 µs 2.16 µs
64 KiB of bytes 1.06 µs 1.06 µs 1.07 µs 1.06 µs 1.02 µs

Against MessagePack, which wipes nothing: a 1,000-record storage write is 2.1x (was 6.7x), a 64 KiB write 1.7x (was 16x), a 1,000-record read 2.6x (was 4.0x), a 64 KiB text read 1.5x (the second UTF-8 pass, which the two-pass design keeps), a 64 KiB bytes read 1.0x.

What each step moved, on the 1,000-record bench unless noted:

Step Commit Storage write Transport write Storage read Transport read
1. Repeat-key check as it writes 16e1752 −47% (map −29%) ±2% ±2%
2. Word-wise growth wipe d8630c7 −17% −10%
3. Buffer f5ef70f −25% (64 KiB −89%) −10% (64 KiB −89%)
4. Buffer is Controlled f9a01b0 ±3% ±3%
5. No path allocation per field d395fe2 ±2% −37% −9% (record −18%) −10%
6. Kind from the tag 47041b8 −14% (64 KiB text −26%) −14%
7. SeenKeys in both validators 6a8791d ±2% ±2% −18% −23%

Step 3's A/B also found that distinct, the map's key check from step 1, wiped 17 KB of untouched spare capacity a byte at a time on every map write; its scratch is a Buffer now.

One behaviour change in the writer: MapWriter::end now treats a malformed key identity as unreachable! where the old read-back path returned an Unwritable error. MapKey::identity cannot fail, so no input reaches it; a writer bug there would now panic rather than error.

A review of the branch (two reviewers, standards and spec, posted below) led to nine follow-up commits: the Kani unwind bound for the widened wipe_raw harness, tests for the mutants CI found surviving (a Buffer's drop wipes its allocation, risky_inner_mut reaches its bytes, a null, bool or float read where a string, bytes or struct is expected is refused by its kind), STORAGE.md and ADR 0003 amended for the Buffer return, wipe_vec made crate-private, Buffer::clear and capacity dropped, and the scalar writer's output trait sealed and named ByteSink. Unifying the writer's FieldNames with SeenKeys was tried and reverted: the map write slowed 3% in an A/B, and the two differ in what they hold.

A zeroize audit of the branch (two comments below: source, LLVM IR at O0 to O3 and assembly on AArch64; then x86-64 and fat LTO) found every wipe intact on every target and five open items, now five more commits: Buffer's doc no longer claims more than its own methods deliver (the Controlled methods that lend the Vec can grow it unwiped, which #496's risky_update renames narrow further); tests that growing a Buffer by push or extend_from_slice frees the old block wiped; the Vec<u8> leaf sink checks in debug builds that it was sized with max_len, and the leaf doc examples use presized wiping sinks; STORAGE.md records that the wipes run on drop, so panic=abort leaves a partly written blob unwiped; and reserve tells the compiler it made room, so Vec's own reallocating growth is compiled out of every Buffer push path (storage writer unchanged, transport map write about 6% slower, accepted).

Checks on every commit: cargo fmt --check, workspace clippy with -D warnings on all targets, the tests of vitaminc-protected (the buffer and wipe tests also under Miri in both models), vitaminc-aead, vitaminc-aead-frame and vitaminc-aead-napi with the node addon, cargo +1.88.0 check as CI's MSRV job runs it, and docs with -D warnings. After the follow-ups: cargo kani -p vitaminc-protected verifies all 4 harnesses, and cargo-mutants on buffer.rs (30 caught of 30) and on found in serde.rs (11 caught, 1 unviable). storage::to_vec keeps a debug_assert_eq! that validates its blob, so the tests still check that the writer and reader agree.

Merge Danger

Door: two-way for the code, one-way for the API

Nothing on disk changes: the bytes every writer produces are byte-for-byte what they were, and the storage profile's conformance corpus and its Kani proofs pass unchanged. The wipe_raw proof in vitaminc-protected was widened to a 64-byte buffer so the word-wise run is covered, and passes. The code can be reverted commit by commit; each stands alone.

Three commits are breaking (!), for a major bump through release-plz:

  • to_frame, decrypt_to_frame, serde::to_vec and storage::to_vec return Buffer where they returned Protected<Vec<u8>>. A caller that read the bytes through risky_ref gets a &Vec<u8> as before; one that wants the Vec calls risky_unwrap.
  • FrameWriter::new and at_depth take &mut Buffer, not &mut Vec<u8>.
  • leaf::write is generic over a new sealed ByteSink trait, implemented for Vec<u8> and Buffer.

New public items: vitaminc_protected::Buffer and vitaminc_aead::frame::SeenKeys, the last because the storage validator in another crate needs it.

Blast Radius: callers of the four writers

Those callers change one line. The aead-napi addon is updated in this branch. The Go reader and writer are untouched. The Performance tables in packages/aead-frame/README.md predate this branch and say so; they are to be re-measured before the release.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-28T00:13:19.494905Z 6a8791d PR opened
🔒 Security Review ✅ Completed 2026-10-28T00:16:14.613203Z 6a8791d PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

github-actions Bot commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

✅ No CRAP threshold violations

1520 function(s) analyzed · threshold 30

@github-actions

github-actions Bot commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

🧬 Mutation testing (cargo-mutants, --in-diff)

caught missed unviable timeout
95 0 31 0

✅ Every mutant in the changed lines was caught by a test.

@coderdan

Copy link
Copy Markdown
Contributor Author

Code review: standards and spec

Two-axis review of c2664d7..6a8791d, run as two separate reviewers so neither masks the other. Line numbers are at 6a8791d.

Standards

Documented-standard breaches

  1. ADR 0003 and STORAGE.md say to_vec returns the blob Protected. It now returns Buffer (packages/aead-frame/src/storage.rs:452), and packages/aead-frame/STORAGE.md:18,391,412 and docs/adr/0003-*.md:47 still say Protected. STORAGE.md ships as rustdoc through include_str!, so the published docs are wrong. The bytes are unchanged, so the frozen-byte rule holds, but the ADR text needs amending with the change.
  2. Commit scope. f5ef70f3 perf(protected)! touches 20 files in 4 crates and breaks vitaminc_aead::{to_frame, decrypt_to_frame, FrameWriter::new}, both to_vecs in aead-frame, and the napi to_frame; git-cliff will file the breaking change under protected only. d8630c71 perf(protected) and 6a8791d9 perf(aead) likewise edit other crates.
  3. Plain-prose docs. Stale text at packages/aead-frame/tests/serde.rs:15 ("take the frame out of its Protected"); oddly re-wrapped doc lines at packages/aead/src/frame.rs:338-339, packages/aead-frame/src/storage.rs:447-451 and packages/aead-frame/src/serde.rs:1087; packages/aead-frame/README.md:179 leaves a dated promise ("will be re-measured before this is released") in shipped docs.

Smells (judgement calls)

  • Duplicated code: SeenKeys (packages/aead/src/frame.rs:139-175) and the writer's FieldNames (packages/aead-frame/src/storage.rs:587-621) are the same inline-16-then-spill shape. FieldNames could be SeenKeys<'static>.
  • Minimal: pub use locked::wipe_vec (packages/protected/src/lib.rs:45) has no caller outside Buffer; pub(crate) suffices.
  • Naming: public trait Out (packages/aead/src/leaf.rs:88); ByteSink or WriteBytes would say what it is.
  • Repeated switch: found() (packages/aead-frame/src/serde.rs:522-543) re-enumerates every tag constant that leaf already classifies; a tags::kind_of(tag) would serve both.
  • Consistent: packages/aead/src/shape.rs:193-194 converts Buffer back to Protected<Vec<u8>> because PassthroughFrame keeps the old type, so two owning wiped types carry the same plaintext in one crate.

Correctness

  • The Kani proof fails. packages/protected/src/locked/proofs.rs:65 keeps #[kani::unwind(33)] while the harness's check loop now runs 64 iterations over a 64-byte buffer. The bound needs to be 65. kani.yml runs cargo kani -p vitaminc-protected on any change under packages/protected, and the job on this PR is red.
  • wipe_raw is sound: align_offset(16).min(len) handles the unalignable case, 16-byte alignment satisfies u128 everywhere, and head plus words plus tail equals len.
  • Buffer::grow saturates to usize::MAX and then Vec::with_capacity panics "capacity overflow", the same as Vec. overwrite panics only on a caller-computed bad range, and says so.
  • MapWriter::end's unreachable! (storage.rs:425) is genuinely unreachable: MapKey::identity never errors.
  • No panic reachable from blob input was found.

Spec

Checked against PR #534's findings and the agreed design (frozen bytes, Buffer as a Controlled type, two-pass reader kept, three reader steps). Tests run: aead-frame storage unit tests and tests/serde.rs (42), protected buffer and wipe tests (16), all pass. Not run: Miri, clippy, MSRV, Kani.

Missing or partial

  • The PR body says the Kani proofs "pass unchanged". The wipe_raw harness was widened (proofs.rs:64-76, buffer 32 to 64, len up to 56), which is right for the word run, and it does not currently pass (above). The body needs correcting.
  • The per-commit gates are claimed in the body, not shown.

Not asked for

  • wipe_vec exported from the crate root with only in-crate users; the agreed design made Buffer the one door.
  • leaf::Out is a new public, unsealed trait. Needed because leaf::write is public and called from aead-frame with a Buffer, but nothing asked for a third-party-implementable sink; consider sealing it.
  • Buffer::clear, with_capacity, capacity go past the agreed "reads through risky_ref plus a narrow overwrite". with_capacity is used by distinct.
  • distinct sorts its scratch through risky_inner_mut (storage.rs:420), a mutable door the agreed design did not name, though it is a Controlled method.
  • .clone() to .to_vec() churn in timestamp.rs, conformance.rs, legacy_fixture.rs and napi: mechanical, fine.

Implemented and checked

  • Same refusals as before: object keys go only through Container::field with FieldNames; map keys record their byte range and run the unchanged distinct at end. Identity::Node still compares tag and bytes, so an INT8 and a UINT8 key with the same byte stay distinct (test at storage.rs:1201), and maps are not capped at 16. The debug_assert_eq!(validate(..), Ok(())) is at storage.rs:464-468.
  • One behaviour change to call out: MapWriter::end turns Invalid::Malformed into unreachable! (storage.rs:986), where the old refused_own returned an Unwritable error in release. A writer bug now panics instead of erroring. Probably intended; the PR body should say so.
  • The storage reader validates first (storage.rs:240), the transport reader still parses first (serde.rs:344), and UTF-8 is checked on the second pass (serde.rs:1147). found matches the old decode for every tag traced, including TYPED_NULL and LOGICAL_LONG as malformed; next_is_null matches is_plain.
  • SeenKeys is used in both validators (frame.rs:192, storage.rs:341); the test covers before and after the spill.
  • Buffer has no Deref, DerefMut, AsRef, From, Clone or PartialEq, and Debug is redacted.
  • Commits are conventional and scoped, ! on the three breaking ones, no CHANGELOG or Cargo.toml in the diff.
  • Nit: the to_vec doc at storage.rs:438-442 and the to_frame doc at frame.rs:338 have broken line wrapping.

Summary. Standards: 3 breaches, 5 smells, 1 correctness defect; worst is the Kani unwind bound. Spec: 1 partial, 4 scope items, 1 unflagged behaviour change; worst is the body's "proofs pass unchanged" claim.

@coderdan

Copy link
Copy Markdown
Contributor Author

Zeroize audit of this branch (c2664d7...1a5e35c)

Run with the zeroize-audit skill over the branch's diff across vitaminc-protected, vitaminc-aead and vitaminc-aead-frame: source analysis, LLVM IR at O0 to O3, release assembly (AArch64, read by hand), five hand-written proof-of-concept programs run under a watching allocator and independently verified. Headline: every wipe this branch added or changed survives optimisation; one medium finding (ZA-0001) is a documentation guarantee that the Controlled trait's non-risky &mut Vec doors break, which #496's risky_update renames narrow. Fixes for the open items follow in this PR.


Run ID: 20261028115220
Timestamp: 2026-10-28T12:28:10+11:00 (run started 2026-10-28T11:52:35+11:00)
Repository: /Users/dan/Projects/CipherStash/vitaminc
Branch: perf/storage-writer-key-check at HEAD 1a5e35c5
Diff audited: git diff c2664d76...HEAD (25 files, 969 insertions, 440 deletions)
Crates: vitaminc-protected, vitaminc-aead, vitaminc-aead-frame (built with --features serde); aead-napi was read as context only (a type change in the diff)
Compile DB: none (Rust only; Cargo.toml at the repository root)
Toolchain: cargo +nightly 1.98.0-nightly (fbb61be30, 2026-05-26) for the emit scripts and rustdoc JSON; rustc/cargo 1.98.1 for the PoCs; host aarch64-apple-darwin; uv 0.11.12

Configuration:

Setting Value
Optimization levels O0, O1, O2, O3 (IR); O0 and O2 (MIR); O2 and O3 (assembly)
MCP mode off
MCP available no
Assembly analysis enabled (AArch64, experimental, every claim read by hand)
Semantic IR analysis enabled
CFG analysis disabled
Runtime tests disabled
PoC validation mandatory; all 5 generated PoCs built and verified
Config files default.yaml merged under rust.yaml (merged-config.yaml)

Executive Summary

Verdict. Every wipe this branch added or changed survives optimization. wipe_raw keeps its volatile stores and its fence at O0 to O3. Buffer::grow allocates a new block, copies, wipes the whole old block and only then frees it, and it never calls realloc. Buffer's Drop and zeroize wipe the full capacity before the free. The error and unwind paths of storage::to_vec, serde::to_vec and distinct all drop the Buffer, which wipes it. One finding is medium: Buffer's documentation says there is no way to grow it that leaves a copy behind, but the Controlled trait's update, update_with, update_with_ref and inner_mut hand out a &mut Vec<u8> under names that do not say "risky", and growing the Vec through them frees the old block with the plaintext still in it. A verified proof of concept shows this (ZA-0001). The other eleven findings are low, and most are the stack and register copies that STORAGE.md already says are not wiped.

Metric Count
Files scanned 10 (files with findings or sensitive objects)
Translation units analyzed 1 (the cargo workspace, run per package)
Sensitive objects 29
Total findings 12

By Severity

Severity Count
Critical 0
High 0
Medium 1
Low 11

By Confidence

Confidence Count
Confirmed 7
Likely 1
Needs review 4

PoC Validation

Metric Count
PoCs generated 5
PoCs validated (verified by the verifier) 5
Exploitable (confirmed) 4 (ZA-0001, ZA-0002, ZA-0003, ZA-0004)
Not exploitable 1 (ZA-0006)
Compile failures 0
No PoC generated 7 (ZA-0005, ZA-0007 to ZA-0012)

Two of the four exploitable PoCs (ZA-0002 and ZA-0003) confirm behaviour the code documents on purpose, so they raise confidence in the observation, not the severity.

By Category

Category Count
MISSING_SOURCE_ZEROIZE 0
PARTIAL_WIPE 0
NOT_ON_ALL_PATHS 2
OPTIMIZED_AWAY_ZEROIZE 0
SECRET_COPY 6
INSECURE_HEAP_ALLOC 0
STACK_RETENTION 3
REGISTER_SPILL 1
MISSING_ON_ERROR_PATH 0
NOT_DOMINATING_EXITS 0
LOOP_UNROLLED_INCOMPLETE 0

By Status

Status Findings
Open (needs a decision or a change) ZA-0001, ZA-0004, ZA-0007, ZA-0012
Accepted (documented design or documented accepted category) ZA-0002, ZA-0003, ZA-0005, ZA-0006, ZA-0008, ZA-0009, ZA-0010, ZA-0011

Status is kept in report/finding-metadata.json because findings.json has no slot for it.


Sensitive Objects Inventory

29 objects, SO-5000 to SO-5028, found by reading the diff and its callers. semantic_audit.py matched none of them, because its name heuristics look for Key, Secret, Token and similar and none of these types are named that way. Every object is in the diff. "Has wipe" is the wipe status the source analysis recorded.

ID Name Type Location Confidence Heuristic Has Wipe
SO-5000 Buffer struct Buffer(Vec<u8>) packages/protected/src/buffer.rs:45 high explicit: audit scope names it as holding plaintext; type-level Zeroize + Drop + ZeroizeOn yes
SO-5001 Buffer::grow old allocation Vec<u8> (self.0 before growth) packages/protected/src/buffer.rs:92 high explicit: audit scope names it as holding plaintext yes
SO-5002 Buffer::grow new allocation grown Vec<u8> packages/protected/src/buffer.rs:99 high explicit: audit scope names it as holding plaintext yes
SO-5003 Vec returned by Buffer::risky_unwrap Vec<u8> packages/protected/src/buffer.rs:150 high explicit: audit scope names it as holding plaintext no, handed to caller by design
SO-5004 &mut Vec through Controlled::inner_mut / risky_inner_mut / update / update_with / update_with_ref &mut Vec<u8> packages/protected/src/buffer.rs:142 high explicit: audit scope names it as holding plaintext bypassable (ZA-0001)
SO-5005 wipe_raw / wipe_vec (wipe primitive) unsafe fn wipe_raw(*mut u8, usize); fn wipe_vec(&mut Vec<u8>) packages/protected/src/locked/mod.rs:673 high repo's approved volatile wipe alongside zeroize yes (the wipe primitive)
SO-5006 to_frame / to_frame_at frame Buffer packages/aead/src/frame.rs:353 high explicit: audit scope names it as holding plaintext yes
SO-5007 FrameWriter / Count / FrameMap (writers into &mut Buffer) &'o mut Buffer + offsets packages/aead/src/frame.rs:364 high explicit: audit scope names it as holding plaintext yes (by the owning Buffer)
SO-5008 decrypt_to_frame frame Buffer packages/aead/src/walk.rs:208 high explicit: audit scope names it as holding plaintext yes
SO-5009 passthrough frame Buffer -> Protected<Vec<u8>> -> PassthroughFrame(Vec<u8>) packages/aead/src/shape.rs:196 high explicit contract: PassthroughFrame is never wiped copy not wiped, by design (ZA-0003)
SO-5010 sealed leaf plaintext Protected<Vec<u8>> sized by leaf::max_len packages/aead/src/encrypt_sink.rs:258 high explicit: audit scope names it as holding plaintext yes
SO-5011 typed-null leaf plaintext Protected<Vec<u8>> sized by typed_null_len packages/aead/src/encrypt_sink.rs:279 medium explicit: audit scope names it as holding plaintext yes
SO-5012 zero_leaf scratch Vec<u8> packages/aead/src/leaf.rs:261 high not secret n/a (not secret)
SO-5013 ByteSink / leaf::write sealed trait over Vec<u8> and Buffer packages/aead/src/leaf.rs:94 high explicit: audit scope names it as holding plaintext depends on caller (ZA-0004)
SO-5014 SeenKeys struct { few: [&str;16], len, many: Option<HashSet<&str>> } packages/aead/src/frame.rs:150 high explicit: audit scope names it as holding plaintext no copy made
SO-5015 storage::to_vec blob Buffer packages/aead-frame/src/storage.rs:453 high explicit: audit scope names it as holding plaintext yes
SO-5016 distinct integer-key scratch Buffer::with_capacity(n * 17) packages/aead-frame/src/storage.rs:409 high explicit: audit scope names it as holding plaintext yes
SO-5017 distinct node keys Vec<&[u8]> packages/aead-frame/src/storage.rs:410 high explicit: audit scope names it as holding plaintext no copy made
SO-5018 Identity::Int temporaries and sort swap temporaries (bool, u128), [u8;16], [u8;17] on stack/registers packages/aead-frame/src/storage.rs:413 medium explicit: audit scope names it as holding plaintext no, documented (ZA-0008 to ZA-0011)
SO-5019 MapWriter.keys Vec<Range<usize>> packages/aead-frame/src/storage.rs:694 high not secret n/a (not secret)
SO-5020 FieldNames struct { inline: [&'static str;16], len, spilled: Vec<&'static str> } packages/aead-frame/src/storage.rs:597 high not secret: &'static str from serde's SerializeStruct API n/a (not secret)
SO-5021 storage write_display output bytes formatted into the blob Buffer via Appender packages/aead-frame/src/storage.rs:501 high explicit: audit scope names it as holding plaintext heap yes; fmt stack buffers no, documented
SO-5022 transport serde::to_vec frame Buffer packages/aead-frame/src/serde.rs:1107 high explicit: audit scope names it as holding plaintext yes
SO-5023 to_part / SerVariant.parts Buffer per part; Vec<(Option<&'static str>, Buffer)> packages/aead-frame/src/serde.rs:1448 high explicit: audit scope names it as holding plaintext yes
SO-5024 displayed() shown Buffer packages/aead-frame/src/serde.rs:1139 high explicit: audit scope names it as holding plaintext yes
SO-5025 SerMap.key Option<Zeroizing<String>> packages/aead-frame/src/serde.rs:1375 high explicit: audit scope names it as holding plaintext yes
SO-5026 At enum At { Key(&'static str), Index(usize), Entry(usize) } packages/aead-frame/src/serde.rs:206 high not secret n/a (not secret)
SO-5027 Deserializer::found / next_is_null tag peek packages/aead-frame/src/serde.rs:496 high reader n/a (no copy)
SO-5028 aead-napi JsRef::to_frame / EncryptFrame.frame Buffer / Option<Buffer> packages/aead-napi/src/js_ref.rs:70 medium outside the three audited packages; in the diff yes

Findings

Medium Severity

ZA-0001: SECRET_COPY, medium (confirmed, open)

Location: packages/protected/src/buffer.rs:142 (docs at buffer.rs:19-22); packages/protected/src/controlled.rs:194-254
Object: the &mut Vec<u8> that Buffer hands out through Controlled::inner_mut, update, update_with, update_with_ref and risky_inner_mut (SO-5004). Related: SO-5000.

Evidence:

  • [source] Buffer's documentation says every way to add bytes goes through reserve, "so there is no way to grow it that leaves a copy behind". Controlled's provided methods update, update_with and update_with_ref (controlled.rs:194-254) are not named risky and pass &mut Vec<u8> to a closure, and inner_mut is a required public trait method. A closure that pushes or extends past capacity makes Vec reallocate, which frees the old block with the plaintext still in it. No caller in scope does this: the one in-scope user, storage.rs:420 (distinct), sorts in place without growing. Only risky_inner_mut carries a warning in its name.
  • [poc] Verified exploitable PoC (poc_za_0001, exit 0, release O3): Controlled::update(|v: &mut Vec<u8>| v.push(0)) on a Buffer filled to exactly its 64-byte capacity, using only use vitaminc_protected::{Buffer, Controlled} with no unsafe and no risky-named call on the attack path. A watching allocator compared the exact freed pointer and read the block inside dealloc: 64 of 64 bytes still held the secret. The control, growing the same Buffer past capacity with Buffer::push, freed a block with 0 non-zero bytes. realloc is left at GlobalAlloc's default (allocate, copy, free), which models a moving realloc. If the system allocator grows a block in place there is no freed old block, but the caller cannot control whether libmalloc moves, and a move frees the old block unwiped. The PoC runs update only; update_with, update_with_ref, inner_mut and risky_inner_mut share the same &mut Vec<u8> and were not run separately.
  • [ir, asm] Compiler analysis (see "Verified correct" below) shows that Buffer::grow, Drop and zeroize do wipe the full old block at O0 to O3, so the exposure is only through these &mut Vec<u8> doors, not through Buffer's own methods.

Confidence: one source signal plus a verified exploitable PoC: confirmed (the MCP downgrade to needs_review was applied first, then the PoC lifted it).

Recommended Fix: two options.

  1. Narrow the documentation claim so it covers Buffer's own methods only, and name update, update_with, update_with_ref and inner_mut as able to bypass it.
  2. Give Buffer an Inner type that cannot grow: a newtype that exposes &mut [u8] (derefs to a slice) but not push or extend, keeping risky_unwrap as the one way to get the Vec. This keeps the misuse-resistance promise. The cost is consistency, because Buffer's Inner would then differ from the other Controlled types.

Protected<Vec<u8>> shares the same property through the same trait methods, and does not promise otherwise, so option 1 is already true of it; option 2 would be specific to Buffer. Whichever is chosen, a compile-fail or allocator test for update-driven growth would pin it.


Low Severity, Confirmed

ZA-0002: SECRET_COPY, low (confirmed, accepted)

Location: packages/protected/src/buffer.rs:151
Object: Buffer::risky_unwrap (core::mem::take), the Vec<u8> it returns (SO-5003)

Evidence:

  • [source] risky_unwrap takes the Vec out with mem::take, so the Buffer's Drop wipes an empty Vec and the returned Vec<u8> carries the plaintext unwiped. This is the documented design (buffer.rs:146-149: the Vec is freed unwiped unless its new owner wipes it). The one in-scope non-test caller, shape.rs:196, moves it straight into Protected<Vec<u8>> with no heap copy, and that type's zeroize wipes the full capacity. Matches the dangerous-API hit F-RUST-DANG-0001.
  • [poc] Verified exploitable PoC (poc_za_0002, exit 0): the Vec from risky_unwrap was freed with 48 of 48 secret bytes; drop(Buffer) on an identically filled block freed it wiped. Exit 0 is the documented behaviour, so this confirms the observation, not a defect.

Recommended Fix: None required. Keep callers to the move-into-a-wiping-type pattern. A clippy disallowed-methods entry or a review checklist item for Buffer::risky_unwrap outside wrapping into Protected would make new uses visible.


ZA-0003: SECRET_COPY, low (confirmed, accepted)

Location: packages/aead/src/encrypt_sink.rs:338
Object: passthrough frame (Buffer to Protected<Vec<u8>> to PassthroughFrame(Vec<u8>), SO-5009)

Evidence:

  • [source] The passthrough frame is built in a Buffer, moved into Protected<Vec<u8>> at shape.rs:196, then copied with risky_ref().to_vec() into PassthroughFrame(Vec<u8>), which is never wiped. The Protected original is wiped on drop. aead-napi ciphertext.rs:125 copies a Buffer the same way. PassthroughFrame is non-sensitive by contract (frame.rs:271-276: "neither sealed nor authenticated, and never wiped"), so the unwiped copy is by design. The Protected wrapper and the comment at shape.rs:193-194 ("the frame leaves the buffer's rules here") suggest more protection than the data gets, because the bytes end up in the clear either way.
  • [poc] Verified exploitable PoC (poc_za_0003, exit 0), reached through the public vitaminc_aead::plan: exactly two heap allocations, 64 bytes (the Buffer) and 45 bytes (the 5-byte frame header plus the 40-byte marker). The 64-byte block was freed all zero; the 45-byte block was freed holding the marker. So the only unwiped copy on this path is the PassthroughFrame one.

Recommended Fix: Optional. Build PassthroughFrame by moving the Vec (PassthroughFrame(frame.risky_unwrap())) and have frame_passthrough return the Buffer or Vec, which drops one allocation and one copy. No zeroize change is needed while passthrough stays non-sensitive by contract.


ZA-0004: SECRET_COPY, low (confirmed, open)

Location: packages/aead/src/leaf.rs:113
Object: impl ByteSink for Vec<u8>; pub fn write, write_leaf, write_typed_null (SO-5010, SO-5013)

Evidence:

  • [source] leaf::write is public and generic over the sealed ByteSink, which keeps an impl for plain Vec<u8>; write_leaf and write_typed_null take &mut Vec<u8>. A caller that does not size the Vec with max_len first gets Vec reallocation, which frees blocks holding plaintext unwiped, and the Vec itself is not wiped unless the caller wraps it. The doc examples of leaf::write (leaf.rs:206-214, 238-245) write plaintext into Vec::new(), which is exactly the unsafe shape. In-scope callers are safe: encrypt_sink.rs:258 and 279 presize inside Protected, and max_len bounds every scalar kind (checked by hand: decimal 17, timestamp 13, date 5, envelope plus u128 20, text and bytes 8 plus len, all within max_len). The Vec<u8> signature predates the diff (write_leaf already took &mut Vec); the diff added the Buffer path beside it.
  • [poc] Verified exploitable PoC (poc_za_0004, exit 0): the public leaf::write with Logical::Decimal into Vec::new(). Growth 0 to 8 to 16 to 32 freed a 16-byte block holding the flags and two mantissa words (distinctive words 0xD15EA5E5 and 0xC0FFEE11); exactly 2 frees, 1 holding mantissa words, final capacity 32. The control, Buffer::new(), freed nothing because Buffer allocates 64 bytes up front (it does not by itself show a wiped free; ZA-0001's control covers that). As with ZA-0001, a block grown in place would leave no freed copy, but the caller cannot control that. Only write was exercised, not write_leaf or write_typed_null.

Recommended Fix: Make the Vec<u8> sink crate-private and offer the public functions over Buffer (or &mut Protected<Vec<u8>>), or add a debug_assert that the Vec has max_len spare capacity on entry. In either case change the doc examples to use Buffer instead of Vec::new().


ZA-0008: STACK_RETENTION, low (confirmed, accepted)

Location: packages/aead/src/leaf.rs:137
Object: vitaminc_aead::leaf::write::<Buffer> and ::<Vec<u8>> scalar temporaries (SO-5018)

Evidence:

  • [ir] O3 IR, both monomorphs: %_50 = alloca [4 x i8] (date), %_71 = alloca [8 x i8] and %_75 = alloca [4 x i8] (timestamp seconds and nanos), %_92/%_96/%_101/%_107 = alloca [4 x i8] (decimal flags and mantissa words), %1/%2/%3 = alloca [32 x i8] (Int::to_i128 result). Each is bracketed by llvm.lifetime.start and llvm.lifetime.end, with 0 store volatile in the function.
  • [asm] vitaminc-aead.O3.s, leaf::write<Buffer>, decimal path: sub sp, sp, #112; add x8, sp, #16 / bl ...Int7to_i128; ldp x10, x9, [sp, #32] (mantissa); str w20, [sp, #16] (flags) / add x1, sp, #16 / bl ...ByteSink17extend_from_slice, then the same str w22/w8/w23, [sp, #16] for the mantissa words. Exit: ldp x29, x30, [sp, #96] ... add sp, sp, #112 / ret, with no str xzr or stp xzr, xzr to [sp, #16..#48]. The Vec<u8> instance: sub sp, sp, #96, str w19, [sp] / mov x1, sp, epilogue add sp, sp, #96 / ret. O2 has the same stores.
  • This is the category packages/aead-frame/STORAGE.md accepts ("Copies on the stack and in registers are not wiped: ... a scalar while it is encoded or decoded"). Related: ZA-0005.

Recommended Fix: None needed; accepted category. A complete fix would require wiping compiler-managed temporaries, which Rust source cannot reach.


ZA-0009: STACK_RETENTION, low (confirmed, accepted)

Location: packages/aead-frame/src/storage.rs:408
Object: storage::distinct::<vec::IntoIter<MapKey>>, the decoded integer key (SO-5016, SO-5018)

Evidence:

  • [ir] O3 IR: %_2.i = alloca [64 x i8] (sret of leaf::read, holds the decoded key), %int.i = alloca [32 x i8] filled by llvm.memcpy(... i64 32), %0 = alloca [32 x i8] (sret of Int::sign_and_word); %_8.1.i = load i128 then llvm.lifetime.end on all three; 0 store volatile. The identity word goes straight to the heap scratch: store i128 %_8.1.i, ptr %dst.i.i.i.i, align 1, so no [u8;16] or [u8;17] alloca survives.
  • [asm] vitaminc-aead-frame.O3.s, distinct: sub sp, sp, #304; loop LBB2_4: add x8, sp, #64 / bl ...leaf4read; ldp q0, q1, [x26, #16] / stp q0, q1, [sp, #128]; sub x8, x29, #128 / bl ...Int13sign_and_word; ldurb w28, [x29, #-128] / ldp x22, x21, [x29, #-112]; identity to heap with strb w28, [x8, x20] ... stp x22, x21, [x8]. Exit: bl ...Buffer...Drop4drop (the scratch is wiped), then LBB2_35: ldp x29, x30, [sp, #288] ... add sp, sp, #304 / ret, with no zero stores to sp+64..sp+191. O2 is the same.
  • The last integer key decoded by distinct stays in its 304-byte frame after return. Accepted in STORAGE.md ("an integer map key, decoded ..."). Related: ZA-0005.

Recommended Fix: None needed; accepted category.


ZA-0010: STACK_RETENTION, low (confirmed, accepted)

Location: packages/aead-frame/src/storage.rs:422
Object: core::slice::sort::shared::smallsort::insertion_sort_shift_left::<[u8; 17], lt>, called from distinct's integers.sort_unstable() (SO-5018)

Evidence:

  • [ir] O3 IR: %tmp.i = alloca [17 x i8], align 1; llvm.memcpy(... dereferenceable(17) %tmp.i, ... i64 17) ... llvm.lifetime.end(%tmp.i); no volatile store. The same family (ipnsort, quicksort, heapsort over [u8;17]) behaves alike; quicksort has 8 str q0, [sp...].
  • [asm] vitaminc-aead-frame.O3.s: sub sp, sp, #32; LBB17_8: ldr q0, [x11] / str q0, [sp] / ldrb w12, [x11, #16] / strb w12, [sp, #16]; LBB17_2: ldr q0, [sp] / str q0, [x12]; LBB17_15: add sp, sp, #32 / ret, no clear. A 17-byte integer-key identity is left at [sp, #0..#17].

Recommended Fix: None needed; accepted category (core's sort temporaries cannot be reached from Rust). If tightened, sort indices or (bool, u128) pairs held in the Buffer, which still leaves core's own element temporaries; there is no complete source-level fix.


Low Severity, Likely

ZA-0011: REGISTER_SPILL, low (likely, accepted)

Location: packages/aead/src/leaf.rs:137
Object: secret words held in callee-saved registers across <Buffer as ByteSink>::extend_from_slice and Buffer::grow

Evidence:

  • [asm] leaf::write<Buffer>, decimal path, keeps the mantissa magnitude in x22/x23 (subs x22, x10, x8 / sbc x23, x9, x8) across bl ...Buffer...ByteSink17extend_from_slice, whose prologue is stp x22, x21, [sp, #-48]! (vitaminc-aead.O3.s); its epilogue ldp x22, x21, [sp], #48 leaves the saved copy below sp. distinct keeps the identity word in x21/x22 (ldp x22, x21, [x29, #-112]) across bl ...Buffer4grow, whose prologue is stp x22, x21, [sp, #16] (vitaminc-protected.O3.s).
  • Asm only (one kind of signal), so likely. AArch64 asm analysis is experimental; read by hand. Related: ZA-0005, ZA-0008, ZA-0009.

Recommended Fix: None; accepted category (STORAGE.md: copies in registers are not wiped). No source-level fix reaches callee-save slots.


Low Severity, Needs Review

ZA-0005: SECRET_COPY, low (needs_review, accepted)

Location: packages/aead-frame/src/storage.rs:413
Object: stack and register copies in distinct, leaf::write, write_display, displayed and SeenKeys hashing (SO-5018, SO-5021)

Evidence:

  • [source] Copies on the stack and in registers are not wiped: distinct's Identity::Int (bool, u128), the word.to_le_bytes() temporary and sort_unstable's [u8;17] element temporaries (storage.rs:413-422); scalar to_le_bytes temporaries in leaf::write; core::fmt digit buffers in write_display (storage.rs:501) and displayed (serde.rs:1139); the SipHash state over each key when SeenKeys spills to a HashSet (frame.rs:169). STORAGE.md:421-434 documents these as accepted. The repo does not claim stack data is cleaned automatically; it documents the opposite.
  • [ir, asm] Carried by the linked findings ZA-0008, ZA-0009, ZA-0010 and ZA-0011, not duplicated here, so this finding stays at the analyzer's needs_review.
  • No PoC: a Rust PoC cannot observe its own stack after a call without undefined behaviour, and the skill does not support Rust stack-retention PoCs.

Recommended Fix: No source change; the repo's position is that wiping the one named copy would overclaim.


ZA-0006: SECRET_COPY, low (needs_review, accepted)

Location: packages/aead/src/frame.rs:169
Object: SeenKeys spill to HashSet<&str> (SO-5014)

Evidence:

  • [source] Past 16 keys SeenKeys builds a HashSet<&str>. Its table, freed without a wipe, holds (ptr, len) pairs and hashbrown control bytes, which are 7 bits of a randomly keyed SipHash of each plaintext key. No key bytes are copied; the leak is key lengths and a few keyed-hash bits, in a process-local allocation. The behaviour predates the diff (it used a HashSet for every object); the diff makes objects of 16 keys or fewer allocate nothing.
  • [ir] drop_glue::<SeenKeys> in vitaminc-aead.O3.ll frees one hashbrown block of 16-byte (ptr, len) buckets plus control bytes (HashSet<&str>; SeenKeys::insert takes dereferenceable(312) = [&str;16] 256 + len 8 + Option<HashSet> 48).
  • [poc] Verified not-exploitable PoC (poc_za_0006, exit 1, which here is the expected outcome): 20 distinct keys inserted into the public SeenKeys, one freed block of 552 bytes holding 20 words pointing into the key text, 20 words equal to the key length, hashbrown control bytes and no key bytes. 552 = 32 buckets x 16 + 32 control bytes + 8, which confirms hashbrown's group width of 8 on aarch64 (the compiler notes' formula used +16, the x86-64 SSE2 width; the notes were corrected). In real use the keys point into the decoded frame, so the table holds addresses of plaintext, not the plaintext. Severity stays low and the finding is kept, per the PoC rule.

Recommended Fix: None required. std's HashSet offers no way to wipe its table before it is freed; if the key lengths and hash bits ever matter, a sorted Vec<&str> with binary search would avoid hashing key bytes at all.


ZA-0007: NOT_ON_ALL_PATHS, low (needs_review, open)

Location: packages/aead-frame/src/storage.rs:987
Object: every Buffer wipe, which is a Drop (SO-5000, SO-5006, SO-5015)

Evidence:

  • [source] Every Buffer wipe happens in Drop. Under panic=unwind (the workspace default; no profile sets panic), a panic in MapWriter::end's unreachable! (storage.rs:987), in Buffer::overwrite's bounds check (buffer.rs:127) or in Vec::with_capacity's capacity-overflow panic inside grow (buffer.rs:99) unwinds through to_vec or to_frame, and the Buffer is dropped and wiped. Under panic=abort, which the WASM targets use by default (the Go guest is WASM), no Drop runs: the plaintext stays in linear memory until the host discards the instance.
  • [ir] Harness, O3 release: the Err and unwind paths of storage::to_vec and serde::to_vec invoke <Buffer as Drop>::drop and drop_glue::<Buffer>, so the wipe runs under unwind. This confirms the panic=unwind half only. The panic=abort half cannot be shown in IR and is true by construction.
  • No PoC: category not supported for Rust PoCs.

Recommended Fix: No Rust code change. Note in STORAGE.md (Memory and timing) that wipes are Drop-based and do not run under panic=abort, and that a WASM host should discard an instance after a trap.


ZA-0012: NOT_ON_ALL_PATHS, low (needs_review, open; hardening, not a defect)

Location: packages/protected/src/buffer.rs:84
Object: Buffer::reserve, Buffer::push, Buffer::extend_from_slice and the Vec growth fallback after Buffer::grow (SO-5000)

Evidence:

  • [ir, asm] Buffer::push and extend_from_slice call self.0.push and self.0.extend_from_slice after reserve. LLVM cannot see that Buffer::grow (#[inline(never)]) left enough room, so callers keep Vec's own growth path. In leaf::write<Buffer> at O3, Buffer::grow is followed by RawVec<u8>::grow_one / do_reserve_and_handle; the same shape appears in storage::to_vec, distinct and <Buffer as ByteSink>::extend_from_slice (after bl Buffer4grow: cmp x19, x8 / b.ls / bl do_reserve_and_handle). grow_one reaches __rust_realloc through finish_grow, which would free the old block unwiped.
  • The path is unreachable while grow guarantees capacity >= len + additional (capacity is max(len + additional, 2 * len, 64); overflow panics in with_capacity), so no wipe is missed today. The wipe is on the compiled path by an invariant LLVM cannot see, not by proof the compiler can use. Filed as NOT_ON_ALL_PATHS as the closest category.
  • No PoC: the path is dead by the capacity invariant, so there is nothing to trigger.

Recommended Fix: Hardening. After self.grow(additional) in reserve, add unsafe { core::hint::assert_unchecked(self.0.capacity() - self.0.len() >= additional) } so LLVM drops the Vec growth fallback (a small speed gain too). It needs a SAFETY comment tying it to grow's capacity guarantee.


Verified Correct

These are the wipes the branch added or changed that were checked in IR and assembly and found sound. They matter as much as the findings. All IR below is from vitaminc-protected.O3.ll unless stated; O2 is identical, and O0 and O1 were compared (O0 calls core::ptr::write_volatile, O1 inlines the u128 loop and calls a shared write_volatile for the byte loops, O2 and O3 inline all three loops as store volatile). The emit scripts build the audited crate at the stated level but its dependencies at O0; the out-of-repo harness was a true release build and shows the same code shape.

1. wipe_raw and wipe_vec: volatile stores and fence survive O0 to O3

bb7:  store volatile i8 0, ptr %_0.i11, align 1        ; head loop
bb18: store volatile i128 0, ptr %_0.i, align 16       ; word loop
bb27: store volatile i8 0, ptr %_0.i12, align 1        ; tail loop
bb28: fence syncscope("singlethread") seq_cst
      ret void

There is no llvm.memset in the function. Each loop is rolled (one volatile store per iteration) with trip counts umin(len, align_offset), (len - head) >> 4 and (len - head) & 15, so there is no incomplete unrolling. The semantic IR script found 3 volatile stores each and loop_unrolled_wipes: [] for wipe_raw, grow and Drop::drop at O2 and O3. Assembly (O3, O2 the same): strb wzr, [x13], #1 / stp xzr, xzr, [x12], #16 / strb wzr, [x9], #1 / ;MEMBARRIER / ret. MIR at O0 and O2 shows 3 write_volatile calls and 1 compiler_fence in wipe_raw, and wipe_vec calling wipe_raw. wipe_vec is inlined at O1 and above, and its length is the capacity, not the length: in Buffer::grow the wipe length is %out.val1.i = load i64, ptr %self (offset 0, capacity) and that same field is the __rust_dealloc size, while the copy length is offset 16 (length).

2. Buffer::grow: allocate, copy, wipe the whole old block, then free; no realloc

Order in the IR: __rust_alloc(%cap) then llvm.memcpy(new, old, len) then store i64 0 (length) then the three volatile loops over the old block then fence syncscope("singlethread") seq_cst then __rust_dealloc(old, cap, 1) then the store of the new {cap, ptr, len}. Buffer::grow contains no __rust_realloc, finish_grow, grow_one or do_reserve_and_handle at O2 or O3, and __rust_realloc appears nowhere in vitaminc-protected.O3.s. Assembly: bl __rust_alloc / bl _memcpy / str xzr, [x19, #16] / the strb wzr, stp xzr, xzr, strb wzr loops / ;MEMBARRIER / ldr x1, [x19] / bl __rust_dealloc. The only exit before the copy is alloc::raw_vec::handle_error, which diverges and leaves the old Vec untouched and still owned. (The Vec growth that callers keep compiled after grow is ZA-0012.)

3. Buffer Drop and Zeroize::zeroize: full-capacity wipe before the free

<Buffer as Drop>::drop at O3: store i64 0 (length), the store volatile i8 / i128 / i8 loops, fence syncscope("singlethread") seq_cst, __rust_dealloc(ptr, cap, 1), then store i64 0, ptr %self and store ptr inttoptr (i64 1 to ptr) (an empty Vec::new()). Assembly: the loops, then ;MEMBARRIER, then bl __rust_dealloc. Zeroize::zeroize has the same three volatile stores, fence and dealloc at O2 and O3. The stores go to memory that is freed immediately after, which is the case dead-store elimination would remove if the stores were not volatile; they are, and it does not fire.

4. storage::distinct scratch Buffer: dropped, and so wiped, on every exit

The Ok exit and the DuplicateKey exit both invoke <Buffer as Drop>::drop(%integers) (assembly: bl ...Buffer...Drop4drop at both exits), and the unwind landing pad calls drop_glue::<Buffer>. The identity word is stored straight into the scratch (store i128 %_8.1.i, ptr %dst; assembly stp x22, x21, [x8]), so no [u8;17] copy sits in distinct's frame. The nodes Vec freed without a wipe holds &[u8] slices (16-byte pointer and length pairs into the blob), not bytes. What does stay on the stack is ZA-0009, ZA-0010 and ZA-0011.

5. to_vec error and unwind paths drop the Buffer (harness, O3 release)

storage::to_vec::<BTreeMap<u64, String>> (harness.O3.storage_to_vec_u64.ll): the unwind cleanup: pad runs invoke drop_glue::<Buffer>(%out) then invoke <Buffer as Drop>::drop. The Err branch bb8 moves the error to %_0, then invoke <Buffer as Drop>::drop(ptr %out). The Ok branch bb9 does memcpy(%_0+8, %out, 24), so the Buffer is moved out, not freed. The __rust_dealloc calls that follow Buffer::drop free an already emptied Vec (capacity 0 after zeroize), or sit on the path where Buffer::drop itself unwound, which wipe_raw cannot do anything about.
serde::to_vec::<BTreeMap<String, u64>> (harness.O3.serde_to_vec.ll, inlined into @za_serde_to_vec_map): bb7.i (the Err branch) runs invoke <Buffer as Drop>::drop(%out.i); cleanup.body.i (all unwinds, after drop_glue::<SerMap>) runs invoke drop_glue::<Buffer>(%out.i). Map key Strings are wiped with invoke <String as Zeroize>::zeroize before their dealloc.

6. SeenKeys holds pointers and lengths only

drop_glue::<frame::SeenKeys> frees one block of buckets * 16 + buckets + 8 bytes: 16-byte (ptr, len) buckets plus control bytes, for a HashSet<&str>. No key bytes are copied (see ZA-0006, and its PoC's 552 bytes for 32 buckets). The earlier compiler note gave + 16 for the trailing constant; hashbrown's group width is 8 on aarch64 and 16 on x86-64 SSE2, so on this host it is + 8. The note was corrected and the PoC confirms it.

7. Wipes in the surrounding code (source level)

  • FrameWriter::new and at_depth now take &mut Buffer, closing the public path that let a caller supply a bare Vec<u8> (the README example used one).
  • Every ? in to_frame_at, decrypt_to_frame, storage::to_vec, serde::to_vec, to_part and displayed drops the partial Buffer, which wipes it.
  • found and next_is_null now peek at the tag instead of decoding the scalar, which removes a stack copy of the value from the reader (SO-5027).
  • No MISSING_SOURCE_ZEROIZE, PARTIAL_WIPE, INSECURE_HEAP_ALLOC, OPTIMIZED_AWAY_ZEROIZE or LOOP_UNROLLED_INCOMPLETE finding: every heap object that holds plaintext in the diff is wiped over its full capacity, and no path reallocates a plaintext Vec in place (except through the doors in ZA-0001 and ZA-0004).

Test gap (not a finding)

There is no unit test in vitaminc-protected that Buffer::grow wipes the old block. packages/protected/tests/buffer_wipe.rs checks only that Drop frees a zeroed block. The behaviour is covered end to end by packages/aead-frame/tests/storage_wipe.rs and transport_wipe.rs, which use marker-scanning allocators over growth, Display, integer keys, drop and the failure path, and distinct's integer-key wipe is covered by storage_wipe.rs (u128 keys whose little-endian bytes are the marker). A small allocator test next to Buffer that grows past capacity and checks the freed block would put the guarantee at the unit that owns it, and would be the natural test to pair with whichever fix is chosen for ZA-0001.

Out-of-diff observations (not findings)

  • serde.rs:170 Error::custom keeps message.to_string(); a user Serialize that puts a value into its error message would put it in an unwiped String. This predates the diff.
  • MapWriter::keys and FrameMap pre-allocate from the serializer's length hint; a large hint is an allocation-size concern, not a wiping one.

PoC Validation Results

All PoCs are standalone cargo crates built in release (opt-level 3) on aarch64-apple-darwin with a #[global_allocator] that reads the block it is about to free; freed memory is never read. Exit 0 means exploitable, 1 means not exploitable, 2 means inconclusive. The verifier re-ran all five and reproduced the exit codes. Sources are under poc/poc_za_000N/src/main.rs (make -C poc ZA-000N runs one).

Finding Category PoC File Exit Code Result Verified Impact
ZA-0001 SECRET_COPY poc/poc_za_0001/src/main.rs 0 exploitable yes needs_review to confirmed; medium kept
ZA-0002 SECRET_COPY poc/poc_za_0002/src/main.rs 0 exploitable (by design) yes needs_review to confirmed; low, accepted
ZA-0003 SECRET_COPY poc/poc_za_0003/src/main.rs 0 exploitable (by design) yes needs_review to confirmed; low, accepted
ZA-0004 SECRET_COPY poc/poc_za_0004/src/main.rs 0 exploitable yes needs_review to confirmed; low, open
ZA-0005 SECRET_COPY none n/a no_poc n/a stack and register copies cannot be observed from Rust without undefined behaviour
ZA-0006 SECRET_COPY poc/poc_za_0006/src/main.rs 1 not_exploitable yes severity low (it already was); confidence stays needs_review; finding kept
ZA-0007 NOT_ON_ALL_PATHS none n/a no_poc n/a category not supported for Rust PoCs
ZA-0008 to ZA-0010 STACK_RETENTION none n/a no_poc n/a already confirmed by IR and assembly
ZA-0011 REGISTER_SPILL none n/a no_poc n/a stays likely (assembly only)
ZA-0012 NOT_ON_ALL_PATHS none n/a no_poc n/a dead by the capacity invariant; nothing to trigger

Limits that apply to the exploitable results: each PoC models Vec growth as a moving realloc (GlobalAlloc's default allocate, copy, free). A system allocator that grows a block in place would leave no freed old block, but the caller does not control that. The ZA-0001 PoC exercises update only, and the ZA-0004 PoC exercises write only.


Superseded Findings

No findings were superseded in this run.

Three pairs of analyzer findings were merged into one finding each, because the IR and the assembly finding describe the same function and add up to two independent signals. This is a merge, not a supersession.

Merged Into
F-RUST-IR-0001 + F-RUST-ASM-0001 ZA-0008
F-RUST-IR-0002 + F-RUST-ASM-0002 ZA-0009
F-RUST-IR-0003 + F-RUST-ASM-0003 ZA-0010

Source and analyzer ids map to report ids as follows: F-RUST-SRC-0001 to 0007 are ZA-0001 to ZA-0007, F-RUST-ASM-0004 is ZA-0011, and the Phase 2 observation about the Vec growth fallback is ZA-0012.


Confidence Gate Summary

Finding Action Reason
ZA-0001 (SECRET_COPY) Held at needs_review in the interim report, then upgraded to confirmed MCP unavailable (mcp_mode=off), so SECRET_COPY was downgraded from likely to needs_review. A verified exploitable PoC is a second signal alongside the source-traced mechanism.
ZA-0002 (SECRET_COPY) Downgraded to needs_review, then upgraded to confirmed Same rule. The PoC confirms documented behaviour; status stays accepted.
ZA-0003 (SECRET_COPY) Downgraded to needs_review, then upgraded to confirmed Same rule. Status stays accepted.
ZA-0004 (SECRET_COPY) Held at needs_review, then upgraded to confirmed Analyzer confidence was needs_review; the verified exploitable PoC adds a second signal. Status stays open.
ZA-0005 (SECRET_COPY) Held at needs_review Source-only signal; the compiler evidence is carried by ZA-0008 to ZA-0011. No PoC possible.
ZA-0006 (SECRET_COPY) Held at needs_review; severity low The verified PoC returned not exploitable (exit 1), which downgrades severity to low (already low) and keeps the finding.
ZA-0007, ZA-0012 (NOT_ON_ALL_PATHS) Held at needs_review Heuristic and invariant-based; no PoC is supported for the category.
ZA-0008 to ZA-0010 (STACK_RETENTION) Confirmed IR (alloca with lifetime.end and no volatile store) plus assembly (secret bytes on the stack at ret): two signals and both hard evidence requirements met.
ZA-0011 (REGISTER_SPILL) Likely Assembly spill instruction shown (hard requirement met); only one kind of signal.

The OPTIMIZED_AWAY_ZEROIZE hard-evidence rule (an IR diff showing a wipe at O0 and none at O1 or O2) found nothing: no function loses its volatile stores at any level. No override was rejected.


Analysis Coverage

Metric Value
Crates analyzed 3 of 3 in scope (vitaminc-protected, vitaminc-aead, vitaminc-aead-frame with --features serde)
TUs in compile DB n/a (no compile DB; one cargo workspace TU, hash 10e312b4)
TUs analyzed 1 (3 of 3 packages succeeded)
TUs with sensitive objects 1
Agent 1 (MCP resolver) skipped (mcp_mode=off)
Agent 2 (source analyzer) success: 29 objects, 7 findings, 15 dangerous-API hits (14 false positives, 1 folded into ZA-0002), 0 semantic findings
Agent 3 (compiler analyzer) success: 3 of 3 packages for MIR (O0, O2), IR (O0 to O3), asm (O2, O3), plus an out-of-repo harness at release O0 and O3; 555 raw script findings dismissed with reasons in script-triage.json, 7 hand-verified findings retained
Agent 4 (report assembler) success (interim and final)
Agent 5 (PoC generator) success: 5 PoCs written and built; 7 findings with no supported category
PoC verifier success: 5 of 5 verified, exit codes reproduced
Agent 6 (test generator) skipped (enable_runtime_tests=false)

What ran: rustdoc JSON for each crate; semantic_audit.py; find_dangerous_apis.py; MIR, IR and assembly emission; check_mir_patterns.py, check_llvm_patterns.py, check_rust_asm.py; the MIR diff; semantic IR analysis of wipe_raw, grow and Drop::drop; the five PoCs.

What was skipped or not established:

  • validate_rust_toolchain.sh was not run: it fails on the macOS bash 3 (declare -A). The tools and scripts were verified by hand with test -x and test -f, and cargo check --workspace --all-targets --features vitaminc-aead-frame/serde succeeded.
  • MCP was off, so SECRET_COPY findings carry the downgrade described above.
  • CFG analysis was disabled, so no NOT_DOMINATING_EXITS or MISSING_ON_ERROR_PATH finding could be produced; the error-path claims rest on IR inspection of the unwind and Err blocks instead.
  • Runtime tests (phase 7) were not generated.
  • The emit scripts build with the dev profile and apply -C opt-level=N to the audited crate only, so its dependencies are built at dev O0 with debug assertions on. The out-of-repo harness was a true release build (dependencies at O3) and showed the same Buffer code shape, so no difference is expected, but the three crates' own release-profile IR was not emitted.
  • All assembly is AArch64 (the host). The AArch64 path is experimental, and every assembly claim in this report was read by hand. x86-64 assembly was not produced.
  • LTO builds were not checked. Volatile stores and the fence are preserved by LTO, but this was not observed.
  • panic=abort builds were not compiled; ZA-0007's abort half is true by construction.

Two script bugs found along the way (not in the repository under audit):

  1. check_mir_patterns.py and check_rust_asm.py crash on source-analysis/sensitive-objects.json, because it is a dict with an objects key and they expect a list. Workaround: secrets-list.json, the objects array on its own.
  2. The emit scripts concatenate two emitted files, so every function appears twice in each .ll and .s file. Workaround: read the first copy only.

Script noise: 555 raw findings were dismissed. 222 IR "non-volatile memset" hits are zero-initialisation (Zeroed::zeroed, hashbrown control bytes, SipHash state, SeenKeys::default, typed-null padding). 7 IR "volatile stores dropped" hits key on SSA names that change with inlining; a per-function comparison across all four levels found no function that loses its wipe. 37 MIR hits are locals named key or at that are borrowed &str. 289 assembly hits are prologue saves in functions that hold no secret.


Appendix: Evidence Files

Paths are relative to /tmp/zeroize-audit-20261028115220/.

Finding Evidence File Description
ZA-0001 poc/poc_za_0001/src/main.rs; source-analysis/source-findings.json (F-RUST-SRC-0001) Growth through Controlled::update; source trace
ZA-0002 poc/poc_za_0002/src/main.rs; source-analysis/rust-dangerous-api-findings.vitaminc_protected.json risky_unwrap PoC; mem::take hit
ZA-0003 poc/poc_za_0003/src/main.rs Passthrough copy PoC
ZA-0004 poc/poc_za_0004/src/main.rs; rust-compiler-analysis/aead.O3.leaf_write_Vec.s leaf::write into Vec::new(); the Vec<u8> monomorph
ZA-0005 source-analysis/source-findings.json (F-RUST-SRC-0005) Source-level list of stack and register copies
ZA-0006 poc/poc_za_0006/src/main.rs; rust-compiler-analysis/vitaminc-aead.O3.ll SeenKeys table PoC; drop_glue::<SeenKeys>
ZA-0007 rust-compiler-analysis/harness.O3.storage_to_vec_u64.ll; rust-compiler-analysis/harness.O3.serde_to_vec.ll Unwind and Err paths that drop the Buffer
ZA-0008 rust-compiler-analysis/vitaminc-aead.O3.ll, aead.O3.leaf_write_Buffer.ll, vitaminc-aead.O3.s, aead.O3.leaf_write_Buffer.s, aead.O3.leaf_write_Vec.s, vitaminc-aead.O2.s; ir-findings.json (F-RUST-IR-0001), asm-findings.json (F-RUST-ASM-0001) leaf::write IR and assembly
ZA-0009 rust-compiler-analysis/vitaminc-aead-frame.O3.ll, frame.O3.distinct.ll, vitaminc-aead-frame.O3.s, frame.O3.distinct.s, vitaminc-aead-frame.O2.s; ir-findings.json (F-RUST-IR-0002), asm-findings.json (F-RUST-ASM-0002) distinct IR and assembly
ZA-0010 rust-compiler-analysis/vitaminc-aead-frame.O3.ll, vitaminc-aead-frame.O3.s; ir-findings.json (F-RUST-IR-0003), asm-findings.json (F-RUST-ASM-0003) Sort temporary IR and assembly
ZA-0011 rust-compiler-analysis/vitaminc-aead.O3.s, vitaminc-aead-frame.O3.s, vitaminc-protected.O3.s; asm-findings.json (F-RUST-ASM-0004) Callee-saved spill assembly
ZA-0012 rust-compiler-analysis/vitaminc-aead.O3.ll, vitaminc-aead-frame.O3.s, notes.md (Observations) Vec growth fallback after grow
Verified wipes rust-compiler-analysis/vitaminc-protected.{O0,O1,O2,O3}.ll, vitaminc-protected.{O2,O3}.s, vitaminc-protected.{O0,O2}.mir, notes.md wipe_raw, grow, Drop, zeroize
All report/findings.json, report/finding-metadata.json, report/findings.interim.json, report/merge_poc.py; poc/poc_final_results.json, poc/poc_verification.json, poc/poc_manifest.json; rust-compiler-analysis/script-triage.json; source-analysis/rust-notes.md; preflight.json, merged-config.yaml Structured results, triage and run configuration

@coderdan

Copy link
Copy Markdown
Contributor Author

Zeroize audit, part two: x86-64 and fat LTO

The audit above ran on the AArch64 host without LTO, on library builds. This pass repeats its wipe checks on x86_64-apple-darwin (and vitaminc-protected on x86_64-unknown-linux-gnu) without LTO, and with fat LTO at O3 on both x86-64 and AArch64 through a leaf binary that calls the generic storage::to_vec and serde::to_vec with concrete types, so cross-crate inlining happens as it would in a shipping binary. Code generation is host-independent, so no emulation was involved and no x86 binary was run.

Result: every wipe survives on every target and with LTO. No wipe became a memset, no unrolled loop dropped a tail, no wipe lost its caller.

  • wipe_raw: on x86-64 the loops unroll (byte loops by 8, the word loop by 4, each with a remainder loop), giving 23 volatile stores and the fence per wipe; the counts add up to the full length. The 16-byte volatile store lowers to two movq $0. On AArch64 under LTO the three rolled loops stay: strb wzr, [x14], #1 / stp xzr, xzr, [x12], #16 / strb wzr, [x9], #1 / ;MEMBARRIER / bl _free.
  • Buffer::grow: alloc, memcpy, volatile wipe of the full old capacity, fence, free, on all three builds. No __rust_realloc in the function.
  • Buffer drop and zeroize: under LTO the drop is inlined into every drop glue with all its stores, the fence and the free.
  • to_vec error and unwind paths under LTO: the Err block runs the inlined wipe, fence and free; every landing pad reaches the Buffer drop glue before resume.
  • Per-function volatile-store counts, LTO against no-LTO: every function holds a whole number of complete wipes.
  • leaf::write stays a separate called function under LTO (#[inline(never)] held), with smaller stack temporaries once to_i128 and extend_from_slice inline into it.

One new finding, low, in the category STORAGE.md already accepts. On x86-64, distinct also copies the decoded integer key's 16-byte identity word into its own frame and leaves it there at return (movq -152(%rbp), %rax / movq %rax, -136(%rbp)), where AArch64 kept that word in registers. Under LTO distinct inlines into MapWriter::end and the copies move into that frame. The sentence in the first report's "Verified Correct" item 4 that no identity copy sits in distinct's frame holds on AArch64 only.

Also checked: the reserve hint for ZA-0012 (committed below) removes every Vec growth call from the Buffer paths under fat LTO as well; at HEAD before it, leaf::write::<Buffer> alone still carried 27 such calls on the dead path.

Limits: the nightly toolchain has no x86-64 darwin std, so the x86 and LTO builds used stable 1.98.1 where the host pass used nightly; Linux x86-64 covered vitaminc-protected only, without LTO; no panic=abort build, so the unwind-path results hold for panic=unwind; the x86 assembly checker on macOS scans only exported symbols, so distinct and leaf::write::<Vec<u8>> were read by hand.

@cipherstash-bot cipherstash-bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recommendation: 🟡 merge after changes

One item must change before merge: add tests for the two new debug checks on the Vec<u8> sink in leaf.rs, and document the new panic. The other items can wait for a follow-up or are optional: the MapWriter length-hint allocation, the missing storage::to_vec test, and the SeenKeys visibility note. The unsafe code looks sound.

How this review was made
Agent Model Review type Result
claude claude-opus-5-5 test-gap 3 found, 3 posted
claude claude-opus-5-5 rust 3 found, 2 posted
codex gpt-5.6-terra test-gap 1 found, 1 posted
codex gpt-5.6-terra rust 0 found, 0 posted

Synthesis: claude-sonnet-5-5 merged the findings, removed duplicates and dropped findings it could not confirm in the code. 1 posted finding(s) were raised by two or more models.

Plain language: claude-sonnet-5-5 read every comment as a new reader would. 3 comment(s) had a problem that stopped the reader acting; it rewrote 3. It also rewrote the review body.

Stack: position 1 of 2 (:asterisk: #559, #560). *️⃣ marks this pull request.

Context loaded: the description, 1 linked issue(s) and 6 discussion entries.

Findings not posted:

  • Fix in a follow-up: storage::to_vec sizes MapWriter::keys from the length hint of the Serialize impl, so a wrong hint can panic or abort the process. Already raised in this thread.

Comment thread packages/aead/src/leaf.rs Resolved
Comment thread packages/protected/src/buffer.rs Resolved
Comment thread packages/aead/src/frame.rs Resolved

@auxesis auxesis left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice one, thanks @coderdan.

Approving in advance of the test gaps being addressed, and the inevitable rebasing.

… so no blob is read back before it is returned

The storage writer read every blob back before returning it, to find a
repeated key as a reader would. That pass rebuilt a HashSet for every
object and hashed every field name, and was 55% of a 1,000-record write.

An object now compares each field name with the ones it has written,
held on the stack and spilling to the heap past 16, since the names are
the type's own and never data. A map records where each key lies in the
blob and checks them for a repeat when it ends, as before, without
parsing the rest. A debug build still reads the blob back, so the tests
keep checking that the writer and reader agree.

On an M4 on mains power this made a 1,000-record write 47% faster, one
record 41% and a map of 1,000 keys 29%, and left reads unchanged.
…ob is not wiped a byte at a time

zeroize stores one byte per volatile write, so wiping a 64 KiB buffer
took 15 µs, and a frame writer that regrows its buffer paid that for
every old allocation it wiped. wipe_raw, which the locked regions use,
now stores a u128 at a time between a byte-wise head and tail, and falls
back to bytes when the pointer cannot be aligned. The stores stay
volatile, so the guarantee is unchanged: only the width is.

A safe wipe_vec wraps it for a Vec<u8>'s whole allocation, and both
frame writers wipe their outgrown buffers with it. The Kani harness is
widened to cover the word run, and two tests pin the bounds from every
alignment; both pass under Miri's default and strict models.

On an M4 on mains this took 17% off a 1,000-record storage write and
10% off the transport write. The drop of the returned buffer still goes
through zeroize, which the next change addresses.
…xt they hold is wiped sixteen bytes a store

A frame or blob came back as Protected<Vec<u8>>, and the drop of a
Vec<u8> goes through zeroize, which stores one byte per volatile write:
15 µs for a 64 KiB value, and a quarter of a 1,000-record write. That
wipe cannot be changed from here without specialisation, so the type is.

Buffer, in vitaminc-protected, is a Vec<u8> that grows only through
push, extend_from_slice and reserve, which copy into a fresh allocation
and wipe the old one, and whose zeroize and drop wipe it with wipe_vec.
It derefs to a slice for reading and patching, has no Clone, PartialEq
or Debug since its bytes are secret, and risky_into_vec is the one way
out. to_frame, decrypt_to_frame and both to_vecs return Protected<Buffer>
and their writers hold &mut Buffer, so a writer cannot grow its buffer
unwiped; the two private reserve copies are gone. leaf::write takes
either buffer through an Out trait, so the sealed-leaf path is unchanged,
and a passthrough frame converts back to a Vec at its boundary since it
crosses in the clear.

BREAKING: the four writers return Protected<Buffer>, not
Protected<Vec<u8>>; FrameWriter and leaf::write take a Buffer; a caller
that borrowed a slice is unchanged, one that took the Vec calls
risky_into_vec.

Two findings shaped the hot path. Buffer's methods are #[inline], as
Vec's are, or structured writes lose up to 24% to a call per byte; but
leaf::write stays out of line, since inlined into the storage writer it
cost 10 to 15%. And distinct's scratch for integer keys, presized to the
key count, is a Buffer too: as a Zeroizing<Vec<[u8; 17]>> it wiped 17 KB
of untouched spare capacity a byte at a time for a map of 1,000 text
keys, 8 µs of a 38 µs write.

On an M4 on mains, against the previous commit: a 1,000-record storage
write 205 to 153 µs, a map of 1,000 keys 38.1 to 31.9 µs, one record
0.263 to 0.208 µs, and a 64 KiB value 17 to 1.9 µs. Transport writes
gained 10 to 17%. Reads are unchanged.
… read through risky_ref and not a plain slice

A Buffer dereferenced to a slice, so a frame written into one could be
read without a risky word anywhere, out of step with Protected and
Locked in the same crate. Buffer now implements Controlled with a
Vec<u8> inside: reads are risky_ref, taking the Vec is risky_unwrap,
and a length is filled in through overwrite(at, bytes), which hands out
no slice. Its Debug is redacted.

The frame and blob writers return a bare Buffer. It already wipes on
drop, so the Protected around it only added a second risky layer.

BREAKING: to_frame, decrypt_to_frame, serde::to_vec and storage::to_vec
return Buffer, not Protected<Buffer>. Buffer no longer implements Deref,
DerefMut or AsRef<[u8]>; risky_into_vec is risky_unwrap.

Measured on mains, three alternating rounds against the previous
commit: every write within 3%, which is the run-to-run spread.
…laced at it, so reading or writing a field allocates nothing

The map and enum access held each entry's path segment ready while its
value was read, and the transport writer did the same for each struct
field and variant part. That segment owns a String copied from the
type's own static name, so a struct of seven fields cost seven
allocations and frees per read and per write that nothing used unless
an error came back.

The entry's place is now a Copy value holding the static name, and
becomes a path segment only when an error is placed at it.

Measured on mains, three alternating rounds against the previous
commit: records read 9% to 10% faster in both profiles, a single record
16% to 18%, and the transport write of records 37% faster.
… value is decoded once on read and not twice

A string, bytes or map read, and an Option read, first asked what kind
of node came next. That answer was found by decoding the whole scalar,
slicing its payload and checking its UTF-8, and then thrown away before
the same scalar was decoded again for the visitor. The kind is in the
tag byte, so it is read from there, and a tag the validator would not
have passed is reported as malformed, as the decode reported it.

Measured on mains, three alternating rounds against the previous
commit: records read 14% faster in both profiles, a 64 KiB text 26%.
…so validating a record allocates no hash set

Both validators, the frame's and the storage profile's, built a HashSet
per object to refuse a repeated key: an allocation and a SipHash per
field for objects that mostly have a handful. SeenKeys keeps the first
sixteen keys inline and compares them directly, and hashes only once
an object has more, so a host object with thousands of keys is still
checked in linear time. The keys are borrowed from the frame, so
nothing is copied.

Measured on mains, three alternating rounds against the previous
commit: records read 18% faster in the storage profile and 23% in the
transport; writes unchanged.
…nner_mut reaches its bytes, so neither can be removed unnoticed
…g, bytes or a struct are refused by their kind
…e test helper, so the text reads as the rest of the file
…r and reader changes, rather than promise a re-measure
…r calls it and a public Vec wipe invites unwrapped secrets
…o writer uses them and each is more surface to review
…eSink, so it is not an open extension point named Out
…ince the Controlled methods that lend the Vec can grow it unwiped
… so the guarantee is tested at the type that owns it
…max_len, and show presized wiping sinks in the examples, since a growing Vec frees plaintext unwiped
… under panic=abort leaves a partly written blob unwiped
…allocating growth is compiled out of Buffer's push paths

After grow, which the compiler does not look into, Vec's own growth code stayed compiled into push and extend_from_slice in every writer. It could never run, because grow always makes room, but if it ever did it would realloc and free the old allocation unwiped. An assert_unchecked on the spare capacity after reserve lets the compiler drop it: every function that called grow also called RawVec::grow_one or do_reserve_and_handle before (8 of 10 in vitaminc-aead, 6 of 21 in vitaminc-aead-frame) and none does now, with or without fat LTO. A debug_assert of the same condition checks it in debug and Miri builds.

Measured: the storage writer is unchanged; the transport map_1000 write is about 6% slower, accepted for the hardening.
…r panic, so removing one fails a test and a caller with an unsized Vec knows why it panics
…ocate the room, since the room it then promises the compiler rests on that panic
…-frame uses it and it should not read as an API to keep stable
@coderdan

coderdan commented Nov 2, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto main (now at efac5e4), with the review fixes in the threads above. CI is green.

On the review summary's other items:

… it is shared with vitaminc-aead-frame and is not an API
@coderdan
coderdan force-pushed the perf/storage-writer-key-check branch from 642ecea to bd45c10 Compare November 2, 2026 03:54
@coderdan
coderdan merged commit 135fc31 into main Nov 2, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

aead-frame: make the storage and transport writers and readers faster, without changing their bytes

3 participants