Skip to content

fix(security): bound tar lists and nested substitution parsing - #2678

Merged
chaliy merged 10 commits into
mainfrom
codex/tar-files-from-budget
Oct 11, 2026
Merged

chaliy merged 10 commits into
mainfrom
codex/tar-files-from-budget

Conversation

@chaliy

@chaliy chaliy commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor

What changed

Tar file lists now charge input, work, and live retained capacity before reading, decoding, or growing per-name storage. File, stdin, repeated-list, and long-option forms preserve order, duplicates, CRLF handling, and malformed UTF-8 replacement.

Nested command substitutions now consume the enclosing AST depth and preserve its timeout origin. Heap-owned child parsers and private command results, plus outlined nonrecursive parameter parsing and AST assembly, prevent native stack overflow on default 2 MiB test-thread stacks. The existing 100-level ceiling and public AST format remain unchanged.

The feature suite also exposed unreliable embedded CPython module alignment on macOS. Module bytes now retain a cached immutable aligned copy when linking or ASLR does not preserve 64 KiB alignment; the original alignment assertion is unchanged, and byte-preservation and repeated-load tests pass.

Why

A dense tar -T list bypassed live-memory accounting. The full ship gate also reproduced a depth-50 substitution SIGABRT on clean main: child parsers reset depth and retained large debug-build stack frames. Both resource failures must return bounded errors without terminating the host.

Before / After

  • Tar before: a 40 KB dense list allocated a 1,255,365-byte peak under a 64 KiB live budget; the isolated allocator regression failed.
  • Tar after: all list forms return the live-budget error before amplification, stay below the regression's 256 KiB total allocator ceiling including fixed interpreter overhead, create no archive, and allow the same shell to recover. Seven integration regressions and the isolated allocator test pass; finite GNU tar/Bash comparisons preserve normal behavior.
  • Parser before: cargo test depth_50_is_bounded -- --nocapture aborted with SIGABRT on clean main 508967f5; a new four-level substitution with a three-level AST limit incorrectly parsed.
  • Parser after: the original depth-50 regression passes. Depth boundaries, mixed compound/substitution nesting, inherited timeout, oversized caller limits, assignments, arrays, pipelines, lists, and redirects are covered. Depth-120 input returns AST nesting too deep; no nested file write occurs, the shell recovers, and moderate nesting matches /bin/bash stdout, stderr, and exit status.

Validation on 9e72082a: full just pre-pr, just test with serialized failpoints, all-features Clippy, security proptests, rustdoc preview, builtin inventory, and tar/CPython CLI smoke tests pass. All 49 PR checks are green. All 33 Criterion cases completed with default sample counts; the CPU-bounded runner required separate subprocess groups. The 96-case Bash comparison harness has zero errors and 100% expected-output matches on main and this revision; aggregate Bashkit time was 27.71 ms before and 25.06 ms after in this run. Single-operation timing changes were about -1% to +2%. Parallel timings were unstable on the active desktop: initial 100/200-session slowdowns changed substantially in back-to-back repeats, with both unchanged results and large spikes. Fresh 200- and 1,000-session repeats detected no statistically significant change. These measurements do not establish a stable parallel performance change; raw runs and repeat evidence are retained.

Risk

Medium: parser frame layout and private ownership affect shared grammar paths. Existing grammar, quoting, function printing, aliases, and security tests run unchanged. CPython may retain one additional module-sized copy plus less than 64 KiB padding on unaligned hosts. No dependency, stack-size, depth-ceiling, test-skip, or CI-policy changes.

Checklist

  • Tests added or updated
  • Backward compatibility considered (public AST unchanged; internal resource accounting corrected)

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Oct 11, 2026 •

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
bashkit 9e72082 Commit Preview URL

Branch Preview URL
Oct 11 2026, 06:52 PM

chaliy added a commit that referenced this pull request Oct 11, 2026
Additive merge only (no history rewrite): picks up #2678 signal-trap
doc/test updates, #2681 perf numbers, #2682 threat-model doc updates.
Diagnostic-prefix bound (TM-DIAG-PREFIX-01) verified intact after merge;
32/32 diagnostic_prefix tests pass.
@chaliy chaliy changed the title fix(tar): budget files-from storage before allocation fix(security): bound tar lists and nested substitution parsing Oct 11, 2026
@chaliy
chaliy marked this pull request as ready for review October 11, 2026 18:40
@chaliy
chaliy merged commit 92103eb into main Oct 11, 2026
49 checks passed
@chaliy
chaliy deleted the codex/tar-files-from-budget branch October 11, 2026 19:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant