Skip to content

@webcontainer/snapshot runs out of memory on about 17,000 files #2187

Open

Description

@onmax

Describe the bug

snapshot() from @webcontainer/snapshot 0.1.0 runs out of memory on a folder of 17,000 small files. The files hold about 17 MB in total, and Node crashes with JavaScript heap out of memory at a 512 MB heap. A node_modules folder for a mid-sized app has that many files.

uncontrolledSnapshot() keeps every pending readdir, lstat and readFile in one Map and calls await Promise.race(tasks.values()) once per settled task. Each Promise.race adds a reaction to every pending promise, and those reactions stay until that promise settles. With n pending tasks that grows with n², which matches the numbers below.

files heap used after snapshot() time
2,000 482 MB 3 s
4,000 1,901 MB 15 s
17,000, 512 MB heap crash
17,000, 512 MB heap, at most 64 directories at once 17 MB 25 s

Link to the blitz that caused the error

https://stackblitz.com/github/onmax/repros/tree/repro/webcontainer-snapshot-race-memory/webcontainer-snapshot-race-memory-blitz

npm start runs snapshot() on 2,000 and 4,000 files, then the same walk with a bounded queue. In StackBlitz snapshot() takes 2.1 s and 9.4 s, the bounded queue 0.15 s and 0.26 s. process.memoryUsage() reads 0 inside a WebContainer and --max-old-space-size has no effect, so this blitz shows the time, not the heap.

The full repro is https://github.com/onmax/repros/tree/repro/webcontainer-snapshot-race-memory/webcontainer-snapshot-race-memory. Opened in StackBlitz, npm run verify (17,000 files) crashes the browser tab after about a minute, and npm run verify -- --control passes in 0.7 s. The heap error itself needs host Node.

Steps to reproduce

  1. npm install && npm run verify. It writes 17,000 files of 1 KiB and calls snapshot() with --max-old-space-size=512. It exits 1 with the heap error.
  2. npm run verify -- --control runs the same reads and encoder with a bounded queue (patch-control.mjs) and passes.

Expected behavior

Memory stays near the size of the file contents, and snapshot() finishes.

Parity with Local

  • I have run the project in my local machine and I could not reproduce the issue.

Platform

  • OS: Linux
  • Node 24.21.0
  • Version: @webcontainer/snapshot 0.1.0

Additional context

A small queue with a fixed concurrency fixes it. I can open a PR if you want one, but I couldn't find the package's source in this repo.

Activity

  1. github-actions commented on Nov 17, 2026

    @github-actions

    We noticed that you did not provide a StackBlitz or Bolt reproduction link. Please update the issue and add a link to a public project.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions