Skip to content

Count untracked files in one command per batch - #522

Merged
powerfooI merged 2 commits into
powerfooI:mainfrom
thiblahute:upstream/untracked-count
Oct 31, 2026
Merged

powerfooI merged 2 commits into
powerfooI:mainfrom
thiblahute:upstream/untracked-count

Conversation

@thiblahute

@thiblahute thiblahute commented Oct 31, 2026 •

Copy link
Copy Markdown
Collaborator

A workspace refresh asked git for the line count of every untracked file one file at a time. Over SSH each of those commands is a fresh login, so a worktree holding an ignored-but-untracked build tree turns one refresh into thousands of logins: 32,504 untracked files meant 32,504 logins, enough to saturate the remote sshd for the better part of an hour and to keep going after the files were gone.

Changes

  • Untracked line counts run as one shell script per batch of 100 paths (GIT_UNTRACKED_NUMSTAT_BATCH_SIZE), the way the fingerprint and check-attr passes already batch. Concurrency stays at 4 commands.
  • The script ends with exit 0: a loop's status is only its last file's, and a file git cannot read (exit 128) must not cost the rest of its batch their counts.
  • Past 500 untracked files (GIT_UNTRACKED_NUMSTAT_MAX_FILES), Changes lists them without line counts. A tree that large is noise in the panel either way, and counting it is not worth the commands.
  • Concurrent refreshes keep independent results; each browser receives complete line counts. The 500-file cap bounds each request to at most five numstat batches.
  • FEATURES.md: one line on the 500-file cap in the Workspace Inspector section.

Verification

  • bun test server/src/workspace/git-diff.test.ts: 40 pass, 0 fail. New cases:
    • one numstat command for a batch of untracked files, with the counts attributed to the right paths;
    • no numstat command past the cap, all entries listed without counts;
    • overlapping refreshes each return complete counts (regression: the original cancellation ran only 1 of 5 batches);
    • with real git, an unreadable file counted last in the batch leaves the other files' counts in place (fails without the exit 0).
  • bun run format:check on the changed files: clean.
  • bun run lint: clean apart from the existing TerminalView.tsx exhaustive-deps warning.
  • bun run typecheck: passes, including the web build and asset budget check (initial JS 805.0/805 KiB).
  • bun run test:quick with an isolated HOME, on main plus this change and the other small fixes I am sending alongside: 3965 pass, 4 skip, 2 fail. Both failures (useMobileControlsDrag.keyboard.test.ts and the SSH paths test) fail the same way on main on macOS.

Not verified on this branch: the 32,504-file case was observed and fixed against a real SSH host before this rebase; the remote counts were not measured again here.

Compatibility

No API or settings changes. Workspaces with more than 500 untracked files no longer show per-file line counts for them.

A workspace refresh asked git for the line count of every untracked file one
file at a time, and over SSH each of those commands is a fresh login. A
worktree holding an ignored build tree makes the arithmetic ugly: 32,504
untracked files is 32,504 logins for a single refresh, enough to saturate the
far host's sshd for the better part of an hour and to keep going long after
the files themselves are gone.

The counts now ride one shell script per batch of 100 paths, the way the
fingerprint and check-attr passes already do. A file git cannot read no longer
costs the rest of its batch their counts. Past 500 untracked files the Changes
panel lists them without line counts: a tree that large is noise in the panel
either way, and saying so is not worth a round trip per entry.

A refresh can also be overtaken while it is still counting: another browser
asks, a poll lands, the tree moves on under it. Each pass claims a generation
and abandons its remaining batches once a newer pass owns the answer, rather
than spending commands on paths that may not exist any more.
@github-actions github-actions Bot added the enhancement New feature or request label Oct 31, 2026
@powerfooI
powerfooI merged commit 3b30ec7 into powerfooI:main Oct 31, 2026
5 checks passed

Copy link
Copy Markdown
Owner

Reviewed and fixed concurrent-refresh cancellation in 465ab63. The new regression failed against the original implementation (1 of 5 batches) and now returns complete counts for overlapping requests. Removed the global generation map while retaining 100-file batching, concurrency 4, and the 500-file cap. Focused tests: 40 pass, 0 fail.

Combined verification on current main plus all six contributions: bun run format:check, bun run typecheck (including the web build and asset budgets), and bun run test:quick all passed: 3968 pass, 4 conditional skips, 0 fail. Initial JS is 805.4/806 KiB. Native Bun checks and lint passed in the correction commits' pre-commit hooks (existing TerminalView exhaustive-deps warning remains).

@powerfooI powerfooI added the bug Something isn't working label Oct 31, 2026 — with ChatGPT Codex Connector
@powerfooI powerfooI removed the enhancement New feature or request label Oct 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants