Repository navigation
Count untracked files in one command per batch - #522
Merged
Merged
Conversation
A workspace refresh asked git for the line count of every untracked file one file at a time, and over SSH each of those commands is a fresh login. A worktree holding an ignored build tree makes the arithmetic ugly: 32,504 untracked files is 32,504 logins for a single refresh, enough to saturate the far host's sshd for the better part of an hour and to keep going long after the files themselves are gone. The counts now ride one shell script per batch of 100 paths, the way the fingerprint and check-attr passes already do. A file git cannot read no longer costs the rest of its batch their counts. Past 500 untracked files the Changes panel lists them without line counts: a tree that large is noise in the panel either way, and saying so is not worth a round trip per entry. A refresh can also be overtaken while it is still counting: another browser asks, a poll lands, the tree moves on under it. Each pass claims a generation and abandons its remaining batches once a newer pass owns the answer, rather than spending commands on paths that may not exist any more.
Owner
|
Reviewed and fixed concurrent-refresh cancellation in 465ab63. The new regression failed against the original implementation (1 of 5 batches) and now returns complete counts for overlapping requests. Removed the global generation map while retaining 100-file batching, concurrency 4, and the 500-file cap. Focused tests: 40 pass, 0 fail. Combined verification on current main plus all six contributions: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A workspace refresh asked git for the line count of every untracked file one file at a time. Over SSH each of those commands is a fresh login, so a worktree holding an ignored-but-untracked build tree turns one refresh into thousands of logins: 32,504 untracked files meant 32,504 logins, enough to saturate the remote sshd for the better part of an hour and to keep going after the files were gone.
Changes
GIT_UNTRACKED_NUMSTAT_BATCH_SIZE), the way the fingerprint andcheck-attrpasses already batch. Concurrency stays at 4 commands.exit 0: a loop's status is only its last file's, and a file git cannot read (exit 128) must not cost the rest of its batch their counts.GIT_UNTRACKED_NUMSTAT_MAX_FILES), Changes lists them without line counts. A tree that large is noise in the panel either way, and counting it is not worth the commands.Verification
bun test server/src/workspace/git-diff.test.ts: 40 pass, 0 fail. New cases:exit 0).bun run format:checkon the changed files: clean.bun run lint: clean apart from the existingTerminalView.tsxexhaustive-deps warning.bun run typecheck: passes, including the web build and asset budget check (initial JS 805.0/805 KiB).bun run test:quickwith an isolatedHOME, onmainplus this change and the other small fixes I am sending alongside: 3965 pass, 4 skip, 2 fail. Both failures (useMobileControlsDrag.keyboard.test.tsand the SSH paths test) fail the same way onmainon macOS.Not verified on this branch: the 32,504-file case was observed and fixed against a real SSH host before this rebase; the remote counts were not measured again here.
Compatibility
No API or settings changes. Workspaces with more than 500 untracked files no longer show per-file line counts for them.