Skip to content

perf(runtime): import local sources via single tar put_archive#740

Open
pranav-afk wants to merge 2 commits into
usestrix:mainfrom
pranav-afk:fix/tar-local-source-import
Open

perf(runtime): import local sources via single tar put_archive#740
pranav-afk wants to merge 2 commits into
usestrix:mainfrom
pranav-afk:fix/tar-local-source-import

Conversation

@pranav-afk

Copy link
Copy Markdown

Replace the SDK per-file LocalDir copy path that hangs for hours on large repos with one container.put_archive() per source, so sandbox startup finishes quickly and scans can produce reports.

@greptile-apps

greptile-apps Bot commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR moves local source imports from SDK manifest entries to a Docker archive upload path. The main changes are:

  • Splits local sources into tar-copied sources and bind mounts.
  • Builds an in-memory tar for each copied source tree.
  • Uploads each tar with container.put_archive() after session start.
  • Adds unit tests for source splitting, tar contents, upload failures, and ownership changes.

Confidence Score: 4/5

The new local-source import path needs fixes before merging.

  • Large repos can exhaust process memory before upload.
  • Import failures after container start can leave containers running.
  • Read errors while walking the source tree can produce incomplete workspaces.
  • Custom sandbox images can fail on the hard-coded pentester ownership step.

strix/runtime/session_manager.py

Important Files Changed

Filename Overview
strix/runtime/session_manager.py Replaces SDK local directory entries with tar archive construction, Docker upload, and post-upload ownership changes.
strix/runtime/backends.py Updates comments to describe the new split between manifest mounts and post-start local source imports.
strix/runtime/docker_client.py Updates comments around bind mounts and the tar-based local source import path.
tests/test_import_local_sources.py Adds unit tests for archive upload, chown failure, put failure, and Docker container extraction helpers.
tests/test_session_entries.py Updates source-splitting tests and adds tar-builder coverage for files, dotfiles, symlinks, empty directories, and unsafe subdirs.
Prompt To Fix All With AI
Fix the following 4 code review issues. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 4
strix/runtime/session_manager.py:93
**Archive Buffer Exhausts Memory**

When a large local source is copied, `_build_source_tar()` stores the whole tar in `BytesIO` and then returns another full `bytes` copy. The changed path is meant for large repos, but a multi-GB repo or a tree with many small files can now OOM the Strix process before Docker receives the archive.

### Issue 2 of 4
strix/runtime/session_manager.py:277
**Started Container Escapes Cleanup**

`_import_local_sources()` runs after the backend has started the container but before the bundle is cached. If tar creation, `put_archive`, `mkdir`, or `chown` fails, `create_or_reuse()` raises with no cached bundle for `cleanup(scan_id)` to delete, leaving the started container running.

### Issue 3 of 4
strix/runtime/session_manager.py:97
**Unreadable Directories Disappear Silently**

`os.walk()` suppresses directory-read errors when no `onerror` handler is provided. If a local repo contains an unreadable subdirectory, the tar import can succeed while omitting that subtree, so the agent later sees missing files under `/workspace/<subdir>` with no import failure or skip count.

### Issue 4 of 4
strix/runtime/session_manager.py:201-209
**Fixed User Breaks Custom Images**

The new import path always runs `chown -R pentester:pentester` after extracting sources. `create_or_reuse()` accepts arbitrary sandbox images, so an image without that user or group now fails during local-source import even though the container started correctly.

Reviews (1): Last reviewed commit: "perf(runtime): import local sources via ..." | Re-trigger Greptile

Comment thread strix/runtime/session_manager.py Outdated
Comment thread strix/runtime/session_manager.py Outdated
Comment thread strix/runtime/session_manager.py Outdated
Comment thread strix/runtime/session_manager.py Outdated
@aYoung-CS

Copy link
Copy Markdown

WTF? This is a direct copy of my PR #695.

@0xallam 0xallam force-pushed the main branch 2 times, most recently from 44e87ca to daf39a2 Compare July 13, 2026 23:50
Replace the SDK per-file LocalDir copy path that hangs for hours on large repos with one container.put_archive() per source, so sandbox startup finishes quickly and scans can produce reports.
Spill tars to temp files, tear down containers on setup failure, fail on unreadable dirs, and resolve workspace ownership from the image user instead of hard-coding pentester.
@0xallam 0xallam force-pushed the fix/tar-local-source-import branch from c33df59 to d8d9af1 Compare July 13, 2026 23:51
@aYoung-CS

Copy link
Copy Markdown

This is a direct copy of my PR #695!!! You THEIF!!!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants