Skip to content

Version 3 - #215

Closed
groundwater wants to merge 6 commits into
mainfrom
experiment/nio-http2
Closed

Version 3#215
groundwater wants to merge 6 commits into
mainfrom
experiment/nio-http2

Conversation

@groundwater

@groundwater groundwater commented Apr 24, 2026

Copy link
Copy Markdown
Owner

Summary

NIO migration, interactive shell, and Terminal toolbar.

NIO Migration

  • Replaced VsockServer with NIOVsockServer as the only HTTP server
  • Streaming file uploads direct to disk
  • HTTP/2 protocol auto-detection
  • allowRemoteHalfClosure for vsock connections

vmctl shell (new)

  • Interactive PTY shell via WebSocket: vmctl → HostAPIService → GhostTools
  • Login shell via /usr/bin/login -fp for proper environment
  • Write retry on short write/EAGAIN across all layers
  • PTY I/O on dedicated serial queue

Terminal toolbar item (new)

  • Open Terminal: launches Terminal.app with vmctl shell
  • Copy vmctl Command: full absolute path to clipboard
  • vmctl.app copied into VM bundle

Build

  • make debug-dmg: debug config, timestamp versioned, notarized

🤖 Generated with Claude Code

@groundwater groundwater changed the title Experiment: NIO HTTP/2 + WebSocket shell for vmctl Version 3 Apr 27, 2026
@groundwater
groundwater force-pushed the experiment/nio-http2 branch from e86dca4 to f2666e2 Compare April 27, 2026 23:06
groundwater and others added 2 commits April 27, 2026 21:53
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Deployment target: macOS 15 → 26 (project.yml, Package.swift, Makefile vtool)
- Swift tools version 6.2 with strict concurrency fixes
  (nonisolated(unsafe) singletons, @sendable callbacks, async shutdown)
- Drop KqueueVsockProbe (kqueue works with AF_VSOCK on macOS 26)
- Add make debug-dmg: debug config, timestamp versioned, notarized
- Sign debug dylibs in Contents/MacOS/ for notarization
- cli target honors APP_SKIP_XCODE_SIGNING
- Update IPSW catalog to macOS 26.4.1 (25E253)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@groundwater
groundwater force-pushed the experiment/nio-http2 branch 4 times, most recently from 066986c to 0399f05 Compare April 28, 2026 05:12
groundwater and others added 4 commits April 28, 2026 00:49
NIOVsockServer (port 5000):
- HTTP/1.1 + HTTP/2 auto-detection via ProtocolDetector
- StreamDispatcher routes requests synchronously (no async race)
- StreamingFileReceiveHandler writes uploads directly to disk
- RouterBridgeHandler bridges NIO HTTP to existing Router
- allowRemoteHalfClosure for vsock connections
- HTTP/2 responses omit forbidden connection header
- os_log tracing throughout the pipeline

HealthServer (port 5002):
- Writes version JSON on connect, stays open for liveness

EventPushServer (port 5003):
- Tracks single client channel, pushEvent() writes NDJSON via NIO

TunnelServer (port 5001):
- CONNECT handshake via TunnelHandshakeHandler
- Bidirectional TCP bridge via paired GlueHandler instances

Shared types (VsockTypes.swift):
- AF_VSOCK, sockaddr_vm, VsockServerError extracted from old VsockServer

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Guest (GhostTools):
- ShellWebSocketHandler spawns PTY via forkpty, bridges I/O over
  WebSocket binary frames, handles resize/exit via text frames
- Login shell via /usr/bin/login -fp for full environment
- PTY I/O on dedicated serial queue with stale fd guard
- Write retry on EAGAIN, bails out when cleanup invalidates fd

Host proxy (HostAPIService):
- Detects /api/v1/shell and switches to bidirectional byte bridge
- Validates upgrade request forwarding, aborts on failure
- Write retry on all handshake and bridge writes

Client (vmctl):
- WebSocket handshake, raw terminal mode, SIGWINCH/SIGINT handling
- Write retry on WebSocket frames and stdout output
- CPty C module exposes forkpty/openpty to Swift

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Terminal toolbar dropdown:
- Open Terminal: launches Terminal.app with vmctl shell via temp script
- Copy vmctl Command: copies full absolute shell-escaped path

vmctl.app copied into VM bundle's Helper/ directory on launch
so it has a stable known path.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
New VMs use ASIF (Apple Sparse Image Format) for near-native SSD
performance via diskutil image create blank --format ASIF.

Migration for old VMs:
- Detects raw disk format via magic bytes (DiskFormatUtilities)
- Custom NSPanel prompt (avoids NSAlert SwiftUI animation crash)
- NSSavePanel for destination, terminal-style progress window
- diskutil image create from --format ASIF (handles sparsity)
- Cancel terminates diskutil process, synchronized isCancelled
- Verifies output is valid ASIF after conversion
- Boot Now button with orderOut/delayed close pattern
- Old VM never modified, skipHelperCopy for legacy launches
- Warns that snapshots and suspend state are not migrated
- GhostTools.dmg attachment non-fatal (logs error, boots without)
- GhostTools.dmg uses makehybrid format for USB auto-mount

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@groundwater

Copy link
Copy Markdown
Owner Author

Superseded by #228. We backed out the SwiftNIO migration because non-blocking writes on macOS AF_VSOCK silently truncate large responses (~10 MiB ceiling) — see #228 description and .var/macos-vsock-write-loss/ reproducer for the full story. The v3 feature set (vmctl shell, Terminal toolbar, streaming file transfer) is preserved on the blocking server.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant