feat(lora): let a running pod fetch and register a LoRA - #315
Open
gioelecerati wants to merge 1 commit into
Open
feat(lora): let a running pod fetch and register a LoRA#315gioelecerati wants to merge 1 commit into
gioelecerati wants to merge 1 commit into
Conversation
A pod builds its LoRA catalog once, in register_library() at engine init.
Anything trained after that can never be enabled — enable_lora on an id
that wasn't in the boot scan raises "not registered" — so a LoRA a user
trains while their session is open is unreachable, which is exactly when
they want it. /api/loras re-globs the disk per request, so the catalog
would even SHOW a file dropped in later; the engine would still refuse it.
New `add_lora {id, url}`: fetch the file into loras_dir() and register it.
Downloaded on a worker thread — ~170 MB has no business in the dispatch
path, let alone the tick — to a .partial that is renamed on completion, so
a dropped connection can't leave a truncated .safetensors that the next
scan happily registers and the engine then fails to parse. The finished
path is queued onto state.pending_register and drained in before_tick
alongside the existing enable/disable queues, so the catalog mutation
serializes with the pipeline like every other LoRA change.
Registration runs BEFORE the enables in the same drain, deliberately: a
client that adds a LoRA and immediately enables it queues both before the
next tick, and the other order would raise on every such pair.
Registration is metadata-only; weights still materialize inside
enable_lora.
Two guards, because the WS is reachable by anyone holding a session. The
id is sanitized before it becomes a filename and a knob name. The URL must
be https on an allow-listed host (DEMON_LORA_URL_HOSTS) — without that,
add_lora is "make this pod GET an arbitrary URL for me" from inside our
network, which is the whole SSRF shape. Defaults cover the orchestrator
(Studio-trained LoRAs) and Tigris (ones the user uploaded).
There is no failure message on the wire by design: the fetch outlives the
request, and the catalog is the completion signal. Clients watch for the
id to appear, then send enable_lora.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
A pod builds its LoRA catalog once, in
register_library()at engine init. Anything trained after that can never be enabled —enable_loraon an id that wasn't in the boot scan raises "not registered". So a LoRA a user trains while their session is open is unreachable, which is exactly when they want it./api/lorasre-globs the disk per request, so the catalog would even show a file dropped in later. The engine would still refuse it.What
New
add_lora {id, url}command: fetch a.safetensorsintoloras_dir()and register it into the live catalog..partialand renames on completion, so a dropped connection can't leave a truncated file that the next scan registers and the engine then fails to parse.state.pending_register, drained inbefore_tickbeside the existing enable/disable queues, so the catalog mutation serializes with the pipeline like every other LoRA change.enable_lora.Guards
The WS is reachable by anyone holding a session, so:
lora_str_<id>knob name;DEMON_LORA_URL_HOSTS). Without that,add_lorais "make this pod GET an arbitrary URL for me" from inside our network — the classic SSRF shape. Defaults cover the orchestrator (Studio-trained LoRAs) and Tigris (user uploads).Contract
No failure message on the wire, by design: the fetch outlives the request. The catalog is the completion signal — clients watch for the id to appear, then send
enable_lora. Wire types regenerated; the capability-gate and dispatcher-parity invariants thattests/unit/test_wire_contract.pyenforces were verified against this branch.Testing
Not yet exercised against a live pod — this needs a
:warmrebake to reach the fleet, which is deliberately not part of this PR.🤖 Generated with Claude Code