Blazing fast structured decisions for System One open-models self-hosted with a TypeSafe AI compatible API, written in Rust.
tokio,axumandserde, the usual suspects- System One compatible API Spec
candlewithtokenizersrelease candidate!- Dynamic, token-based batching
- SDPA on CPU, Metal, and CUDA
- Flash Attention on Ampere, Ada Lovelace, and Hopper
- Blazing fast inference for ModernBERT and Qwen3.5
- Multimodal decisions from text, images, and video when supported by the model
Rust and Cargo 1.98.1, mandatory. ffmpeg and ffprobe 6.1.1 or higher for video inputs.
If you run on CUDA you also need nvcc. And if you run on Metal, you also need xcode and xcodebuild -downloadComponent MetalToolchain.
cargo install sys1 --features cpu
# cargo install sys1 --no-default-features --features metal
# cargo install sys1 --no-default-features --features cuda
# cargo install sys1 --no-default-features --features cuda,flash-attn-2 # Ampere, Ada Lovelace, or HopperThen run it with any of the supported models (more coming soon!).
convaiinnovations/layafor English text, guardrails, email triageconvaiinnovations/laya-multilingualfor 100+ languages, ~2.2x fasterconvaiinnovations/laya-typed-decisionsfor typed-decisions workflowsCloudflare/cleffor text, image, and video decisionsCloudflare/clef-flashfor text, image, and video decisions (smaller and faster than the 27B variant)
sys1 --model-id convaiinnovations/laya --dtype autoAnd, just query your Jev-compatible API at /v1/systemone (or /v1/decide).
curl http://localhost:3000/v1/systemone \
-H "Content-Type: application/json" \
-d '{
"model": "convaiinnovations/laya",
"state": {
"message": "I was charged twice for invoice 4411. Please refund me today."
},
"questions": {
"route": {
"type": "choice",
"instructions": "Where should this ticket go?",
"criteria": {
"billing": "payments, refunds, invoices",
"bug": "the product is broken",
"account": "login or access"
}
},
"urgency": {
"type": "score",
"instructions": "How urgent is this message?",
"criteria": [
"routine, no rush",
"today",
"urgent",
"critical, about to churn"
]
},
"escalate": {
"type": "noul",
"instructions": "Escalate to a human immediately?"
}
}
}'Or, install the TypeSafe AI SDK for Python, or the TypeSafe AI SDK for Javascript / Typescript.
from typesafe_sdk import Choice, TypeSafeClient
client = TypeSafeClient(base_url="http://localhost:3000", api_key="-")
response = client.system_one(
model="convaiinnovations/laya",
state={"message": "I was charged twice for invoice 4411."},
questions={
"route": Choice(
instructions="Where should this ticket go?",
criteria={"billing": None, "bug": None, "account": None},
),
},
)
print(response.choices["route"].choice)