laya-ts

A self-hosted HTTP server for Laya, the open-source System-1 decision model. It speaks the same request/response format as TypeSafe's hosted Jev API (POST /v1/systemone). Next to it is a benchmark that runs the same requests against both and measures latency and answer deviation.

laya-ts/
├── server/            HTTP server + Dockerfile (runs @receptron/laya on ONNX Runtime, CPU)
├── bench/             benchmark: 479 cases / 2,066 questions, runner, random-data generator
│   ├── cases/random.json      seeded random invoices, tickets, emails, reviews, chats, clauses, orders
│   ├── cases/docs/*.json      examples extracted from docs.typesafe.ai cookbooks, primitives and patterns
│   ├── gen-random.mjs         regenerates cases/random.json
│   ├── run.mjs                runs cases against local and/or live, writes results/<timestamp>/
│   └── fetch-docs.sh          downloads the source doc pages (not committed)
└── Makefile

Requirements: Docker, plus Node.js ≥ 20 for the benchmark (it has no npm dependencies). Budget 2–4 GB RAM for the server.

Quick start

make build     # build the image (≈ 900 MB)
make up        # start on :8080; first start downloads the ≈ 1.7 GB model into the laya-cache volume
make wait      # block until the model is loaded
make smoke     # send one example request
make help      # list every target

API

POST /v1/systemone (alias /v1/system_one)

{
  "state": "Help! My payouts have been failing for 3 days.",
  "questions": {
    "department": { "type": "choice", "instructions": "Which team should handle this?",
                    "criteria": { "billing": "Payments, invoicing, refunds", "technical": "Bugs, outages", "sales": "Pricing" } },
    "frustration": { "type": "score", "instructions": "How frustrated is the customer?", "criteria": ["Calm", "Frustrated", "Very angry"] },
    "is_urgent":   { "type": "noul",  "instructions": "Does this convey urgency?" }
  }
}

The response is {"model": "laya", "answers": {...}, "usage": {...}}, with the same answer shapes as the TypeSafe API: choice + probabilities + confidence, score + legend + probabilities + confidence, or noul. A model field in the request is accepted and ignored. The x-inference-ms response header gives the server-side time.

GET /health returns {"status": "loading" | "ready" | "error", "queued": n}, with a 503 until the model is ready.

Errors: 400 malformed request · 401 bad key (only if LAYA_API_KEY is set) · 413 body too large · 422 the model rejected the question (e.g. options longer than 192 tokens) · 429 queue full · 503 still loading.

Configuration (environment)

Variable	Default
PORT	8080
LAYA_API_KEY	–	if set, requests need Authorization: Bearer <key>
LAYA_MAX_BATCH	4	questions per forward pass; bounds memory (8 used ≈ 3.4 GB on long states)
LAYA_MAX_QUEUE	64	pending requests before 429
LAYA_THREADS	all cores	ONNX Runtime intra-op threads
LAYA_SUBFOLDER	–	multilingual for the multilingual checkpoint
LAYA_REVISION	main	pin a Hugging Face commit for reproducible weights
LAYA_MODEL_DIR	–	use a local ONNX bundle instead of downloading
MAX_BODY_BYTES	5000000

Make variables: PORT, MEMORY (container limit, default 4g), IMAGE, NAME, VOLUME, e.g. make up PORT=9000 MEMORY=6g.

Run without Docker: make dev (the model is cached under ~/.cache/receptron-laya).

Performance notes

Inference is serialized (ONNX Runtime already uses every core). Laya encodes the state once per question (each question's sequence is question header + state, up to 512 tokens), so cost grows with questions × state length. On 2 vCPUs, a short three-question ticket takes ≈ 0.35 s warm; a 62-question request over a 512-token document takes ≈ 5 min. The hosted Jev API does this on GPUs. States longer than 512 tokens are truncated (Jev accepts much longer states).

Benchmark

make bench-gen                              # regenerate random cases (seed 42, 150 cases)
make bench                                  # all cases against the local server (~70 min on 2 vCPUs)
make bench-quick                            # first 25 cases
make bench-random / make bench-docs         # one half of the suite

export TYPESAFE_API_KEY=...                 # https://console.typesafe.ai/keys
make bench-live                             # local vs live Jev on the same requests
make bench BENCH_ARGS="--targets live --concurrency 8"   # live only, throughput
make bench BENCH_ARGS="--filter guardrails --repeat 3"   # one group, 3 timed calls each

make bench includes the live target automatically when TYPESAFE_API_KEY is set. Each run writes bench/results/<timestamp>/ with report.md, summary.json and results.jsonl (every request, answer and timing). See bench/README.md for the case format, metrics and where each case came from.

Credits

• Model: convaiinnovations/laya (Apache 2.0).
• Runtime: @receptron/laya (MIT), the Node.js/ONNX port.
• API format and cookbook examples: TypeSafe. This project is not affiliated with TypeSafe.