diff --git a/.github/workflows/pricing-sync.yml b/.github/workflows/pricing-sync.yml index d0a982a..7d6297a 100644 --- a/.github/workflows/pricing-sync.yml +++ b/.github/workflows/pricing-sync.yml @@ -1,26 +1,30 @@ name: Pricing Sync # Refresh the bundled model-price table (cli/core/pricing/models.json) from -# models.dev and open a PR when prices changed. Runs monthly; run it by hand +# models.dev and open a PR when prices changed. Runs weekly; run it by hand # (workflow_dispatch) after a model launch or a price change. # # The commit is `fix(cli)`, which releases a patch — the table is compiled into # the binaries, so a refresh only reaches users when they update. That release # is the point of the job. # -# This mirrors the cloud's own monthly pricing-sync. The two tables are -# deliberately allowed to drift: the cloud is authoritative for cost and -# re-prices historical rows, while this copy exists so `dira status` / -# `dira report` have a number to show offline. `est_cost_usd` on the wire is -# contract-documented as a label, never a billing base, so a stale table here -# never moves an invoice. +# Weekly, not the cloud's own monthly cadence: claude-fable-5-1 launched +# 2026-09-01, and a monthly job would leave it estimated at the sonnet fallback +# ($3/$15) against a real $10/$50 for up to a month. Weekly also caps a +# transient models.dev outage at a week of staleness instead of a month. The +# two tables are still deliberately allowed to drift regardless of cadence: the +# cloud is authoritative for cost and re-prices historical rows, while this +# copy exists so `dira status` / `dira report` have a number to show offline. +# `est_cost_usd` on the wire is contract-documented as a label, never a billing +# base, so a stale table here never moves an invoice. # -# models.dev being down just fails the run (no PR); retry manually or wait for -# next month — pricing freshness is never critical. +# The fetch step below retries through transient failures and stalls on its +# own; models.dev being down for longer than that retry budget still fails the +# run with no PR, but the next scheduled run is a week away, not a month. on: schedule: - - cron: "0 6 1 * *" # monthly, 1st at 06:00 UTC + - cron: "0 6 * * 1" # weekly, Mondays at 06:00 UTC workflow_dispatch: permissions: @@ -49,20 +53,56 @@ jobs: - uses: Swatinem/rust-cache@v2 - name: refresh the table from models.dev shell: bash - # Mirrors `just pricing-sync`. Writes via a temp file so a failed fetch - # or a rejected payload can never truncate the vendored table. + # Mirrors `just pricing-sync`. + # + # Build before anything touches a pipe. This is the actual fix for the + # 2026-09-02 outage (run 33498614567: `curl: (18) Transferred a partial + # file`, then an EOF parse error on the truncated JSON). The old step + # piped curl straight into `cargo run`, which spent ~2 minutes compiling + # before reading a single byte — the pipe buffer filled, curl blocked, + # and models.dev dropped the idle connection. The job's reported 2m13s + # runtime was entirely build time, and the sync binary's own truncation + # guard caught the bad payload, so the failure was silent: no bad table, + # but also no PR and no signal. Building the binary first means nothing + # is ever sitting in a pipe while cargo compiles. + # + # --speed-limit 1024 --speed-time 60 turns a stalled transfer (under + # 1KB/s for 60s) into a retried error instead of a multi-minute hang — + # exactly the failure mode above, now caught even if something else + # downstream of curl gets slow again. + # + # Still writes via a temp file and only `mv`s once `pricing_sync` exits + # 0, so a rejected payload can never truncate the vendored table — that + # guarantee is unchanged, just split across the raw fetch and the + # tool's output instead of living in a single piped command. + # + # The trailing `cli/core/pricing/models.json` argument hands the sync + # binary the current table so a refresh only ever appends: it can no + # longer drop a model id that models.dev has stopped publishing. The + # append-only behavior lives in the binary; this step just wires the + # argument through. run: | set -euo pipefail - tmp="$(mktemp)" - trap 'rm -f "$tmp"' EXIT - curl -fsSL https://models.dev/api.json \ - | cargo run -q -p dira-core --bin pricing_sync > "$tmp" + cargo build -q -p dira-core --bin pricing_sync + raw="$(mktemp)"; tmp="$(mktemp)" + trap 'rm -f "$raw" "$tmp"' EXIT + curl -fsSL --retry 5 --retry-all-errors --retry-delay 5 \ + --connect-timeout 20 --max-time 300 --speed-limit 1024 --speed-time 60 \ + -o "$raw" https://models.dev/api.json + cargo run -q -p dira-core --bin pricing_sync -- cli/core/pricing/models.json \ + < "$raw" > "$tmp" mv "$tmp" cli/core/pricing/models.json - name: validate the refreshed table # The pricing tests assert the table parses, covers every harness # family, and still splits generations correctly — so a bad refresh # fails here rather than shipping. - run: cargo test -p dira-core --lib pricing tokens + # + # The `--` is load-bearing: `cargo test` takes a single filter, so the + # bare `… --lib pricing tokens` this used to run died on "unexpected + # argument 'tokens'" before a test could execute. Nothing noticed, + # because the fetch above had never once succeeded far enough to reach + # this step. Past `--`, libtest takes both filters. + run: cargo test -p dira-core --lib -- pricing tokens - name: open PR uses: peter-evans/create-pull-request@v8 with: diff --git a/.grok/rules/zavet.md b/.grok/rules/zavet.md index f94d6e1..f4d7c59 100644 --- a/.grok/rules/zavet.md +++ b/.grok/rules/zavet.md @@ -84,6 +84,8 @@ agent context at session start. Keep it short and non-negotiable. - DIRASH-0030 — Full-content knowledge sync is opted into by its own prompt, never implied by linking (active) - DIRASH-0031 — One backoff ladder lives in dira_core; callers own their attempt budget (active) - DIRASH-0032 — A record's first-sight triple is repaired as a unit, from recorded facts (active) +- DIRASH-0035 — A newer schema is refused loudly, never run against (active) +- DIRASH-0036 — A pricing refresh appends and never drops a vendored id (active) ### Living specs (.zavet/specs/ — keep current while you work) diff --git a/.zavet/INDEX.md b/.zavet/INDEX.md index ec1eef0..57cda74 100644 --- a/.zavet/INDEX.md +++ b/.zavet/INDEX.md @@ -39,6 +39,8 @@ a handful of documents. The decisions block below is regenerated by - **DIRASH-0030** — Full-content knowledge sync is opted into by its own prompt, never implied by linking (active) - **DIRASH-0031** — One backoff ladder lives in dira_core; callers own their attempt budget (active) - **DIRASH-0032** — A record's first-sight triple is repaired as a unit, from recorded facts (active) +- **DIRASH-0035** — A newer schema is refused loudly, never run against (active) +- **DIRASH-0036** — A pricing refresh appends and never drops a vendored id (active) ## Specs diff --git a/.zavet/decisions/DIRASH-0036-a-pricing-refresh-appends-and-never-drops-a-vendored-id.md b/.zavet/decisions/DIRASH-0036-a-pricing-refresh-appends-and-never-drops-a-vendored-id.md new file mode 100644 index 0000000..8541cf4 --- /dev/null +++ b/.zavet/decisions/DIRASH-0036-a-pricing-refresh-appends-and-never-drops-a-vendored-id.md @@ -0,0 +1,70 @@ +--- +id: DIRASH-0036 +title: A pricing refresh appends and never drops a vendored id +status: active +guards: + - cli/core/src/bin/pricing_sync.rs + - cli/core/pricing/ +checks: + - the table still carries one id per harness family :: cargo test -p dira-core --lib -- pricing tokens +origin: session +verified: true +--- + +## Decision + +`pricing_sync` takes the table it is replacing as an optional positional +argument, and with it the refresh is append-only: a key models.dev still +publishes gets the fresh price, a key models.dev has stopped publishing keeps +its last-known price. The only supported way for an id to leave +`cli/core/pricing/models.json` is an explicit `null` in `overrides.json`. +Both the workflow and `just pricing-sync` pass the argument; omitting it is a +clean regenerate, kept for a from-scratch run. + +## Why + +models.dev prunes ids as vendors retire them, and a regenerate-from-scratch +sync turns that into silent data loss. The cloud's counterpart re-prices +historical `token_usage` rows against its copy of this table; once a key is +gone the resolve cascade has nothing to fall back to — there is no +`gemini-3` family key under `gemini-3-pro-preview` — so those rows become +permanently unpriceable. The refresh of 2026-09 would have dropped four +Gemini ids this way, and it is what broke the cloud's canary test. + +Retention also makes the canaries honest. Pinning literal ids in a test is +only reasonable when a re-sync cannot empty them out; before this, a rotting +canary looked like an upstream rename and the tempting fix was to loosen the +assertion, which is how the test stopped guarding the models dira actually +observes. + +The cost is that a genuinely wrong price can no longer be corrected by +waiting for upstream to fix it — the entry persists until someone suppresses +it. That is the right trade: `overrides.json` already exists for exactly this, +and a wrong price is a visible, fixable estimate, while a missing one is +unrecoverable history. + +## Rejected + +- **Let ids drop and re-point the canaries each time** — the simplest diff, + but it accepts the cloud's history loss and guarantees the canaries rot + again on the next vendor rename. +- **Hand-copy dropped ids into `overrides.json`** — uses the documented + escape hatch, but it is manual work on every upstream prune and nothing + detects a prune that nobody noticed. +- **Fall back to a clean regenerate when the path argument is missing or + unreadable** — a typo'd path would drop every retained key with no + warning, which is the exact failure this decision exists to prevent. A bad + path is a hard error. + +## Agent directives + +- Never remove an entry from `cli/core/pricing/models.json` by hand, and + never "clean up" ids the catalog no longer carries. Suppress with a `null` + in `overrides.json` instead, and say why. +- The sanity gates in `pricing_sync` (`catalog.len() < 10`, missing + providers) are load-bearing under retention: a truncated catalog no longer + shows up as a shrunken table, it shows up as prices that quietly stop + moving. Do not soften them. +- The canary test asserts table **membership**, not `resolve().is_some()`. + The cascade's prefix step will answer for a missing id out of a shorter + sibling and hide the gap. diff --git a/AGENTS.md b/AGENTS.md index f5ca4c2..a956f71 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -91,6 +91,8 @@ agent context at session start. Keep it short and non-negotiable. - DIRASH-0030 — Full-content knowledge sync is opted into by its own prompt, never implied by linking (active) - DIRASH-0031 — One backoff ladder lives in dira_core; callers own their attempt budget (active) - DIRASH-0032 — A record's first-sight triple is repaired as a unit, from recorded facts (active) +- DIRASH-0035 — A newer schema is refused loudly, never run against (active) +- DIRASH-0036 — A pricing refresh appends and never drops a vendored id (active) ### Living specs (.zavet/specs/ — keep current while you work) diff --git a/cli/core/pricing/models.json b/cli/core/pricing/models.json index 56e4b21..25976c0 100644 --- a/cli/core/pricing/models.json +++ b/cli/core/pricing/models.json @@ -1,5 +1,5 @@ { - "$comment": "Generated by `just pricing-sync` from https://models.dev/api.json — do not hand-edit; corrections go in overrides.json. Prices are USD per 1M tokens. Scope: tool-calling, cost-bearing models from the providers behind the supported harnesses, minus dated aliases the resolver already reaches by stripping the pin. The cloud keeps its own copy and is authoritative; this one only labels local views and may drift between monthly refreshes.", + "$comment": "Generated by `just pricing-sync` from https://models.dev/api.json — do not hand-edit; corrections go in overrides.json. Prices are USD per 1M tokens. Scope: tool-calling, cost-bearing models from the providers behind the supported harnesses, minus dated aliases the resolver already reaches by stripping the pin. Append-only: a refresh updates the price of any key upstream still publishes but never drops a key upstream stops publishing, so the cloud's re-pricing cascade always has a key to resolve historical usage against. To actually remove an entry, add an explicit null override in overrides.json — that is the only supported way a key leaves this table. The cloud keeps its own copy and is authoritative; this one only labels local views and may drift between weekly refreshes.", "models": { "claude-3-5-haiku": { "cacheRead": 0.08, @@ -13,6 +13,12 @@ "input": 10.0, "output": 50.0 }, + "claude-fable-5-1": { + "cacheRead": 0.25, + "cacheWrite": 12.5, + "input": 10.0, + "output": 50.0 + }, "claude-haiku-4-5": { "cacheRead": 0.1, "cacheWrite": 1.25, @@ -191,19 +197,24 @@ "output": 2.5 }, "gemini-3.6-flash": { - "cacheRead": 0.15, - "input": 1.5, - "output": 7.5 + "cacheRead": 0.075, + "input": 0.75, + "output": 3.75 + }, + "gemini-3.7-flash": { + "cacheRead": 0.075, + "input": 0.75, + "output": 3.75 }, "gemini-flash-latest": { - "cacheRead": 0.15, - "input": 1.5, - "output": 9.0 + "cacheRead": 0.075, + "input": 0.75, + "output": 3.75 }, "gemini-flash-lite-latest": { - "cacheRead": 0.025, - "input": 0.25, - "output": 1.5 + "cacheRead": 0.03, + "input": 0.3, + "output": 2.5 }, "gemini-robotics-er-1.6-preview": { "input": 1.0, @@ -388,10 +399,10 @@ "output": 180.0 }, "gpt-5.6": { - "cacheRead": 0.5, - "cacheWrite": 6.25, - "input": 5.0, - "output": 30.0 + "cacheRead": 0.4, + "cacheWrite": 5.0, + "input": 4.0, + "output": 20.0 }, "gpt-5.6-luna": { "cacheRead": 0.02, @@ -400,10 +411,10 @@ "output": 1.2 }, "gpt-5.6-sol": { - "cacheRead": 0.5, - "cacheWrite": 6.25, - "input": 5.0, - "output": 30.0 + "cacheRead": 0.4, + "cacheWrite": 5.0, + "input": 4.0, + "output": 20.0 }, "gpt-5.6-terra": { "cacheRead": 0.2, @@ -436,6 +447,11 @@ "input": 2.0, "output": 6.0 }, + "grok-4.6": { + "cacheRead": 0.5, + "input": 2.0, + "output": 6.0 + }, "grok-build-0.1": { "cacheRead": 0.2, "input": 1.0, @@ -491,6 +507,11 @@ "input": 0.3, "output": 1.2 }, + "muse-spark-1.2": { + "cacheRead": 0.15, + "input": 1.25, + "output": 4.25 + }, "o1": { "cacheRead": 7.5, "input": 15.0, diff --git a/cli/core/src/bin/pricing_sync.rs b/cli/core/src/bin/pricing_sync.rs index 6210154..e024e90 100644 --- a/cli/core/src/bin/pricing_sync.rs +++ b/cli/core/src/bin/pricing_sync.rs @@ -4,8 +4,14 @@ //! **stdout**, so the fetch stays in the caller (`just pricing-sync`) and this //! binary needs no HTTP client — the same shape as `sign_vector`. //! -//! curl -fsSL https://models.dev/api.json \ -//! | cargo run -q -p dira-core --bin pricing_sync > cli/core/pricing/models.json +//! curl -fsSL https://models.dev/api.json -o catalog.json +//! cargo run -q -p dira-core --bin pricing_sync -- cli/core/pricing/models.json \ +//! < catalog.json > models.json.new +//! +//! Fetch to a file, never straight down a pipe into `cargo run`: cargo spends +//! minutes compiling before it reads a byte, the pipe buffer fills, and the +//! server drops the connection it has been holding open for nothing. That is +//! the 2026-09-02 outage, and `just pricing-sync` is the invocation to copy. //! //! Only the providers behind the harnesses dira tracks are kept: the full //! catalog carries the same model under dozens of resellers at *different* @@ -18,6 +24,18 @@ //! Exits non-zero if the payload doesn't look like the catalog, so a truncated //! or error response can never overwrite a good table with a bad one. //! +//! Takes an optional positional argument: the path to the table this refresh +//! is replacing. With it, the refresh is append-only — a key upstream still +//! publishes gets the fresh price, a key upstream has quietly dropped keeps +//! its last-known price instead of vanishing. That matters because the +//! cloud's counterpart re-prices historical `token_usage` rows against this +//! table, and a dropped key leaves that resolve cascade with no family key to +//! fall back to: the rows become permanently unpriceable. Deliberately +//! removing a bad entry stays an explicit `null` in `overrides.json`, not a +//! side effect of upstream silently unpublishing it. Without the argument the +//! binary does a clean regenerate exactly as before, so it stays usable from +//! scratch (first run, tests) without carrying merge logic along for the ride. +//! //! Deliberately mirrors the cloud's `scripts/pricing-sync.ts`. The two tables //! are allowed to drift between refreshes: the cloud is authoritative and //! re-prices historical rows, while this copy only labels local views. @@ -40,12 +58,37 @@ fn num(v: Option<&Value>) -> Option { } fn main() -> Result<(), Box> { + // The previous table, when given, is what makes this an append-only + // refresh instead of a clean regenerate — see the module doc. Missing or + // unreadable is a hard error, never a silent fall-through to a clean + // regenerate: a typo'd path would otherwise drop every retained key + // without so much as a warning, exactly the bug this binary exists to fix. + let existing_path = std::env::args().nth(1); + let existing_models: Map = match &existing_path { + None => Map::new(), + Some(path) => { + let raw = std::fs::read_to_string(path) + .map_err(|e| format!("reading existing table at {path}: {e}"))?; + let parsed: Value = serde_json::from_str(&raw) + .map_err(|e| format!("parsing existing table at {path}: {e}"))?; + parsed + .get("models") + .and_then(Value::as_object) + .cloned() + .ok_or_else(|| format!("existing table at {path} has no \"models\" object"))? + } + }; + let mut raw = String::new(); std::io::stdin().read_to_string(&mut raw)?; let catalog: Map = serde_json::from_str(&raw)?; // A catalog this small means a truncated or error payload, not a real - // shrink — bail rather than publish it. + // shrink — bail rather than publish it. This gate matters more now than + // it used to: with retention, a partial catalog can no longer be caught + // downstream by "the table shrank" — missing keys just quietly keep the + // previous refresh's price instead. Without this check that failure mode is silent + // instead of loud, which is the one thing this binary must never be. if catalog.len() < 10 { return Err(format!("catalog looks truncated: {} providers", catalog.len()).into()); } @@ -128,6 +171,22 @@ fn main() -> Result<(), Box> { } } + // Append-only merge: every key still standing in `models` at this point + // came from the fresh sync above and already has this run's price, so it + // must win outright. A key only the previous table knows about is one + // upstream has quietly stopped publishing — fill the gap with its + // last-known entry instead of letting it disappear. This has to run after + // the provider loop (so freshness always wins) and before the dated-alias + // prune below (so a retained key is pruned by the same rule a fresh one + // would be, not exempted from it). + let mut retained: Vec = Vec::new(); + for (key, entry) in &existing_models { + if !models.contains_key(key) { + models.insert(key.clone(), entry.clone()); + retained.push(key.clone()); + } + } + // Drop dated aliases whose undated form is already present at the same // price: the resolver's cascade strips `-20251001` before looking up, so the // entry can never be reached and only bloats the bundle. Kept when the @@ -143,15 +202,31 @@ fn main() -> Result<(), Box> { for k in &redundant { models.remove(k); } + // A retained key can itself get pruned here (its undated form showed up + // fresh at the same price this run) — drop it from the report too, since + // it no longer needs a reviewer's eyes as a surviving gap-fill. + retained.retain(|k| models.contains_key(k)); + retained.sort(); eprintln!( "pricing sync: {} models from {considered} tool-calling, cost-bearing entries across {}, \ {collisions} price collisions resolved by provider precedence, \ - {} redundant dated aliases dropped", + {} redundant dated aliases dropped, \ + {} keys retained because upstream no longer publishes them", models.len(), PROVIDERS.join(", "), - redundant.len() + redundant.len(), + retained.len() ); + if !retained.is_empty() { + // One per line rather than joined into the summary above — a wide + // comma list is easy to skim past, and this is exactly the list a + // refresh PR reviewer needs to actually look at. + eprintln!("retained (upstream dropped these ids):"); + for key in &retained { + eprintln!(" {key}"); + } + } let mut out = Map::new(); out.insert( @@ -159,9 +234,13 @@ fn main() -> Result<(), Box> { "Generated by `just pricing-sync` from https://models.dev/api.json — do not hand-edit; \ corrections go in overrides.json. Prices are USD per 1M tokens. Scope: tool-calling, \ cost-bearing models from the providers behind the supported harnesses, minus dated \ - aliases the resolver already reaches by stripping the pin. The cloud keeps its own \ - copy and is authoritative; this one only labels local views and may drift between \ - monthly refreshes." + aliases the resolver already reaches by stripping the pin. Append-only: a refresh \ + updates the price of any key upstream still publishes but never drops a key upstream \ + stops publishing, so the cloud's re-pricing cascade always has a key to resolve \ + historical usage against. To actually remove an entry, add an explicit null override \ + in overrides.json — that is the only supported way a key leaves this table. The cloud \ + keeps its own copy and is authoritative; this one only labels local views and may \ + drift between weekly refreshes." .into(), ); out.insert("providers".into(), PROVIDERS.into()); diff --git a/cli/core/src/pricing.rs b/cli/core/src/pricing.rs index 237ecdd..33fa20f 100644 --- a/cli/core/src/pricing.rs +++ b/cli/core/src/pricing.rs @@ -288,17 +288,36 @@ mod tests { } } + /// Pinning literal ids here used to be brittle — a monthly re-sync could drop + /// one and the test would rot silently until something noticed prices were + /// missing. The sync is now append-only (it never drops a vendored id), so + /// pinning is safe, and doing so buys something a looser family/substring + /// check does not: a whole provider vanishing from the catalog fails here, + /// not just a stale id. One id per supported harness family, plus the full + /// Anthropic generation lineup dira actually observes today. #[test] fn the_bundled_table_parses_and_covers_every_harness_family() { let t = table(); assert!(t.len() > 50, "table looks truncated: {} entries", t.len()); for id in [ + // Anthropic (Claude Code): every generation in active use. + "claude-opus-5", "claude-opus-4-8", "claude-sonnet-5", "claude-haiku-4-5", "claude-fable-5", + "claude-fable-5-1", + // One id per remaining supported harness family. + "gpt-5.3-codex", // openai/Codex + "gemini-3.1-pro-preview", // google/Gemini CLI + "grok-4.5", // xai ] { - assert!(resolve(id).is_some(), "{id} must resolve"); + // Presence in the table, not `resolve().is_some()`: the cascade's + // prefix step would answer for a missing id out of a shorter + // sibling and hide the gap. That is exactly how `claude-fable-5-1` + // read as priced while it was absent — it matched `claude-fable-5` + // and inherited a cache-read rate four times its own. + assert!(t.contains_key(id), "{id} must be a table entry"); } } diff --git a/cli/core/src/tokens.rs b/cli/core/src/tokens.rs index 82f7000..7136069 100644 --- a/cli/core/src/tokens.rs +++ b/cli/core/src/tokens.rs @@ -386,6 +386,12 @@ not-json /// `claude-fable-5` used to match nothing and fall through to the sonnet-shaped /// fallback, which under-estimated it by ~3×. It is a real family with its own /// price, and it must not be reported as unpriced. + /// + /// `claude-fable-5-1` must be pinned separately, not folded into the + /// `claude-fable-5` case above: its `cache_read` is 0.25, a quarter of + /// `claude-fable-5`'s 1.0, while every other rate matches. If the two ever + /// collapsed onto one table key, cache reads on the newer model would be + /// priced 4x too high. #[test] fn fable_is_priced_and_recognised() { let p = pricing_for("claude-fable-5"); @@ -394,6 +400,13 @@ not-json (10.0, 50.0, 1.0, 12.5) ); assert_eq!(pricing_family("claude-fable-5"), Some("claude-fable-5")); + + let p1 = pricing_for("claude-fable-5-1"); + assert_eq!( + (p1.input, p1.output, p1.cache_read, p1.cache_write), + (10.0, 50.0, 0.25, 12.5) + ); + assert_eq!(pricing_family("claude-fable-5-1"), Some("claude-fable-5-1")); } #[test] @@ -407,6 +420,7 @@ not-json "claude-sonnet-5", "claude-haiku-4-5", "claude-fable-5", + "claude-fable-5-1", "grok", ] { assert!(pricing_family(m).is_some(), "{m}"); diff --git a/justfile b/justfile index d88408f..b78bf51 100644 --- a/justfile +++ b/justfile @@ -49,18 +49,33 @@ contract: contract-schema vector # NOT a contract artifact and deliberately NOT part of `just ci`: it needs the # network, and the table is an estimate for local display only. The cloud keeps # its own copy, is authoritative, and re-prices historical rows — the two are -# allowed to drift between refreshes. Runs monthly in CI; run it by hand after a -# model launch or a price change. +# allowed to drift between refreshes. Runs weekly in CI (a monthly cadence once +# left claude-fable-5-1 estimated at the sonnet fallback price for weeks after its +# 2026-09-01 launch); run it by hand too after a model launch or a price change. # -# Writes via a temp file so a failed fetch or a rejected payload can never -# truncate the vendored table. +# Build before the fetch: this used to pipe curl straight into `cargo run`, +# which meant curl's connection sat idle for the ~2 minutes cargo spent +# compiling before it read a byte — the pipe buffer filled, curl blocked, and +# models.dev dropped the connection (the 2026-09-02 CI outage, run 33498614567). +# Building the binary first guarantees nothing is ever waiting in a pipe. +# +# curl's --speed-limit/--speed-time turns a stalled transfer into a retried +# error instead of a multi-minute hang, and --retry covers ordinary transient +# failures. Still writes via a temp file and only `mv`s once `pricing_sync` +# exits 0, so a failed fetch or a rejected payload can never truncate the +# vendored table. The trailing path argument hands the binary the existing +# table so a refresh only ever appends, never drops a retired model id. pricing-sync: #!/usr/bin/env bash set -euo pipefail - tmp="$(mktemp)" - trap 'rm -f "$tmp"' EXIT - curl -fsSL https://models.dev/api.json \ - | cargo run -q -p dira-core --bin pricing_sync > "$tmp" + cargo build -q -p dira-core --bin pricing_sync + raw="$(mktemp)"; tmp="$(mktemp)" + trap 'rm -f "$raw" "$tmp"' EXIT + curl -fsSL --retry 5 --retry-all-errors --retry-delay 5 \ + --connect-timeout 20 --max-time 300 --speed-limit 1024 --speed-time 60 \ + -o "$raw" https://models.dev/api.json + cargo run -q -p dira-core --bin pricing_sync -- cli/core/pricing/models.json \ + < "$raw" > "$tmp" mv "$tmp" cli/core/pricing/models.json echo "wrote cli/core/pricing/models.json"