Skip to content

fix(cli): make the pricing refresh append-only and ship Fable 5.1 - #137

Merged
azlekov merged 3 commits into
developfrom
fix/pricing-sync-retention
Sep 2, 2026
Merged

fix(cli): make the pricing refresh append-only and ship Fable 5.1#137
azlekov merged 3 commits into
developfrom
fix/pricing-sync-retention

Conversation

@azlekov

@azlekov azlekov commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

The monthly Pricing Sync job has been failing, and claude-fable-5-1 (launched 2026-09-01) was being priced at the sonnet fallback — $3/$15 against a real $10/$50.

The outage

Run 33498614567 died with curl: (18) Transferred a partial file, then an EOF parse error. The step piped a 4.4 MB catalog straight into cargo run, which spends ~2 minutes compiling before it reads a byte: the pipe buffer filled, curl blocked, and models.dev dropped the connection. The job's 2m13s runtime was entirely build time. The sync binary's truncation guard worked, so the failure was silent — no bad table, but no PR and no signal either.

Now: build first, fetch to a file, then feed it in. curl gains retries and --speed-limit/--speed-time so a stall becomes a retried error rather than a multi-minute hang.

While in there: the validation step cargo test --lib pricing tokens never actually ran — cargo test takes one filter, so it died on unexpected argument 'tokens'. Nothing noticed, because the fetch had never once got that far.

The append-only change (DIRASH-0036)

models.dev prunes ids as vendors retire them, and a regenerate-from-scratch sync turns that into silent data loss. This refresh alone would have dropped gemini-3-pro-preview, gemini-2.0-flash, gemini-2.0-flash-lite and gemini-robotics-er-1.6-preview. The cloud re-prices historical token_usage rows against its copy of this table; once a key is gone the cascade has nothing to fall back to, and those rows become permanently unpriceable.

pricing_sync now takes the table it replaces as a positional argument and merges over it. Removal becomes an explicit null in overrides.json.

That also makes pinning ids in the canary honest, so it now names every Anthropic generation in active use plus one id per remaining harness family — and asserts table membership rather than resolve().is_some(). The cascade's prefix step answers for a missing id out of a shorter sibling and hides the gap: claude-fable-5-1 read as priced while absent, matching claude-fable-5 and inheriting a cache-read rate four times its own.

Table diff

Added claude-fable-5-1 (10/50, cacheRead 0.25), gemini-3.7-flash, grok-4.6, muse-spark-1.2. Prices drop on gpt-5.6/gpt-5.6-sol (5/30 → 4/20) and the Gemini flash line. Nothing removed — the four retired Gemini ids are retained at last-known price.

Cadence goes weekly (0 6 * * 1).

Verification

just ci passes. cargo test -p dira-core --lib -- pricing tokens (the workflow's own step) is green at 22/22. Retention was proven by running the binary with and without the path argument against a saved catalog, and a deliberately truncated catalog still exits non-zero with the table untouched.

Both workflows check out develop, so this can only be exercised after merge — trigger Pricing Sync by hand then; it should go green and open no PR, since the table it would write is already committed.

🤖 Generated with Claude Code

azlekov and others added 3 commits September 2, 2026 18:01
… an id

`pricing_sync` now takes the table it is replacing as an optional positional
argument and merges over it: a key upstream still publishes gets the fresh
price, a key upstream has dropped keeps its last-known one. Removing an entry
becomes an explicit null in overrides.json, the only supported way out.

Four Gemini ids would have vanished from this refresh alone. The cloud
re-prices historical token_usage rows against its copy of this table, and a
dropped key leaves that cascade nothing to resolve against, so those rows
become permanently unpriceable.

Retention is also what makes pinning ids in the canary honest, so the test
now names every Anthropic generation in active use plus one id per remaining
harness family, and asserts table membership rather than resolve().is_some()
— the cascade's prefix step answers for a missing id out of a shorter sibling
and hides the gap. That is not hypothetical: claude-fable-5-1 read as priced
while absent, matching claude-fable-5 and inheriting a cache-read rate four
times its own.

Why: a regenerate-from-scratch sync turns an upstream prune into silent,
unrecoverable data loss.
Rejected: re-pointing canaries each time upstream renames a model — accepts
the history loss and guarantees the same rot next quarter.
Refs: DIRASH-0036

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Asen Lekov <asenlekoff@gmail.com>
…binary

The 2026-09-02 run failed with `curl: (18) Transferred a partial file` and an
EOF parse error on the truncated JSON. The step piped a 4.4 MB catalog
straight into `cargo run`, which spends about two minutes compiling before it
reads a byte: the pipe buffer filled, curl blocked, and models.dev dropped the
connection it had been holding open for nothing. The job's 2m13s runtime was
entirely build time.

Build first so nothing waits in a pipe, then fetch to a file, then feed it in.
curl also gains retries and --speed-limit/--speed-time, which turns a stalled
transfer into a retried error rather than a multi-minute hang. The temp-file-
then-mv guarantee is unchanged.

The validation step never actually ran: `cargo test --lib pricing tokens`
takes a single filter and died on "unexpected argument 'tokens'" before any
test executed, which nothing noticed because the fetch had never once got that
far. Past `--`, libtest takes both.

Cadence goes weekly. claude-fable-5-1 launched 2026-09-01, and a monthly job
leaves a new flagship estimated at the sonnet fallback rate for up to a month.

Why: a fetch whose reader is still compiling is a stalled fetch.
Refs: DIRASH-0036

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Asen Lekov <asenlekoff@gmail.com>
Adds claude-fable-5-1 (10/50, cacheRead 0.25), gemini-3.7-flash, grok-4.6 and
muse-spark-1.2. Prices drop on gpt-5.6 and gpt-5.6-sol (5/30 to 4/20),
gemini-3.6-flash and gemini-flash-latest (1.5/9 to 0.75/3.75); gemini-flash-
lite-latest rises to 0.3/2.5.

Nothing was removed: four Gemini ids upstream no longer publishes
(gemini-2.0-flash, gemini-2.0-flash-lite, gemini-3-pro-preview,
gemini-robotics-er-1.6-preview) are retained at their last-known price.

claude-fable-5-1's cache-read rate is a quarter of claude-fable-5's, so until
this landed the resolver's prefix step priced its cache reads 4x too high.

Refs: DIRASH-0036

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Asen Lekov <asenlekoff@gmail.com>
@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Knowledge layer

Check Result
Decision checks passed
Guard trailers passed

Spec currency is reported in the job summary and never fails a build, the same severity the local pre-commit hook uses.

@azlekov
azlekov merged commit be52623 into develop Sep 2, 2026
8 checks passed
@azlekov
azlekov deleted the fix/pricing-sync-retention branch September 2, 2026 15:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant