Show model cost tiers in the LLM picker

[FadhlanR]

Background and Goal

Give end users a glanceable sense of how expensive each AI model is, as a dollar-sign tier (Free, $, $$, $$$, $$$$) shown in the AI assistant model picker (LLMSelect) — in every row and beside the selected model. Closes CS-12195.

The tier is derived from the pricing that already lives on OpenRouterModel catalog cards; there was no cost-tier concept before this.

Where to start

• packages/host/app/utils/model-cost.ts — the pure tier logic. A blended 3:1 input:output price per million tokens (the industry-standard "blended" cost), mapped to fixed, human-round bands (≤$1 → $, ≤$5 → $$, ≤$20 → $$$, >$20 → $$$$; 0 → Free; unknown pricing → no badge). Fully unit-tested.
• packages/host/app/components/matrix/room.gts — pickerModelIds + loadModelCostTiers fill a @tracked map the picker reads; see the reactivity note below.
• packages/host/app/components/ai-assistant/llm-select.gts — LLMOption.costTierLabel + the badge markup/styling (a muted chip, distinct from the green per-token prices on the model cards).

Key decisions and non-obvious mechanics

• No card-schema change. ModelConfiguration and OpenRouterModel are correlated only by the modelId string (no link), so the picker searches the OpenRouter catalog realm (config.resolvedOpenRouterRealmURL) by modelId and derives the tier client-side. Degrades to no badge when the realm is unconfigured or a model has no catalog card.
• Why a task, not a trackedFunction/resource. The lookup can't run inside a reactive computation: store.search hydrates cards (mutating tracked store state) and the tier derivation reads those cards' pricing fields, so the computation self-invalidates into an endless re-run and never settles (the picker just never shows a badge). Instead a restartableTask gated on a content key (realm + sorted model ids) does the search off the render path and writes a @tracked map — content-keyed so the live matrix churn behind pickerModelIds (a fresh usedLLMs array each sync) can't retrigger it. Verified working in the running app.
• Tests cover the fallback-model branch, the system-card branch (badge attaches to a ModelConfiguration via its modelId), the uncatalogued-model case (no badge), and graceful degradation when the catalog search fails.

Screenshot

🤖 Generated with Claude Code

[github-actions]

Preview deployments

• Host staging preview

• Host production preview

Host Test Results

    1 files  ± 0      1 suites  ±0   2h 54m 12s ⏱️ -28s
3 823 tests +13  3 809 ✅ +13  14 💤 ±0  0 ❌ ±0 
3 842 runs  +13  3 828 ✅ +13  14 💤 ±0  0 ❌ ±0 

Results for commit 333af8b. ± Comparison against earlier commit 9dbab75.

Realm Server Test Results

    1 files  ±    0      1 suites  ±0   14m 34s ⏱️ + 3m 16s
2 032 tests ±    0  2 031 ✅  -     1  0 💤 ±0  1 ❌ +1 
4 222 runs  +2 111  4 221 ✅ +2 110  0 💤 ±0  1 ❌ +1 

Results for commit 333af8b. ± Comparison against earlier commit 9dbab75.

For more details on these errors, see this check.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9dbab7557b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Reject partially numeric pricing strings

When OpenRouter supplies a malformed value with a numeric prefix, such as "0.000003 USD" or "1x", parseFloat accepts the prefix and the picker displays a potentially incorrect tier instead of treating the price as unknown as intended. Parse the entire value (for example with Number) and reject negative or otherwise invalid prices before calculating the tier.

Useful? React with 👍 / 👎.

[lukemelia]

Any reason not to make this a computed on the OpenRouterModel card or the ModelConfiguration card?

[habdelra]

[Claude Code 🤖] This review focuses on the one open question on the PR — whether the tier derivation belongs on a card as a computed field — and on the two places this change could quietly go wrong: the reactivity design in room.gts, and the robustness of the pure tier math. The visual/markup change in llm-select.gts reads cleanly and I have nothing to add there.

Bottom line: The tier logic and picker wiring are sound and well-tested, and the restartableTask design correctly sidesteps a real reactivity trap rather than papering over it. Nothing I found blocks on its own; the requested-changes call is the maintainer's, and it turns on the architectural question below. Two small items: a robustness gap in parsePrice (inline), and the branch currently has a merge conflict in environment.js (GitHub reports the PR as un-mergeable).

Answering the open review — "any reason not to make this a computed on OpenRouterModel or ModelConfiguration?"

The two candidate homes are not equivalent, and neither removes the machinery in room.gts. Laying out the mechanics so the decision is on the facts:

• ModelConfiguration — not viable as a computeVia. ModelConfiguration (in packages/base/system-card.gts) carries only modelId, toolsSupported, reasoningEffort, inputModalities — no pricing, and no link to OpenRouterModel. The two cards are correlated by the modelId string alone. A computed there would have to do an async cross-realm lookup to find the matching catalog card, which computeVia can't express.

• OpenRouterModel — idiomatic and a genuine improvement for the math. Pricing lives on this card, and it already computes toolsSupported, inputModalities, and cardTitle via computeVia from its own fields, so a costTier/costTierLabel computed alongside them is right at home. The win is real: the tier becomes an indexed search-doc field (queryable/sortable, and the ~40 lines of model-cost.ts derivation move onto the card and out of the host), and the host's search would just read instance.costTier instead of reading pricing and computing.

• But it does not eliminate the task/search in room.gts. The reactivity problem this PR solves is about running store.search inside a reactive computation — not about where the tier arithmetic lives. Because picker rows are ModelConfiguration cards / fallback id strings (not OpenRouterModel cards) and the only correlation is the modelId string with no link, the host still has to search the OpenRouter realm by modelId and join client-side. So the restartableTask + content-key gating stays regardless of where the derivation lives.

Net: a computed on OpenRouterModel is the better home for the derivation and matches the existing computeds; a computed on ModelConfiguration isn't achievable without a schema change (a linksTo OpenRouterModel). The reactivity design is orthogonal to that choice and is sound either way. My suggestion, for what it's worth: move the tier math to an OpenRouterModel computed (delete model-cost.ts's role in the host, have the task read the computed field), and keep the task exactly as written.

What lands right

• The reactivity design is the hard part and it's done correctly — not a trackedFunction that would loop forever. Detail and the two load-bearing invariants are in the inline note on the task.
• Graceful degradation is real, not asserted. Unconfigured realm, uncatalogued model, and a failing catalog search all resolve to "no badge," and the failure path deliberately avoids writing the tracked map so it can't spin into a retry loop while the catalog is down. The integration tests exercise all three branches.
• The pure tier logic is well-specified and well-tested — the 3:1 blend, inclusive fixed bounds, and the unknown-vs-Free distinction are all pinned by unit tests, and I re-derived the representative-model cases (e.g. sonnet 0.000003/0.000015 → blended 6 → $$$; o1 → 26.25 → $$$$) — they check out.

Recommendations

1. Decide the architectural question above — an OpenRouterModel.costTier computed is the clean move for the derivation; ModelConfiguration isn't viable without a link. (Body, above.)
2. Harden parsePrice against numeric-prefixed / negative strings so it matches the file's own stated contract — Number() + a >= 0 guard. Non-blocking; not reachable with today's OpenRouter data. (Inline on model-cost.ts; this is the Codex P2, verified.)
3. Optionally add a negative-space unit test for a numeric-prefix price string, and a one-line reactivity caveat near llmsForSelectMenu. (Inline notes.)

Adjacent, out of scope

• Merge conflict. The branch conflicts with main in packages/host/config/environment.js (both sides changed it since the branch point) — needs a merge/rebase before it can land. Purely mechanical, not a code issue.
• Badge staleness within a session — the load's content key omits pricing, so a mid-session catalog price change won't refresh a badge. Reasonable for a cosmetic badge; called out in the inline note so it's a conscious trade.

---

Generated by Claude Code

[habdelra]

[Claude Code 🤖] Non-blocking (robustness). parseFloat accepts a leading numeric prefix and returns a finite number for inputs the file header intends to treat as unknown — so parsePrice can classify a malformed price as valid and produce a wrong badge, which the top-of-file comment explicitly says it exists to prevent ("a present-yet-unparseable price makes the whole cost unknown — better no badge than a wrong tier").

Verified (Node):

• parseFloat("0.000003 USD") → 0.000003 (finite → treated as a real price)
• parseFloat("1x") → 1
• parseFloat("-0.001") → -0.001 (finite) → blendedCostPerMillion goes negative → modelCostTier hits the blended <= 0 branch and renders Free for a negative price.

Number() rejects all three (Number("0.000003 USD") / Number("1x") → NaN), so the header's contract is met by construction. A >= 0 guard also closes the negative-price → Free path:

Suggested change

	let n = parseFloat(price);
	return Number.isFinite(n) ? n : 'invalid';
	let n = Number(price);
	return Number.isFinite(n) && n >= 0 ? n : 'invalid';

'' and null are already returned as 'absent' above this line, so Number('') → 0 never reaches here.

Two caveats so this is fairly scoped:

• Not reachable with real data today. OpenRouter emits clean decimal strings ("0.0000006", "0"), so no live model trips this — it's defensive, not a live bug. This is the finding Codex raised; the above is the verification.
• Consistent with existing style. openrouter-model.gts's formatPrice also uses parseFloat, so keeping parseFloat here wouldn't be out of place — but this file is the one that documents the stricter contract, which is why Number() fits it better.

Test-coverage note (non-blocking): the "unparseable → undefined" unit test only covers fully non-numeric strings ('n/a', 'abc', 'garbage'), which parseFloat already rejects. A numeric-prefix case like modelCostTierLabel('0.000003 USD', '0') → undefined would pin the exact behavior the header promises and would fail against the current parseFloat.

---

Generated by Claude Code

[habdelra]

[Claude Code 🤖] Confirmation (nothing to change) + one limitation worth naming.

The choice to run this as a restartableTask gated on a plain (untracked) content key, rather than a trackedFunction/resource, is correct and worth keeping. this.store.search(...) hydrates cards — which mutates tracked store state — and the tier derivation then reads those hydrated cards' pricing fields; inside a reactive computation that read-after-write self-invalidates and never settles, so the badge would never appear. Two guard rails keep the current shape safe, and it's worth being explicit about them because they're load-bearing:

1. The only tracked write, this.modelCostTierByModelId = tiers, happens after await this.store.search(...) — i.e. in a later microtask — so even though llmsForSelectMenu reads that same tracked field, the write lands in a separate runloop and can't trip Ember's backtracking-rerender assertion.
2. lastModelCostTierKey is deliberately untracked, so setting it on the synchronous render path (ensureModelCostTiersLoaded) invalidates nothing and can't retrigger the getter.

This stops being safe the moment either (a) llmsForSelectMenu reads any of the task's own tracked state (this.loadModelCostTiers.isRunning / .last / .performCount), or (b) anything on the synchronous path in ensureModelCostTiersLoaded writes tracked state. A one-line note to that effect near the getter would help the next editor who's tempted to add a spinner.

Limitation to name (non-blocking): the content key is realm + sorted model ids only — pricing is not part of it. So if the OpenRouter catalog updates a model's pricing while the model-id set stays the same, the badge won't refresh within the session (only a new/removed model id, or a page reload, forces a re-derive). Given badges are cosmetic and catalog pricing rarely moves mid-session, that's a reasonable trade — flagging it so it reads as a deliberate choice rather than an oversight.

---

Generated by Claude Code

[FadhlanR]

[Claude Code 🤖] Good call — I moved the derivation onto OpenRouterModel as computeVia fields (costTier + costTierLabel), reading its own pricing, right alongside the existing toolsSupported / inputModalities / cardTitle computeds. The tier is now an indexed, queryable/sortable field and the host-side math is gone — the picker just reads instance.costTierLabel off the catalog search.

On the two candidate homes:

ModelConfiguration — deliberately not. Why we treat price differently from toolsSupported, even though both could be plain fields there, comes down to ownership and staleness:

• toolsSupported is a configuration decision the config card owns — a policy ("should we send tools to this model?") a curator sets and may deliberately override. Low-volatility, authored. Note it isn't synced from anywhere today; it's hand-set in each ModelConfiguration.
• price is externally-owned live data — OpenRouter is the single source of truth, it changes without anyone touching the config, and a curator has no reason to author or override it. Copying it onto ModelConfiguration makes a snapshot that silently drifts, and this feature's contract is explicitly "better no badge than a wrong tier" — so a stale price badge is exactly the failure mode to avoid.
• There's also no machinery to hang it on: there's no OpenRouterModel → ModelConfiguration sync anywhere (they only correlate by the modelId string), so it wouldn't be "one more synced field" — it'd mean building that bridge. If we ever want pricing on ModelConfiguration, the right shape is a linksTo OpenRouterModel so it stays live — a bigger modeling change than this badge warrants.

OpenRouterModel — done, and the right home for the math. Pricing already lives here and the card already derives its own computeds, so costTier is idiomatic. It doesn't remove the small host-side search (picker rows are ModelConfiguration / fallback ids with no link, so the host still looks up the catalog by modelId) — but that search now just reads the computed field.

Also hardened parsePrice (Number() + a >= 0 guard) so numeric-prefixed/negative strings read as unknown rather than a wrong tier, per the Codex note.

[lukemelia]

[Claude Code 🤖] Posting a recommendation on behalf of Luke:

On the model-cost placement — since OpenRouterModel is the only real consumer of the cost-tier math (the picker in room.gts just reads the derived costTierLabel field off catalog search results), I'd lean toward co-locating the logic in the card module rather than keeping it in runtime-common. Most people never touch this card, so the concern is best localized to it instead of parking a domain-specific pricing helper in the shared package everyone loads and has to skim past.

To keep coverage, the idiomatic path is a co-located live test (openrouter-model.test.gts exporting runTests()), which can import the card's exports directly via a relative import.

One important caveat if we go this way: the CI live-test job (ci-host.yaml) discovers *.test.gts from only the single realm in REALM_URL (currently the skills realm), so a test in the openrouter realm won't run in CI unless we also add that realm to the live-test matrix. We should do both in the same change — otherwise the banding edge cases (NaN / negative price / the 3:1 blend) would pass locally but silently never run in CI, which is worse than no test.

Not a blocker on this PR — the current runtime-common + fast unit-test approach is a perfectly fine default. This is a direction for a follow-up if/when we want the concern fully localized to the card.

[lukemelia]

See my comment about moving model-cost.ts' code out of runtime-common and into the realm.