[HDX-5081] Formula support across API surfaces

[wrn14897]

Summary

Stacked on #2953 ([HDX-5132] event-source formulas) — base is warren/formula-support-for-events-metrics. Once #2953 merges, this branch will be rebased onto main and retargeted.

Plumbs chart formula configs (HDX-5078/5079/5080/5132) through every remaining surface that persists or accepts chart configs, so formulas created anywhere round-trip and query correctly everywhere — on metric and log/trace event sources.

External dashboards API v2

• formulas (letter-ref arithmetic over the tile's select items, e.g. A / (A + B) * 100) and showOperandSeries are accepted on line / stacked_bar / table / number builder tiles and round-trip through POST → GET → PUT.
• Number tiles follow the editor's contract: with formulas they carry multiple operand select items, support a single formula, and persist showOperandSeries: false internally (never exposed on the external surface).
• Write-time validation via a new shared validateChartConfigFormulas helper in common-utils dashboardValidation (same rules as the chart editor's validateChartForm): expression parse + series-ref range checks, asRatio mutual exclusion, number-tile single-formula cap.
• Source-kind gate: formula tiles must reference a metric, log, or trace source, via the shared isFormulaSourceKind predicate (moved from the chart editor into common-utils core/utils, app re-exports — the editor gating and the server gates cannot drift). Other kinds (e.g. session) are rejected. On updates the gate is scoped to changed formula tiles (mirroring the heatmap gate), so a source whose kind changed after acceptance doesn't block unrelated dashboard edits.
• OpenAPI spec regenerated with the new Formula component schema.

MCP

• clickstack_save_dashboard / clickstack_patch_dashboard tile schemas declare formulas / showOperandSeries with LLM-facing descriptions (previously the strict Zod schemas silently stripped them).
• clickstack_patch_dashboard re-validates the merged tile through the external tile schema, so patches can no longer persist configs an equivalent create/full-update rejects (also closes a pre-existing bypass of the asRatio/two-select rule).
• clickstack_query_tile computes formula columns for both the composed metric path and the inline event path.
• The dashboards query-guide prompt gains a == FORMULAS == section with metric and log/trace examples.

Alerts

• Alert-on-formula-tile support landed with HDX-5080 ([HDX-5080] Chart editor UI for metric formulas #2909); this PR adds a checkAlerts int test for an event (log) formula tile proving the threshold evaluates the formula value (25 = 1 error / 4 total × 100), not an operand.

CLI

• shared/tileConfig.ts delegated its stale local convertToNumberChartConfig / convertToTableChartConfig ports to the shared formula-aware common-utils implementations (the web already imports these), so formula tiles render identically to the web — including operand hiding on number tiles. packages/cli/AGENTS.md pipeline notes updated.

How to test on Vercel preview

N/A — non-UI change (API/MCP/CLI surfaces only; the packages/app edits are behavior-identical re-exports).

Testing

• make ci-lint, make ci-unit — green across all packages (incl. [HDX-5132] Formula support for log/trace event sources #2953's editor tests, which now consume the re-exported predicates).
• Integration (via make dev-int):
  • external-api/__tests__/dashboards.int.test.ts — 187 passed, incl. a Metric formulas (HDX-5081) suite: per-display-type round-trips (metric + trace event source), PUT preservation, source-kind-flip scoping, and rejections (unknown series ref, malformed expression, asRatio + formulas, multiple formulas on number, multi-select number without formulas, formulas on a session source).
  • mcp/__tests__/dashboards — 131 passed, incl. saving formula tiles through MCP, query_tile computing the metric formula (10 / 200 × 100 = 5) and the log event formula (3 / 12 × 100 = 25) with operands hidden, patch-path validation parity, and session-source rejection.
  • tasks/checkAlerts/__tests__/checkAlerts.int.test.ts — 173 passed, incl. the new event-formula tile alert.
• New common-utils unit tests for validateChartConfigFormulas.

References

• Linear Issue: HDX-5081
• Related PRs: [HDX-5132] Formula support for log/trace event sources #2953 (HDX-5132 event-source formulas — base of this stack), [HDX-5078] Formula expression model: schema + parser/validator #2872 (HDX-5078 schema/parser), [HDX-5079] Render formulas in the composed metric query #2908 (HDX-5079 query rendering), [HDX-5080] Chart editor UI for metric formulas #2909 (HDX-5080 editor UI + alert support)

[changeset-bot]

🦋 Changeset detected

Latest commit: ddfd42d

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 5 packages

Name	Type
@hyperdx/common-utils	Minor
@hyperdx/api	Minor
@hyperdx/cli	Patch
@hyperdx/app	Minor
@hyperdx/otel-collector	Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
hyperdx-oss	Ready	Preview	Aug 21, 2026 3:14pm
hyperdx-storybook	Ready	Preview	Aug 21, 2026 3:14pm

[greptile-apps]

Greptile Summary

The PR extends formula support through external dashboard APIs, MCP dashboard tools, alert evaluation, and CLI chart rendering.

• Adds shared formula expression, operand-reference, display-type, and source-kind validation.
• Preserves formula configuration across dashboard create, read, update, patch, and query paths.
• Revalidates merged MCP tile patches through the external tile schema.
• Scopes update-time source-kind checks to formula tiles whose formulas or source changed.
• Adds OpenAPI documentation and integration coverage for metric and event-source formulas.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains; the MCP patch path now rejects invalid formula configurations before persistence, and unchanged formula tiles no longer block unrelated dashboard updates after a source-kind change.

Important Files Changed

Filename	Overview
packages/api/src/mcp/tools/dashboards/patchDashboard.ts	Revalidates merged tile patches with the external schema before shared validation and persistence, resolving the prior formula-validation bypass.
packages/api/src/routers/external-api/v2/utils/dashboards.ts	Round-trips formula fields and limits update-time source-kind validation to new or changed formula tiles.
packages/api/src/utils/zod.ts	Adds formula fields and shared cross-field validation to supported external builder tile schemas.
packages/common-utils/src/dashboardValidation.ts	Centralizes formula parsing, operand-reference, asRatio, and number-tile validation.
packages/api/src/mcp/tools/dashboards/schemas.ts	Exposes formula configuration in MCP schemas while leaving cross-field enforcement to the external tile schema.
packages/cli/src/shared/tileConfig.ts	Delegates number and table transformations to shared formula-aware chart utilities.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
    Input[External API or MCP tile input] --> Schema[External tile schema]
    Schema --> FormulaValidation[Shared formula validation]
    FormulaValidation --> SourceGate[Formula source-kind gate]
    SourceGate --> Persist[(Dashboard configuration)]
    Persist --> Query[Metric or event query path]
    Query --> FormulaResult[Computed formula series]
    Persist --> Alerts[Alert evaluation]
    Persist --> CLI[CLI chart rendering]

Loading

_{Reviews (9): Last reviewed commit: "chore: simplify comments and drop ticket..." | Re-trigger Greptile}

[github-actions]

E2E Test Results

✅ All tests passed • 308 passed • 1 skipped • 1058s

Status	Count
✅ Passed	308
❌ Failed	0
⚠️ Flaky	0
⏭️ Skipped	1

Tests ran across 4 shards in parallel.

View full report →

[github-actions]

🔴 Tier 4 — Critical

Touches authentication, tenancy data models, the public API or shipped database config — or substantially changes the query rendering engine, background tasks, the OTel pipeline, image build, or release CI.

Why this tier:

• Critical-path files (2) — tenancy, public API, or shipped database config:
  • packages/api/src/routers/external-api/v2/dashboards.ts
  • packages/api/src/routers/external-api/v2/utils/dashboards.ts
• Cross-layer change: touches frontend (packages/app) + backend (packages/api) + shared utils (packages/common-utils)

Review process: Deep review from a domain expert. Synchronous walkthrough may be required.
SLA: Schedule synchronous review within 2 business days.

Stats

• Production files changed: 11
• Production lines changed: 690 (+ 1003 in test files, excluded from tier calculation)
• Critical-path lines changed: 242
• Branch: warren/HDX-5081-formula-api-surfaces
• Author: wrn14897

To override this classification, remove the review/tier-4 label and apply a different review/tier-* label. Manual overrides are preserved on subsequent pushes.

[github-actions]

Deep Review

✅ No critical issues found. This is a well-constructed, contract-disciplined change. Formula validation is single-sourced through validateChartConfigFormulas, every MCP write path funnels through the same external superRefine (including the new patch re-parse that closes a real bypass), user expressions reach SQL only via a parsed/validated AST (never raw interpolation), and the round-trip serialization for line/stacked_bar/table/number checks out. No P0/P1 issues. One P2 test gap and a set of P3 polish items follow.

🟡 P2 — recommended

• packages/api/src/routers/external-api/v2/utils/dashboards.ts:1146 — The filterChangedFormulaTiles branch that re-validates when only sourceId changes (formulas byte-identical) — the asymmetry that is the entire purpose of this helper — has no test; the sole update-scoping test re-enters the gate by changing formulas, exercising a different branch.
  • Fix: Add a PUT integration test that keeps a formula tile's formulas identical but repoints it to a formula-incapable source and asserts a 400, plus a companion repointing to a compatible source asserting 200.
  • _{testing, correctness}

🔵 P3 nitpicks (12)

• packages/api/src/utils/zod.ts:576 — Formulas posted on unsupported builder display types (pie/bar/heatmap/etc.) are silently stripped by the discriminated-union re-parse rather than rejected, and an invalid formula there raises a parse error against a field that then disappears; the shared isFormulaDisplayType gate exists but is never invoked server-side.
  • Fix: Call isFormulaDisplayType(displayType) in the tile-config refinement and emit an explicit "formulas are not supported on this display type" issue before running validateChartConfigFormulas.
  • _correctness
• packages/api/src/utils/zod.ts:262 — The external formulas cap (.max(10)) is stricter than the unbounded internal SharedChartSettingsSchema.formulas, so a line/stacked_bar/table tile authored in the editor with >10 formulas serializes on GET but is rejected when that same body is submitted via PUT, breaking round-trip.
  • Fix: Align the caps — add a matching .max(10) (plus editor guard) to the internal schema, or raise the external cap so any editor-persistable config round-trips.
  • _api-contract
• packages/api/src/utils/zod.ts:386 — The number-tile schema reuses the shared externalFormulasSchema (.max(10)) while the OpenAPI NumberBuilderChartConfig.formulas documents maxItems: 1; the effective single-formula cap comes only from validateChartConfigFormulas, so zod.ts and the spec disagree at the array level.
  • Fix: Give the number tile a dedicated .max(1) formulas schema so the Zod array bound matches the documented contract.
  • _api-contract
• packages/api/src/mcp/tools/dashboards/schemas.ts:273 — mcpFormulaSchema hand-rolls expression/alias/numberFormat and re-hardcodes .max(1024) instead of deriving from MetricFormulaSchema, contradicting the "cannot drift" reuse this PR applies on the external surface.
  • Fix: Derive the MCP formula schema from MetricFormulaSchema (or at least share the max-length constant) so the persisted shape cannot diverge on the MCP surface.
  • _{kieran-typescript}
• packages/api/src/routers/external-api/v2/utils/dashboards.ts:1112 — filterChangedFormulaTiles is a near-verbatim copy of filterChangedHeatmapTiles (same map-build, guard, existing-lookup, and source-compare skeleton); two copies of this subtle gate-change-detection algorithm will drift.
  • Fix: Extract a shared filterChangedGatedTiles(requestTiles, existingTiles, { matches, changed }) helper and implement both variants on top of it.
  • _{maintainability}
• packages/api/src/routers/external-api/v2/utils/dashboards.ts:1076 — getFormulaTilesWithIncompatibleSources is a near-verbatim copy of getHeatmapTilesWithIncompatibleSources; only the tile-match and source-compat predicates differ.
  • Fix: Extract getTilesWithIncompatibleSources(sources, tiles, { matches, isCompatible }) and pass predicates from each caller.
  • _{maintainability}
• packages/cli/src/shared/tileConfig.ts:4 — The file-header @source still points convertToNumberChartConfig/convertToTableChartConfig at ChartUtils.tsx, but after this PR both delegate to the common-utils transforms; the per-function docblocks were updated, the header was not.
  • Fix: Update the header to note the number/table transforms delegate to @hyperdx/common-utils/dist/core/utils.
  • _{maintainability}
• packages/api/src/routers/external-api/v2/utils/dashboards.ts:1134 — The !isBuilderSavedChartConfig(existingConfig) branch (raw-SQL/PromQL tile converted to a builder tile carrying formulas) is not exercised by any update test.
  • Fix: Add a PUT test that replaces a raw-SQL tile (same id) with a builder formula tile on an incompatible source and asserts rejection.
  • _testing
• packages/api/src/routers/external-api/v2/utils/dashboards.ts:1129 — The existing === undefined branch (a brand-new formula tile added during a PUT) is only reachable on update and is untested.
  • Fix: Add a PUT test appending a new formula tile on an incompatible source and asserting rejection.
  • _testing
• packages/api/src/mcp/prompts/dashboards/content.ts:55 — buildCreateDashboardPrompt, which drives the clickstack_save_dashboard workflow, never mentions formulas; all guidance lives in the query-guide prompt, so an agent on the create path discovers formulas only via the tool field description.
  • Fix: Add a short FORMULAS note or cross-reference to the create prompt's tile-type guide, mirroring how asRatio is surfaced.
  • _agent-native
• packages/api/src/mcp/tools/dashboards/schemas.ts:296 — The formulas description and prompt rules omit the .max(10) cap and the constant-only-expression rejection, so an LLM learns them only by round-tripping through a runtime error.
  • Fix: Add one line noting "up to 10 formulas; every expression must reference at least one series letter."
  • _agent-native
• packages/common-utils/src/core/renderChartConfig.ts:818 (pre-existing; not introduced by this diff) — A select-item alias is emitted as a SQL column name via UNSAFE_RAW_SQL without quote-doubling, so an alias containing a double-quote breaks out of the quoted identifier; the new formula alias/expression path correctly routes through quotedColumnName and is safe.
  • Fix: Route the select-item alias through quotedColumnName (or the parameterized { Identifier } form) instead of UNSAFE_RAW_SQL.
  • _security

---

Reviewers (9): correctness, security, api-contract, testing, maintainability, project-standards, agent-native, kieran-typescript, learnings-researcher. The adversarial reviewer was dispatched but did not return (output stalled shortly after launch); its scope — round-trip fidelity, source-kind-gate bypass, and number-tile edge cases — was independently spot-checked during synthesis and corroborated the security/correctness conclusions (no injection path; gate is a feature gate, not a tenant boundary; number-tile select branching has no data-loss path).

Testing gaps:

• filterChangedFormulaTiles: the source-only-change and raw-SQL→builder-conversion branches (the ones unique to the update path vs. create) lack exercising tests.
• No test covers a line/stacked_bar/table formula tile with showOperandSeries left unset (default-true render path); only the false path is exercised.
• No test asserts a select-item or formula alias containing a double-quote is safely escaped in generated SQL.
• No test that a number tile carrying showOperandSeries in the request body is silently stripped (rather than rejected), locking in that behavior as intentional.

Note: OpenAPI regeneration byte-sync could not be verified in-environment (yarn docgen blocked); CI lint:openapi regenerates and lints, so drift would surface there. @hyperdx/app is edited but absent from the changeset — defensible as a behavior-preserving re-export refactor under the "no release warranted" exception.

[jordan-simonovski]

LGTM! Nice work on this one.

[pulpdrew]

Stamp