Skip to content

feat: cross-run lineage via reserved run attributes#2153

Draft
rchasman wants to merge 7 commits into
vercel:mainfrom
rchasman:feat/cross-run-lineage-rootid
Draft

feat: cross-run lineage via reserved run attributes#2153
rchasman wants to merge 7 commits into
vercel:mainfrom
rchasman:feat/cross-run-lineage-rootid

Conversation

@rchasman
Copy link
Copy Markdown

@rchasman rchasman commented May 29, 2026

Problem

When one run starts another (a recurring schedule that re-arms itself, a fan-out, any daisy chain), the runs have no relationship to each other, so you cannot list or cancel a chain as a unit. I hit this building a scheduler on Workflow.

Approach

Two layers on the attributes mechanism from #2088:

  1. Attributes you can set at creation and filter by. Set them when you start a run; list({ attributes }) returns runs holding every pair. Tag related runs, list them as a group.
  2. Lineage, built on layer 1: start() records reserved $rootRunId / $parentRunId, so a daisy chain of any depth groups under one root, listable and cancellable via list({ attributes: { $rootRunId } }).

Scope (commits are ordered so this is easy)

The first four commits are layer 1: attributes-at-creation plus the list filter, across core/world/world-local/world-postgres/world-vercel. They are self-contained, complete #2088, and are useful on their own. The last two commits add the start() lineage layer and its e2e. If you would rather take the lineage auto-recording slower, dropping those two leaves a clean attributes-filter PR. Happy either way.

Caveat: the Vercel world needs the runs API to adopt attributes

world-local and world-postgres implement the whole feature. The Vercel world talks to the hosted runs API, where #2088 shipped only a write-only attributes endpoint (set yes, filter no). For this to work end to end on the hosted world, the runs API needs two additions:

  • persist attributes provided at run creation, so start()-set attributes (including $rootRunId) are stored, and
  • accept an attributes filter on the list endpoint (/v2/runs).

Both are server-side and outside this repo. Until they land, world-vercel throws on an attributes filter rather than silently returning every run, and attributes set at creation take effect only once the create path stores them. The world-testing /runs route is a reference for the filter.

Testing

core, world-local, and world-vercel suites green; an end-to-end daisy chain groups under one $rootRunId through the real runtime and queue.

@changeset-bot
Copy link
Copy Markdown

changeset-bot Bot commented May 29, 2026

🦋 Changeset detected

Latest commit: 0a03200

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 21 packages
Name Type
@workflow/world Minor
@workflow/core Minor
@workflow/world-local Minor
@workflow/world-postgres Minor
@workflow/world-vercel Patch
@workflow/cli Patch
@workflow/vitest Patch
@workflow/web-shared Patch
@workflow/web Patch
@workflow/world-testing Patch
@workflow/builders Patch
@workflow/next Patch
@workflow/nitro Patch
workflow Minor
@workflow/astro Patch
@workflow/nest Patch
@workflow/rollup Patch
@workflow/sveltekit Patch
@workflow/vite Patch
@workflow/nuxt Patch
@workflow/ai Major

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel
Copy link
Copy Markdown
Contributor

vercel Bot commented May 29, 2026

@rchasman is attempting to deploy a commit to the Vercel Labs Team on Vercel.

A member of the Team first needs to authorize it.

Copy link
Copy Markdown
Contributor

@vercel vercel Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional Suggestion:

In the resilient-start path, rootId is omitted from the run_started event's eventData, causing cross-run lineage to be lost when run_created fails and the run is bootstrapped from the queue.

Fix on Vercel

@pranaygp
Copy link
Copy Markdown
Contributor

since Run is serialized, returning a run already gets handled in observability and you can click the run ID to go straight to the next run ID

now that #2088 has been shipped in v5 too, you can also use attributes to have parent and child runs link to each other by setting the run IDs as you want. We could have a reserved $parentRunId attribute too for lineage tracking

wdyt @rchasman ?

@rchasman rchasman force-pushed the feat/cross-run-lineage-rootid branch from bb30012 to e0b62e9 Compare May 29, 2026 05:07
@rchasman rchasman changed the title feat: cross-run lineage via propagated rootId feat: cross-run lineage via reserved run attributes May 29, 2026
@rchasman
Copy link
Copy Markdown
Author

Re-spun onto attributes per your suggestion. start() records $rootId and $parentRunId as reserved attributes (#2088); the one addition that makes it useful is an attributes filter on ListWorkflowRunsParams, so a lineage is queryable: list({ attributes: { $rootId } }).

Good call.. Going through attributes turned out cleaner than the column: lineage is preserved across lifecycle updates for free (attributes already carry forward), and the same filter generalizes to any attribute, not just lineage.

I record both $parentRunId (the edge you mentioned, and the basis for a tree view later) and $rootId (the group key, so grouping is one filter instead of an N-hop walk).

Also fixed the resilient-start path the review bot flagged: attributes now flow through run_started too.

Proven end-to-end with a real daisy-chain; unit + e2e green, existing suites unaffected. Still world-local only and one runs.get per nested start that could thread through the step context for zero I/O, both noted in the PR.

rchasman added 7 commits May 30, 2026 14:28
Completes the attributes feature from vercel#2088. Attributes can now be set when
a run is created (start options, CreateWorkflowRunRequest, the run_created and
resilient run_started event data, and queue RunInput) and queried via an
attributes filter on ListWorkflowRunsParams, so related runs can be tagged and
listed as a group.

Signed-off-by: Roey D. Chasman <rchasman@gmail.com>
Store attributes passed at run creation and honor the attributes filter in
runs.list (match runs holding every requested key/value). Because they ride on
the attributes mechanism, they are preserved across lifecycle updates for free.

Signed-off-by: Roey D. Chasman <rchasman@gmail.com>
Honor the attributes filter via jsonb containment (@>), matching world-local.
Without it the param was silently ignored and the query returned every run.

Signed-off-by: Roey D. Chasman <rchasman@gmail.com>
Attribute filtering executes server-side in the platform /v2/runs endpoint,
which does not accept it yet. Forwarding the filter would be silently ignored
and return every run, so throw instead of returning wrong results.

Signed-off-by: Roey D. Chasman <rchasman@gmail.com>
start() records reserved $rootRunId and $parentRunId attributes on each run: a
run with no parent is its own root, and a run started from inside another run
inherits the parent's $rootRunId, so a daisy chain of any depth groups under one
root. Built on the attributes filter above, so the chain is listable and
cancellable as a unit via list({ attributes: { $rootRunId } }).

Signed-off-by: Roey D. Chasman <rchasman@gmail.com>
Prove the full path: a real daisy-chain through the runtime and queue groups
under one $rootRunId, listable via the attributes filter, with an unrelated run
excluded. The /runs route translates ?rootRunId= into the attribute filter.

Signed-off-by: Roey D. Chasman <rchasman@gmail.com>
Signed-off-by: Roey D. Chasman <rchasman@gmail.com>
@rchasman rchasman force-pushed the feat/cross-run-lineage-rootid branch from e0b62e9 to 0a03200 Compare May 30, 2026 22:48
@rchasman
Copy link
Copy Markdown
Author

@pranaygp here is where the PR landed and why, since the purpose evolved into two clear layers:

Layer 1: make attributes filterable. #2088 lets you set attributes mid-run, but not at creation, and there is no way to query by them. This adds both: set attributes when you start a run, and filter list() by them. On its own that completes #2088. You can tag related runs (tenant, user, ...) and list them as a group, no lineage concept required.

Layer 2: reserved lineage params. The original PR did this with a dedicated rootId column. On your nudge I moved it onto the attributes mechanism: start() now records $rootRunId (the root grouping) and $parentRunId (the parent edge you suggested exposing as a reserved attribute too), so a chain of any depth groups under one id, listable and cancellable via the layer-1 filter.

I ordered the commits so layer 1 lands first and layer 2 is the last two. If you would rather take the reserved lineage params slower, dropping those two leaves a clean attributes-filter PR. Either way the filter is the foundation and the lineage is a reserved convention on top of it.

(world-local and world-postgres implement the filter; world-vercel throws until the hosted runs API accepts attributes at creation and on list, noted in the description.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants