Skip to content

[e2e] event-log-race-repro: classify polled run failures from structured error#2196

Draft
VaguelySerious wants to merge 2 commits into
stablefrom
peter/repro-classify-failed-runs
Draft

[e2e] event-log-race-repro: classify polled run failures from structured error#2196
VaguelySerious wants to merge 2 commits into
stablefrom
peter/repro-classify-failed-runs

Conversation

@VaguelySerious
Copy link
Copy Markdown
Member

Follow-up to #2195 (the classify fix landed on the branch just after #2195 was squash-merged, so it didn't make it onto stable).

A failed WorkflowRun exposes its reason as error: { code, message } and has no top-level errorCode. The repro poller was calling classifyFailure(runData.errorCode) — always undefined — so every polled run failure collapsed to an uncategorised, detail-less other (gating, but with nothing to act on).

This reads runData.error.code / .message, so:

  • USER_ERROR / RUNTIME_ERROR / CORRUPTED_EVENT_LOG are classified correctly (vs. lumped into other).
  • The 🚨 regression row shows why the run failed instead of just "uncategorised".

Test-only; empty changeset.

🤖 Generated with Claude Code

VaguelySerious and others added 2 commits June 1, 2026 12:49
…issing field

A failed WorkflowRun exposes its reason as `error: { code, message }` and has no
top-level `errorCode`, so the poller's `classifyFailure(runData.errorCode)` was
always passing `undefined` — collapsing every polled failure to an
uncategorised, detail-less `other`. Read `runData.error.code`/`.message` so
USER_ERROR/RUNTIME_ERROR/CORRUPTED_EVENT_LOG are classified correctly and the
regression row shows why the run failed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(cherry picked from commit d2a59d4)
@changeset-bot
Copy link
Copy Markdown

changeset-bot Bot commented Jun 1, 2026

🦋 Changeset detected

Latest commit: 7100ccb

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 0 packages

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 1, 2026

🧪 E2E Test Results

Some tests failed

Summary

Passed Failed Skipped Total
❌ ▲ Vercel Production 899 2 67 968
✅ 💻 Local Development 970 0 86 1056
✅ 📦 Local Production 970 0 86 1056
✅ 🐘 Local Postgres 970 0 86 1056
✅ 🪟 Windows 88 0 0 88
❌ 🌍 Community Worlds 134 88 0 222
✅ 📋 Other 492 0 36 528
Total 4523 90 361 4974

❌ Failed Tests

▲ Vercel Production (2 failed)

hono (1 failed):

vite (1 failed):

🌍 Community Worlds (88 failed)

mongodb (12 failed):

  • hookWorkflow is not resumable via public webhook endpoint | wrun_01KT1D0KYZASJBB8NRN9BXVHQK
  • webhookWorkflow | wrun_01KT1D0RST3GDTHQ91FZF7QW3N
  • sleepingWorkflow | wrun_01KT1D2EMJREEDA6TVG4N42ZCY
  • outputStreamWorkflow no startIndex (reads all chunks)
  • outputStreamWorkflow negative startIndex (reads from end)
  • outputStreamWorkflow - getTailIndex and getStreamChunks getTailIndex returns correct index after stream completes
  • outputStreamWorkflow - getTailIndex and getStreamChunks getTailIndex returns -1 before any chunks are written
  • outputStreamWorkflow - getTailIndex and getStreamChunks getStreamChunks returns same content as reading the stream
  • outputStreamInsideStepWorkflow - getWritable() called inside step functions | wrun_01KT1D5FAXNM1GDEB9F5BTA41W
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_01KT1D9SDDKVNS21CAVM4FY8FH
  • pages router sleepingWorkflow via pages router
  • resilient start: addTenWorkflow completes when run_created returns 500 | wrun_01KT1DFR3S8PTQZPYP3VB5W4KD

redis (9 failed):

  • hookWorkflow is not resumable via public webhook endpoint | wrun_01KT1D0KYZASJBB8NRN9BXVHQK
  • sleepingWorkflow | wrun_01KT1D2EMJREEDA6TVG4N42ZCY
  • outputStreamWorkflow negative startIndex (reads from end)
  • outputStreamWorkflow - getTailIndex and getStreamChunks getTailIndex returns correct index after stream completes
  • outputStreamWorkflow - getTailIndex and getStreamChunks getTailIndex returns -1 before any chunks are written
  • outputStreamWorkflow - getTailIndex and getStreamChunks getStreamChunks returns same content as reading the stream
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_01KT1D9SDDKVNS21CAVM4FY8FH
  • pages router sleepingWorkflow via pages router
  • resilient start: addTenWorkflow completes when run_created returns 500 | wrun_01KT1DFR3S8PTQZPYP3VB5W4KD

turso-dev (1 failed):

  • dev e2e should rebuild on imported step dependency change

turso (66 failed):

  • addTenWorkflow | wrun_01KT1CZGDVYV8YB9R3JNH5K7GF
  • addTenWorkflow | wrun_01KT1CZGDVYV8YB9R3JNH5K7GF
  • wellKnownAgentWorkflow (.well-known/agent) | wrun_01KT1D1KAQAFBT9JMW1WSAKMVT
  • should work with react rendering in step
  • promiseAllWorkflow | wrun_01KT1CZPMYPKFGM9PR80YE4EHD
  • promiseRaceWorkflow | wrun_01KT1CZWB6YVJCAXFX2VYSBQQ8
  • promiseAnyWorkflow | wrun_01KT1CZYDHRXZ88MSC41A90G8F
  • importedStepOnlyWorkflow | wrun_01KT1D223E5VJG9XVG04MPQ370
  • readableStreamWorkflow | wrun_01KT1D00BHY0N42H1NFBGHXDPQ
  • hookWorkflow | wrun_01KT1D0CVXMHZCA7CCJ6205BQY
  • hookWorkflow is not resumable via public webhook endpoint | wrun_01KT1D0KYZASJBB8NRN9BXVHQK
  • webhookWorkflow | wrun_01KT1D0RST3GDTHQ91FZF7QW3N
  • sleepingWorkflow | wrun_01KT1D2EMJREEDA6TVG4N42ZCY
  • parallelSleepWorkflow | wrun_01KT1D2XWXBBRTKHJPDJV92RMY
  • nullByteWorkflow | wrun_01KT1D32QK510AK41SZYY6AYP7
  • workflowAndStepMetadataWorkflow | wrun_01KT1D34Q5WH5Q2Y3H0V5H5QMK
  • outputStreamWorkflow no startIndex (reads all chunks)
  • outputStreamWorkflow positive startIndex (skips first chunk)
  • outputStreamWorkflow negative startIndex (reads from end)
  • outputStreamWorkflow - getTailIndex and getStreamChunks getTailIndex returns correct index after stream completes
  • outputStreamWorkflow - getTailIndex and getStreamChunks getTailIndex returns -1 before any chunks are written
  • outputStreamWorkflow - getTailIndex and getStreamChunks getStreamChunks returns same content as reading the stream
  • outputStreamInsideStepWorkflow - getWritable() called inside step functions | wrun_01KT1D5FAXNM1GDEB9F5BTA41W
  • fetchWorkflow | wrun_01KT1D5YHK74ZXN0PQ180H2PPT
  • promiseRaceStressTestWorkflow | wrun_01KT1D61NCHT0MGZY1MXK0SAK2
  • error handling error propagation workflow errors nested function calls preserve message and stack trace
  • error handling error propagation workflow errors cross-file imports preserve message and stack trace
  • error handling error propagation step errors basic step error preserves message and stack trace
  • error handling error propagation step errors cross-file step error preserves message and function names in stack
  • error handling retry behavior regular Error retries until success
  • error handling retry behavior FatalError fails immediately without retries
  • error handling retry behavior RetryableError respects custom retryAfter delay
  • error handling retry behavior maxRetries=0 disables retries
  • error handling catchability FatalError can be caught and detected with FatalError.is()
  • error handling not registered WorkflowNotRegisteredError fails the run when workflow does not exist
  • error handling not registered StepNotRegisteredError fails the step but workflow can catch it
  • error handling not registered StepNotRegisteredError fails the run when not caught in workflow
  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_01KT1D9DVK8BB9KPHNHTSHKKC1
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_01KT1D9SDDKVNS21CAVM4FY8FH
  • hookDisposeTestWorkflow - hook token reuse after explicit disposal while workflow still running | wrun_01KT1DA89G49C63H2NC57FEB7B
  • stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars) | wrun_01KT1DAPX5S6RKPDNMRNJ4J3KE
  • stepFunctionWithClosureWorkflow - step function with closure variables passed as argument | wrun_01KT1DAZ6T860W3VYBCDF7SQ88
  • closureVariableWorkflow - nested step functions with closure variables | wrun_01KT1DB4BG0JH53WCFDVFTHBZV
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step | wrun_01KT1DB6JJH7WSX8J7KWDJJY8Y
  • health check (queue-based) - workflow and step endpoints respond to health check messages
  • health check (CLI) - workflow health command reports healthy endpoints
  • pathsAliasWorkflow - TypeScript path aliases resolve correctly | wrun_01KT1DBN6RAKGQJDD4HXQ8NFWP
  • Calculator.calculate - static workflow method using static step methods from another class | wrun_01KT1DBTCRYNCFCWM49KTNS1WR
  • AllInOneService.processNumber - static workflow method using sibling static step methods | wrun_01KT1DC0H400RJAE0Q0C964HKR
  • ChainableService.processWithThis - static step methods using this to reference the class | wrun_01KT1DC6PWYQFNVQGEVHJ7HKDG
  • thisSerializationWorkflow - step function invoked with .call() and .apply() | wrun_01KT1DCD43XWGNGHH2ADFC8BSY
  • customSerializationWorkflow - custom class serialization with WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE | wrun_01KT1DCKD9TYECP0XD7QZZRDNT
  • instanceMethodStepWorkflow - instance methods with "use step" directive | wrun_01KT1DCSGFZWH62T3NSZ3ZMEZ6
  • crossContextSerdeWorkflow - classes defined in step code are deserializable in workflow context | wrun_01KT1DD8A6EBJ8A3TGSG6TMGPW
  • stepFunctionAsStartArgWorkflow - step function reference passed as start() argument | wrun_01KT1DDFPA3QJ33P82QYQRF985
  • cancelRun - cancelling a running workflow | wrun_01KT1DDQAF949HCHQ4R9D4MC0Z
  • cancelRun via CLI - cancelling a running workflow | wrun_01KT1DE0G61PB2J9D1CY13Y7PN
  • pages router addTenWorkflow via pages router
  • pages router promiseAllWorkflow via pages router
  • pages router sleepingWorkflow via pages router
  • hookWithSleepWorkflow - hook payloads delivered correctly with concurrent sleep | wrun_01KT1DEEN84WQ6W0C4705W56DG
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration | wrun_01KT1DF1PZV686NRS8TVJ15V1H
  • sleepWithSequentialStepsWorkflow - sequential steps work with concurrent sleep (control) | wrun_01KT1DFDCE1E6R6MDCYP1HRWP1
  • importMetaUrlWorkflow - import.meta.url is available in step bundles | wrun_01KT1DFKPYBVKF1AK6VGE0TM9X
  • metadataFromHelperWorkflow - getWorkflowMetadata/getStepMetadata work from module-level helper (#1577) | wrun_01KT1DFP58EY6JKCYA46QQMPSK
  • resilient start: addTenWorkflow completes when run_created returns 500 | wrun_01KT1DFR3S8PTQZPYP3VB5W4KD

Details by Category

❌ ▲ Vercel Production
App Passed Failed Skipped
✅ astro 81 0 7
✅ example 81 0 7
✅ express 81 0 7
✅ fastify 81 0 7
❌ hono 80 1 7
✅ nextjs-turbopack 86 0 2
✅ nextjs-webpack 86 0 2
✅ nitro 81 0 7
✅ nuxt 81 0 7
✅ sveltekit 81 0 7
❌ vite 80 1 7
✅ 💻 Local Development
App Passed Failed Skipped
✅ astro-stable 82 0 6
✅ express-stable 82 0 6
✅ fastify-stable 82 0 6
✅ hono-stable 82 0 6
✅ nextjs-turbopack-canary 69 0 19
✅ nextjs-turbopack-stable 88 0 0
✅ nextjs-webpack-canary 69 0 19
✅ nextjs-webpack-stable 88 0 0
✅ nitro-stable 82 0 6
✅ nuxt-stable 82 0 6
✅ sveltekit-stable 82 0 6
✅ vite-stable 82 0 6
✅ 📦 Local Production
App Passed Failed Skipped
✅ astro-stable 82 0 6
✅ express-stable 82 0 6
✅ fastify-stable 82 0 6
✅ hono-stable 82 0 6
✅ nextjs-turbopack-canary 69 0 19
✅ nextjs-turbopack-stable 88 0 0
✅ nextjs-webpack-canary 69 0 19
✅ nextjs-webpack-stable 88 0 0
✅ nitro-stable 82 0 6
✅ nuxt-stable 82 0 6
✅ sveltekit-stable 82 0 6
✅ vite-stable 82 0 6
✅ 🐘 Local Postgres
App Passed Failed Skipped
✅ astro-stable 82 0 6
✅ express-stable 82 0 6
✅ fastify-stable 82 0 6
✅ hono-stable 82 0 6
✅ nextjs-turbopack-canary 69 0 19
✅ nextjs-turbopack-stable 88 0 0
✅ nextjs-webpack-canary 69 0 19
✅ nextjs-webpack-stable 88 0 0
✅ nitro-stable 82 0 6
✅ nuxt-stable 82 0 6
✅ sveltekit-stable 82 0 6
✅ vite-stable 82 0 6
✅ 🪟 Windows
App Passed Failed Skipped
✅ nextjs-turbopack 88 0 0
❌ 🌍 Community Worlds
App Passed Failed Skipped
✅ mongodb-dev 5 0 0
❌ mongodb 57 12 0
✅ redis-dev 5 0 0
❌ redis 60 9 0
❌ turso-dev 4 1 0
❌ turso 3 66 0
✅ 📋 Other
App Passed Failed Skipped
✅ e2e-local-dev-nest-stable 82 0 6
✅ e2e-local-dev-tanstack-start-stable 82 0 6
✅ e2e-local-postgres-nest-stable 82 0 6
✅ e2e-local-postgres-tanstack-start-stable 82 0 6
✅ e2e-local-prod-nest-stable 82 0 6
✅ e2e-local-prod-tanstack-start-stable 82 0 6

📋 View full workflow run


Some E2E test jobs failed:

  • Vercel Prod: failure
  • Local Dev: success
  • Local Prod: success
  • Local Postgres: success
  • Windows: success

Check the workflow run for details.

⚠️ Community world tests failed (non-blocking):

  • Community Worlds: failure

Check the workflow run for details.

@vercel
Copy link
Copy Markdown
Contributor

vercel Bot commented Jun 1, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
example-nextjs-workflow-turbopack Ready Ready Preview, Comment Jun 1, 2026 10:54am
example-nextjs-workflow-webpack Ready Ready Preview, Comment Jun 1, 2026 10:54am
example-workflow Ready Ready Preview, Comment Jun 1, 2026 10:54am
workbench-astro-workflow Ready Ready Preview, Comment Jun 1, 2026 10:54am
workbench-express-workflow Ready Ready Preview, Comment Jun 1, 2026 10:54am
workbench-fastify-workflow Ready Ready Preview, Comment Jun 1, 2026 10:54am
workbench-hono-workflow Ready Ready Preview, Comment Jun 1, 2026 10:54am
workbench-nitro-workflow Ready Ready Preview, Comment Jun 1, 2026 10:54am
workbench-nuxt-workflow Ready Ready Preview, Comment Jun 1, 2026 10:54am
workbench-sveltekit-workflow Ready Ready Preview, Comment Jun 1, 2026 10:54am
workbench-tanstack-start-workflow Ready Ready Preview, Comment Jun 1, 2026 10:54am
workbench-vite-workflow Ready Ready Preview, Comment Jun 1, 2026 10:54am
workflow-docs Ready Ready Preview, Comment, Open in v0 Jun 1, 2026 10:54am
workflow-swc-playground Ready Ready Preview, Comment Jun 1, 2026 10:54am
workflow-tarballs Ready Ready Preview, Comment Jun 1, 2026 10:54am
workflow-web Ready Ready Preview, Comment Jun 1, 2026 10:54am

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant