Skip to content

Add serde compliance tooling and improve custom class serialization DX#1552

Draft
TooTallNate wants to merge 2 commits intomainfrom
nrajlich/serde-dx-improvements
Draft

Add serde compliance tooling and improve custom class serialization DX#1552
TooTallNate wants to merge 2 commits intomainfrom
nrajlich/serde-dx-improvements

Conversation

@TooTallNate
Copy link
Copy Markdown
Member

Summary

  • Add serde compliance checker library (analyzeSerdeCompliance) to @workflow/builders for programmatic analysis of custom class serialization correctness
  • Add build-time warnings when serde-registered classes have Node.js built-in imports remaining in the workflow bundle
  • Add workflow transform <file> CLI command for inspecting SWC transform output with optional --check-serde analysis
  • Implement workflow validate CLI command (replacing the "Coming soon" stub) with serde compliance scanning
  • Add Serde Analysis panel to the SWC playground that automatically appears when serde patterns are detected
  • Update workflow skill file (v1.4 → v1.5) with comprehensive custom class serialization documentation, fixing the incorrect claim that class instances are "not supported"

Motivation

Custom class serde is difficult for both humans and AI agents to implement retroactively. See vercel-labs/just-bash#172 where an AI agent:

  • Defined serde methods outside the class body (not detected by SWC)
  • Used dynamic imports with @vite-ignore to avoid sandbox restrictions
  • Wrote manual hydration shims bypassing the serde system
  • Reimplemented class registry registration manually

These changes provide validation tooling and correct documentation to prevent these mistakes.

- Add serde compliance checker library to @workflow/builders
- Add build-time warnings for serde classes with Node.js imports in workflow bundle
- Add 'workflow transform' CLI command for inspecting SWC output
- Implement 'workflow validate' CLI command with serde compliance checks
- Add serde analysis panel to SWC playground
- Update workflow skill with custom class serialization documentation
@changeset-bot
Copy link
Copy Markdown

changeset-bot bot commented Mar 29, 2026

🦋 Changeset detected

Latest commit: 2b3680a

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 16 packages
Name Type
@workflow/cli Patch
@workflow/builders Patch
workflow Patch
@workflow/world-testing Patch
@workflow/astro Patch
@workflow/nest Patch
@workflow/next Patch
@workflow/nitro Patch
@workflow/rollup Patch
@workflow/sveltekit Patch
@workflow/vite Patch
@workflow/vitest Patch
@workflow/ai Patch
@workflow/nuxt Patch
@workflow/core Patch
@workflow/web-shared Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel
Copy link
Copy Markdown
Contributor

vercel bot commented Mar 29, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
example-nextjs-workflow-turbopack Ready Ready Preview, Comment Mar 30, 2026 0:19am
example-nextjs-workflow-webpack Ready Ready Preview, Comment Mar 30, 2026 0:19am
example-workflow Ready Ready Preview, Comment Mar 30, 2026 0:19am
workbench-astro-workflow Ready Ready Preview, Comment Mar 30, 2026 0:19am
workbench-express-workflow Ready Ready Preview, Comment Mar 30, 2026 0:19am
workbench-fastify-workflow Ready Ready Preview, Comment Mar 30, 2026 0:19am
workbench-hono-workflow Ready Ready Preview, Comment Mar 30, 2026 0:19am
workbench-nitro-workflow Ready Ready Preview, Comment Mar 30, 2026 0:19am
workbench-nuxt-workflow Ready Ready Preview, Comment Mar 30, 2026 0:19am
workbench-sveltekit-workflow Ready Ready Preview, Comment Mar 30, 2026 0:19am
workbench-vite-workflow Ready Ready Preview, Comment Mar 30, 2026 0:19am
workflow-swc-playground Ready Ready Preview, Comment Mar 30, 2026 0:19am
1 Skipped Deployment
Project Deployment Actions Updated (UTC)
workflow-docs Skipped Skipped Mar 30, 2026 0:19am

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 29, 2026

🧪 E2E Test Results

Some tests failed

Summary

Passed Failed Skipped Total
❌ ▲ Vercel Production 785 61 67 913
✅ 💻 Local Development 818 0 178 996
✅ 📦 Local Production 818 0 178 996
✅ 🐘 Local Postgres 818 0 178 996
✅ 🪟 Windows 75 0 8 83
❌ 🌍 Community Worlds 130 59 24 213
✅ 📋 Other 207 0 42 249
Total 3651 120 675 4446

❌ Failed Tests

▲ Vercel Production (61 failed)

astro (7 failed):

  • DurableAgent e2e core multiple sequential tool calls
  • DurableAgent e2e onFinish fires constructor + stream callbacks in order with event data
  • readableStreamWorkflow | wrun_01KMY1QFSE0QHJXVYMJGKYHXGX | 🔍 observability
  • hookWorkflow | wrun_01KMY1QSNT1ZXR9GTW1C3FKK8M | 🔍 observability
  • error handling retry behavior FatalError fails immediately without retries
  • instanceMethodStepWorkflow - instance methods with "use step" directive | wrun_01KMY27HXXXJYNVT42FEGHBMG8 | 🔍 observability
  • stepFunctionAsStartArgWorkflow - step function reference passed as start() argument | wrun_01KMY2858BNYNS54S0K44YZ7FK | 🔍 observability

example (7 failed):

  • DurableAgent e2e experimental_onToolCallFinish (GAP) completes but callbacks are not called (GAP)
  • DurableAgent e2e prepareCall (GAP) completes but prepareCall is not applied (GAP)
  • readableStreamWorkflow | wrun_01KMY1QFSE0QHJXVYMJGKYHXGX | 🔍 observability
  • parallelSleepWorkflow | wrun_01KMY1RZGDD7D975RQBHC4RHB6 | 🔍 observability
  • outputStreamInsideStepWorkflow - getWritable() called inside step functions | wrun_01KMY1WX2WZ11FHNDKTY12VCNJ | 🔍 observability
  • error handling error propagation step errors cross-file step error preserves message and function names in stack
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration | wrun_01KMY29MMBKW53R60WB4D42EWC | 🔍 observability

express (7 failed):

  • DurableAgent e2e core multiple sequential tool calls
  • nullByteWorkflow | wrun_01KMY1S3PM32VFQFGYTBHA0KA6 | 🔍 observability
  • workflowAndStepMetadataWorkflow | wrun_01KMY1S5NW6HW801H49A38RC2Y | 🔍 observability
  • outputStreamWorkflow negative startIndex (reads from end)
  • promiseRaceStressTestWorkflow | wrun_01KMY1XE48HQ3MAVJBTDBJJ4XS | 🔍 observability
  • customSerializationWorkflow - custom class serialization with WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE | wrun_01KMY25Q2C7KR2VWN4QK49DJQY | 🔍 observability
  • instanceMethodStepWorkflow - instance methods with "use step" directive | wrun_01KMY27HXXXJYNVT42FEGHBMG8 | 🔍 observability

fastify (2 failed):

  • DurableAgent e2e experimental_onStepStart (GAP) completes but callbacks are not called (GAP)
  • outputStreamWorkflow negative startIndex (reads from end)

hono (5 failed):

  • DurableAgent e2e instructions string instructions are passed to the model
  • DurableAgent e2e timeout completes within timeout
  • hookWorkflow is not resumable via public webhook endpoint | wrun_01KMY1R426BGR32N2WPB4GMR8J | 🔍 observability
  • outputStreamWorkflow positive startIndex (skips first chunk)
  • crossContextSerdeWorkflow - classes defined in step code are deserializable in workflow context | wrun_01KMY27WNNFET6VGH06EYYCME0 | 🔍 observability

nextjs-turbopack (4 failed):

  • DurableAgent e2e onFinish fires constructor + stream callbacks in order with event data
  • addTenWorkflow | wrun_01KMY1PXPFDW41CM1N3H4S7RZR | 🔍 observability
  • parallelSleepWorkflow | wrun_01KMY1RZGDD7D975RQBHC4RHB6 | 🔍 observability
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_01KMY2105J42BRDDA32F34ZD0W | 🔍 observability

nextjs-webpack (7 failed):

  • DurableAgent e2e core multiple sequential tool calls
  • DurableAgent e2e onStepFinish fires constructor + stream callbacks in order with step data
  • DurableAgent e2e tool approval (GAP) completes but needsApproval is not checked (GAP)
  • importedStepOnlyWorkflow | wrun_01KMY1RDEESXS80TMSSMCCM2YS | 🔍 observability
  • hookWorkflow is not resumable via public webhook endpoint | wrun_01KMY1R426BGR32N2WPB4GMR8J | 🔍 observability
  • outputStreamWorkflow - getTailIndex and getStreamChunks getTailIndex returns correct index after stream completes
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step | wrun_01KMY22MMCQTJ4SZ5B880KAN4R | 🔍 observability

nitro (6 failed):

  • DurableAgent e2e core tool error recovery
  • DurableAgent e2e instructions string instructions are passed to the model
  • hookWorkflow is not resumable via public webhook endpoint | wrun_01KMY1R426BGR32N2WPB4GMR8J | 🔍 observability
  • webhookWorkflow | wrun_01KMY1RCMFDWQNXP8YX8WHHZQA | 🔍 observability
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step | wrun_01KMY22MMCQTJ4SZ5B880KAN4R | 🔍 observability
  • instanceMethodStepWorkflow - instance methods with "use step" directive | wrun_01KMY27HXXXJYNVT42FEGHBMG8 | 🔍 observability

nuxt (7 failed):

  • DurableAgent e2e experimental_onStart (GAP) completes but callbacks are not called (GAP)
  • promiseAllWorkflow | wrun_01KMY1Q53ASHCGJK6T1ZQWEQJD | 🔍 observability
  • promiseRaceWorkflow | wrun_01KMY1QAMADMWN8836FA013BWQ | 🔍 observability
  • promiseAnyWorkflow | wrun_01KMY1QDQBSV6E9698GDB2WRG1 | 🔍 observability
  • error handling retry behavior maxRetries=0 disables retries
  • crossContextSerdeWorkflow - classes defined in step code are deserializable in workflow context | wrun_01KMY27WNNFET6VGH06EYYCME0 | 🔍 observability
  • sleepWithSequentialStepsWorkflow - sequential steps work with concurrent sleep (control) | wrun_01KMY29ZF86C23P8BK8QSDEDRN | 🔍 observability

sveltekit (4 failed):

  • DurableAgent e2e core multiple sequential tool calls
  • DurableAgent e2e experimental_onStepStart (GAP) completes but callbacks are not called (GAP)
  • outputStreamWorkflow positive startIndex (skips first chunk)
  • error handling not registered StepNotRegisteredError fails the run when not caught in workflow

vite (5 failed):

  • DurableAgent e2e prepareCall (GAP) completes but prepareCall is not applied (GAP)
  • outputStreamWorkflow no startIndex (reads all chunks)
  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_01KMY20MSRTD9QVMHQYRX2NZTT | 🔍 observability
  • Calculator.calculate - static workflow method using static step methods from another class | wrun_01KMY236N5ZGNDGPQH2K6985HW | 🔍 observability
  • customSerializationWorkflow - custom class serialization with WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE | wrun_01KMY25Q2C7KR2VWN4QK49DJQY | 🔍 observability
🌍 Community Worlds (59 failed)

mongodb (3 failed):

  • hookWorkflow is not resumable via public webhook endpoint | wrun_01KMY1R426BGR32N2WPB4GMR8J
  • webhookWorkflow | wrun_01KMY1RCMFDWQNXP8YX8WHHZQA
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_01KMY2105J42BRDDA32F34ZD0W

redis (2 failed):

  • hookWorkflow is not resumable via public webhook endpoint | wrun_01KMY1R426BGR32N2WPB4GMR8J
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_01KMY2105J42BRDDA32F34ZD0W

turso (54 failed):

  • addTenWorkflow | wrun_01KMY1PXPFDW41CM1N3H4S7RZR
  • addTenWorkflow | wrun_01KMY1PXPFDW41CM1N3H4S7RZR
  • wellKnownAgentWorkflow (.well-known/agent) | wrun_01KMY1R3JWCJAEEEJNN8796R16
  • should work with react rendering in step
  • promiseAllWorkflow | wrun_01KMY1Q53ASHCGJK6T1ZQWEQJD
  • promiseRaceWorkflow | wrun_01KMY1QAMADMWN8836FA013BWQ
  • promiseAnyWorkflow | wrun_01KMY1QDQBSV6E9698GDB2WRG1
  • importedStepOnlyWorkflow | wrun_01KMY1RDEESXS80TMSSMCCM2YS
  • hookWorkflow | wrun_01KMY1QSNT1ZXR9GTW1C3FKK8M
  • hookWorkflow is not resumable via public webhook endpoint | wrun_01KMY1R426BGR32N2WPB4GMR8J
  • webhookWorkflow | wrun_01KMY1RCMFDWQNXP8YX8WHHZQA
  • sleepingWorkflow | wrun_01KMY1RK7J6ZBK6G81KD630HA1
  • parallelSleepWorkflow | wrun_01KMY1RZGDD7D975RQBHC4RHB6
  • nullByteWorkflow | wrun_01KMY1S3PM32VFQFGYTBHA0KA6
  • workflowAndStepMetadataWorkflow | wrun_01KMY1S5NW6HW801H49A38RC2Y
  • fetchWorkflow | wrun_01KMY1XAPT13X7H5T7BA6BKF7N
  • promiseRaceStressTestWorkflow | wrun_01KMY1XE48HQ3MAVJBTDBJJ4XS
  • error handling error propagation workflow errors nested function calls preserve message and stack trace
  • error handling error propagation workflow errors cross-file imports preserve message and stack trace
  • error handling error propagation step errors basic step error preserves message and stack trace
  • error handling error propagation step errors cross-file step error preserves message and function names in stack
  • error handling retry behavior regular Error retries until success
  • error handling retry behavior FatalError fails immediately without retries
  • error handling retry behavior RetryableError respects custom retryAfter delay
  • error handling retry behavior maxRetries=0 disables retries
  • error handling catchability FatalError can be caught and detected with FatalError.is()
  • error handling not registered WorkflowNotRegisteredError fails the run when workflow does not exist
  • error handling not registered StepNotRegisteredError fails the step but workflow can catch it
  • error handling not registered StepNotRegisteredError fails the run when not caught in workflow
  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_01KMY20MSRTD9QVMHQYRX2NZTT
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_01KMY2105J42BRDDA32F34ZD0W
  • hookDisposeTestWorkflow - hook token reuse after explicit disposal while workflow still running | wrun_01KMY21KAM2N8GACGTA4ZT350X
  • stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars) | wrun_01KMY225EB9ME5E3C08Q48P7EB
  • stepFunctionWithClosureWorkflow - step function with closure variables passed as argument | wrun_01KMY22CVSN3K7TKJH652BK98W
  • closureVariableWorkflow - nested step functions with closure variables | wrun_01KMY22JPJVF3KMSC1ZX9SMJ9F
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step | wrun_01KMY22MMCQTJ4SZ5B880KAN4R
  • health check (queue-based) - workflow and step endpoints respond to health check messages
  • pathsAliasWorkflow - TypeScript path aliases resolve correctly | wrun_01KMY231VRY6M5AYG4G2CEPF6S
  • Calculator.calculate - static workflow method using static step methods from another class | wrun_01KMY236N5ZGNDGPQH2K6985HW
  • AllInOneService.processNumber - static workflow method using sibling static step methods | wrun_01KMY255F0XCRH552R2S2M8HEF
  • ChainableService.processWithThis - static step methods using this to reference the class | wrun_01KMY25BAXAS0E0BVZGNGZRE0V
  • thisSerializationWorkflow - step function invoked with .call() and .apply() | wrun_01KMY25H9HT9V0MBQS4WW00RCN
  • customSerializationWorkflow - custom class serialization with WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE | wrun_01KMY25Q2C7KR2VWN4QK49DJQY
  • instanceMethodStepWorkflow - instance methods with "use step" directive | wrun_01KMY27HXXXJYNVT42FEGHBMG8
  • crossContextSerdeWorkflow - classes defined in step code are deserializable in workflow context | wrun_01KMY27WNNFET6VGH06EYYCME0
  • stepFunctionAsStartArgWorkflow - step function reference passed as start() argument | wrun_01KMY2858BNYNS54S0K44YZ7FK
  • cancelRun - cancelling a running workflow | wrun_01KMY28D9GP21YN8K4YVA9XHW6
  • cancelRun via CLI - cancelling a running workflow | wrun_01KMY28NV23M0T07J9SDF6NBX9
  • pages router addTenWorkflow via pages router
  • pages router promiseAllWorkflow via pages router
  • pages router sleepingWorkflow via pages router
  • hookWithSleepWorkflow - hook payloads delivered correctly with concurrent sleep | wrun_01KMY290NJKR6N7H7ZVMCHASRS
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration | wrun_01KMY29MMBKW53R60WB4D42EWC
  • sleepWithSequentialStepsWorkflow - sequential steps work with concurrent sleep (control) | wrun_01KMY29ZF86C23P8BK8QSDEDRN

Details by Category

❌ ▲ Vercel Production
App Passed Failed Skipped
❌ astro 69 7 7
❌ example 69 7 7
❌ express 69 7 7
❌ fastify 74 2 7
❌ hono 71 5 7
❌ nextjs-turbopack 77 4 2
❌ nextjs-webpack 74 7 2
❌ nitro 70 6 7
❌ nuxt 69 7 7
❌ sveltekit 72 4 7
❌ vite 71 5 7
✅ 💻 Local Development
App Passed Failed Skipped
✅ astro-stable 69 0 14
✅ express-stable 69 0 14
✅ fastify-stable 69 0 14
✅ hono-stable 69 0 14
✅ nextjs-turbopack-canary 58 0 25
✅ nextjs-turbopack-stable 75 0 8
✅ nextjs-webpack-canary 58 0 25
✅ nextjs-webpack-stable 75 0 8
✅ nitro-stable 69 0 14
✅ nuxt-stable 69 0 14
✅ sveltekit-stable 69 0 14
✅ vite-stable 69 0 14
✅ 📦 Local Production
App Passed Failed Skipped
✅ astro-stable 69 0 14
✅ express-stable 69 0 14
✅ fastify-stable 69 0 14
✅ hono-stable 69 0 14
✅ nextjs-turbopack-canary 58 0 25
✅ nextjs-turbopack-stable 75 0 8
✅ nextjs-webpack-canary 58 0 25
✅ nextjs-webpack-stable 75 0 8
✅ nitro-stable 69 0 14
✅ nuxt-stable 69 0 14
✅ sveltekit-stable 69 0 14
✅ vite-stable 69 0 14
✅ 🐘 Local Postgres
App Passed Failed Skipped
✅ astro-stable 69 0 14
✅ express-stable 69 0 14
✅ fastify-stable 69 0 14
✅ hono-stable 69 0 14
✅ nextjs-turbopack-canary 58 0 25
✅ nextjs-turbopack-stable 75 0 8
✅ nextjs-webpack-canary 58 0 25
✅ nextjs-webpack-stable 75 0 8
✅ nitro-stable 69 0 14
✅ nuxt-stable 69 0 14
✅ sveltekit-stable 69 0 14
✅ vite-stable 69 0 14
✅ 🪟 Windows
App Passed Failed Skipped
✅ nextjs-turbopack 75 0 8
❌ 🌍 Community Worlds
App Passed Failed Skipped
✅ mongodb-dev 5 0 0
❌ mongodb 55 3 8
✅ redis-dev 5 0 0
❌ redis 56 2 8
✅ turso-dev 5 0 0
❌ turso 4 54 8
✅ 📋 Other
App Passed Failed Skipped
✅ e2e-local-dev-nest-stable 69 0 14
✅ e2e-local-postgres-nest-stable 69 0 14
✅ e2e-local-prod-nest-stable 69 0 14

📋 View full workflow run


Some E2E test jobs failed:

  • Vercel Prod: failure
  • Local Dev: success
  • Local Prod: success
  • Local Postgres: success
  • Windows: success

Check the workflow run for details.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 29, 2026

📊 Benchmark Results

📈 Comparing against baseline from main branch. Green 🟢 = faster, Red 🔺 = slower.

workflow with no steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
💻 Local 🥇 Nitro 0.044s (-17.1% 🟢) 1.006s (-1.0%) 0.962s 10 1.00x
💻 Local Express 0.045s (~) 1.005s (~) 0.961s 10 1.02x
🐘 Postgres Express 0.065s (-0.8%) 1.011s (~) 0.946s 10 1.49x
🐘 Postgres Nitro 0.065s (+1.4%) 1.012s (~) 0.947s 10 1.49x
workflow with 1 step

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
💻 Local 🥇 Express 1.134s (~) 2.005s (~) 0.871s 10 1.00x
💻 Local Nitro 1.135s (~) 2.006s (~) 0.871s 10 1.00x
🐘 Postgres Express 1.147s (~) 2.010s (~) 0.863s 10 1.01x
🐘 Postgres Nitro 1.157s (~) 2.011s (~) 0.854s 10 1.02x
workflow with 10 sequential steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Express 10.897s (~) 11.019s (~) 0.122s 3 1.00x
💻 Local Nitro 10.937s (~) 11.024s (~) 0.086s 3 1.00x
💻 Local Express 10.941s (~) 11.024s (~) 0.083s 3 1.00x
🐘 Postgres Nitro 10.949s (~) 11.357s (+3.1%) 0.408s 3 1.00x
workflow with 25 sequential steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Express 14.553s (~) 15.023s (~) 0.470s 4 1.00x
🐘 Postgres Nitro 14.603s (~) 15.028s (~) 0.424s 4 1.00x
💻 Local Express 14.980s (~) 15.031s (~) 0.050s 4 1.03x
💻 Local Nitro 14.985s (~) 15.280s (+1.7%) 0.296s 4 1.03x
workflow with 50 sequential steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Express 13.865s (~) 14.021s (~) 0.155s 7 1.00x
🐘 Postgres Nitro 13.987s (~) 14.454s (~) 0.467s 7 1.01x
💻 Local Nitro 16.661s (+1.7%) 17.031s (~) 0.370s 6 1.20x
💻 Local Express 16.706s (~) 17.030s (~) 0.324s 6 1.20x
Promise.all with 10 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 1.267s (-0.5%) 2.010s (~) 0.743s 15 1.00x
🐘 Postgres Express 1.273s (~) 2.011s (~) 0.738s 15 1.00x
💻 Local Nitro 1.494s (-1.9%) 2.006s (~) 0.512s 15 1.18x
💻 Local Express 1.524s (-1.7%) 2.005s (~) 0.481s 15 1.20x
Promise.all with 25 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 2.334s (+0.5%) 3.011s (~) 0.677s 10 1.00x
🐘 Postgres Express 2.385s (+1.3%) 3.011s (~) 0.627s 10 1.02x
💻 Local Express 2.943s (+0.7%) 3.760s (+12.5% 🔺) 0.817s 8 1.26x
💻 Local Nitro 3.154s (+8.3% 🔺) 3.884s (+21.1% 🔺) 0.731s 8 1.35x
Promise.all with 50 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 3.463s (~) 4.011s (~) 0.547s 8 1.00x
🐘 Postgres Express 3.467s (-0.9%) 4.012s (~) 0.544s 8 1.00x
💻 Local Nitro 8.307s (+9.7% 🔺) 8.773s (+9.4% 🔺) 0.465s 4 2.40x
💻 Local Express 8.330s (+3.6%) 9.021s (+2.8%) 0.691s 4 2.41x
Promise.race with 10 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 1.257s (-0.9%) 2.011s (~) 0.754s 15 1.00x
🐘 Postgres Express 1.259s (~) 2.010s (~) 0.751s 15 1.00x
💻 Local Nitro 1.536s (+0.6%) 2.005s (~) 0.469s 15 1.22x
💻 Local Express 1.578s (+3.6%) 2.007s (~) 0.428s 15 1.26x
Promise.race with 25 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 2.333s (-1.2%) 3.011s (~) 0.678s 10 1.00x
🐘 Postgres Express 2.335s (-1.1%) 3.011s (~) 0.675s 10 1.00x
💻 Local Nitro 3.014s (+7.1% 🔺) 3.760s (+17.2% 🔺) 0.745s 8 1.29x
💻 Local Express 3.414s (+12.2% 🔺) 4.009s (+3.2%) 0.595s 8 1.46x
Promise.race with 50 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 3.459s (~) 4.012s (~) 0.554s 8 1.00x
🐘 Postgres Express 3.470s (~) 4.012s (~) 0.541s 8 1.00x
💻 Local Nitro 8.730s (+10.9% 🔺) 9.026s (+6.0% 🔺) 0.296s 4 2.52x
💻 Local Express 9.009s (+4.0%) 9.274s (+2.8%) 0.265s 4 2.60x
workflow with 10 sequential data payload steps (10KB)

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Express 0.821s (-0.6%) 1.007s (~) 0.186s 60 1.00x
🐘 Postgres Nitro 0.845s (+3.9%) 1.007s (~) 0.162s 60 1.03x
💻 Local Express 1.007s (+2.2%) 1.505s (+34.9% 🔺) 0.498s 40 1.23x
💻 Local Nitro 1.025s (+1.0%) 1.400s (-18.6% 🟢) 0.376s 43 1.25x
workflow with 25 sequential data payload steps (10KB)

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Express 1.951s (-2.4%) 2.151s (-7.1% 🟢) 0.200s 42 1.00x
🐘 Postgres Nitro 2.024s (+6.3% 🔺) 2.609s (+28.5% 🔺) 0.585s 35 1.04x
💻 Local Nitro 3.028s (-0.7%) 3.609s (-3.2%) 0.582s 25 1.55x
💻 Local Express 3.049s (+1.0%) 3.759s (-1.1%) 0.711s 24 1.56x
workflow with 50 sequential data payload steps (10KB)

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 4.027s (+4.8%) 4.705s (+15.4% 🔺) 0.678s 26 1.00x
🐘 Postgres Express 4.048s (~) 4.704s (+3.0%) 0.656s 26 1.01x
💻 Local Express 9.087s (-0.5%) 9.556s (-2.4%) 0.470s 13 2.26x
💻 Local Nitro 9.131s (+2.4%) 9.865s (+5.8% 🔺) 0.734s 13 2.27x
workflow with 10 concurrent data payload steps (10KB)

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 0.282s (-2.3%) 1.008s (~) 0.726s 60 1.00x
🐘 Postgres Express 0.288s (+2.3%) 1.008s (~) 0.720s 60 1.02x
💻 Local Express 0.582s (-0.9%) 1.004s (~) 0.422s 60 2.07x
💻 Local Nitro 0.587s (+3.9%) 1.004s (~) 0.417s 60 2.09x
workflow with 25 concurrent data payload steps (10KB)

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 0.503s (-1.1%) 1.007s (~) 0.504s 90 1.00x
🐘 Postgres Express 0.509s (+1.4%) 1.007s (~) 0.499s 90 1.01x
💻 Local Express 2.501s (-2.5%) 3.009s (~) 0.508s 30 4.97x
💻 Local Nitro 2.546s (+9.8% 🔺) 3.009s (~) 0.463s 30 5.06x
workflow with 50 concurrent data payload steps (10KB)

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Express 0.792s (~) 1.009s (~) 0.216s 119 1.00x
🐘 Postgres Nitro 0.793s (+2.4%) 1.017s (+0.8%) 0.224s 119 1.00x
💻 Local Express 11.129s (-0.8%) 11.572s (-3.0%) 0.443s 11 14.05x
💻 Local Nitro 11.198s (+8.2% 🔺) 11.938s (+9.9% 🔺) 0.741s 11 14.13x
Stream Benchmarks (includes TTFB metrics)
workflow with stream

💻 Local Development

World Framework Workflow Time TTFB Slurp Wall Time Overhead Samples vs Fastest
💻 Local 🥇 Nitro 0.205s (-1.5%) 1.003s (~) 0.012s (+19.4% 🔺) 1.018s (~) 0.813s 10 1.00x
💻 Local Express 0.207s (~) 1.003s (~) 0.012s (+9.0% 🔺) 1.018s (~) 0.811s 10 1.01x
🐘 Postgres Nitro 0.213s (+3.4%) 0.990s (~) 0.001s (+18.2% 🔺) 1.012s (~) 0.799s 10 1.04x
🐘 Postgres Express 0.222s (+9.3% 🔺) 0.994s (~) 0.002s (+41.7% 🔺) 1.011s (~) 0.789s 10 1.08x
stream pipeline with 5 transform steps (1MB)

💻 Local Development

World Framework Workflow Time TTFB Slurp Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Express 0.614s (~) 1.023s (+2.1%) 0.006s (+22.2% 🔺) 1.043s (+1.9%) 0.429s 59 1.00x
🐘 Postgres Nitro 0.620s (+2.8%) 1.002s (~) 0.005s (+3.9%) 1.022s (~) 0.403s 59 1.01x
💻 Local Nitro 0.741s (-1.7%) 1.009s (~) 0.009s (-3.1%) 1.024s (~) 0.283s 59 1.21x
💻 Local Express 0.743s (+2.8%) 1.010s (~) 0.009s (+6.5% 🔺) 1.023s (~) 0.280s 59 1.21x
10 parallel streams (1MB each)

💻 Local Development

World Framework Workflow Time TTFB Slurp Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 0.972s (+0.6%) 1.219s (+4.8%) 0.000s (+324.5% 🔺) 1.234s (+2.8%) 0.261s 49 1.00x
🐘 Postgres Express 0.975s (-0.9%) 1.296s (+4.4%) 0.000s (+2.1%) 1.315s (+4.4%) 0.340s 47 1.00x
💻 Local Nitro 1.274s (+4.2%) 2.022s (~) 0.000s (-9.1% 🟢) 2.025s (~) 0.751s 30 1.31x
💻 Local Express 1.296s (+2.9%) 2.022s (~) 0.000s (~) 2.025s (~) 0.730s 30 1.33x
fan-out fan-in 10 streams (1MB each)

💻 Local Development

World Framework Workflow Time TTFB Slurp Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 1.717s (-1.9%) 2.066s (-1.5%) 0.000s (-66.7% 🟢) 2.078s (-1.6%) 0.362s 29 1.00x
🐘 Postgres Express 1.849s (+2.2%) 2.141s (~) 0.000s (-100.0% 🟢) 2.160s (-0.6%) 0.311s 28 1.08x
💻 Local Express 3.553s (-2.6%) 4.033s (-3.2%) 0.001s (+22.2% 🔺) 4.037s (-3.2%) 0.484s 15 2.07x
💻 Local Nitro 3.632s (+6.9% 🔺) 4.167s (+4.9%) 0.001s (+42.2% 🔺) 4.171s (+4.9%) 0.539s 15 2.12x

Summary

Fastest Framework by World

Winner determined by most benchmark wins

World 🥇 Fastest Framework Wins
💻 Local Nitro 12/21
🐘 Postgres Nitro 12/21
Fastest World by Framework

Winner determined by most benchmark wins

Framework 🥇 Fastest World Wins
Express 🐘 Postgres 18/21
Nitro 🐘 Postgres 17/21
Column Definitions
  • Workflow Time: Runtime reported by workflow (completedAt - createdAt) - primary metric
  • TTFB: Time to First Byte - time from workflow start until first stream byte received (stream benchmarks only)
  • Slurp: Time from first byte to complete stream consumption (stream benchmarks only)
  • Wall Time: Total testbench time (trigger workflow + poll for result)
  • Overhead: Testbench overhead (Wall Time - Workflow Time)
  • Samples: Number of benchmark iterations run
  • vs Fastest: How much slower compared to the fastest configuration for this benchmark

Worlds:

  • 💻 Local: In-memory filesystem world (local development)
  • 🐘 Postgres: PostgreSQL database world (local development)
  • ▲ Vercel: Vercel production/preview deployment
  • 🌐 Turso: Community world (local development)
  • 🌐 MongoDB: Community world (local development)
  • 🌐 Redis: Community world (local development)
  • 🌐 Jazz: Community world (local development)

📋 View full workflow run


Some benchmark jobs failed:

  • Local: success
  • Postgres: success
  • Vercel: failure

Check the workflow run for details.

};

if (flags.strict && totalIssues > 0) {
this.exit(1);
Copy link
Copy Markdown
Contributor

@vercel vercel bot Mar 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When --json and --strict flags are both used with validation issues, this.exit(1) throws an ExitError before return output, so the JSON output is never returned to oclif for serialization.

Fix on Vercel

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant