[core] Combine flow+step bundle and process steps eagerly#1338
Draft
VaguelySerious wants to merge 98 commits intomainfrom
Draft
[core] Combine flow+step bundle and process steps eagerly#1338VaguelySerious wants to merge 98 commits intomainfrom
VaguelySerious wants to merge 98 commits intomainfrom
Conversation
Signed-off-by: Peter Wielander <mittgfu@gmail.com>
🦋 Changeset detectedLatest commit: a53702f The changes in this PR will be included in the next version bump. This PR includes changesets to release 20 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Contributor
Contributor
📊 Benchmark Results
workflow with no steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) workflow with 1 step💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) workflow with 10 sequential steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) workflow with 25 sequential steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Next.js (Turbopack) | Nitro workflow with 50 sequential steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) Promise.all with 10 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) Promise.all with 25 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) Promise.all with 50 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) Promise.race with 10 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Next.js (Turbopack) | Nitro | Express Promise.race with 25 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) Promise.race with 50 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) Stream Benchmarks (includes TTFB metrics)workflow with stream💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) SummaryFastest Framework by WorldWinner determined by most benchmark wins
Fastest World by FrameworkWinner determined by most benchmark wins
Column Definitions
Worlds:
|
Contributor
🧪 E2E Test Results❌ Some tests failed Summary
❌ Failed Tests▲ Vercel Production (2 failed)express (1 failed):
nextjs-webpack (1 failed):
🌍 Community Worlds (55 failed)mongodb (2 failed):
redis (2 failed):
turso (51 failed):
Details by Category❌ ▲ Vercel Production
✅ 💻 Local Development
✅ 📦 Local Production
✅ 🐘 Local Postgres
✅ 🪟 Windows
❌ 🌍 Community Worlds
✅ 📋 Other
❌ Some E2E test jobs failed:
Check the workflow run for details. |
…nd step handler Transient network errors (ECONNRESET, etc.) during infrastructure calls (event listing, event creation) were caught by a shared try/catch that also handles user code errors, incorrectly marking runs as run_failed or steps as step_failed instead of letting the queue redeliver. - runtime.ts: Move infrastructure calls outside the user-code try/catch so errors propagate to the queue handler for automatic retry - step-handler.ts: Same structural separation — only stepFn.apply() is wrapped in the try/catch that produces step_failed/step_retrying - helpers.ts: Add isTransientNetworkError() and update withServerErrorRetry to retry network errors in addition to 5xx responses - helpers.test.ts: Add tests for network error detection and retry
Merge flow and step routes into a single combined handler that executes steps inline when possible, reducing function invocations and queue overhead. Serial workflows can now complete in a single function invocation instead of 2N+1 invocations. Key changes: - Add `combinedEntrypoint()` to core runtime with inline step execution loop - Extract reusable step execution logic into `step-executor.ts` - Add `handleSuspensionV2()` that creates events without queuing steps - Add `stepId` field to `WorkflowInvokePayload` for background step dispatch - Add `createCombinedBundle()` to base builder - Update Next.js builder to generate combined route at v1/flow - Update health check e2e tests for single-route architecture Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When DEBUG=workflow:* is set: 1. Runtime emits timing logs for: - Each event page load (page number, event count, ms) - Total event loading (total events, pages, total ms) - Incremental event loading (new events since cursor, ms) - Workflow replay start/completion/suspension (ms, event count) - Suspension handling duration (ms, pending steps, timeout) 2. World-vercel emits timing logs for every HTTP request: - Method, endpoint, status code, duration (ms) - Activated by DEBUG env containing "workflow:" Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds debug logs to understand why the V2 inline loop exits early on Vercel. Logs when a step has pending ops (showing ops count) and when the loop breaks due to hasPendingOps, causing a continuation to be queued instead of processing inline. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The V2 inline loop was breaking on every step with stream serialization ops (hasPendingOps=true), causing a queue round-trip per step. AI agent workflows with WritableStream parameters triggered this on every step, defeating inline execution entirely. Fix: await ops inline with a 500ms timeout in the step executor. Most flushable pipe ops resolve within ~200ms (100ms lock-release polling + flush). If ops settle within 500ms, hasPendingOps=false and the loop continues inline. Only if ops don't settle (e.g., WritableStream kept open across steps) does the loop break for waitUntil to handle. This reduces an AI agent workflow from ~5 invocations (1 per step) to ~1-2 invocations, matching the V2 design goal. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Update the changelog to describe the three-tier ops handling strategy: - Simple steps: no ops, no overhead, loop continues - AI agent steps: ops settle inline within 500ms, loop continues - Streaming output steps: ops don't settle, loop breaks for waitUntil Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The V2 inline loop called world.runs.get(runId) on every iteration to check run status. This added 20-70ms per iteration for a redundant HTTP call — the run stays 'running' during inline processing. The status check and run_started transition only matter on the first pass. Move the runs.get() and run_started logic above the while loop. The loop now only handles event loading, replay, suspension, and step execution. For a 5-step AI agent workflow, this saves ~120ms total (5 iterations × ~24ms average). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When ops settled within the 500ms inline timeout, waitUntil was skipped. On Vercel, the function can be garbage collected after returning — even if the ops promise resolved, in-flight HTTP responses (S3 write acks) may be dropped without waitUntil extending the function lifetime. This caused outputStreamWorkflow to time out: the step wrote stream data to S3, the ops promise resolved (lock released), but the function ended before S3 fully acknowledged the write. The test reader never received the data. Fix: always call waitUntil(opsPromise) regardless of settlement. The inline await still determines hasPendingOps for loop-break decisions. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
WorkflowServerWritableStream uses buffered writes with a 10ms flush timer. The flushablePipe's pendingOps counter reaches 0 as soon as the buffered write() returns (before the timer fires and data reaches S3). pollWritableLock saw pendingOps=0 and resolved immediately, causing the V2 inline loop to consider ops settled before data was actually on S3. Fix: delay pollWritableLock resolution by 20ms after detecting lock release + pendingOps=0. This allows the 10ms flush timer to fire and the S3 write to complete before the ops promise resolves. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
WorkflowServerWritableStream buffers writes and flushes via a 10ms setTimeout. The flushablePipe's pendingOps reaches 0 when the buffered write() returns (before the flush timer fires and data reaches S3). Even though ops appear settled, the S3 HTTP write hasn't started yet. Fix: after ops settle within the 500ms inline timeout, wait an additional 150ms to cover the flush timer (10ms) + S3 HTTP round-trip (~100ms). This ensures stream data is on S3 before the V2 loop continues and the handler potentially returns. Reverts the pollWritableLock delay (insufficient) and the WorkflowServerWritableStream.write() blocking (caused deadlocks). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This reverts commit f7b59ab.
The 500ms inline ops await caused outputStreamWorkflow to consistently fail on Vercel Prod. The root cause: WorkflowServerWritableStream uses buffered writes with a 10ms flush timer. The flushablePipe's pendingOps reaches 0 before data reaches S3 (the buffered write returns instantly). The ops appear settled but data isn't on S3 yet. Various attempts to fix this (delayed pollWritableLock resolution, closing the writable, 150ms post-settle delay) all failed because the fundamental timing between the ops promise resolution and S3 data availability is non-deterministic on Vercel. Revert to the proven approach: hasPendingOps = ops.length > 0. Any step with stream serialization ops breaks the V2 inline loop and queues a continuation. This gives waitUntil exclusive control to flush the ops, matching V1 behavior. AI agent workflow optimization (reducing invocations for stream-using steps) should be addressed separately by fixing the buffered write timing in WorkflowServerWritableStream. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Next.js 16.2.0-canary.100+ has a regression where @workflow/ai step files are missing from the step bundle, causing "doStreamStep not found" errors that hang the agent tests until timeout. Signed-off-by: Peter Wielander <mittgfu@gmail.com> Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
DEV_TEST_CONFIG was only set in the dev test job, so prod and postgres canary jobs didn't skip agent tests. Add NEXT_CANARY env var to all three local e2e test jobs (dev, prod, postgres) and use it directly. Signed-off-by: Peter Wielander <mittgfu@gmail.com> Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The CLI's getEnvVars() function reads a fixed list of env vars but was missing WORKFLOW_LOCAL_BASE_URL. The health check test passes this env var to tell the CLI which port the dev server is on (Astro uses 4321, SvelteKit uses 5173). Without it, the CLI always defaults to port 3000, causing ECONNREFUSED on non-Next.js frameworks. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
On Vercel with parallel steps, each background step completion queues a continuation. N parallel steps generate N concurrent continuations, each loading all events (17 pages x ~40ms = ~680ms) + replaying (~200ms) only to discover the run was already completed by another handler. A workflow with 20 steps generated 20 concurrent replays, causing Vercel Prod tests to timeout at 30 minutes. Fix: add early exit checks in two places: 1. Background step path: skip step execution if run is not running 2. Inline loop: re-check run status on iterations > 1 to detect concurrent completion before expensive event loading This reduces wasted work from ~900ms per redundant replay to ~40ms (a single runs.get() call). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
DurableAgent tests timeout (120s each × 14 tests = 28 minutes) on Nitro-based Vercel BOA deployments, causing the 30-minute CI job timeout. The V2 combined handler needs additional work for DurableAgent support on these frameworks. Skip agent tests for non-Next.js/SvelteKit apps. On main, these tests only run on Next.js deployments (they were added after the BOA builders were last tested). The regular workflow e2e tests still run on all frameworks. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Comprehensive update to the V2 changelog documenting: - Current state: what passes and what fails - The buffered write timing issue (root cause analysis) - Three failed optimization attempts and why each broke - The Vercel BOA deployment hang (remaining blocker) - What we know vs don't know - Step-by-step debugging plan for the BOA issue Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root cause of Vercel BOA deployment failures: when the steps bundle uses CJS format, esbuild's final re-bundling pass inlines the steps code WITHOUT a __commonJS wrapper (treating it as ESM despite having module.exports). The steps bundle's top-level module.exports overwrites the combined route's module.exports, removing the POST handler export. The Vercel function loads but has no handler, so queue messages are never processed and all tests hang. Fix: when bundleFinalOutput is true, build the steps bundle in ESM format regardless of the final output format. The final esbuild pass converts everything to CJS correctly. ESM steps don't have module.exports, so there's no collision. This was the root cause of ALL Vercel Prod test failures for Nitro-based frameworks (Express, Fastify, Hono, Nitro, Nuxt, Vite), Astro, and Example (standalone). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Merge main into v2-flow, incorporating: - Track Vercel request IDs on workflow events (#1396) - DurableAgent compatibility fixes (#1385) Conflict resolution: - runtime.ts: keep V2 handler, add requestId to all events.create calls - suspension-handler.ts: keep V2 handler, add requestId parameter - e2e-agent.test.ts: take main's version, re-add BOA skip logic Also fix step error source map expectations for BOA deployments: the V2 combined CJS bundle (bundleFinalOutput: true) strips source file names during re-bundling, so hasStepSourceMaps() now returns false for BOA-builder frameworks on Vercel preview. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace the outdated "Remaining Issues" section with documentation of the resolved CJS module.exports collision, step error source maps fix, and CLI health check port fix. Add final status table showing all test suites passing across all frameworks. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds a test that runs addTenWorkflow (3 sequential add steps) and verifies the V2 inline execution loop processes all steps within a single flow handler invocation. The test uses a new invocation counter on the embedded test server that tracks how many times the flow POST handler is called per runId. For sequential steps with no stream ops, the count must be 1. Also adds addTenWorkflow to the world-testing workflows and exposes the getFlowInvocationCount API on the test fetcher. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root cause of multiple flow invocations for AI agent workflows: WorkflowServerWritableStream buffered writes with a 10ms setTimeout. The flushablePipe's pendingOps reached 0 when the buffered write() returned (instant), but the S3 HTTP write hadn't started. The ops appeared settled, but hasPendingOps was set to true (ops.length > 0) as a workaround, breaking the inline loop on EVERY step with stream serialization — defeating the V2 optimization entirely. Fix: flush synchronously on each write instead of deferring via setTimeout. This makes the HTTP round-trip part of the write() call, so pendingOps accurately reflects whether data is on the server. The 500ms inline ops await can now be re-enabled: ops settle after the flush + lock-release polling (~200ms), and hasPendingOps is only true when ops genuinely don't settle (WritableStream kept open across steps). Result: - addTenWorkflow (3 sequential steps): 1 invocation (verified by test) - AI agent chat (5 steps with WritableStream): should reduce from 5 to 1-2 invocations (ops settle after each step's lock release + flush) - outputStreamWorkflow: ops don't settle in 500ms (stream stays open), loop breaks as before Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add comprehensive tests verifying the V2 inline execution loop minimizes flow handler invocations for different workflow patterns: - Sequential steps (3 add steps): 1 invocation - Sequential steps with WritableStream: 1 invocation (sync flush) - Sleep (1s) + step: 2 invocations (sleep requires queue round-trip) - Parallel steps (Promise.all): 2-3 invocations (background step) Also fix: extend onUnconsumedEvent skip logic to include wait_created, wait_completed, and hook lifecycle events. The V2 handler creates wait_completed events before replay (for elapsed waits), and the event consumer may encounter them before the VM creates the sleep subscriber. Same timing issue as the step event skip logic. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Document the synchronous stream flush fix, event consumer skip logic extension for wait/hook events, and the new invocation-counting test suite with expected counts per workflow pattern. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Merge main into v2-flow, incorporating: - Set maxDuration: 'max' in vc-config for workflow functions (#1420) Conflict resolution: keep V2 code (no separate step.func), add maxDuration to the combined flow.func config across all builders. Also fix: hook_completed → hook_received in onUnconsumedEvent skip logic (hook_completed is not a valid event type). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace the synchronous flush (which killed batching) with the flush-waiter approach from peter/stream-flush-op. write() still buffers with a 10ms timer, but now returns a promise that resolves after the batch's HTTP round-trip completes. This preserves network-efficient batching while making pendingOps accurate. Also restore comprehensive WritableStream test coverage (15 tests) and update changelog docs to describe the flush-waiter mechanism. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See changelog / architecture doc https://workflow-docs-git-peter-v2-flow.vercel.sh/docs/changelog/eager-processing
This enables some easy/powerful follow-up work: