implement event-sourced architecture #621
Draft
+8,198
−1,932
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
🦋 Changeset detectedLatest commit: 91fc888 The changes in this PR will be included in the next version bump. This PR includes changesets to release 18 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Contributor
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
🧪 E2E Test Results❌ Some tests failed Summary
❌ Failed Tests🌍 Community Worlds (156 failed)mongodb (39 failed):
redis (39 failed):
starter (39 failed):
turso (39 failed):
Details by Category✅ ▲ Vercel Production
✅ 💻 Local Development
✅ 📦 Local Production
✅ 🐘 Local Postgres
✅ 🪟 Windows
❌ 🌍 Community Worlds
|
This was referenced Dec 16, 2025
Collaborator
Author
This stack of pull requests is managed by Graphite. Learn more about stacking. |
pranaygp
force-pushed
the
pranaygp/perf-phase-3b-atomic-events
branch
from
6ebd4c5 to
2e46b8a
Compare
pranaygp
force-pushed
the
pranaygp/12-04-perf_parallelize_suspension_handler_for_high-concurrency
branch
from
eece359 to
290e879
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
This PR introduces a performance optimization for event creation by adding a createBatch() method to the World interface. The implementation enables atomic batch creation of multiple events, significantly improving the wait completion logic in the runtime from O(n²) to O(n) complexity.
Key Changes
- Added
events.createBatch()method to the World interface for creating multiple events in a single operation - Implemented batch creation across three storage backends (world-vercel, world-postgres, world-local) with backend-specific optimizations
- Optimized runtime wait completion logic using Set-based correlation ID lookup and batch event creation
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
packages/world/src/interfaces.ts |
Added createBatch() method signature with JSDoc documentation to the Storage events interface |
packages/world-vercel/src/storage.ts |
Integrated batch event creation into the storage adapter |
packages/world-vercel/src/events.ts |
Implemented createWorkflowRunEventBatch() using parallel API calls via Promise.all |
packages/world-postgres/src/storage.ts |
Implemented batch creation using a single INSERT query with multiple values for optimal database performance |
packages/world-local/src/storage.ts |
Implemented sequential batch creation to maintain monotonic ULID ordering for filesystem storage |
packages/core/src/runtime.ts |
Refactored wait completion to use Set-based lookup and batch event creation, improving from O(n²) to O(n) complexity |
.changeset/brave-dots-bake.md |
Added changeset documenting the performance improvement across all affected packages |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
When resumeHook() is called on a legacy run (specVersion < 2), the hook_received event was previously rejected. This adds support for storing hook_received events on legacy runs without entity mutation, matching the behavior of wait_completed. Co-Authored-By: Claude Opus 4.5 <[email protected]>
![vercel[bot]](https://avatars.githubusercontent.com/in/8329?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Additional Suggestion:
Missing version.ts module - file was imported but didn't exist after genversion removal
View Details
📝 Patch Details
diff --git a/packages/world-vercel/src/utils.ts b/packages/world-vercel/src/utils.ts
index baf9566..5e3eeeb 100644
--- a/packages/world-vercel/src/utils.ts
+++ b/packages/world-vercel/src/utils.ts
@@ -3,7 +3,7 @@ import { getVercelOidcToken } from '@vercel/oidc';
import { WorkflowAPIError } from '@workflow/errors';
import { type StructuredError, StructuredErrorSchema } from '@workflow/world';
import type { z } from 'zod';
-import { version } from './version.js';
+import { version } from './version';
export interface APIConfig {
baseUrl?: string;
Analysis
The file packages/world-vercel/src/utils.ts was importing a non-existent version.js module at line 6, which is actively used in the getUserAgent() function to generate User-Agent headers. The version.ts file appears to have been auto-generated by genversion but was removed during cleanup. The fix involved:
- Creating packages/world-vercel/src/version.ts with an exported version constant matching the package.json version (4.0.1-beta.27)
- Updating the import statement in utils.ts from './version.js' to './version' (standard TypeScript import format)
This aligns with the turbo.json build configuration that lists src/version.ts as an expected build output, and ensures the User-Agent header can be properly constructed with the package version.
|
|
||
| // ============================================================================= | ||
| // Legacy workflow events (deprecated, use run_* events instead) | ||
| // ============================================================================= |
Member
There was a problem hiding this comment.
This could use an equivalent end comment, otherwise CreateEventSchema might be misinterpreted as legacy code
| * Wire format schema for step in event results. | ||
| * Handles error deserialization from wire format. | ||
| */ | ||
| const StepWireSchema = StepSchema.omit({ |
Member
There was a problem hiding this comment.
Duplicate with the schema in workflow-vercel/src/steps.ts?
| "dev": "tsc --watch", | ||
| "clean": "tsc --build --clean && rm -rf dist", | ||
| "test": "vitest", | ||
| "typecheck": "genversion --es6 src/version.ts && tsc --noEmit" |
Member
There was a problem hiding this comment.
genversion but packages/world-vercel/src/utils.ts still relies on the version file existing to report the user agent
Member
There was a problem hiding this comment.
This is also breaking builds. I'll push a fix for now and we can discuss later
| @@ -1,576 +1,11 @@ | |||
| import path from 'node:path'; | |||
Member
There was a problem hiding this comment.
Can we move the contents of runs-storage.ts, steps-storage.ts, etc. into this file? It'd be nice to see the actual diff, and I don't know if there's a point to have them spread out. We can split it into smaller files in a separate PR if it's just a tidy
Signed-off-by: Peter Wielander <[email protected]>
Contributor
There was a problem hiding this comment.
Additional Suggestion:
Missing version.ts module - file was imported but didn't exist after genversion removal
View Details
📝 Patch Details
diff --git a/packages/world-vercel/src/utils.ts b/packages/world-vercel/src/utils.ts
index baf9566..5e3eeeb 100644
--- a/packages/world-vercel/src/utils.ts
+++ b/packages/world-vercel/src/utils.ts
@@ -3,7 +3,7 @@ import { getVercelOidcToken } from '@vercel/oidc';
import { WorkflowAPIError } from '@workflow/errors';
import { type StructuredError, StructuredErrorSchema } from '@workflow/world';
import type { z } from 'zod';
-import { version } from './version.js';
+import { version } from './version';
export interface APIConfig {
baseUrl?: string;
Analysis
The file packages/world-vercel/src/utils.ts was importing a non-existent version.js module at line 6, which is actively used in the getUserAgent() function to generate User-Agent headers. The version.ts file appears to have been auto-generated by genversion but was removed during cleanup. The fix involved:
- Creating packages/world-vercel/src/version.ts with an exported version constant matching the package.json version (4.0.1-beta.27)
- Updating the import statement in utils.ts from './version.js' to './version' (standard TypeScript import format)
This aligns with the turbo.json build configuration that lists src/version.ts as an expected build output, and ensures the User-Agent header can be properly constructed with the package version.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Pranay:
corresponding workflow-server PR: https://github.com/vercel/workflow-server/pull/154
important: This is a big change to the way workflows work since everything is now event sourced, I introduced new events types,
and changed the shape of the step object (lastKnownError -> error and startedAt -> firstStartedAt). New event logs that use this published version ofworkflowwill be incompatible with previous workflow version event logs. This doesn't affect the runtime of workflows since those are deployment pegged - but this does affect observability since the event shape looks different and the world spec has changed. The web-shared package just needs to be compatible with viewing workflow runs of the old schema for this to work correctly (which I believe it does, but please double check @VaguelySerious if I missed anything).The currently failing e2e tests on vercel world are related to the CLI I believe (slack x-ref). However once we merged the workflow-server PR, we can drop the env var changes on the vercel deployments for PR so that this PR points to the main prod deployment, again and then I'll re-run e2e tests to make sure they work :)
I Also added a new docs page with diagrams to explain the event sourcing and state machine lifecycles (preview link):
small: I also removed the unused run paused/resumed stuff which we've never used to simplify
Summary
Implement event-sourced architecture for runs, steps, and hooks:
run_created,run_started,run_completed,run_failed,run_cancelled)step_retryingevent for non-fatal step failures that will be retriedfatalfield fromstep_failedevent (step_failed now implies terminal failure)lastKnownErrortoerrorfor consistency with serverevents.create()step_createdevent for earlier detectionrun_paused/run_resumedevents andpausedstatusThis makes the system faster, easier to reason about, and resilient to data inconsistencies.
Test plan
🤖 Generated with Claude Code