Skip to content

feat(eve): stamp every stream event with a stable id - #1239

Open
AndrewBarba wants to merge 9 commits into
mainfrom
barba/stable-stream-event-ids
Open

feat(eve): stamp every stream event with a stable id#1239
AndrewBarba wants to merge 9 commits into
mainfrom
barba/stable-stream-event-ids

Conversation

@AndrewBarba

@AndrewBarba AndrewBarba commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator

Refs #1193.

Every session stream event now carries a durable meta.id: an evt_-prefixed ULID, stamped once immediately before the event is written to the durable stream, alongside the existing meta.at. The persisted record and every authored hook observe the same envelope, and re-reading the stream returns the same id for the same event.

That gives consumers a stable key for an individual emission. Reconnecting from a cursor, rewinding to startIndex=0, and merging a saved event log with the prefix a live stream replays are all safe to ingest, and two distinct events with identical payloads stay distinct. EveAgentStore (and so the React, Vue, and Svelte bindings) applies each persisted event once, and the dev TUI renders a subagent transcript once when its child stream reopens.

  • session stream protocol bumped to version 20; StampedHandleMessageStreamEvent exposed across the stream, client, channel, hook, and frontend APIs
  • dependency-free, Web Crypto-backed monotonic ULID generator, vendored because the ulid package's Node build hard-imports node:crypto and this module is bundled into browser clients, the Nitro server, and the workflow step sandbox
  • id-keyed event deduper with an unbounded window, so a full-session rewind cannot cascade back in
  • extension compatibility contracts, public documentation, and a changeset for the new envelope

Breaking: stream readers return StampedHandleMessageStreamEvent, which guarantees meta is present, so initialEvents on useEveAgent/EveAgentStore needs that type instead of HandleMessageStreamEvent[]. Events persisted before stream version 20 carry meta.at and no meta.id, so rewinding into a session that started before this release yields events whose id is absent despite the type. eve passes those through rather than dropping them and they cannot be deduplicated; the exposure ends when those sessions do. meta.id stays required rather than becoming optional for a transitional case.

Why an id and not an index

#1193 asks for a zero-based durable index on every event, visible to hooks. eve cannot provide that, for three independent reasons:

  • The log never tells the writer where a chunk landed. world.streams.write and writeMulti both return Promise<void>. The index is assigned server-side by append order and is observable only afterwards — through getInfo for the tail, or by counting on read.
  • One session stream has several concurrent writers, in different workflow runs. The per-turn child run writes to the driver run's stream through a cross-run writable, and the subagent event proxy, runtime-action dispatch, cancellation settle, and terminal failure steps each revive their own writable. There is no shared counter to increment, and a durable one would need a serialized read-modify-write per event — for every token delta.
  • A writer-side counter would be wrong anyway. Writes are batched with up to 1000 chunks in flight, and a retried step leaves the failed attempt's chunks in the log while re-emitting from state restored out of its step input. Counter and physical position diverge on the first retry.

Stamping a true index means serializing every emission through a synchronous append round-trip, which is what the batching pipeline exists to avoid.

The goal behind #1193 — checkpoint, reconnect, process any replay idempotently — is met in two pieces. Readers already have absolute position: startIndex is an absolute event count, getTailIndex() resolves the tail, and the TypeScript client carries the cursor across reconnects. Consumers with no cursor to count against — hooks, and anything merging a saved log with a live stream — get meta.id. The limitation that remains is that meta.id is an identity, not an ordering key: where id > $cursor is not a lossless cursor, because ids minted in different steps come from different processes and clocks.

What meta.id does not cover

A retried durable step re-emits. eve runs each step up to four times; if one is interrupted partway, whatever it already wrote stays on the stream and the new attempt emits its own events with their own ids. Both attempts carry the same turnId, stepIndex, and sequence, but they are distinct events and no field records which attempt finished.

This is intended. A retry re-invokes the model, so one coordinate can carry different text on each attempt; a key that collapsed retries would keep the abandoned attempt and drop the one whose messages are in durable history. Replaying a completed step emits nothing, so crash recovery, redeploys, and resuming a parked turn add nothing to the stream.

Testing

  • ULID encoding against the reference implementation's golden vectors, monotonicity, validation, and clock rollback
  • distinct ids per emission, and stable ids across a workflow stream rewind
  • id-keyed deduplication for identical payloads, long full-session replays, and unstamped pre-v20 events
  • EveAgentStore overlap and reset behavior
  • TUI replay handling for the parent stream and for restarted subagent child pumps
  • stamped hook delivery and extension compatibility fixtures
  • an agent-basic-runtime e2e eval asserting every event served over HTTP carries a well-formed evt_ id, and that a startIndex=0 rewind replays those events under the same ids in the same order

PR Checklist

  • I linked an issue with prior discussion confirming this change is wanted
  • I ran the relevant checks from CONTRIBUTING.md
  • I added tests and documentation where relevant
  • I added a changeset if this touches the published eve package
  • DCO sign-off passes for every commit (git commit --signoff)

Every session stream event now carries `meta.id`, a unique `evt_`-prefixed
ULID minted once at emission. The id is stable across reconnects, rewinds,
tail reads, and replays, so a consumer can key durable work on it — for
example `INSERT ... ON CONFLICT (id) DO NOTHING` in a hook — instead of
guessing identity from payload content.

Events are stamped before the channel adapter runs, so the adapter, the
persisted chunk, and hooks all observe the same id.

Client consumers now use it. `EveAgentStore` no longer double-applies an
`initialEvents` prefix that the live stream replays, which also protects the
React, Vue, and Svelte bindings and any user-authored reducer. The dev TUI
drops re-delivered chunks up front, which fixes a subagent section that
rendered its child transcript twice when the child stream reopened, and
retires four content-comparison replay heuristics that could silently drop a
new model call whose text prefixed the previous one.

Bumps the message stream version to 20 and the hook, dynamicTool,
dynamicSkill, and dynamicInstructions capability epochs. All four changed
only additively, so prior-epoch extensions stay compatible and are retained.

Signed-off-by: Andrew Barba <barba@hey.com>
@vercel

vercel Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
eve-docs Ready Ready Preview, Comment, Open in v0 Jul 28, 2026 3:09pm
eve-docs-1644 Ready Ready Preview, Comment, Open in v0 Jul 28, 2026 3:09pm
eve-docs-4759 Ready Ready Preview, Comment, Open in v0 Jul 28, 2026 3:09pm

@github-actions

github-actions Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Bundle + Package Summary: apps/fixtures/weather-agent

Key takeaways

  • Bundle warning: function payloads grew 12.9%; runtime dependency undici@7.28.0 added.
  • Package delta: install footprint 67.19 MB -> 68.84 MB (+1.65 MB ⚠️).
  • Runtime delta: function payloads 14.63 MB -> 16.52 MB (+1.88 MB ⚠️); 2 changed payloads.
  • Dependency delta: 1 runtime dep added.

❌ Bundle Warning: Action Will Fail

This action will fail because the bundle warning policy was exceeded. Add the acknowledge-bundle-warning label to acknowledge the regression and allow the check to pass without regenerating this report.

Area Warning Details
Runtime Total function bytes 14.63 MB -> 16.52 MB; +1.88 MB ⚠️ (12.9%) over limit 10.0%
Package Runtime dependency added New runtime dependency undici@7.28.0 was added. Runtime dependencies increase install footprint; prefer a vendored devDependency when possible.

Delta vs main (a69faae)

Area Metric Baseline Current Delta
Package Packed tarball 7.42 MB 7.43 MB +9.8 kB ⚠️
Package Unpacked publish size 27.92 MB 27.95 MB +27.6 kB ⚠️
Package Installed footprint 67.19 MB 68.84 MB +1.65 MB ⚠️
Package Published files 2785 2794 +9
Package Installed files 6044 6263 +219
Runtime Unique function payloads 2 2 0
Runtime Total function bytes 14.63 MB 16.52 MB +1.88 MB ⚠️
Runtime Public routes 11 11 0
Dependency manifest changes vs main (a69faae)

Runtime dependencies

  • Added: undici@7.28.0
Changed function payloads vs main (a69faae) (2)
Function Status Baseline Current Delta Route changes
functions/__server.func changed 7.32 MB 8.26 MB +942.1 kB ⚠️ none
functions/.well-known/workflow/v1/flow.func changed 7.32 MB 8.26 MB +942.1 kB ⚠️ none

eve init install

Metric Baseline Current Delta
Installed footprint 105.59 MB 107.24 MB +1.65 MB ⚠️
Installed packages 122 123 +1
dependencies 4 4 0
devDependencies 2 2 0
Dependency package bytes 42.29 MB 42.32 MB +27.6 kB ⚠️
devDependency package bytes 5.04 MB 5.04 MB 0 B ➖
eve init dependency changes vs main (a69faae)

dependencies

  • Changed: ai@^7.0.34 -> ai@^7.0.38, eve@file:eve-0.27.7.tgz -> eve@file:eve-0.27.8.tgz
Build Metadata
  • Preset: vercel
  • Nitro: nitro@3.0.260610-beta
  • Output directory: apps/fixtures/weather-agent/.vercel/output
  • Build metadata timestamp: 2026-07-28T15:10:23.798Z
  • Route aliases: 11 public, 1 internal (12 total aliases)
  • Vercel routes in config: 14
  • Severity legend: 🔴 dominant/large, 🟠 notable, 🟡 watch, ⚪ small
Package Drill-Down

Package Details

  • Package: eve@0.27.8
  • Package directory: packages/eve
  • Tarball: 7.43 MB (eve-0.27.8.tgz)
  • Unpacked payload: 27.95 MB across 2794 published files
  • Installed footprint: 68.84 MB across 6263 installed files
  • Installed root package: 26.67 MB
  • Installed dependencies: 42.17 MB
  • Runtime dependencies: 2
  • Peer dependencies: 5 (4 optional)

Installed footprint is measured from an isolated temporary npm install of the packed tarball.

Heavy installed dependencies

  • eve: 26.67 MB (38.7%)
  • @rolldown/binding-linux-x64-gnu: 18.96 MB (27.5%)
  • ai: 6.51 MB (9.5%)
  • zod: 5.04 MB (7.3%)
  • nitro: 2.41 MB (3.5%)
Publish payload breakdown
Published file size
🔴 dist/src/compiled/shadcn-registry/index.js       [########################] 13.15 MB 47.1%
🟠 dist/src/compiled/experimental-ai-sdk-code-mo... [###.....................] 1.51 MB 5.4%
🟡 dist/src/compiled/@vercel/sandbox/index.js       [#.......................] 632.4 kB 2.3%
🟡 dist/src/compiled/_chunks/workflow/undici-C2Z... [#.......................] 502.4 kB 1.8%
🟡 dist/src/compiled/@chat-adapter/slack/index.js   [#.......................] 440.5 kB 1.6%
🔴 Other published files                            [#####################...] 11.72 MB 41.9%
Installed footprint breakdown
Installed package size
🔴 eve                             [########################] 26.67 MB 38.7%
🔴 @rolldown/binding-linux-x64-gnu [#################.......] 18.96 MB 27.5%
🔴 ai                              [######..................] 6.51 MB 9.5%
🔴 zod                             [#####...................] 5.04 MB 7.3%
🟠 nitro                           [##......................] 2.41 MB 3.5%
🟠 undici                          [#.......................] 1.62 MB 2.4%
🔴 Other installed packages        [#######.................] 7.63 MB 11.1%
Runtime dependencies (2)
Package Range Notes
nitro 3.0.260610-beta
undici 7.28.0
Peer dependencies (5)
Package Range Notes
@opentelemetry/api ^1.0.0 optional peer
ai catalog:
braintrust ^3.0.0 optional peer
just-bash ^3.0.0 optional peer
microsandbox ^0.5.0 optional peer
eve init install drill-down

eve init install details

  • Command: eve init my-agent
  • Package manager: npm
  • Installed footprint: 107.24 MB across 8131 installed files
  • Installed packages: 123 total (117 transitive-only)
  • dependencies: 4 direct packages totaling 42.32 MB
  • devDependencies: 2 direct packages totaling 5.04 MB
  • Other transitive package files: 59.88 MB

Installed footprint is measured from an isolated temporary eve init my-agent using the current packed eve tarball.

Heavy installed dependencies

  • @typescript/typescript-linux-x64: 27.95 MB (26.1%)
  • eve: 26.67 MB (24.9%)
  • @rolldown/binding-linux-x64-gnu: 18.96 MB (17.7%)
  • zod: 9.00 MB (8.4%)
  • ai: 6.51 MB (6.1%)
Installed footprint breakdown
Installed package size
🔴 @typescript/typescript-linux-x64 [########################] 27.95 MB 26.1%
🔴 eve                              [#######################.] 26.67 MB 24.9%
🔴 @rolldown/binding-linux-x64-gnu  [################........] 18.96 MB 17.7%
🔴 zod                              [########................] 9.00 MB 8.4%
🔴 ai                               [######..................] 6.51 MB 6.1%
🟠 @types/node                      [##......................] 2.54 MB 2.4%
🔴 Other installed packages         [#############...........] 15.61 MB 14.6%
dependencies (4)
Package Range Installed size Share
@vercel/connect 0.4.2 135.8 kB 0.1%
ai ^7.0.38 6.51 MB 6.1%
eve file:eve-0.27.8.tgz 26.67 MB 24.9%
zod 4.4.3 9.00 MB 8.4%
devDependencies (2)
Package Range Installed size Share
@types/node 24.x 2.54 MB 2.4%
typescript 7.0.2 2.50 MB 2.3%
Function Drill-Down

Payload Size Graph

Unique function payload size and share of total
🔴 functions/.well-known/workflow/v1/flow.func     [########################] 8.26 MB 50.0%
🔴 functions/__server.func                         [########################] 8.26 MB 50.0%

Top Function Payloads

🟠 functions/.well-known/workflow/v1/flow.func • 1 public route • 8.26 MB
Metric Value
Public routes /.well-known/workflow/v1/flow
Runtime nodejs24.x
Handler index.mjs
Payload 8.26 MB
Function files 8.26 MB across 44 files
Traced dependencies 0 B
Signal 🟠 Bundled file _chunks/runtime-artifacts.mjs is 1.58 MB (19.1%)

🟠 🔎 Dependency Analysis

📦 Bundled files:

Bundled file size
🟠 _chunks/runtime-artifacts.mjs [#############...........] 1.58 MB 19.1%
🟠 index.mjs                     [##########..............] 1.18 MB 14.3%
🟡 _libs/undici.mjs              [########................] 925.6 kB 11.2%
🟡 _chunks/world-vercel.mjs      [#######.................] 900.3 kB 10.9%
🟡 _chunks/sandbox.mjs           [######..................] 767.6 kB 9.3%
🟠 Other bundled files           [########################] 2.91 MB 35.2%

🧾 Vercel Config

{
  "handler": "index.mjs",
  "launcherType": "Nodejs",
  "shouldAddHelpers": false,
  "supportsResponseStreaming": true,
  "runtime": "nodejs24.x",
  "maxDuration": "max",
  "experimentalTriggers": [
    {
      "type": "queue/v2beta",
      "topic": "__eve776561746865722d6167656e74_wkf_workflow_*",
      "consumer": "default",
      "retryAfterSeconds": 5,
      "initialDelaySeconds": 0
    }
  ],
  "environment": {
    "WORKFLOW_PRECONDITION_GUARD": "1"
  }
}

🟠 functions/__server.func • 10 public routes, 1 internal alias • 8.26 MB
Metric Value
Public routes /
/eve/v1/callback/[token]
/eve/v1/connections/[name]/callback/[token]
/eve/v1/health
/eve/v1/info
/eve/v1/session
/eve/v1/session/[sessionId]
/eve/v1/session/[sessionId]/cancel
/eve/v1/session/[sessionId]/stream
/eve/v1/session/reset
Internal aliases /__server
Runtime nodejs24.x
Handler index.mjs
Payload 8.26 MB
Function files 8.26 MB across 44 files
Traced dependencies 0 B
Signal 🟠 Bundled file _chunks/runtime-artifacts.mjs is 1.58 MB (19.1%)

🟠 🔎 Dependency Analysis

📦 Bundled files:

Bundled file size
🟠 _chunks/runtime-artifacts.mjs [#############...........] 1.58 MB 19.1%
🟠 index.mjs                     [##########..............] 1.18 MB 14.3%
🟡 _libs/undici.mjs              [########................] 925.6 kB 11.2%
🟡 _chunks/world-vercel.mjs      [#######.................] 900.3 kB 10.9%
🟡 _chunks/sandbox.mjs           [######..................] 767.6 kB 9.3%
🟠 Other bundled files           [########################] 2.91 MB 35.2%

🧾 Vercel Config

{
  "handler": "index.mjs",
  "launcherType": "Nodejs",
  "shouldAddHelpers": false,
  "supportsResponseStreaming": true,
  "runtime": "nodejs24.x"
}

Build Timing: e2e/fixtures/agent-tools-sandbox

This is an informational timing measurement inside eve build, from preflight through publication. Output-size measurement and profile writing are excluded.

Build mode: deployable Vercel build with sandbox template prewarm included.

  • Build pipeline: 1.86 s -> 1.82 s (-43.0 ms) vs main (a69faae).
  • Timing is informational: shared GitHub runners are too variable for a hard timing budget.
Detailed phase timings vs `main (a69faae)`
Phase Baseline Current Delta
extension.check 11.8 ms 5.0 ms -6.8 ms
project.resolve 9.2 ms 6.5 ms -2.7 ms
workspace.create 1.0 ms 1.2 ms +0.2 ms
host.prepare 391.2 ms 181.2 ms -210.0 ms
vercel.service-prefix.resolve 2.0 ms 2.0 ms 0.0 ms
nitro.create 194.5 ms 195.0 ms +0.5 ms
sandbox.prewarm 176.5 ms 204.1 ms +27.6 ms
nitro.cache.prepare 0.3 ms 0.3 ms 0.0 ms
nitro.prepare 0.8 ms 0.9 ms +0.1 ms
nitro.public-assets 0.8 ms 0.8 ms 0.0 ms
nitro.prerender 0.6 ms 0.4 ms -0.2 ms
nitro.bundle 1.07 s 1.22 s +148.1 ms
nitro.cache.write 0.3 ms 0.3 ms 0.0 ms
agent-summary.emit 0.5 ms 0.5 ms 0.0 ms
nitro.close 0.1 ms 0.1 ms 0.0 ms
output.publish 3.5 ms 3.6 ms +0.1 ms
workspace.remove 2.2 ms 2.1 ms -0.1 ms

Review pass over the stamped-event-id work.

Correctness:

- Remove the event deduper's bounded window. Once a stream exceeded the
  capacity, rewinding to the start re-admitted the evicted id, which evicted
  the next, cascading until the whole replay was applied again. Every caller
  already retains an object per event, so the cap bought nothing.
- Tolerate a missing envelope. Events written before stream version 20 have
  no `meta`, so rewinding into an older part of a live session threw instead
  of admitting the event.
- Stop promising that `meta.id` makes hook side effects idempotent. A durable
  step that is interrupted re-runs and re-emits with fresh ids, so
  `on conflict (id) do nothing` does not dedupe a retry. The guarantee is
  scoped to re-reading an already persisted stream.
- Stop promising a total order. Ids are minted per process, so two steps on
  different machines can sort either way; drop the `where id > $cursor`
  pagination guidance in favour of stream position.
- Narrow `RouteContext.agent.getEventStream()`, which still returned
  unstamped events.
- Mark the changeset `minor`: `initialEvents` now requires stamped events.

Structure:

- Extract the ULID generator to `#shared/ulid.ts` with a `createUlidFactory()`
  for callers needing isolated monotonic state, leaving `event-id.ts` as the
  `evt_` prefix layer. Records why the generator is in-repo rather than an npm
  package, and pins the timestamp encoding with golden vectors cross-checked
  against the reference implementation.
- Revert `callAdapterEventHandler` to its original signature and stamp after
  the adapter runs. Adapter handlers only ever receive `event.data`, so they
  never observed the envelope the comments claimed; the parameter change was
  churn across thirteen files.

Also drops development notes, redundant assertions, and an unreachable
`padStart`, and removes the test-only override parameter from
`stampMessageStreamEvent`.

Signed-off-by: Andrew Barba <barba@hey.com>
…vent-ids

# Conflicts:
#	packages/eve/src/channel/schedule.test.ts
#	packages/eve/src/channel/send.test.ts
#	packages/eve/src/channel/session.ts
#	packages/eve/src/channel/types.ts
#	packages/eve/src/client/open-stream.ts
The ULID in the event envelope example decoded to 2025-07-28 while the
`at` beside it read 2026-07-27. The section links the ULID spec and
describes the id as a millisecond timestamp, so the mismatch is
reachable by anyone who follows that.

Signed-off-by: Andrew Barba <barba@hey.com>
meta and meta.at have existed since the first release; version 20 adds
meta.id alone. The docs, the changeset, and the deduper comment all said
older events arrive with no envelope, which would lead a reader to guard
on the wrong thing. The dedupe test built that same impossible shape, so
it covered a case that cannot occur while leaving the real one — an
envelope holding only at — untested. Both are covered now.

The runtime read was already correct: event.meta?.id is undefined either
way.

Signed-off-by: Andrew Barba <barba@hey.com>
The envelope docs framed re-delivery as purely a reconnect concern and
left the retry limitation to a trailing sentence. Three gaps mattered.

Stream writes are batched, and a batch that fails partway re-sends pages
that already landed, so the durable log can hold the same chunk twice.
That, not just reconnects, is why keying on the id is required.

Replay and retry read as synonyms and behave oppositely: replaying a
completed step serves the journaled result and emits nothing, while an
interrupted step re-runs and re-emits under new ids. Only the second
produces duplicates, and the docs never separated them.

The hooks guide told authors to key idempotency on the turn and step
coordinates. That is right for gating a once-per-turn side effect, since
a retry restores those coordinates. It is wrong for stored content: the
retry re-invokes the model, so one coordinate can carry different text
per attempt, and conflict-on-coordinates keeps the abandoned attempt and
drops the one that finished.

Signed-off-by: Andrew Barba <barba@hey.com>
The legacy envelope literal typechecks with a direct assertion, so the
cast through `unknown` was hiding nothing and tripped the mechanical
invariant guard.

Signed-off-by: Andrew Barba <barba@hey.com>
The deduper claimed a partially failed write batch can leave the same
chunk in the durable log twice. It cannot: a rejected batch tears the
pipe down without re-flushing the retained buffer, and nothing retries
`writeMulti`, so a partial page failure loses chunks rather than
duplicating them. Drop the claim from the deduper and the docs.

Also stop advising `event.meta?.id`. `meta.id` is a required field, so
the optional read narrows to `string` and cannot detect the pre-v20
events it was meant to guard. The changeset, the concepts page, and the
field doc now state the exposure plainly instead.

Add an `agent-basic-runtime` e2e eval covering the wire format: every
event served over HTTP carries a well-formed `evt_` id, and a
`startIndex=0` rewind replays those events under the same ids in order.

Prune the comments introduced with this feature down to the non-obvious
why, and drop the narration of earlier iterations.

Signed-off-by: Andrew Barba <barba@hey.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant