feat(think): extract transport from Think into a separate "host" DO#1962
Draft
cjol wants to merge 1 commit into
Draft
feat(think): extract transport from Think into a separate "host" DO#1962cjol wants to merge 1 commit into
cjol wants to merge 1 commit into
Conversation
…streaming ingest() This PR makes a two-Durable-Object topology the standard way to connect Think agents to the outside world: a user-owned host owns the transport, and the agent exposes a single streaming `ingest()` entry point over Workers RPC. It includes a reference Telegram host built on the Chat SDK, verified end to end against production Telegram and Workers AI. Relates to #1856, #1894, #1946. ## Why - Transport concerns (webhook routing, messenger delivery, threading, connection config) live inside `Think` today, entangled with the turn machinery. App code cannot intercept its own channel: slash commands on a Telegram bot have no seam (#1946 asks for connection config, #1856 for routing external HTTP), and every transport change risks the turn pipeline. - Extending Think inward was prototyped first: an `onChannelEvent` interception hook, channel-registered HTTP routes on the agent, and a static worker-level routing manifest. All three were rejected. Each grew agent-side API for what is host business, and the hostless topology they preserved was not worth its surface area. - The chosen contract inverts ownership: the host owns transport end to end and calls the agent over native DO RPC. `ingest()` has one variant, a byte stream. Hosts that want wait semantics buffer on their side (`collectIngestReply`), so the interface does not carry three modes. - Durable reply delivery is now a host concern and is deliberately omitted from this PR. The deprecated Think-owned messenger runtime keeps its fiber-based recovery unchanged; hosts own their retry story for now. - `deliverNotice()` is not implemented for host-driven channels yet (they have no delivery surface; it throws). Out-of-turn agent-to-host pushes are follow-up work. - The built-in web WebSocket surface is untouched. The intent is to eventually re-express it as a host too and delete that transport code from Think. ## Public API Surface | Symbol | Kind | Notes | | --- | --- | --- | | `Think.ingest(input)` | method | Experimental. Primary invocation route for host-owned transports. Thin facade over `runTurn()`; returns an NDJSON-framed `ReadableStream<Uint8Array>` that crosses the RPC boundary. | | `IngestInput`, `IngestStreamEvent`, `IngestReply` | types | The wire contract: `delta`, `done`, and `error` frames. | | `decodeIngestStream(stream)` | function | NDJSON decode helper for hosts. | | `collectIngestReply(stream)` | function | Wait semantics by buffering. | | `ChannelDefinition.kind`, `ChannelDefinition.ingress` | change | Now optional (additive). A definition with neither is pure behaviour policy: `instructions`, `tools`, `maxTurns`. | Deprecated with no behavior change: `getMessengers()`, `messengerChannel()`, `ChannelKind`, `ChannelIngress`, and the `kind`, `ingress`, `capabilities`, `conversation`, `delivery` channel fields. Existing messenger apps keep working. ## Architectural Changes Before, the agent DO owned the transport: ``` Telegram -> webhook -> Think DO (route matching, Chat SDK singleton, delivery fibers, turn pipeline, transcript) ``` After, transport is app code in a host, and the agent is a callee: ``` Telegram -> Host worker (Chat SDK adapter, ChatStateDO for threading/ dedupe/locks, slash commands, delivery) -> getAgentByName(env.HostedAgent, thread).ingest(...) -> Agent DO (channel policy + turn pipeline + transcript) ``` Things this makes easy that were hard or impossible before: - Slash commands: `chat.onSlashCommand("/help", ...)` in the host answers without waking the agent DO or running a model turn. Previously there was no seam at all between the webhook and the model. - Richer messenger experiences: the full Chat SDK surface (modals, reactions, thread subscriptions, typing, post+edit streaming) is ordinary app code in the host rather than whatever the framework chose to expose. - Connection tuning: concurrency, debounce, and adapter options are the app's own `Chat` constructor arguments, which is what #1946 asked for. ## Code Changes - `packages/think/src/ingest.ts` (new): the wire contract and the stream producer, kept out of `think.ts` so the host-facing contract is reviewable in isolation. The stream is an observation tap: the turn starts eagerly and completes even if the caller never reads or cancels. Consumer backpressure is deliberately not propagated to the model turn, matching the existing web WebSocket surface; the buffer is bounded by one turn's output, and a cancelled stream stops buffering immediately without aborting the turn. - `packages/think/src/think.ts`: adds the `ingest()` method (with a TSDoc invariant that it must stay a thin facade over `runTurn()` and never grow its own turn semantics), defaults omitted `kind` to `"custom"` at the read sites, and adds the deprecation notice on `getMessengers()`. The turn pipeline, recovery, and messenger runtime are otherwise untouched. - `packages/think/src/channels/index.ts`: `ingress` and `kind` become optional, policy-only definitions validate, `kind: "messenger"` without webhook ingress throws, deprecation notices on the transport-flavoured surface. - `examples/channel-host-telegram`: the reference host. Chat SDK Telegram bot in the worker with state in its own DO via the community [`chat-state-cloudflare-do`](https://chat-sdk.dev/adapters/community/cloudflare-do) adapter, which should be vendored into this repo before the pattern is promoted. One agent instance per Telegram thread; replies stream from `ingest()` into `thread.post` for post+edit streaming. - `docs/think/channels.md`: rewritten around behaviour policy and `ingest()`, with an entry-point map (ingest for external events, web WS for browser chat, `runTurn()`/`chat()` in process, deprecated messenger webhooks). `docs/think/messengers.md` gains a deprecation banner. ## Compatibility - All changes are additive or deprecation notices. `getMessengers()` apps behave identically. - `ingest()` is experimental; its wire format may change while Think is experimental. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
🦋 Changeset detectedLatest commit: 130235c The changes in this PR will be included in the next version bump. This PR includes changesets to release 2 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
agents
@cloudflare/ai-chat
@cloudflare/codemode
create-think
hono-agents
@cloudflare/shell
@cloudflare/think
@cloudflare/voice
@cloudflare/worker-bundler
commit: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR makes a two-Durable-Object topology the standard way to connect Think agents to the outside world: a user-owned host owns the transport, and the agent exposes a single streaming
ingest()entry point over Workers RPC. It includes a reference Telegram host built on the Chat SDK, verified end to end against production Telegram and Workers AI. Relates to #1856, #1894, #1946.Why
Thinktoday, entangled with the turn machinery. App code cannot intercept its own channel: slash commands on a Telegram bot have no seam (Question: could Think expose messenger concurrency configuration? #1946 asks for connection config, No supported way to route an external HTTP request (e.g. OAuth callback) to a messenger sub-agent (facet) — getAgentByName wakes a different empty DO #1856 for routing external HTTP), and every transport change risks the turn pipeline. Each problem could be fixed independently, but together point to a broader solution.onChannelEventinterception hook, channel-registered HTTP routes on the agent, and a static worker-level routing manifest. All three were rejected. Each grew agent-side API for what is host business, and the hostless topology they preserved was not worth its surface area.ingest()has one variant, a byte stream. Hosts that want wait semantics buffer on their side (collectIngestReply), so the interface does not carry three modes.deliverNotice()is not implemented for host-driven channels yet (they have no delivery surface; it throws). Out-of-turn agent-to-host pushes are follow-up work.Public API Surface
Think.ingest(input)runTurn(); returns an NDJSON-framedReadableStream<Uint8Array>that crosses the RPC boundary.IngestInput,IngestStreamEvent,IngestReplydelta,done, anderrorframes.decodeIngestStream(stream)collectIngestReply(stream)ChannelDefinition.kind,ChannelDefinition.ingressinstructions,tools,maxTurns.Deprecated with no behavior change:
getMessengers(),messengerChannel(),ChannelKind,ChannelIngress, and thekind,ingress,capabilities,conversation,deliverychannel fields. Existing messenger apps keep working.Architectural Changes
Before, the agent DO owned the transport:
After, transport is app code in a host, and the agent is a callee:
Things this makes easy that were hard or impossible before:
chat.onSlashCommand("/help", ...)in the host answers without waking the agent DO or running a model turn. Previously there was no seam at all between the webhook and the model.Chatconstructor arguments, which is what Question: could Think expose messenger concurrency configuration? #1946 asked for.Code Changes
packages/think/src/ingest.ts(new): the wire contract and the stream producer, kept out ofthink.tsso the host-facing contract is reviewable in isolation. The stream is an observation tap: the turn starts eagerly and completes even if the caller never reads or cancels. Consumer backpressure is deliberately not propagated to the model turn, matching the existing web WebSocket surface; the buffer is bounded by one turn's output, and a cancelled stream stops buffering immediately without aborting the turn.packages/think/src/think.ts: adds theingest()method (with a TSDoc invariant that it must stay a thin facade overrunTurn()and never grow its own turn semantics), defaults omittedkindto"custom"at the read sites, and adds the deprecation notice ongetMessengers(). The turn pipeline, recovery, and messenger runtime are otherwise untouched.packages/think/src/channels/index.ts:ingressandkindbecome optional, policy-only definitions validate,kind: "messenger"without webhook ingress throws, deprecation notices on the transport-flavoured surface.examples/channel-host-telegram: the reference host. Chat SDK Telegram bot in the worker with state in its own DO via the communitychat-state-cloudflare-doadapter, which should be vendored into this repo before the pattern is promoted. One agent instance per Telegram thread; replies stream fromingest()intothread.postfor post+edit streaming.docs/think/channels.md: rewritten around behaviour policy andingest(), with an entry-point map (ingest for external events, web WS for browser chat,runTurn()/chat()in process, deprecated messenger webhooks).docs/think/messengers.mdgains a deprecation banner.Compatibility
getMessengers()apps behave identically.ingest()is experimental; its wire format may change while Think is experimental.