Skip to content

feat(think): extract transport from Think into a separate "host" DO#1962

Draft
cjol wants to merge 1 commit into
mainfrom
feat/ingest-streaming
Draft

feat(think): extract transport from Think into a separate "host" DO#1962
cjol wants to merge 1 commit into
mainfrom
feat/ingest-streaming

Conversation

@cjol

@cjol cjol commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

This PR makes a two-Durable-Object topology the standard way to connect Think agents to the outside world: a user-owned host owns the transport, and the agent exposes a single streaming ingest() entry point over Workers RPC. It includes a reference Telegram host built on the Chat SDK, verified end to end against production Telegram and Workers AI. Relates to #1856, #1894, #1946.

Why

  • Transport concerns (webhook routing, messenger delivery, threading, connection config) live inside Think today, entangled with the turn machinery. App code cannot intercept its own channel: slash commands on a Telegram bot have no seam (Question: could Think expose messenger concurrency configuration? #1946 asks for connection config, No supported way to route an external HTTP request (e.g. OAuth callback) to a messenger sub-agent (facet) — getAgentByName wakes a different empty DO #1856 for routing external HTTP), and every transport change risks the turn pipeline. Each problem could be fixed independently, but together point to a broader solution.
  • Extending Think inward was prototyped first: an onChannelEvent interception hook, channel-registered HTTP routes on the agent, and a static worker-level routing manifest. All three were rejected. Each grew agent-side API for what is host business, and the hostless topology they preserved was not worth its surface area.
  • The chosen contract inverts ownership: the host owns transport end to end and calls the agent over native DO RPC. ingest() has one variant, a byte stream. Hosts that want wait semantics buffer on their side (collectIngestReply), so the interface does not carry three modes.
  • Durable reply delivery is now a host concern and is deliberately omitted from this PR. The deprecated Think-owned messenger runtime keeps its fiber-based recovery unchanged; hosts own their retry story for now.
  • deliverNotice() is not implemented for host-driven channels yet (they have no delivery surface; it throws). Out-of-turn agent-to-host pushes are follow-up work.
  • The built-in web WebSocket surface is untouched. The intent is to eventually re-express it as a host too and delete that transport code from Think.

Public API Surface

Symbol Kind Notes
Think.ingest(input) method Experimental. Primary invocation route for host-owned transports. Thin facade over runTurn(); returns an NDJSON-framed ReadableStream<Uint8Array> that crosses the RPC boundary.
IngestInput, IngestStreamEvent, IngestReply types The wire contract: delta, done, and error frames.
decodeIngestStream(stream) function NDJSON decode helper for hosts.
collectIngestReply(stream) function Wait semantics by buffering.
ChannelDefinition.kind, ChannelDefinition.ingress change Now optional (additive). A definition with neither is pure behaviour policy: instructions, tools, maxTurns.

Deprecated with no behavior change: getMessengers(), messengerChannel(), ChannelKind, ChannelIngress, and the kind, ingress, capabilities, conversation, delivery channel fields. Existing messenger apps keep working.

Architectural Changes

Before, the agent DO owned the transport:

Telegram -> webhook -> Think DO (route matching, Chat SDK singleton,
                        delivery fibers, turn pipeline, transcript)

After, transport is app code in a host, and the agent is a callee:

Telegram -> Host worker (Chat SDK adapter, ChatStateDO for threading/
            dedupe/locks, slash commands, delivery)
              -> getAgentByName(env.HostedAgent, thread).ingest(...)
                   -> Agent DO (channel policy + turn pipeline + transcript)

Things this makes easy that were hard or impossible before:

  • Slash commands: chat.onSlashCommand("/help", ...) in the host answers without waking the agent DO or running a model turn. Previously there was no seam at all between the webhook and the model.
  • Richer messenger experiences: the full Chat SDK surface (modals, reactions, thread subscriptions, typing, post+edit streaming) is ordinary app code in the host rather than whatever the framework chose to expose.
  • Connection tuning: concurrency, debounce, and adapter options are the app's own Chat constructor arguments, which is what Question: could Think expose messenger concurrency configuration? #1946 asked for.

Code Changes

  • packages/think/src/ingest.ts (new): the wire contract and the stream producer, kept out of think.ts so the host-facing contract is reviewable in isolation. The stream is an observation tap: the turn starts eagerly and completes even if the caller never reads or cancels. Consumer backpressure is deliberately not propagated to the model turn, matching the existing web WebSocket surface; the buffer is bounded by one turn's output, and a cancelled stream stops buffering immediately without aborting the turn.
  • packages/think/src/think.ts: adds the ingest() method (with a TSDoc invariant that it must stay a thin facade over runTurn() and never grow its own turn semantics), defaults omitted kind to "custom" at the read sites, and adds the deprecation notice on getMessengers(). The turn pipeline, recovery, and messenger runtime are otherwise untouched.
  • packages/think/src/channels/index.ts: ingress and kind become optional, policy-only definitions validate, kind: "messenger" without webhook ingress throws, deprecation notices on the transport-flavoured surface.
  • examples/channel-host-telegram: the reference host. Chat SDK Telegram bot in the worker with state in its own DO via the community chat-state-cloudflare-do adapter, which should be vendored into this repo before the pattern is promoted. One agent instance per Telegram thread; replies stream from ingest() into thread.post for post+edit streaming.
  • docs/think/channels.md: rewritten around behaviour policy and ingest(), with an entry-point map (ingest for external events, web WS for browser chat, runTurn()/chat() in process, deprecated messenger webhooks). docs/think/messengers.md gains a deprecation banner.

Compatibility

  • All changes are additive or deprecation notices. getMessengers() apps behave identically.
  • ingest() is experimental; its wire format may change while Think is experimental.

…streaming ingest()

This PR makes a two-Durable-Object topology the standard way to connect Think agents to the outside world: a user-owned host owns the transport, and the agent exposes a single streaming `ingest()` entry point over Workers RPC. It includes a reference Telegram host built on the Chat SDK, verified end to end against production Telegram and Workers AI. Relates to #1856, #1894, #1946.

## Why

- Transport concerns (webhook routing, messenger delivery, threading, connection config) live inside `Think` today, entangled with the turn machinery. App code cannot intercept its own channel: slash commands on a Telegram bot have no seam (#1946 asks for connection config, #1856 for routing external HTTP), and every transport change risks the turn pipeline.
- Extending Think inward was prototyped first: an `onChannelEvent` interception hook, channel-registered HTTP routes on the agent, and a static worker-level routing manifest. All three were rejected. Each grew agent-side API for what is host business, and the hostless topology they preserved was not worth its surface area.
- The chosen contract inverts ownership: the host owns transport end to end and calls the agent over native DO RPC. `ingest()` has one variant, a byte stream. Hosts that want wait semantics buffer on their side (`collectIngestReply`), so the interface does not carry three modes.
- Durable reply delivery is now a host concern and is deliberately omitted from this PR. The deprecated Think-owned messenger runtime keeps its fiber-based recovery unchanged; hosts own their retry story for now.
- `deliverNotice()` is not implemented for host-driven channels yet (they have no delivery surface; it throws). Out-of-turn agent-to-host pushes are follow-up work.
- The built-in web WebSocket surface is untouched. The intent is to eventually re-express it as a host too and delete that transport code from Think.

## Public API Surface

| Symbol | Kind | Notes |
| --- | --- | --- |
| `Think.ingest(input)` | method | Experimental. Primary invocation route for host-owned transports. Thin facade over `runTurn()`; returns an NDJSON-framed `ReadableStream<Uint8Array>` that crosses the RPC boundary. |
| `IngestInput`, `IngestStreamEvent`, `IngestReply` | types | The wire contract: `delta`, `done`, and `error` frames. |
| `decodeIngestStream(stream)` | function | NDJSON decode helper for hosts. |
| `collectIngestReply(stream)` | function | Wait semantics by buffering. |
| `ChannelDefinition.kind`, `ChannelDefinition.ingress` | change | Now optional (additive). A definition with neither is pure behaviour policy: `instructions`, `tools`, `maxTurns`. |

Deprecated with no behavior change: `getMessengers()`, `messengerChannel()`, `ChannelKind`, `ChannelIngress`, and the `kind`, `ingress`, `capabilities`, `conversation`, `delivery` channel fields. Existing messenger apps keep working.

## Architectural Changes

Before, the agent DO owned the transport:

```
Telegram -> webhook -> Think DO (route matching, Chat SDK singleton,
                        delivery fibers, turn pipeline, transcript)
```

After, transport is app code in a host, and the agent is a callee:

```
Telegram -> Host worker (Chat SDK adapter, ChatStateDO for threading/
            dedupe/locks, slash commands, delivery)
              -> getAgentByName(env.HostedAgent, thread).ingest(...)
                   -> Agent DO (channel policy + turn pipeline + transcript)
```

Things this makes easy that were hard or impossible before:

- Slash commands: `chat.onSlashCommand("/help", ...)` in the host answers without waking the agent DO or running a model turn. Previously there was no seam at all between the webhook and the model.
- Richer messenger experiences: the full Chat SDK surface (modals, reactions, thread subscriptions, typing, post+edit streaming) is ordinary app code in the host rather than whatever the framework chose to expose.
- Connection tuning: concurrency, debounce, and adapter options are the app's own `Chat` constructor arguments, which is what #1946 asked for.

## Code Changes

- `packages/think/src/ingest.ts` (new): the wire contract and the stream producer, kept out of `think.ts` so the host-facing contract is reviewable in isolation. The stream is an observation tap: the turn starts eagerly and completes even if the caller never reads or cancels. Consumer backpressure is deliberately not propagated to the model turn, matching the existing web WebSocket surface; the buffer is bounded by one turn's output, and a cancelled stream stops buffering immediately without aborting the turn.
- `packages/think/src/think.ts`: adds the `ingest()` method (with a TSDoc invariant that it must stay a thin facade over `runTurn()` and never grow its own turn semantics), defaults omitted `kind` to `"custom"` at the read sites, and adds the deprecation notice on `getMessengers()`. The turn pipeline, recovery, and messenger runtime are otherwise untouched.
- `packages/think/src/channels/index.ts`: `ingress` and `kind` become optional, policy-only definitions validate, `kind: "messenger"` without webhook ingress throws, deprecation notices on the transport-flavoured surface.
- `examples/channel-host-telegram`: the reference host. Chat SDK Telegram bot in the worker with state in its own DO via the community [`chat-state-cloudflare-do`](https://chat-sdk.dev/adapters/community/cloudflare-do) adapter, which should be vendored into this repo before the pattern is promoted. One agent instance per Telegram thread; replies stream from `ingest()` into `thread.post` for post+edit streaming.
- `docs/think/channels.md`: rewritten around behaviour policy and `ingest()`, with an entry-point map (ingest for external events, web WS for browser chat, `runTurn()`/`chat()` in process, deprecated messenger webhooks). `docs/think/messengers.md` gains a deprecation banner.

## Compatibility

- All changes are additive or deprecation notices. `getMessengers()` apps behave identically.
- `ingest()` is experimental; its wire format may change while Think is experimental.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@changeset-bot

changeset-bot Bot commented Jul 17, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 130235c

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages
Name Type
@cloudflare/think Minor
@cloudflare/agent-think Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@pkg-pr-new

pkg-pr-new Bot commented Jul 17, 2026

Copy link
Copy Markdown

Open in StackBlitz

agents

npm i https://pkg.pr.new/agents@1962

@cloudflare/ai-chat

npm i https://pkg.pr.new/@cloudflare/ai-chat@1962

@cloudflare/codemode

npm i https://pkg.pr.new/@cloudflare/codemode@1962

create-think

npm i https://pkg.pr.new/create-think@1962

hono-agents

npm i https://pkg.pr.new/hono-agents@1962

@cloudflare/shell

npm i https://pkg.pr.new/@cloudflare/shell@1962

@cloudflare/think

npm i https://pkg.pr.new/@cloudflare/think@1962

@cloudflare/voice

npm i https://pkg.pr.new/@cloudflare/voice@1962

@cloudflare/worker-bundler

npm i https://pkg.pr.new/@cloudflare/worker-bundler@1962

commit: 130235c

@cjol cjol changed the title feat(think): two-DO topology: host-owned transport drives agents via streaming ingest() feat(think): extract transport from Think into a separate "host" DO Jul 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant