Skip to content
Open
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions docs/chat/streaming.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,44 @@ TanStack AI implements the [AG-UI Protocol](https://docs.ag-ui.com/introduction)

> **Tip:** Some models expose their internal reasoning as thinking content that streams before the response. See [Thinking & Reasoning](./thinking-content).

### Type-Safe Tool Call Events

When you pass typed tools (defined with `toolDefinition()` and Zod schemas) to `chat()`, the stream chunks automatically carry type information for tool call events. The `toolName` field narrows to the union of your tool name literals, and the `input` field on `TOOL_CALL_END` events is typed as the union of your tool input schemas:

```typescript
import { chat, toolDefinition } from "@tanstack/ai";
import { openaiText } from "@tanstack/ai-openai";
import { z } from "zod";

const weatherTool = toolDefinition({
name: "get_weather",
description: "Get weather for a location",
inputSchema: z.object({
location: z.string(),
unit: z.enum(["celsius", "fahrenheit"]).optional(),
}),
});

const stream = chat({
adapter: openaiText("gpt-5.2"),
messages,
tools: [weatherTool],
});

for await (const chunk of stream) {
if (chunk.type === "TOOL_CALL_END") {
chunk.toolName; // ✅ typed as "get_weather" (not string)
chunk.input; // ✅ typed as { location: string; unit?: "celsius" | "fahrenheit" }
}
}
```

Without typed tools, `toolName` defaults to `string` and `input` defaults to `unknown` — the same behavior as before. The type narrowing is automatic when you use `toolDefinition()` with Zod schemas.

> **Note:** When multiple tools are provided, `input` is typed as the union of all tool input types. Checking `toolName === 'get_weather'` does not narrow `input` to that specific tool's input type — if you need per-tool discrimination, use a type guard after the `toolName` check.

> **Tip:** The typed stream chunk type is exported as `TypedStreamChunk<TTools>` if you need to annotate variables or function parameters. When used without type arguments, `TypedStreamChunk` is equivalent to `StreamChunk`.
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Outdated

### Thinking Chunks

Thinking/reasoning is represented by AG-UI events `STEP_STARTED` and `STEP_FINISHED`. They stream separately from the final response text:
Expand Down
36 changes: 35 additions & 1 deletion docs/reference/type-aliases/StreamChunk.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,41 @@ title: StreamChunk
type StreamChunk = AGUIEvent;
```

Defined in: [types.ts:976](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L976)
Defined in: [types.ts:989](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L989)

Chunk returned by the SDK during streaming chat completions.
Uses the AG-UI protocol event format.

# Type Alias: TypedStreamChunk

```ts
type TypedStreamChunk<TTools extends ReadonlyArray<Tool<any, any, any>> = ReadonlyArray<Tool<any, any, any>>>
```

Defined in: [types.ts:1033](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L1033)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Update the TypedStreamChunk source anchor.

This Defined in link still points to types.ts:1033, but the alias is declared at packages/typescript/ai/src/types.ts Line 1048 in this PR, so the reference page is already stale. As per coding guidelines, docs/**/*.md: Use Markdown for documentation in the docs/ directory with auto-generated docs via TypeDoc.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@docs/reference/type-aliases/StreamChunk.md` at line 23, The "Defined in"
source anchor for the TypedStreamChunk entry is stale; update the anchor in
docs/reference/type-aliases/StreamChunk.md to point to the current declaration
location packages/typescript/ai/src/types.ts at line 1048 (or to the exact
URL/permalink generated by TypeDoc for TypedStreamChunk), ensuring the link text
and URL reflect the new file and line; update the markdown so it uses the
correct Markdown link format and matches the repo's TypeDoc-generated anchor for
TypedStreamChunk.


A variant of `StreamChunk` parameterized by the tools array. When specific tool types are provided (e.g. from `chat({ tools: [myTool] })`):

- `TOOL_CALL_START` and `TOOL_CALL_END` events have `toolName` narrowed to the union of known tool name literals.
- `TOOL_CALL_END` events have `input` typed as the union of tool input types.

When tools are untyped or absent, `TypedStreamChunk` degrades to the same type as `StreamChunk`.

This is the type returned by `chat()` when streaming is enabled (the default). You don't typically need to reference it directly unless annotating function parameters or return types.

```ts
import type { TypedStreamChunk } from "@tanstack/ai";
import { toolDefinition } from "@tanstack/ai";

// Given tools created with toolDefinition():
const weatherTool = toolDefinition({ name: "get_weather", description: "...", inputSchema: /* Zod schema */ });
const searchTool = toolDefinition({ name: "search", description: "...", inputSchema: /* Zod schema */ });

// Without type args — equivalent to StreamChunk
type Chunk = TypedStreamChunk;

// With specific tools — tool call events are typed
type TypedChunk = TypedStreamChunk<[typeof weatherTool, typeof searchTool]>;
```

See [Streaming - Type-Safe Tool Call Events](../../chat/streaming) for a practical walkthrough.
2 changes: 2 additions & 0 deletions docs/tools/tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,8 @@ const inputSchema: JSONSchema = {

> **Note:** When using JSON Schema, TypeScript will infer `any` for input/output types since JSON Schema cannot provide compile-time type information. Zod schemas are recommended for full type safety.

> **Tip:** Type safety from Zod schemas extends beyond tool execution — when you iterate over the stream returned by `chat()`, tool call events have typed `toolName` and `input` fields too. See [Type-Safe Tool Call Events](../chat/streaming#type-safe-tool-call-events).

## Tool Definition

Tools are defined using `toolDefinition()` from `@tanstack/ai`:
Expand Down
64 changes: 64 additions & 0 deletions examples/ts-react-chat/src/routes/api.tanchat.ts
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,70 @@ const loggingMiddleware: ChatMiddleware = {
},
}

// ===========================
// TypedStreamChunk showcase — type-safe tool call events
// ===========================
//
// When `chat()` receives tools with typed schemas, the returned stream
// carries type information on TOOL_CALL_START and TOOL_CALL_END events.
// No casts, no `as any` — just narrow by `chunk.type` and everything is typed.

const tools = [
getGuitars,
recommendGuitarToolDef,
addToCartToolServer,
addToWishListToolDef,
getPersonalGuitarPreferenceToolDef,
compareGuitars,
calculateFinancing,
searchGuitars,
] as const

async function typedStreamShowcase() {
const stream = chat({
adapter: openaiText('gpt-4o'),
messages: [
{ role: 'user' as const, content: 'Recommend an acoustic guitar' },
],
tools,
})

for await (const chunk of stream) {
switch (chunk.type) {
case 'TOOL_CALL_START':
// ✅ chunk.toolName is typed as the union of all tool name literals:
// 'getGuitars' | 'recommendGuitar' | 'addToCart' | 'addToWishList'
// | 'getPersonalGuitarPreference' | 'compareGuitars'
// | 'calculateFinancing' | 'searchGuitars'
//
// ❌ Without TypedStreamChunk, this would just be `string`
console.log(`Tool call started: ${chunk.toolName}`)
break

case 'TOOL_CALL_END':
// ✅ chunk.toolName — same typed literal union as above
// ✅ chunk.input — union of all tool input types, inferred from Zod schemas:
// | {}
// | { id: string | number }
// | { guitarId: string; quantity: number }
// | { guitarId: string }
// | { guitarIds: number[] }
// | { guitarId: number; months: number }
// | { query: string }
console.log(`Tool call ended: ${chunk.toolName}`, chunk.input)
break

case 'TEXT_MESSAGE_CONTENT':
// Non-tool events are unaffected — still fully typed
console.log(chunk.delta)
break
}
}
}

// Suppress unused warning — this is a showcase, not called at runtime
void typedStreamShowcase

export const Route = createFileRoute('/api/tanchat')({
server: {
handlers: {
Expand Down
39 changes: 28 additions & 11 deletions packages/typescript/ai/src/activities/chat/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ import type {
ToolCallArgsEvent,
ToolCallEndEvent,
ToolCallStartEvent,
TypedStreamChunk,
} from '../../types'
import type {
ChatMiddleware,
Expand All @@ -69,11 +70,15 @@ export const kind = 'text' as const
* @template TAdapter - The text adapter type (created by a provider function)
* @template TSchema - Optional Standard Schema for structured output
* @template TStream - Whether to stream the output (default: true)
* @template TTools - The tools array type for type-safe tool call events in the stream
*/
export interface TextActivityOptions<
TAdapter extends AnyTextAdapter,
TSchema extends SchemaInput | undefined,
TStream extends boolean,
TTools extends ReadonlyArray<Tool<any, any, any>> = ReadonlyArray<
Tool<any, any, any>
>,
> {
/** The text adapter to use (created by a provider function like openaiText('gpt-4o')) */
adapter: TAdapter
Expand All @@ -87,7 +92,7 @@ export interface TextActivityOptions<
/** System prompts to prepend to the conversation */
systemPrompts?: TextOptions['systemPrompts']
/** Tools for function calling (auto-executed when called) */
tools?: TextOptions['tools']
tools?: TTools
/** Controls the randomness of the output. Higher values make output more random. Range: [0.0, 2.0] */
temperature?: TextOptions['temperature']
/** Nucleus sampling parameter. The model considers tokens with topP probability mass. */
Expand Down Expand Up @@ -125,7 +130,7 @@ export interface TextActivityOptions<
outputSchema?: TSchema
/**
* Whether to stream the text result.
* When true (default), returns an AsyncIterable<StreamChunk> for streaming output.
* When true (default), returns an AsyncIterable<TypedStreamChunk<TTools>> for streaming output.
* When false, returns a Promise<string> with the collected text content.
*
* Note: If outputSchema is provided, this option is ignored and the result
Expand Down Expand Up @@ -186,9 +191,12 @@ export function createChatOptions<
TAdapter extends AnyTextAdapter,
TSchema extends SchemaInput | undefined = undefined,
TStream extends boolean = true,
TTools extends ReadonlyArray<Tool<any, any, any>> = ReadonlyArray<
Tool<any, any, any>
>,
>(
options: TextActivityOptions<TAdapter, TSchema, TStream>,
): TextActivityOptions<TAdapter, TSchema, TStream> {
options: TextActivityOptions<TAdapter, TSchema, TStream, TTools>,
): TextActivityOptions<TAdapter, TSchema, TStream, TTools> {
return options
}

Expand All @@ -200,16 +208,22 @@ export function createChatOptions<
* Result type for the text activity.
* - If outputSchema is provided: Promise<InferSchemaType<TSchema>>
* - If stream is false: Promise<string>
* - Otherwise (stream is true, default): AsyncIterable<StreamChunk>
* - Otherwise (stream is true, default): AsyncIterable<TypedStreamChunk<TTools>>
*
* When tools with typed schemas are provided, the stream chunks include
* type-safe `toolName` and `input` fields on tool call events.
*/
export type TextActivityResult<
TSchema extends SchemaInput | undefined,
TStream extends boolean = true,
TTools extends ReadonlyArray<Tool<any, any, any>> = ReadonlyArray<
Tool<any, any, any>
>,
> = TSchema extends SchemaInput
? Promise<InferSchemaType<TSchema>>
: TStream extends false
? Promise<string>
: AsyncIterable<StreamChunk>
: AsyncIterable<TypedStreamChunk<TTools>>

// ===========================
// ChatEngine Implementation
Expand Down Expand Up @@ -1374,9 +1388,12 @@ export function chat<
TAdapter extends AnyTextAdapter,
TSchema extends SchemaInput | undefined = undefined,
TStream extends boolean = true,
TTools extends ReadonlyArray<Tool<any, any, any>> = ReadonlyArray<
Tool<any, any, any>
>,
>(
options: TextActivityOptions<TAdapter, TSchema, TStream>,
): TextActivityResult<TSchema, TStream> {
options: TextActivityOptions<TAdapter, TSchema, TStream, TTools>,
): TextActivityResult<TSchema, TStream, TTools> {
const { outputSchema, stream } = options

// If outputSchema is provided, run agentic structured output
Expand All @@ -1387,7 +1404,7 @@ export function chat<
SchemaInput,
boolean
>,
) as TextActivityResult<TSchema, TStream>
) as TextActivityResult<TSchema, TStream, TTools>
}

// If stream is explicitly false, run non-streaming text
Expand All @@ -1398,13 +1415,13 @@ export function chat<
undefined,
false
>,
) as TextActivityResult<TSchema, TStream>
) as TextActivityResult<TSchema, TStream, TTools>
}

// Otherwise, run streaming text (default)
return runStreamingText(
options as unknown as TextActivityOptions<AnyTextAdapter, undefined, true>,
) as TextActivityResult<TSchema, TStream>
) as TextActivityResult<TSchema, TStream, TTools>
}

/**
Expand Down
Loading
Loading