streaming.md

title	Streaming
id	streaming-responses
order	2

TanStack AI supports streaming responses for real-time chat experiences. Streaming allows you to display responses as they're generated, rather than waiting for the complete response.

How Streaming Works

When you use chat(), it returns an async iterable stream of chunks:

import { chat } from "@tanstack/ai";
import { openaiText } from "@tanstack/ai-openai";

const stream = chat({
  adapter: openaiText("gpt-5.2"),
  messages,
});

// Stream contains chunks as they arrive
for await (const chunk of stream) {
  console.log(chunk); // Process each chunk
}

Server-Side Streaming

Convert the stream to an HTTP response using toServerSentEventsResponse:

import { chat, toServerSentEventsResponse } from "@tanstack/ai";
import { openaiText } from "@tanstack/ai-openai";

export async function POST(request: Request) {
  const { messages } = await request.json();

  const stream = chat({
    adapter: openaiText("gpt-5.2"),
    messages,
  });

  // Convert to HTTP response with proper headers
  return toServerSentEventsResponse(stream);
}

Client-Side Streaming

The useChat hook automatically handles streaming:

import { useChat, fetchServerSentEvents } from "@tanstack/ai-react";

const { messages, sendMessage, isLoading } = useChat({
  connection: fetchServerSentEvents("/api/chat"),
});

// Messages update in real-time as chunks arrive
messages.forEach((message) => {
  // Message content updates incrementally
});

Stream Events (AG-UI Protocol)

TanStack AI implements the AG-UI Protocol for streaming. Stream events contain different types of data:

AG-UI Events

• RUN_STARTED - Emitted when a run begins
• TEXT_MESSAGE_START/CONTENT/END - Text content streaming lifecycle
• TOOL_CALL_START/ARGS/END - Tool invocation lifecycle
• STEP_STARTED/STEP_FINISHED - Thinking/reasoning steps
• RUN_FINISHED - Run completion with finish reason and usage
• RUN_ERROR - Error occurred during the run

Tip: Some models expose their internal reasoning as thinking content that streams before the response. See Thinking & Reasoning.

Type-Safe Tool Call Events

When you pass typed tools (defined with toolDefinition() and Zod schemas) to chat(), the stream chunks automatically carry type information for tool call events. The toolName field narrows to the union of your tool name literals, and the input field on TOOL_CALL_END events is typed as the union of your tool input schemas:

import { chat, toolDefinition } from "@tanstack/ai";
import { openaiText } from "@tanstack/ai-openai";
import { z } from "zod";

const weatherTool = toolDefinition({
  name: "get_weather",
  description: "Get weather for a location",
  inputSchema: z.object({
    location: z.string(),
    unit: z.enum(["celsius", "fahrenheit"]).optional(),
  }),
});

const stream = chat({
  adapter: openaiText("gpt-5.2"),
  messages,
  tools: [weatherTool],
});

for await (const chunk of stream) {
  if (chunk.type === "TOOL_CALL_END") {
    chunk.toolName; // ✅ typed as "get_weather" (not string)
    chunk.input;    // ✅ typed as { location: string; unit?: "celsius" | "fahrenheit" }
  }
}

Without typed tools, toolName defaults to string and input defaults to unknown — the same behavior as before. The type narrowing is automatic when you use toolDefinition() with Zod schemas.

Note: When multiple tools are provided, input is typed as the union of all tool input types. Checking toolName === 'get_weather' does not narrow input to that specific tool's input type — if you need per-tool discrimination, use a type guard after the toolName check.

Tip: The typed stream chunk type is exported as TypedStreamChunk<TTools> if you need to annotate variables or function parameters. When used without type arguments, TypedStreamChunk is equivalent to StreamChunk.

Thinking Chunks

Thinking/reasoning is represented by AG-UI events STEP_STARTED and STEP_FINISHED. They stream separately from the final response text:

for await (const chunk of stream) {
  if (chunk.type === "STEP_FINISHED") {
    console.log("Thinking:", chunk.content); // Accumulated thinking content
    console.log("Delta:", chunk.delta); // Incremental thinking token
  }
}

Thinking content is automatically converted to ThinkingPart in UIMessage objects. It is UI-only and excluded from messages sent back to the model.

Connection Adapters

TanStack AI provides connection adapters for different streaming protocols:

Server-Sent Events (SSE)

import { useChat, fetchServerSentEvents } from "@tanstack/ai-react";

const { messages } = useChat({
  connection: fetchServerSentEvents("/api/chat"),
});

HTTP Stream

import { useChat, fetchHttpStream } from "@tanstack/ai-react";

const { messages } = useChat({
  connection: fetchHttpStream("/api/chat"),
});

Custom Stream

import { stream } from "@tanstack/ai-react";

const { messages } = useChat({
  connection: stream(async (messages, data, signal) => {
    // Custom streaming implementation
    const response = await fetch("/api/chat", {
      method: "POST",
      body: JSON.stringify({ messages, ...data }),
      signal,
    });
    // Return async iterable
    return processStream(response);
  }),
});

Monitoring Stream Progress

You can monitor stream progress with callbacks:

const { messages } = useChat({
  connection: fetchServerSentEvents("/api/chat"),
  onChunk: (chunk) => {
    console.log("Received chunk:", chunk);
  },
  onFinish: (message) => {
    console.log("Stream finished:", message);
  },
});

Cancelling Streams

Cancel ongoing streams:

const { stop } = useChat({
  connection: fetchServerSentEvents("/api/chat"),
});

// Cancel the current stream
stop();

Best Practices

1. Handle loading states - Use isLoading to show loading indicators
2. Handle errors - Check error state for stream failures
3. Cancel on unmount - Clean up streams when components unmount
4. Optimize rendering - Batch updates if needed for performance
5. Show progress - Display partial content as it streams

Next Steps

• Connection Adapters - Learn about different connection types
• API Reference - Explore streaming APIs