Skip to content
Open
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
b8c1615
feat(ai): add type-safe tool call events to chat() stream
AlemTuzlak Apr 15, 2026
a2c2f9c
ci: apply automated fixes
autofix-ci[bot] Apr 15, 2026
5a37cb8
feat(ai): make tool call events a discriminated union for per-tool in…
AlemTuzlak Apr 16, 2026
5e3ebd7
ci: apply automated fixes
autofix-ci[bot] Apr 16, 2026
9a1df7e
docs: improve type-safe tool call event documentation
AlemTuzlak Apr 16, 2026
d46cd6d
fix(ai): resolve tsc errors in discriminated union types and fix docs
AlemTuzlak Apr 16, 2026
883fe15
Merge remote-tracking branch 'origin/main' into worktree-serialized-e…
AlemTuzlak Apr 24, 2026
f2aaac0
fix(ai): restore discriminated union narrowing on typed tool-call events
AlemTuzlak Apr 24, 2026
53e7203
Merge branch 'main' into worktree-serialized-exploring-wall
AlemTuzlak May 25, 2026
2b0bd96
fix(ai): address Round 1 CR findings on type-safe stream chunks
AlemTuzlak May 25, 2026
f66ad73
docs: add TaggedCustomEvent reference page
AlemTuzlak May 25, 2026
c42100f
docs(ai): fix TypedStreamChunk JSDoc to match no-typed-tools fallback
AlemTuzlak May 25, 2026
c768cce
refactor(ai): include TTools in streaming-structured chat() return cast
AlemTuzlak May 25, 2026
22d188a
Merge branch 'main' into worktree-serialized-exploring-wall
AlemTuzlak May 26, 2026
3569aa0
feat(ai): narrow stream chunks via Pick<AGUI*Event, …> + EventType li…
AlemTuzlak May 26, 2026
0a9c07e
fix(ai): switch chunk.type from EventType enum literal to plain strin…
AlemTuzlak May 26, 2026
573e413
ci: apply automated fixes
autofix-ci[bot] May 26, 2026
cff54ce
feat(ai): narrow CustomEvent to TaggedCustomEvent even without typed …
AlemTuzlak May 26, 2026
de5f152
docs+feat: update model refs to gpt-5.2 and preserve tool name litera…
AlemTuzlak May 26, 2026
821a3d0
ci: apply automated fixes
autofix-ci[bot] May 26, 2026
c0b0feb
feat(ai): expose typed tool output on TOOL_CALL_END events
AlemTuzlak May 26, 2026
550b102
ci: apply automated fixes
autofix-ci[bot] May 26, 2026
57c46ff
chore(ai): fix lint errors in @tanstack/ai
AlemTuzlak May 26, 2026
71262b6
Merge branch 'main' into worktree-serialized-exploring-wall
tombeckenham Jun 2, 2026
5f47380
Merge remote-tracking branch 'origin/main' into worktree-serialized-e…
tombeckenham Jun 2, 2026
1317dd6
ci: apply automated fixes
autofix-ci[bot] Jun 2, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions docs/chat/streaming.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,44 @@ TanStack AI implements the [AG-UI Protocol](https://docs.ag-ui.com/introduction)

> **Tip:** Some models expose their internal reasoning as thinking content that streams before the response. See [Thinking & Reasoning](./thinking-content).

### Type-Safe Tool Call Events

When you pass typed tools (defined with `toolDefinition()` and Zod schemas) to `chat()`, the stream chunks automatically carry type information for tool call events. The `toolName` field narrows to the union of your tool name literals, and the `input` field on `TOOL_CALL_END` events is typed as the union of your tool input schemas:

```typescript
import { chat, toolDefinition } from "@tanstack/ai";
import { openaiText } from "@tanstack/ai-openai";
import { z } from "zod";

const weatherTool = toolDefinition({
name: "get_weather",
description: "Get weather for a location",
inputSchema: z.object({
location: z.string(),
unit: z.enum(["celsius", "fahrenheit"]).optional(),
}),
});

const stream = chat({
adapter: openaiText("gpt-5.2"),
messages,
tools: [weatherTool],
});

for await (const chunk of stream) {
if (chunk.type === "TOOL_CALL_END") {
chunk.toolName; // ✅ typed as "get_weather" (not string)
chunk.input; // ✅ typed as { location: string; unit?: "celsius" | "fahrenheit" }
}
}
```

Without typed tools, `toolName` defaults to `string` and `input` defaults to `unknown` — the same behavior as before. The type narrowing is automatic when you use `toolDefinition()` with Zod schemas.

> **Note:** When multiple tools are provided, `input` is typed as the union of all tool input types. Checking `toolName === 'get_weather'` does not narrow `input` to that specific tool's input type — if you need per-tool discrimination, use a type guard after the `toolName` check.

> **Tip:** The typed stream chunk type is exported as `TypedStreamChunk<TTools>` if you need to annotate variables or function parameters. When used without type arguments, `TypedStreamChunk` is equivalent to `StreamChunk`.
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Outdated

### Thinking Chunks

Thinking/reasoning is represented by AG-UI events `STEP_STARTED` and `STEP_FINISHED`. They stream separately from the final response text:
Expand Down
36 changes: 35 additions & 1 deletion docs/reference/type-aliases/StreamChunk.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,41 @@ title: StreamChunk
type StreamChunk = AGUIEvent;
```

Defined in: [types.ts:976](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L976)
Defined in: [types.ts:989](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L989)

Chunk returned by the SDK during streaming chat completions.
Uses the AG-UI protocol event format.

# Type Alias: TypedStreamChunk

```ts
type TypedStreamChunk<TTools extends ReadonlyArray<Tool<any, any, any>> = ReadonlyArray<Tool<any, any, any>>>
```

Defined in: [types.ts:1033](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L1033)
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Outdated

A variant of `StreamChunk` parameterized by the tools array. When specific tool types are provided (e.g. from `chat({ tools: [myTool] })`):

- `TOOL_CALL_START` and `TOOL_CALL_END` events have `toolName` narrowed to the union of known tool name literals.
- `TOOL_CALL_END` events have `input` typed as the union of tool input types.

When tools are untyped or absent, `TypedStreamChunk` degrades to the same type as `StreamChunk`.

This is the type returned by `chat()` when streaming is enabled (the default). You don't typically need to reference it directly unless annotating function parameters or return types.

```ts
import type { TypedStreamChunk } from "@tanstack/ai";
import { toolDefinition } from "@tanstack/ai";

// Given tools created with toolDefinition():
const weatherTool = toolDefinition({ name: "get_weather", description: "...", inputSchema: /* Zod schema */ });
const searchTool = toolDefinition({ name: "search", description: "...", inputSchema: /* Zod schema */ });

// Without type args — equivalent to StreamChunk
type Chunk = TypedStreamChunk;

// With specific tools — tool call events are typed
type TypedChunk = TypedStreamChunk<[typeof weatherTool, typeof searchTool]>;
```

See [Streaming - Type-Safe Tool Call Events](../../chat/streaming) for a practical walkthrough.
2 changes: 2 additions & 0 deletions docs/tools/tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,8 @@ const inputSchema: JSONSchema = {

> **Note:** When using JSON Schema, TypeScript will infer `any` for input/output types since JSON Schema cannot provide compile-time type information. Zod schemas are recommended for full type safety.

> **Tip:** Type safety from Zod schemas extends beyond tool execution — when you iterate over the stream returned by `chat()`, tool call events have typed `toolName` and `input` fields too. See [Type-Safe Tool Call Events](../chat/streaming#type-safe-tool-call-events).

## Tool Definition

Tools are defined using `toolDefinition()` from `@tanstack/ai`:
Expand Down
64 changes: 64 additions & 0 deletions examples/ts-react-chat/src/routes/api.tanchat.ts
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,70 @@ const loggingMiddleware: ChatMiddleware = {
},
}

// ===========================
// TypedStreamChunk showcase — type-safe tool call events
// ===========================
//
// When `chat()` receives tools with typed schemas, the returned stream
// carries type information on TOOL_CALL_START and TOOL_CALL_END events.
// No casts, no `as any` — just narrow by `chunk.type` and everything is typed.

const tools = [
getGuitars,
recommendGuitarToolDef,
addToCartToolServer,
addToWishListToolDef,
getPersonalGuitarPreferenceToolDef,
compareGuitars,
calculateFinancing,
searchGuitars,
] as const

async function typedStreamShowcase() {
const stream = chat({
adapter: openaiText('gpt-4o'),
messages: [
{ role: 'user' as const, content: 'Recommend an acoustic guitar' },
],
tools,
})

for await (const chunk of stream) {
switch (chunk.type) {
case 'TOOL_CALL_START':
// ✅ chunk.toolName is typed as the union of all tool name literals:
// 'getGuitars' | 'recommendGuitar' | 'addToCart' | 'addToWishList'
// | 'getPersonalGuitarPreference' | 'compareGuitars'
// | 'calculateFinancing' | 'searchGuitars'
//
// ❌ Without TypedStreamChunk, this would just be `string`
console.log(`Tool call started: ${chunk.toolName}`)
break

case 'TOOL_CALL_END':
// ✅ chunk.toolName — same typed literal union as above
// ✅ chunk.input — union of all tool input types, inferred from Zod schemas:
// | {}
// | { id: string | number }
// | { guitarId: string; quantity: number }
// | { guitarId: string }
// | { guitarIds: number[] }
// | { guitarId: number; months: number }
// | { query: string }
console.log(`Tool call ended: ${chunk.toolName}`, chunk.input)
break

case 'TEXT_MESSAGE_CONTENT':
// Non-tool events are unaffected — still fully typed
console.log(chunk.delta)
break
}
}
}

// Suppress unused warning — this is a showcase, not called at runtime
void typedStreamShowcase

export const Route = createFileRoute('/api/tanchat')({
server: {
handlers: {
Expand Down
39 changes: 28 additions & 11 deletions packages/typescript/ai/src/activities/chat/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ import type {
ToolCallArgsEvent,
ToolCallEndEvent,
ToolCallStartEvent,
TypedStreamChunk,
} from '../../types'
import type {
ChatMiddleware,
Expand All @@ -69,11 +70,15 @@ export const kind = 'text' as const
* @template TAdapter - The text adapter type (created by a provider function)
* @template TSchema - Optional Standard Schema for structured output
* @template TStream - Whether to stream the output (default: true)
* @template TTools - The tools array type for type-safe tool call events in the stream
*/
export interface TextActivityOptions<
TAdapter extends AnyTextAdapter,
TSchema extends SchemaInput | undefined,
TStream extends boolean,
TTools extends ReadonlyArray<Tool<any, any, any>> = ReadonlyArray<
Tool<any, any, any>
>,
> {
/** The text adapter to use (created by a provider function like openaiText('gpt-4o')) */
adapter: TAdapter
Expand All @@ -87,7 +92,7 @@ export interface TextActivityOptions<
/** System prompts to prepend to the conversation */
systemPrompts?: TextOptions['systemPrompts']
/** Tools for function calling (auto-executed when called) */
tools?: TextOptions['tools']
tools?: TTools
/** Controls the randomness of the output. Higher values make output more random. Range: [0.0, 2.0] */
temperature?: TextOptions['temperature']
/** Nucleus sampling parameter. The model considers tokens with topP probability mass. */
Expand Down Expand Up @@ -125,7 +130,7 @@ export interface TextActivityOptions<
outputSchema?: TSchema
/**
* Whether to stream the text result.
* When true (default), returns an AsyncIterable<StreamChunk> for streaming output.
* When true (default), returns an AsyncIterable<TypedStreamChunk<TTools>> for streaming output.
* When false, returns a Promise<string> with the collected text content.
*
* Note: If outputSchema is provided, this option is ignored and the result
Expand Down Expand Up @@ -186,9 +191,12 @@ export function createChatOptions<
TAdapter extends AnyTextAdapter,
TSchema extends SchemaInput | undefined = undefined,
TStream extends boolean = true,
TTools extends ReadonlyArray<Tool<any, any, any>> = ReadonlyArray<
Tool<any, any, any>
>,
>(
options: TextActivityOptions<TAdapter, TSchema, TStream>,
): TextActivityOptions<TAdapter, TSchema, TStream> {
options: TextActivityOptions<TAdapter, TSchema, TStream, TTools>,
): TextActivityOptions<TAdapter, TSchema, TStream, TTools> {
return options
}

Expand All @@ -200,16 +208,22 @@ export function createChatOptions<
* Result type for the text activity.
* - If outputSchema is provided: Promise<InferSchemaType<TSchema>>
* - If stream is false: Promise<string>
* - Otherwise (stream is true, default): AsyncIterable<StreamChunk>
* - Otherwise (stream is true, default): AsyncIterable<TypedStreamChunk<TTools>>
*
* When tools with typed schemas are provided, the stream chunks include
* type-safe `toolName` and `input` fields on tool call events.
*/
export type TextActivityResult<
TSchema extends SchemaInput | undefined,
TStream extends boolean = true,
TTools extends ReadonlyArray<Tool<any, any, any>> = ReadonlyArray<
Tool<any, any, any>
>,
> = TSchema extends SchemaInput
? Promise<InferSchemaType<TSchema>>
: TStream extends false
? Promise<string>
: AsyncIterable<StreamChunk>
: AsyncIterable<TypedStreamChunk<TTools>>

// ===========================
// ChatEngine Implementation
Expand Down Expand Up @@ -1374,9 +1388,12 @@ export function chat<
TAdapter extends AnyTextAdapter,
TSchema extends SchemaInput | undefined = undefined,
TStream extends boolean = true,
TTools extends ReadonlyArray<Tool<any, any, any>> = ReadonlyArray<
Tool<any, any, any>
>,
>(
options: TextActivityOptions<TAdapter, TSchema, TStream>,
): TextActivityResult<TSchema, TStream> {
options: TextActivityOptions<TAdapter, TSchema, TStream, TTools>,
): TextActivityResult<TSchema, TStream, TTools> {
const { outputSchema, stream } = options

// If outputSchema is provided, run agentic structured output
Expand All @@ -1387,7 +1404,7 @@ export function chat<
SchemaInput,
boolean
>,
) as TextActivityResult<TSchema, TStream>
) as TextActivityResult<TSchema, TStream, TTools>
}

// If stream is explicitly false, run non-streaming text
Expand All @@ -1398,13 +1415,13 @@ export function chat<
undefined,
false
>,
) as TextActivityResult<TSchema, TStream>
) as TextActivityResult<TSchema, TStream, TTools>
}

// Otherwise, run streaming text (default)
return runStreamingText(
options as unknown as TextActivityOptions<AnyTextAdapter, undefined, true>,
) as TextActivityResult<TSchema, TStream>
) as TextActivityResult<TSchema, TStream, TTools>
}

/**
Expand Down
Loading
Loading