From 5a82e915919350f60c1511b5cb2298f9af6c3da7 Mon Sep 17 00:00:00 2001 From: Tom Beckenham <34339192+tombeckenham@users.noreply.github.com> Date: Tue, 2 Jun 2026 10:02:23 +1000 Subject: [PATCH 1/3] docs(chat): document the `fetcher` chat transport for server functions PR #512 added a first-class `fetcher` option to `useChat`/`ChatClient` for wiring a TanStack Start server function (that returns an SSE `Response`) into chat, but the connection-adapters page only covered the `stream()` path. Add a "Server Functions via `fetcher`" section, a "Pick a Transport" row, and a keyword so the new capability is discoverable. Co-Authored-By: Claude Opus 4.8 (1M context) --- docs/chat/connection-adapters.md | 35 ++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/docs/chat/connection-adapters.md b/docs/chat/connection-adapters.md index 46b79a2fb..cdc0910ad 100644 --- a/docs/chat/connection-adapters.md +++ b/docs/chat/connection-adapters.md @@ -12,6 +12,7 @@ keywords: - websocket - rpc - server functions + - fetcher - streaming transport - fetchServerSentEvents - subscribe send @@ -29,6 +30,7 @@ This page covers every supported transport, when to pick which, and how to build | An environment that blocks SSE (some edge runtimes, strict proxies) | [`fetchHttpStream`](#http-streaming-ndjson) | | React Native or Expo | [`xhrHttpStream`](#react-native-and-expo) by default, [`xhrServerSentEvents`](#react-native-and-expo) for SSE, or [`fetchHttpStream`](#http-streaming-ndjson) only when streaming `fetch` is available | | A TanStack Start (or other) server function that already returns an async iterable | [`stream`](#server-functions-and-direct-async-iterables) | +| A TanStack Start server function that returns an SSE `Response` (`toServerSentEventsResponse(...)`) | [`fetcher`](#server-functions-via-fetcher) | | An RPC framework like Cap'n Web, gRPC-Web, or tRPC | [`rpcStream`](#rpc-streams) | | A single long-lived WebSocket (or BroadcastChannel, postMessage, shared worker) serving many runs | [Custom `subscribe` / `send` adapter](#persistent-transports-websockets-and-friends) | | Standard SSE but with custom fetch wrapping (auth refresh, retries) | [`fetchServerSentEvents` with `fetchClient`](#custom-fetch-client) | @@ -175,6 +177,39 @@ The factory receives the conversation messages plus any per-request `data` you p > **Tip:** `stream()` is **request-scoped**. The factory is invoked once per `sendMessage`, the iterable runs to completion, and the connection closes. If you need a single long-lived channel that multiplexes many sends — for example a WebSocket — use [`subscribe` / `send`](#persistent-transports-websockets-and-friends) instead. +## Server Functions via `fetcher` + +When your server function returns an SSE `Response` — the common case for a [TanStack Start](https://tanstack.com/start) handler that ends with `toServerSentEventsResponse(...)` — use the top-level `fetcher` option instead of a connection adapter. `fetcher` is a sibling of `connection` (provide exactly one), and it accepts a plain async function. It mirrors the `fetcher` option on the [generation hooks](../media/generation-hooks). + +```typescript +// server/chat.server.ts +import { createServerFn } from "@tanstack/react-start"; +import { chat, toServerSentEventsResponse } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; +import type { UIMessage } from "@tanstack/ai"; + +export const chatFn = createServerFn({ method: "POST" }) + .inputValidator((data: { messages: Array }) => data) + .handler(({ data }) => + toServerSentEventsResponse( + chat({ adapter: openaiText("gpt-5.1"), messages: data.messages }), + ), + ); +``` + +```typescript +import { useChat } from "@tanstack/ai-react"; +import { chatFn } from "./server/chat.server"; + +const { messages, sendMessage } = useChat({ + fetcher: ({ messages }, { signal }) => chatFn({ data: { messages }, signal }), +}); +``` + +The fetcher receives `{ messages, data, threadId, runId }` plus an `AbortSignal` (triggered by `stop()` or when a send is superseded). Return either a `Response` — whose SSE body the chat client parses for you — or an `AsyncIterable`, which is yielded directly. Both sync and `Promise`-wrapped returns are accepted. + +> **Tip:** Reach for `fetcher` when your server function returns a `Response`; reach for [`stream()`](#server-functions-and-direct-async-iterables) when it returns an `AsyncIterable` directly. `stream()`'s factory is typed as `() => AsyncIterable`, so a `Promise` won't typecheck there — that's exactly the gap `fetcher` fills. Aborts, retries, and `live: true` work the same on both. + ## RPC Streams `rpcStream()` is identical in behavior to `stream()` but reads better at call sites that hand off to an RPC client. Use it when integrating with Cap'n Web, gRPC-Web, tRPC subscriptions, or any RPC framework that already returns an async iterable: From 0793ab7ed9dcd60972b7f921af087b0653ecbab7 Mon Sep 17 00:00:00 2001 From: Tom Beckenham <34339192+tombeckenham@users.noreply.github.com> Date: Tue, 2 Jun 2026 10:10:54 +1000 Subject: [PATCH 2/3] docs(chat): correct fetcher/stream comparison MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Drop the "retries" claim (no such feature — the only `retry` in the client is SSE `retry:` control-line parsing) and fix the `stream()` factory description (it returns an `AsyncIterable`; it isn't a zero-arg factory). State the verifiable structural fact instead: `fetcher` normalizes to the same request-scoped adapter as `stream()`. Verified the server snippet typechecks without an `as any` cast. Co-Authored-By: Claude Opus 4.8 (1M context) --- docs/chat/connection-adapters.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/chat/connection-adapters.md b/docs/chat/connection-adapters.md index cdc0910ad..5e53872c6 100644 --- a/docs/chat/connection-adapters.md +++ b/docs/chat/connection-adapters.md @@ -208,7 +208,7 @@ const { messages, sendMessage } = useChat({ The fetcher receives `{ messages, data, threadId, runId }` plus an `AbortSignal` (triggered by `stop()` or when a send is superseded). Return either a `Response` — whose SSE body the chat client parses for you — or an `AsyncIterable`, which is yielded directly. Both sync and `Promise`-wrapped returns are accepted. -> **Tip:** Reach for `fetcher` when your server function returns a `Response`; reach for [`stream()`](#server-functions-and-direct-async-iterables) when it returns an `AsyncIterable` directly. `stream()`'s factory is typed as `() => AsyncIterable`, so a `Promise` won't typecheck there — that's exactly the gap `fetcher` fills. Aborts, retries, and `live: true` work the same on both. +> **Tip:** Reach for `fetcher` when your server function returns a `Response`; reach for [`stream()`](#server-functions-and-direct-async-iterables) when it returns an `AsyncIterable` directly. `stream()`'s factory must return an `AsyncIterable`, so a `Promise` won't typecheck there — that's exactly the gap `fetcher` fills. Internally `fetcher` normalizes to the same request-scoped adapter as `stream()`, so everything downstream — `stop()`/abort, error handling, and tool calls — behaves identically. ## RPC Streams From ef0bcfab058b722d0bec400cfe576cf55961bbc1 Mon Sep 17 00:00:00 2001 From: Tom Beckenham <34339192+tombeckenham@users.noreply.github.com> Date: Tue, 2 Jun 2026 10:19:05 +1000 Subject: [PATCH 3/3] docs(chat): clarify fetcher vs stream is async-vs-sync, not Response-vs-iterable MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A `fetcher` can also return an `AsyncIterable` (sync or `Promise`-wrapped), not just a `Response` — the previous tip implied a clean Response/iterable split. The real distinction is that `stream()`'s factory must return the iterable *synchronously*, so an async server-function call (always a `Promise`) needs `fetcher` regardless of what it resolves to. Also corrects the pre-existing `stream()` section, which listed "TanStack Start server functions" as a `stream()` use case — those are async and hit the #509 type error; redirect them to `fetcher` and reframe `stream()` as the synchronous in-process / RSC / test path. Co-Authored-By: Claude Opus 4.8 (1M context) --- docs/chat/connection-adapters.md | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/docs/chat/connection-adapters.md b/docs/chat/connection-adapters.md index 5e53872c6..62532170d 100644 --- a/docs/chat/connection-adapters.md +++ b/docs/chat/connection-adapters.md @@ -29,8 +29,8 @@ This page covers every supported transport, when to pick which, and how to build | A normal HTTP server and want the default | [`fetchServerSentEvents`](#server-sent-events-sse) | | An environment that blocks SSE (some edge runtimes, strict proxies) | [`fetchHttpStream`](#http-streaming-ndjson) | | React Native or Expo | [`xhrHttpStream`](#react-native-and-expo) by default, [`xhrServerSentEvents`](#react-native-and-expo) for SSE, or [`fetchHttpStream`](#http-streaming-ndjson) only when streaming `fetch` is available | -| A TanStack Start (or other) server function that already returns an async iterable | [`stream`](#server-functions-and-direct-async-iterables) | -| A TanStack Start server function that returns an SSE `Response` (`toServerSentEventsResponse(...)`) | [`fetcher`](#server-functions-via-fetcher) | +| Code that **synchronously** returns an `AsyncIterable` (in-process `chat()`, an RSC stream, tests) | [`stream`](#server-functions-and-direct-async-iterables) | +| An **async** call — a TanStack Start server function or any `Promise`-returning function — resolving to a `Response` or an `AsyncIterable` | [`fetcher`](#server-functions-via-fetcher) | | An RPC framework like Cap'n Web, gRPC-Web, or tRPC | [`rpcStream`](#rpc-streams) | | A single long-lived WebSocket (or BroadcastChannel, postMessage, shared worker) serving many runs | [Custom `subscribe` / `send` adapter](#persistent-transports-websockets-and-friends) | | Standard SSE but with custom fetch wrapping (auth refresh, retries) | [`fetchServerSentEvents` with `fetchClient`](#custom-fetch-client) | @@ -160,14 +160,15 @@ walkthrough, see [Quick Start: React Native](../getting-started/quick-start-reac ## Server Functions and Direct Async Iterables -When your client can call into your server without going over HTTP — TanStack Start server functions, RSC streams, in-process tests — skip the transport entirely. `stream()` takes a factory that returns an `AsyncIterable` and wires it straight into the client: +When your client can call into your server without going over HTTP — RSC streams, in-process tests, a direct in-process `chat()` call — skip the transport entirely. `stream()` takes a factory that returns an `AsyncIterable` **synchronously** and wires it straight into the client. (A [TanStack Start](https://tanstack.com/start) server function returns a `Promise`, so it needs [`fetcher`](#server-functions-via-fetcher), not `stream()` — see the next section.) ```typescript import { useChat, stream } from "@tanstack/ai-react"; import { chatServerFn } from "./server/chat.server"; -// `chatServerFn` is a server function that returns an AsyncIterable, -// e.g. the result of `chat({ adapter, model, messages })` on the server. +// `chatServerFn` is an in-process server-side function that synchronously +// returns an AsyncIterable — e.g. the result of +// `chat({ adapter, model, messages })` on the server. const { messages } = useChat({ connection: stream((messages, data) => chatServerFn({ messages, ...data })), }); @@ -179,7 +180,7 @@ The factory receives the conversation messages plus any per-request `data` you p ## Server Functions via `fetcher` -When your server function returns an SSE `Response` — the common case for a [TanStack Start](https://tanstack.com/start) handler that ends with `toServerSentEventsResponse(...)` — use the top-level `fetcher` option instead of a connection adapter. `fetcher` is a sibling of `connection` (provide exactly one), and it accepts a plain async function. It mirrors the `fetcher` option on the [generation hooks](../media/generation-hooks). +When you call into your server with an **async** function — the universal case for a [TanStack Start](https://tanstack.com/start) server function, which always returns a `Promise` — use the top-level `fetcher` option instead of a connection adapter. `fetcher` is a sibling of `connection` (provide exactly one), and it accepts a plain async function. It mirrors the `fetcher` option on the [generation hooks](../media/generation-hooks). The most common shape is a handler that ends with `toServerSentEventsResponse(...)` and resolves to a `Response`: ```typescript // server/chat.server.ts @@ -206,9 +207,9 @@ const { messages, sendMessage } = useChat({ }); ``` -The fetcher receives `{ messages, data, threadId, runId }` plus an `AbortSignal` (triggered by `stop()` or when a send is superseded). Return either a `Response` — whose SSE body the chat client parses for you — or an `AsyncIterable`, which is yielded directly. Both sync and `Promise`-wrapped returns are accepted. +The fetcher receives `{ messages, data, threadId, runId }` plus an `AbortSignal` (triggered by `stop()` or when a send is superseded). Return a `Response` — whose SSE body the chat client parses for you — **or** an `AsyncIterable`, which is yielded directly. If your server function returns the stream itself (instead of wrapping it in a `Response`), the fetcher handles that too. Sync and `Promise`-wrapped returns are both accepted. -> **Tip:** Reach for `fetcher` when your server function returns a `Response`; reach for [`stream()`](#server-functions-and-direct-async-iterables) when it returns an `AsyncIterable` directly. `stream()`'s factory must return an `AsyncIterable`, so a `Promise` won't typecheck there — that's exactly the gap `fetcher` fills. Internally `fetcher` normalizes to the same request-scoped adapter as `stream()`, so everything downstream — `stop()`/abort, error handling, and tool calls — behaves identically. +> **Tip:** The choice between `fetcher` and [`stream()`](#server-functions-and-direct-async-iterables) is about **async vs sync**, not `Response`-vs-iterable — both can yield an `AsyncIterable`. `stream()`'s factory must return that iterable **synchronously**, so a server-function call (which returns a `Promise`) won't typecheck there — that's the gap `fetcher` fills ([issue #509](https://github.com/TanStack/ai/issues/509)). Use `stream()` when you can hand back an async iterable synchronously (in-process `chat()`, an RPC client, tests); use `fetcher` for anything you have to `await`. Both normalize to the same request-scoped adapter, so `stop()`/abort, error handling, and tool calls behave identically. ## RPC Streams