diff --git a/docs/chat/connection-adapters.md b/docs/chat/connection-adapters.md index 46b79a2fb..62532170d 100644 --- a/docs/chat/connection-adapters.md +++ b/docs/chat/connection-adapters.md @@ -12,6 +12,7 @@ keywords: - websocket - rpc - server functions + - fetcher - streaming transport - fetchServerSentEvents - subscribe send @@ -28,7 +29,8 @@ This page covers every supported transport, when to pick which, and how to build | A normal HTTP server and want the default | [`fetchServerSentEvents`](#server-sent-events-sse) | | An environment that blocks SSE (some edge runtimes, strict proxies) | [`fetchHttpStream`](#http-streaming-ndjson) | | React Native or Expo | [`xhrHttpStream`](#react-native-and-expo) by default, [`xhrServerSentEvents`](#react-native-and-expo) for SSE, or [`fetchHttpStream`](#http-streaming-ndjson) only when streaming `fetch` is available | -| A TanStack Start (or other) server function that already returns an async iterable | [`stream`](#server-functions-and-direct-async-iterables) | +| Code that **synchronously** returns an `AsyncIterable` (in-process `chat()`, an RSC stream, tests) | [`stream`](#server-functions-and-direct-async-iterables) | +| An **async** call — a TanStack Start server function or any `Promise`-returning function — resolving to a `Response` or an `AsyncIterable` | [`fetcher`](#server-functions-via-fetcher) | | An RPC framework like Cap'n Web, gRPC-Web, or tRPC | [`rpcStream`](#rpc-streams) | | A single long-lived WebSocket (or BroadcastChannel, postMessage, shared worker) serving many runs | [Custom `subscribe` / `send` adapter](#persistent-transports-websockets-and-friends) | | Standard SSE but with custom fetch wrapping (auth refresh, retries) | [`fetchServerSentEvents` with `fetchClient`](#custom-fetch-client) | @@ -158,14 +160,15 @@ walkthrough, see [Quick Start: React Native](../getting-started/quick-start-reac ## Server Functions and Direct Async Iterables -When your client can call into your server without going over HTTP — TanStack Start server functions, RSC streams, in-process tests — skip the transport entirely. `stream()` takes a factory that returns an `AsyncIterable` and wires it straight into the client: +When your client can call into your server without going over HTTP — RSC streams, in-process tests, a direct in-process `chat()` call — skip the transport entirely. `stream()` takes a factory that returns an `AsyncIterable` **synchronously** and wires it straight into the client. (A [TanStack Start](https://tanstack.com/start) server function returns a `Promise`, so it needs [`fetcher`](#server-functions-via-fetcher), not `stream()` — see the next section.) ```typescript import { useChat, stream } from "@tanstack/ai-react"; import { chatServerFn } from "./server/chat.server"; -// `chatServerFn` is a server function that returns an AsyncIterable, -// e.g. the result of `chat({ adapter, model, messages })` on the server. +// `chatServerFn` is an in-process server-side function that synchronously +// returns an AsyncIterable — e.g. the result of +// `chat({ adapter, model, messages })` on the server. const { messages } = useChat({ connection: stream((messages, data) => chatServerFn({ messages, ...data })), }); @@ -175,6 +178,39 @@ The factory receives the conversation messages plus any per-request `data` you p > **Tip:** `stream()` is **request-scoped**. The factory is invoked once per `sendMessage`, the iterable runs to completion, and the connection closes. If you need a single long-lived channel that multiplexes many sends — for example a WebSocket — use [`subscribe` / `send`](#persistent-transports-websockets-and-friends) instead. +## Server Functions via `fetcher` + +When you call into your server with an **async** function — the universal case for a [TanStack Start](https://tanstack.com/start) server function, which always returns a `Promise` — use the top-level `fetcher` option instead of a connection adapter. `fetcher` is a sibling of `connection` (provide exactly one), and it accepts a plain async function. It mirrors the `fetcher` option on the [generation hooks](../media/generation-hooks). The most common shape is a handler that ends with `toServerSentEventsResponse(...)` and resolves to a `Response`: + +```typescript +// server/chat.server.ts +import { createServerFn } from "@tanstack/react-start"; +import { chat, toServerSentEventsResponse } from "@tanstack/ai"; +import { openaiText } from "@tanstack/ai-openai"; +import type { UIMessage } from "@tanstack/ai"; + +export const chatFn = createServerFn({ method: "POST" }) + .inputValidator((data: { messages: Array }) => data) + .handler(({ data }) => + toServerSentEventsResponse( + chat({ adapter: openaiText("gpt-5.1"), messages: data.messages }), + ), + ); +``` + +```typescript +import { useChat } from "@tanstack/ai-react"; +import { chatFn } from "./server/chat.server"; + +const { messages, sendMessage } = useChat({ + fetcher: ({ messages }, { signal }) => chatFn({ data: { messages }, signal }), +}); +``` + +The fetcher receives `{ messages, data, threadId, runId }` plus an `AbortSignal` (triggered by `stop()` or when a send is superseded). Return a `Response` — whose SSE body the chat client parses for you — **or** an `AsyncIterable`, which is yielded directly. If your server function returns the stream itself (instead of wrapping it in a `Response`), the fetcher handles that too. Sync and `Promise`-wrapped returns are both accepted. + +> **Tip:** The choice between `fetcher` and [`stream()`](#server-functions-and-direct-async-iterables) is about **async vs sync**, not `Response`-vs-iterable — both can yield an `AsyncIterable`. `stream()`'s factory must return that iterable **synchronously**, so a server-function call (which returns a `Promise`) won't typecheck there — that's the gap `fetcher` fills ([issue #509](https://github.com/TanStack/ai/issues/509)). Use `stream()` when you can hand back an async iterable synchronously (in-process `chat()`, an RPC client, tests); use `fetcher` for anything you have to `await`. Both normalize to the same request-scoped adapter, so `stop()`/abort, error handling, and tool calls behave identically. + ## RPC Streams `rpcStream()` is identical in behavior to `stream()` but reads better at call sites that hand off to an RPC client. Use it when integrating with Cap'n Web, gRPC-Web, tRPC subscriptions, or any RPC framework that already returns an async iterable: