One answer, every surface

The same agent answers a React app, an AG-UI client, a WhatsApp bot and an MCP client. You write a tool once. What each surface gets depends on what it can draw, and the server decides that per request.

tool execute ──► domain output ──────────────────────────────► the model, MCP callers
     │
     └─ present / ctx.emitUi ──► component { name, props, version }
                                   │  validated against the server catalog
                                   │  intersected with the client's uiCapabilities
                                   ├─ supported   ──► `ui` frame (+ fallbackText) ──► React, AG-UI `agora.ui`
                                   └─ unsupported ──► `text` frame (the component's text) ──► any channel

A tool result has three layers, and each surface takes the ones it can use:

Backend and frontendCopy Anchor Link

Every piece below runs on one of two sides. Each code block on this page names its side and the file it belongs in.

The backend owns everything the model and the database touch: which components exist (the catalog), what each tool returns, validation, negotiation, the text fallback and the HTTP routes. AgentModule mounts the routes itself; you write no controller for the chat. The frontend only draws: it maps a component name to a React component, says which of them it can draw, and renders the stream it reads from those routes.

End to end: a React chat that draws OrderSummaryCopy Anchor Link

One component, one tool, one chat page. The backend half is the same for every surface on this page; only the frontend half is specific to the React chat.

Backend — NestJSCopy Anchor Link

1. The component definition, the contract every surface reads:

import { createComponent } from '@dudousxd/nestjs-agent-core/genui';
import { z } from 'zod';

export const orderSummary = createComponent({
  name: 'OrderSummary',
  title: 'Order summary',
  description: 'One order: status and total.',
  version: 1,
  props: z.object({ id: z.string(), status: z.string(), totalCents: z.number().int() }),
  fallbackText: ({ id, status, totalCents }) =>
    `*Order ${id}* — ${status}, ${(totalCents / 100).toFixed(2)}`,
});

2. The catalog:

import { defineCatalog } from '@dudousxd/nestjs-agent-core/genui';
import { BUILTIN_COMPONENTS } from '@dudousxd/nestjs-agent-core/genui/builtins';
import { orderSummary } from './order-summary';

export const catalog = defineCatalog([...BUILTIN_COMPONENTS, orderSummary.definition]);

3. The tool returns domain data and presents it with the component:

import { AiTool, type AiToolCtx } from '@dudousxd/nestjs-agent';
import { z } from 'zod';
import { orderSummary } from '../genui/order-summary';
import { type Order, OrdersService } from './orders.service';

@AiTool({
  description: 'Look up one of the customer’s orders.',
  input: z.object({ orderId: z.string() }),
})
export class OrderStatusTool { // name: orderStatus, kind: read
  constructor(private readonly orders: OrdersService) {}

  execute({ orderId }: { orderId: string }, ctx: AiToolCtx) {
    return this.orders.findForCustomer(orderId, ctx.actor.id);
  }

  present(order: Order) {
    return orderSummary({ id: order.id, status: order.status, totalCents: order.totalCents });
  }
}

The model reads execute's return value on every surface. present (or a direct ctx.emitUi) is the only part whose delivery differs.

4. The module. AgentModule mounts the agent's routes under path (default agent), resolves the caller with actorResolver, and puts every route behind guards. AgentGenuiModule gives the server its catalog. The tool is an ordinary provider:

import { Module } from '@nestjs/common';
import { AgentModule, requestUserActorResolver } from '@dudousxd/nestjs-agent';
import { AgentGenuiModule } from '@dudousxd/nestjs-agent/genui';
import { catalog } from './genui/catalog';
import { OrderStatusTool } from './orders/order-status.tool';
import { OrdersService } from './orders/orders.service';
import { SessionGuard } from './auth/session.guard';

@Module({
  imports: [
    AgentModule.forRoot({
      model,                                     // see Getting started
      actorResolver: requestUserActorResolver(), // who is calling: reads req.user
      guards: [SessionGuard],                    // your guard, on every agent route
      // path: 'agent',                          // the default: routes under /agent
    }),
    AgentGenuiModule.forRoot({ catalog }),
  ],
  providers: [OrdersService, OrderStatusTool],
})
export class AppModule {}

A guard that sets req.user is enough for requestUserActorResolver(); without actorResolver the routes are anonymous, one identity per browser. See Identity & Authorization.

Frontend — ReactCopy Anchor Link

5. The renderer, an ordinary React component whose props are the definition's props:

export function OrderSummary({ id, status, totalCents }: { id: string; status: string; totalCents: number }) {
  return (
    <section>
      <h3>Order {id}</h3>
      <p>{status} — {(totalCents / 100).toFixed(2)}</p>
    </section>
  );
}

6. The page mounts the connection, hands it the registry, and says what it can draw:

import { AgentProvider, ChatInput, MessageList, useAgentChat } from '@dudousxd/nestjs-agent-react';
import type { GenuiRegistry } from '@dudousxd/nestjs-agent-react/genui';
import { catalog } from '../../api/src/genui/catalog'; // optional: validate props before drawing
import { OrderSummary } from './genui/order-summary';

// Keys are the catalog's component names.
const registry: GenuiRegistry = { OrderSummary };

export function ChatPage() {
  return (
    <AgentProvider
      genui={{ registry, catalog }}
      uiCapabilities={{ components: [{ name: 'OrderSummary', version: 1 }] }}
    >
      <Chat />
    </AgentProvider>
  );
}

function Chat() {
  const chat = useAgentChat();
  return (
    <>
      <MessageList messages={chat.messages} status={chat.status} />
      <ChatInput onSubmit={(text) => chat.sendMessage({ text })} />
    </>
  );
}

MessageList draws every pushed component inside the provider. For your own markup, render each ui block with <GenerativeUI part={block} /> (see Frontend).

With this page, OrderSummary is drawn. Any other catalog component a tool presents (a builtin DataTable, say) arrives as its text, and the genui tools offer the model only OrderSummary, because the page declared nothing else.

How the two halves connectCopy Anchor Link

Per surfaceCopy Anchor Link

Declaring what a client can drawCopy Anchor Link

A client sends the components it has renderers for, each with the exact version it implements. It is written by the frontend (or by the backend code that speaks for a channel, such as a webhook) and applied by the backend:

uiCapabilities: {
  components: [
    { name: 'DataTable', version: 1 },
    { name: 'Chart', version: 1 },
  ],
}

• Omitted: nothing is negotiated. Every component of the authorized catalog is offered and drawn, which is how clients behaved before negotiation existed.
• components: []: text only.
• A list: the server intersects it with the catalog it would use anyway (the static one, or the resolver's for this tenant). A component matches only by name and version, where a definition without version is version 1. The list never adds a component or grants a permission.

The shape is strict: only components, each entry only name (a valid component name) and a positive integer version, no duplicates. Anything else is a 400 Invalid UI capabilities on POST /agent/chat, and a 400 on AG-UI.

Where the declaration travels:

• POST /agent/chat body field uiCapabilities.
• React: <AgentProvider uiCapabilities={…}> once, or useAgentChat({ uiCapabilities }) per chat. The chat's option wins. Every send carries it. The React client does not derive the list from your genui registry, so pass it explicitly.
• AG-UI: forwardedProps.uiCapabilities. agUiChatStream forwards the send's uiCapabilities there.
• In-process: AgentService.send({ …, uiCapabilities }) and AgentService.chat(…).
• A queued message stores the declaration and runs under it. An action proposal stores it in its execution context, so a result that lands later is negotiated for the client that asked.

What changes for the modelCopy Anchor Link

The genui tools describe themselves per turn from the negotiated catalog:

• The tree tool (ui__render) and the ui__show tool list only declared components — the tree tool's input schema too — and are removed when none are left.
• With mode: 'per-component', a tool (ui__show_<name>) whose component the client did not declare is removed from that turn's tool list.
• A call to a tool removed this way is refused as an unknown tool, even if the model names it.
• A streaming tree is previewed only for a client that draws every node of it; anyone else gets the final push (or its text) alone.

With components: [] the model sees no UI tool at all and answers in prose.

What changes for your toolsCopy Anchor Link

All of this happens in the Nest process; the frontend receives only the result. ctx.emitUi and a tool's present hook go through the same negotiation when the server has a catalog (AgentGenuiModule):

1. The props are validated against the server definition (name, version, schema). An unknown component or invalid props throws.
2. If the client declared the component, it is pushed as a ui frame carrying fallbackText, the definition's text for those props.
3. If not, the component's text is written as a text frame instead. The text is stored as an assistant message after the tool step and added to the model's context, so the conversation reads the same on reload.

A tree is all or nothing: if any node's component is not declared, the whole tree is sent as text (treeToText).

fallbackText is the component definition's fallbackText(props). With none, or when it throws, the text is the props as a JSON code block, preceded by *title* when the props have a title. Text is Slack mrkdwn-compatible. For a nicer per-channel rendering, register channel renderers on a component registry and call registry.render(presentation, 'whatsapp'). See Custom components and renderers per channel.

Web over AG-UICopy Anchor Link

Backend. Add the adapter next to the native routes. It mounts POST /agent/ag-ui, behind the same actorResolver, guards and ownership checks:

import { AgentModule, agUiAdapter } from '@dudousxd/nestjs-agent';

AgentModule.forRoot({
  // … as above
  adapters: [agUiAdapter()],
});

The route reads forwardedProps.uiCapabilities (a malformed value is a 400) and negotiates exactly as above. A drawable component arrives as the CUSTOM event agora.ui with { id, component, props, version?, fallbackText?, componentVersions?, toolCallId?, partial? }, so an AG-UI client that cannot draw it after all can still show the text. An undeclared one arrives as text, inside the run's TEXT_MESSAGE_* events.

Frontend. Any AG-UI client posts a RunAgentInput to that URL with uiCapabilities in forwardedProps. The React chat above sends its turns over AG-UI by replacing one method of the backend, openChatStream, with agUiChatStream. It forwards the send's uiCapabilities, staged attachments and regenerate; everything else stays on the native routes:

import { AgentClient, type ChatStreamRequest, agUiChatStream } from '@dudousxd/nestjs-agent-react';

export const readCookie = (name: string) =>
  decodeURIComponent(document.cookie.match(new RegExp(`(?:^|; )${name}=([^;]*)`))?.[1] ?? '');

/** Turns over AG-UI; threads, queue, decisions, uploads, cancel and re-attach over REST. */
export class AgUiAgentClient extends AgentClient {
  override openChatStream(request: ChatStreamRequest) {
    return agUiChatStream(request, {
      url: '/agent/ag-ui',
      // Not the provider's getHeaders: built here, per send.
      headers: { 'X-XSRF-TOKEN': readCookie('XSRF-TOKEN') },
    });
  }
}

import { AgUiAgentClient, readCookie } from './agent-backend';

const backend = new AgUiAgentClient({ getHeaders: () => ({ 'X-XSRF-TOKEN': readCookie('XSRF-TOKEN') }) });

<AgentProvider
  backend={backend}
  genui={{ registry, catalog }}
  uiCapabilities={{ components: [{ name: 'OrderSummary', version: 1 }] }}
>
  <Chat />
</AgentProvider>;

agUiChatStream does not read the provider's connection settings: url is the full path (base URL and prefix included), headers are its own headers, and it fetches with credentials: 'same-origin' unless you pass a fetch of your own. See AG-UI.

What AG-UI costs youCopy Anchor Link

AG-UI is a second door onto the same runs, not a richer one. The server side (tools, authorization, quota, durable runs, memory) is identical; what differs is what the wire carries and what a request can ask for. The native stream is POST /agent/chat plus GET /agent/chat/:runId/stream; AG-UI is POST /agent/ag-ui. "React" below means useAgentChat over a backend whose openChatStream is agUiChatStream, which keeps the REST routes for everything but the turn.

Choose AG-UI when the client is someone else's (CopilotKit, @ag-ui/client, a Python or .NET agent UI). For your own React app, the native stream carries more and needs no adapter.

A text channel in-processCopy Anchor Link

Backend only. There is no frontend of yours: the channel's app is the client. A messaging webhook resolves its own actor, sends with an empty capability list, and reads the answer back when the run ends:

@Injectable()
export class WhatsAppInbound {
  constructor(private readonly agent: AgentService) {}

  async handle(actor: Actor, threadId: string, text: string): Promise<string | undefined> {
    const sent = await this.agent.send({
      actor,
      threadId,
      message: text,
      uiCapabilities: { components: [] }, // text only
    });
    // With actionApprovalMode: 'independent', "yes" / "no #id" decide a pending action here
    // (the words are configurable: actionProposalText).
    if ('proposalDecision' in sent) return sent.text;
    if (sent.queued) return undefined; // runs after the turn in progress
    for await (const _chunk of this.agent.subscribe(sent.runId)) {
      // ends when the run settles
    }
    const thread = await this.agent.getThread(actor, sent.threadId);
    // Reply with the assistant messages stored after the user's message.
    return lastAssistantText(thread);
  }
}

lastAssistantText is your code. Under independent approvals an action tool does not hold the run open waiting for a click: the turn ends with the pending action, the user answers in a later message, and the result is posted to the thread when it runs. With the default blocking approvals, a text channel has to decide the call through AgentService.approve / reject itself.

To send a table as an image instead of text, take the stored component off the message (it is in ui, with its fallbackText) and render it with createReactServerRenderer(…).images(item) in your delivery code. Receiving a component never sends an attachment by itself.

For WhatsApp (Evolution API, Cloud API) and Telegram, @dudousxd/nestjs-agent-channels does all of this for you: it verifies and deduplicates the webhook, answers in the background, converts the reply's markdown and splits it at the channel's limit, attaches the person's media, asks questions as text, and offers proposals as buttons.

MCPCopy Anchor Link

The MCP server exposes the same registry, with no conversation behind a call:

• tools/call returns the tool's domain output as JSON text. That is the reason to keep execute's output complete and leave the visuals to present.
• ctx.emitUi resolves to an id and discards the component. present runs, and what it returns is discarded too. No text fallback is produced.
• action tools are not exposed unless actions: 'execute'. When they are, their preflight still runs before execution.

History and renderer driftCopy Anchor Link

Every stored ui component keeps its version and fallbackText, and a tree keeps componentVersions for its nodes. When a renderer is later missing, its version changed, its props no longer validate or it throws, React's GenerativeUI shows your fallback. Without one it shows the stored fallbackText. A tree with stored text and no fallback falls back as a whole.

RelatedCopy Anchor Link

• Generative UI: the catalog, AgentGenuiModule and the React provider.
• Component registries and server rendering: present, table/chart, HTML/PNG/PDF.
• Action proposals: approvals that work on every surface, including text.
• AG-UI and the MCP server.