Generative UI

Some answers are a table, a chart or a checklist, not prose. Generative UI lets the server push a component into the assistant message: a name, props, and optionally a version. The library carries it and persists it. How it looks is always up to your app.

server                      stream                          client
catalog ──► tool validates ──► { kind: 'ui', id, component, ──► registry[component]
            ctx.emitUi          props, version?,                 (your renderer)
                                fallbackText? }
                                   │
                                   └─► persisted on the message → reload shows the same thing

The whole setup is four pieces of consumer code:

1. A catalog file, shared by server and browser.
2. AgentGenuiModule.forRoot({ catalog }) on the server.
3. Optionally, your own tools calling ctx.emitUi.
4. genui={{ registry, catalog }} on the client's <AgentProvider>. Nothing per message.

For result-driven UI, use the tool’s optional present hook. See Component registries and server rendering for functional/class tools, custom components, shared React renderers, and WhatsApp image/PDF delivery.

The same component reaches clients that cannot draw it as text. A client declares the components it renders (uiCapabilities), and the server sends the rest as their fallbackText. See One answer, every surface.

When no catalog component fits a one-off interactive answer, the opt-in sandbox lets the model write a small HTML/CSS/JS view, drawn in an isolated iframe. And any A2UI client can draw the same components: see A2UI.

1. A catalogCopy Anchor Link

import { defineCatalog, defineComponent } from '@dudousxd/nestjs-agent-core/genui';
import { BUILTIN_COMPONENTS, LAYOUT_COMPONENTS } from '@dudousxd/nestjs-agent-core/genui/builtins';
import { z } from 'zod';

const DealCard = defineComponent({
  name: 'DealCard',
  title: 'Deal',
  description: 'One deal in the pipeline: name, stage, amount.',
  props: z.object({ name: z.string(), stage: z.string(), amount: z.number() }),
  fallbackText: (p) => `*${p.name}* — ${p.stage} (${p.amount})`,
  version: 1,
});

export const catalog = defineCatalog([...BUILTIN_COMPONENTS, ...LAYOUT_COMPONENTS, DealCard]);

@dudousxd/nestjs-agent-core/genui is isomorphic: it imports nothing server-only, so the React app imports this same file to validate props before drawing them.

BUILTIN_COMPONENTS and LAYOUT_COMPONENTS are optional, ready-made definitions (no visuals): a table, a chart, KPI cards, sources, a checklist, a timeline, code, a diff and a callout, plus Stack, Card, Heading, Text, Badge, Link and Image. Their props, text fallbacks, tools, and how to pick or override them are in Builtin components.

Props may be any Standard Schema or a JSON Schema object. JSON Schema is plain data, so components can come from a database, for example components each tenant defines and versions for itself (see per-request catalogs). JSON Schema props go through a small built-in validator by default. Pass ajvValidator(new Ajv()) to defineCatalog for the whole spec.

fallbackText is what a channel that cannot draw the component shows (Slack, email, WhatsApp, a CLI): componentToText(catalog, component, props). The server stores it on every pushed component, and sends it instead of the component to a client whose uiCapabilities do not list it. version (default 1) is what a client's declaration has to match.

2. The server: AgentGenuiModuleCopy Anchor Link

import { AgentModule } from '@dudousxd/nestjs-agent';
import { AgentGenuiModule } from '@dudousxd/nestjs-agent/genui';
import { catalog } from './catalog';

@Module({
  imports: [
    AgentModule.forRoot({ /* … */ }),
    AgentGenuiModule.forRoot({ catalog }),
  ],
})
export class AppModule {}

That registers the tools the model uses to push components, each call validated against the catalog. A refused call reaches the model as a tool error naming each problem, and the model can fix it. There are two shapes.

Tree (the default), one tool, ui__render, whose input is a nested layout composed from the catalog:

AgentGenuiModule.forRoot({
  catalog,
  treeToolName: 'renderResult',
  treeInstructions: 'Call this after reading the data, to present the answer.',
  terminal: true,
});

{ "type": "Card", "props": { "title": "Signups" }, "children": [
  { "type": "KpiCards", "props": { "items": [{ "label": "New", "value": 12 }] } },
  { "type": "DataTable", "props": { "columns": [{ "key": "day", "label": "Day" }], "rows": [] } }
] }

The tree tool's description carries catalogToModelText(catalog), so the model sees every component, its props and which ones take children. Each node is validated. A valid tree is pushed as one frame, component: 'genui:tree', props: { root }. A tree of a single component with no children — { "type": "DataTable", "props": { … } } — is pushed as that component, exactly as its per-component tool would push it (same frame, version and fallbackText).

The tool's input schema is exact: below the root, every node is a union by type (through $defs / $ref) whose variants carry their component's own props schema, and only components that take children have children. The root itself is a plain object listing every type and every props schema, because OpenAI and Anthropic both refuse a union at the top level of a tool's parameters. A component whose props schema is not self-contained (it carries its own $ref / $defs) is described as a plain object. A provider that refuses $ref in tool parameters takes treeSchema: 'loose' — one generic node shape, the props described in the tool's text only. Whichever schema the model sees, the server validates every call.

When the model gets the shape wrong. Where what it meant is unambiguous, the server takes it as meant rather than refusing the call (validateTree): a tree, children or props sent as a JSON string (with a stray trailing }, or > written for a key's :); an element that left out its envelope — { "props": { "html": … } } or the bare props { "html": …, "css": … } — when exactly ONE component's props schema fits those keys (every key one of its properties, every required one present); props written beside the type; a whole element wrapped in one more key ({ "props": { "type": …, "props": … } }, { "root": "{…}" }), or the component named under component / root. Anything else is refused with a message naming the shape. A client that draws a call from its arguments (CopilotKit renders tool calls; an OpenUI artifact renderer reads them) applies the same reading with normalizeTreeInput(catalog, args) from @dudousxd/nestjs-agent-core/genui, so it draws what the server accepted. The tool's description also shows the envelope with an example built from your catalog.

A component with a tool of its own. componentTools: ['Sandbox'] keeps tree mode and ALSO registers a flat tool for each named standalone component — ui__sandbox, whose input IS the sandbox's props (prefix: componentToolPrefix, default ui__). A big component a model writes on its own has no envelope to leave out there; it streams as it does in a tree, may still be nested in a ui__render tree, and the tree tool's description points the model at it.

Per component (mode: 'per-component'), one tool per component, whose input is the props: ui__show_data_table, ui__show_deal_card, … internal components and layout components (children: true, e.g. Stack, Card) get no tool — flat props cannot carry children. The generic ui__show tool leaves them out too. Reach for it with small models that compose trees poorly, or when each tool should carry its own exact schema.

terminal: true ends the turn once the call succeeds. When the UI is the answer, there is no model call to narrate it.

Options: mode, treeSchema, streaming, streamingThrottleMs, terminal, treeToolName, treeInstructions, treeLimits, componentTools, componentToolPrefix, namePrefix, showTool, showInstructions, roles, ability, presentation, resolver. To build them from config:

AgentGenuiModule.forRootAsync({
  imports: [ConfigModule],
  inject: [ConfigService],
  useFactory: (config: ConfigService) => ({ catalog, streaming: config.get('GENUI_STREAMING') }),
});

The catalog is injectable anywhere — @InjectGenuiCatalog() catalog: Catalog (token GENUI_CATALOG) — and a test swaps it with overrideProvider(GENUI_CATALOG).useValue(testCatalog); the tools are built from the injected one.

Per-request catalogsCopy Anchor Link

When components differ per tenant (defined and versioned in a database), give the module a resolver. It is consulted on every call (validation, and the version stamped on the frame) and every time a turn's tool list is described to the model, so nothing is rebuilt at boot:

@Injectable()
export class TenantCatalogResolver extends GenuiCatalogResolver {
  constructor(private readonly components: ComponentRepository) {
    super();
  }
  async resolve({ tenant }: GenuiCatalogScope): Promise<Catalog> {
    // Cache here: it runs once per turn per genui tool, and once per call.
    return defineCatalog([...BUILTIN_COMPONENTS, ...(await this.components.forTenant(tenant))]);
  }
}

AgentGenuiModule.forRootAsync({
  imports: [ComponentsModule],
  useFactory: () => ({ mode: 'tree', showTool: true, terminal: true }),
  resolver: TenantCatalogResolver,
});

resolve({ actor, threadId, tenant, agentName }) — tenant is actor.tenantRef. With a resolver, the catalog option is only the boot-time one (it names the per-component tools) and may be omitted.

A boot-time tool list cannot name a component that only one tenant has, so reach tenant components through the tree tool or the generic show tool (showTool: true, named ui__show): one tool taking { component, props }, whose description and input schema list the tenant's components for that turn. Per-component tools still work for components every tenant shares; one the resolved catalog lacks is left out of that turn's tool list, and refused if called anyway. The same per-turn narrowing applies to the client's uiCapabilities.

In a test, overrideProvider(GenuiCatalogResolver).useValue({ resolve: () => testCatalog }).

Drawing a tree while the model writes itCopy Anchor Link

A model-composed layout is one ui__render call, and its arguments stream for as long as the model writes them — seconds, for a dashboard. By default nothing is drawn until the call has run; then the validated tree appears whole. With streaming: 'partial' the layout fills in as it is written:

AgentGenuiModule.forRoot({ catalog, streaming: 'partial' }); // tree mode, the default

The server reads the streaming arguments (tool-input-delta) as far as they go and pushes the tree so far as ui frames marked partial: true, under the id the final push uses (<toolCallId>:ui:0). Each one replaces the last in place; the call's own push, validated as always, replaces the last preview. What a partial frame is, and is not:

• Not validated. A preview's props are whatever the model has written so far. Only the final tree goes through the catalog, and an invalid one is refused to the model as before (it retries under a new call). A preview the call does not replace — the tree was invalid, or degrades to text for this client — is withdrawn with a partial frame whose props are {}, before the call's outcome.
• Not persisted. The message keeps the final tree only; a reloaded thread draws it. Under the durable runner the previews a step showed ride its journaled result, so a replay withdraws the same ones.
• Not for text. A preview carries no fallbackText, a text channel skips it, and a turn whose client cannot draw the tree (its UI capabilities leave a node out) gets no previews at all — only the final text.
• Throttled. At most one preview every streamingThrottleMs (100 ms) per call, and only when it changed, so a long tree costs the stream (and the Redis or SQL sink buffering it) a bounded number of snapshots.

A node appears once its type has arrived and names a component this client draws. Every node of a partial tree carries its position as id (root, root.0, root.1.2 — the same rule that names the final tree's nodes, so a renderer keyed on it never remounts), and incomplete: true while the model is still writing it:

{ "kind": "ui", "id": "c1:ui:0", "component": "genui:tree", "partial": true, "version": 1,
  "props": { "root": { "id": "root", "type": "Card", "props": { "title": "Sales" }, "incomplete": true,
    "children": [{ "id": "root.0", "type": "KpiCards", "props": { "items": [{ "label": "Rev" }] }, "incomplete": true }] } } }

Per component. A component that cannot draw half its data opts out on its definition:

defineComponent({ ...Chart, streaming: 'complete' });

While the model writes it, such a node is sent as a placeholder, { id, type, props: {}, incomplete: true, held: true }, without its children; it is drawn once its whole subtree has arrived and its props pass its schema. streaming: 'partial' on a definition does the opposite — previews a component in a tree that defaults to complete — and a complete layout above it is drawn (incomplete, its children so far) once its own props are whole, instead of holding it back. The default stays complete because a renderer written for validated props would be handed half-written ones: opting in is saying your renderers cope (see Streaming trees on the client).

Previews need a model stream that carries the call's arguments as they are written. aiSdkModel does; the OpenCode engine does not (its server sends a tool call's input whole), so there a tree appears once, when the call completes.

The mechanism is a hook any tool can implement: ToolHandler.previewInput(scope) answers how to preview a streamed call (render(partialInput) → a frame, undefined for nothing new, null to withdraw), and the loop does the parsing (parsePartialJson, exported from the core), the throttling and the withdrawal. The wire format is the AdonisJS sibling's (@adonis-agora/agent), so either client draws either server's previews.

Money and units in what a tool returnsCopy Anchor Link

A tool's output is what the model reads, and a model handed totalCents: 12999 sometimes says "$12,999". Return the amount already formatted for people, beside the raw value a component takes — { total: '$129.99', totalCents: 12999 } — and push the raw value to the component (its props are typed). The same goes for any unit a model could misread (bytes, basis points, milliseconds): format it in the data, not in a prompt line the model may skip.

3. Pushing from your own tools: ctx.emitUiCopy Anchor Link

Any lib tool can push a component. The generated genui tools call this for you.

@AiTool({ description: 'Weekly forecast for a city.', input: z.object({ city: z.string() }) })
export class ForecastTool { // name: forecast, kind: read
  async execute({ city }: { city: string }, ctx: AiToolCtx) {
    const days = await this.weather.week(city);
    const { id } = await ctx.emitUi('Chart', {
      type: 'line', xKey: 'day', series: [{ key: 'maxC' }], data: days,
    });
    return { shown: id, days: days.length }; // what the model is told
  }
}

• Always there. ctx.emitUi is on every AiToolCtx. Where there is no conversation to push into (the MCP server, a bare registry.invoke) it is a no-op that still resolves to an id, so a tool never checks for it.
• Validated by the server. With AgentGenuiModule installed, the component must be in the catalog (or the resolver's catalog) and its props must validate, or the call throws. A component the client did not declare in uiCapabilities goes out as its text instead. See What changes for your tools.
• Live. The frame streams while the tool runs, in the inline runner, the durable runner, and on the worker that serves a dispatched tool step.
• Persisted. When the step's tools have settled, the loop writes every component of the step onto the assistant message (AgentStore.setMessageUi, which the Drizzle, MikroORM and in-memory stores implement). The model turn's own frames come first, then the tools' pushes in call order. A reload shows each one right after the tool call that pushed it.
• Updates in place. id defaults to <toolCallId>:ui:<n>. Pass your own id to update one component across pushes, for example streaming rows into a table.
• Replay-safe. The pushes ride the tool step's journaled result. A durable replay re-runs nothing, so it neither re-streams nor re-persists them. A tool that pushes nothing journals exactly what it always did.

Another runnerCopy Anchor Link

A backend that is not this library's loop (a sandboxed agent, an OpenCode runner) pushes the same way. It writes { kind: 'ui', id, component, props, version?, toolCallId? } between the call's announcement and its outcome, with an id derived from the call so a retry replaces it. It also persists the list on the assistant message's ui field. See the stream protocol.

4. The client: genui on <AgentProvider>Copy Anchor Link

import { AgentProvider } from '@dudousxd/nestjs-agent-react';
import type { GenuiRegistry } from '@dudousxd/nestjs-agent-react/genui';
import { catalog } from './catalog';

const registry: GenuiRegistry = { DataTable: MyTable, Chart: MyChart, Card: MyCard, Text: MyText };

<AgentProvider
  genui={{
    registry,
    catalog,
    fallback: ({ reason, item }) => <Unknown name={item.component} reason={reason} />,
  }}
>
  <App />
</AgentProvider>;

That's all: MessageItem and MessageList draw every pushed component inside the provider, with no renderUi per message. genui takes exactly what <GenuiProvider> takes — GenuiProvider from @dudousxd/nestjs-agent-react/genui is the same thing as a standalone building block, for an app without AgentProvider or a subtree with its own registry.

• Renderers receive the props spread, and children when they are a layout node in a tree.
• catalog validates before drawing, synchronously when the schema allows, so nothing flashes.
• resolveComponent(name, version) covers components the registry lacks, such as tenant components at the exact version a message was rendered with. It is cached, and it may be async (loading shows meanwhile).
• Every item, and every tree node, has its own error boundary. An unknown component, invalid props or a renderer that throws each end up in your fallback. Without a fallback, the item's stored fallbackText is shown.
• Tell the server what this app draws with <AgentProvider uiCapabilities={{ components: [...] }}>. It is not derived from registry. Omitted, the server offers and sends every catalog component.
• genui:tree frames render node by node through the same registry.

A refused call the model fixedCopy Anchor Link

When a ui__render (or any tool) call fails and the model's next tool call is the same tool and does not fail, the transcript leaves the failed one out (retriedCallIds) — while the retry runs and once it succeeds — so the person sees the view, not the refusal before it. A failure the model moved on from, or whose retry failed too, stays. useChatTranscript({ hideRetriedFailures: false }) (or buildTranscriptBlocks's option) keeps every attempt, for an operator's view.

Streaming trees on the clientCopy Anchor Link

A partial tree frame renders node by node like a final one, keyed by node position, so nodes stay mounted while they fill in and when the final frame replaces the preview. A node the model is still writing is not validated against the catalog (its props are half there); its renderer can ask with useGenuiNode(), which answers { id, type, incomplete, held } inside a tree and null outside one:

import { useGenuiNode } from '@dudousxd/nestjs-agent-react/genui';

function KpiCards({ items = [] }: { items?: Kpi[] }) {
  const node = useGenuiNode();
  if (node?.incomplete && items.length === 0) return <KpiSkeleton />;
  return <Kpis items={items.filter((item) => item?.label)} />;
}

Props of an incomplete node can be missing or half-built (an array still growing, its last item partial), so a renderer opted into previews defaults what it reads. A held node (streaming: 'complete') never reaches its renderer: <GenerativeUI> draws placeholder instead — a node, or (node) => ReactNode — defaulting to loading, and settable once on <GenuiProvider> (or genui on <AgentProvider>). A withdrawn preview draws nothing, and the transcript drops a preview whose call settled without replacing it. incomplete only ever means "the model has not finished writing this node"; it is not a loading state for data.

Only the nodes that changed render again. The chat reaches React as a fresh copy on every update (each token, each partial frame), so the transcript and the tree renderer share structure with the previous frame: a node, a pushed component or a props object equal to the one before keeps that object, and its renderer does not run. A node that grew, or whose incomplete or held flag changed, is a new object, and so are its ancestors. A heavy renderer (a chart that measures itself, a table) therefore needs no memo of its own to keep up with a fast stream. Inline fallback, loading, placeholder and onError values are read when a node renders, so passing new ones does not re-render the tree either. For a loading state for slow data (a tool that pushes a skeleton and replaces it), and for skeletons that keep the layout steady, see Loading states and skeletons.

For your own transcript markup, <GenerativeUI part={block} /> draws one component with the provider's settings (any prop you pass wins), useGenerativeUI(part) gives you the state for custom chrome, and useAmbientRenderUi() (main entry) returns the provider's renderer. An explicit renderUi on MessageItem still wins.

json-render. If your app already renders json-render specs, send tree frames through it: import GenuiProvider from @dudousxd/nestjs-agent-react/genui/json-render (optional peer @json-render/react >= 0.21) and add jsonRender:

import { GenuiProvider } from '@dudousxd/nestjs-agent-react/genui/json-render';

<GenuiProvider registry={registry} catalog={catalog} jsonRender>  {/* or jsonRender={myJsonRenderRegistry} */}
  <App />
</GenuiProvider>;