Attachments & multimodal

A user can attach an image or PDF to a chat message so a vision-capable model sees it natively. The library stays provider-agnostic: it passes a URL straight through as the model's image/file part and never fetches bytes itself — turning an uploaded file into a model-reachable URL is the app's job, behind the attachment-staging seam.

The attachment shapeCopy Anchor Link

Every attachment is a MessageAttachment:

interface MessageAttachment {
  mediaId: string      // stable id in your media store — provenance + replay key
  url: string          // a URL the model provider can fetch at turn time
  contentType: string  // image/* → image part, otherwise → file part
  name: string         // original filename
}

A client sends attachments on the POST /agent/chat body:

{ "message": "What's wrong with this diagram?", "attachments": [ { "mediaId": "…", "url": "…", "contentType": "image/png", "name": "arch.png" } ] }

They're persisted with the message and replayed as-is; the AI SDK adapter renders each as a native model content part (image/* → image part, otherwise → file part).

Turning attachments onCopy Anchor Link

One option, attachments, names where uploads live — and that is the bring-your-own-storage seam:

import { defineConfig, attachmentStores } from '@adonis-agora/agent'

export default defineConfig({
  model: () => aiSdkModel(openai('gpt-4o')),
  attachments: attachmentStores.media(), // @adonis-agora/media: any Drive disk (S3, GCS, R2, fs)
})

• attachmentStores.media(options?) keeps uploads in @adonis-agora/media (an optional peer — register @adonis-agora/media/media_provider and configure config/media.ts): a media row per file in its own collection (agent-attachments), owned by the actor, on the media library's default disk or disk. Options: maxBytes, allowedContentTypes, collection, ownerType, disk, visibility ('private' → signed, short-lived urls; 'public' → the disk's url), urlExpiresInSeconds, resolveUrl(record, { disk }) (a CDN or proxy route), canAccess({ record, actor, allowed }) (widen or narrow who may attach a file; by default its uploader, or anyone whose OWN thread already holds a message carrying it — a fork, a regenerate; that second half is re-derived from the surviving messages on every send, through the agent store's optional referencedMediaIds, which both shipped stores implement). On a disk that cannot sign, the model gets the bytes inline as a data: url (logged once — fine in dev, not in production).
• attachmentStores.memory({ maxBytes?, allowedContentTypes? }) — tests and demos: the bytes become a data: url.
• Your own store — any AttachmentStagingStore: stage(input) stores an upload and answers the attachment; resolve({ mediaId, actor }) answers the attachment with a url the model can fetch now, or null when the actor may not use it; optional describe() declares { maxBytes, allowedContentTypes }.

Omit attachments and they are off: no upload route, and a send naming one answers 501.

The upload route and the sendCopy Anchor Link

With a store configured, POST /agent/attachments takes a multipart file, checks it against the store's limits — 413 over the size cap, 415 for a type not on the list — and answers the MessageAttachment ({ mediaId, url, contentType, name }). GET /agent/config reports the same limits (attachments: { enabled, upload, maxBytes, allowedContentTypes, maxPerMessage } — upload is 'resumable' when the routes below are mounted, else 'multipart'), which is where @adonis-agora/agent/react's useAttachments takes its defaults from.

Resumable uploads (tus)Copy Anchor Link

With attachmentStores.media() over a media library that has uploads.resumable in config/media.ts (and its tus routes.enabled), large files upload in chunks and resume after a dropped connection. The agent's routes never carry a byte:

These routes are mounted only when the media library is resumable, and then GET /agent/config reports upload: 'resumable'. The multipart route stays. On the client it is one line — see React → Attachments:

import { mediaAttachments } from '@adonis-agora/agent/react/media'

<AgentProvider attachments={{ upload: mediaAttachments() }}>

An upload is not attachable until it completes: a send naming a pending mediaId is refused like an unknown one. If routes.collection is set on the media library's tus routes, its acceptsMimeTypes also gates these uploads.

A send then names uploads by id only:

POST /agent/chat
{ "message": "what's in this?", "attachments": [{ "mediaId": "3f2c…" }] }

That is the one shape — an entry carrying anything besides mediaId is refused with 400, and at most 10 per message. The server resolves each id through the store for the calling actor (403 for one it may not use), so the url the model fetches is never taken from the request. GET /agent/threads/:id returns the attachments on the user message with the url re-minted by mediaId on every read, so an old turn's signed link has not expired.

The content-type allow-listCopy Anchor Link

The default is an exact-match list of seven types — what multimodal providers commonly accept as native image or file parts:

['image/png', 'image/jpeg', 'image/gif', 'image/webp', 'application/pdf', 'text/plain', 'text/csv']

There is no wildcard and no prefix rule. A store's allowedContentTypes replaces the list rather than extending it, so spread what you want to keep. Widen it only to types your model provider actually renders.

Listing and collecting staged uploadsCopy Anchor Link

An upload belongs to no thread until it is sent, so a thread read cannot show it. Two things answer "what has this person uploaded":

• GET /agent/attachments — the caller's own staged files, newest first (50), metadata only: { mediaId, name, contentType, sizeBytes, createdAt }, no url (one is minted only when a turn or a thread read resolves the file). Mounted with the other attachment routes; 501 when the store keeps no inventory.
• AgentService.listAttachments(actor, { limit }) — the same, in process.

Uploads that were staged and never sent pile up. AgentService.collectableAttachments is the sweep's read half: the actor's staged files that are older than a cutoff you choose and that no message references — neither a message in one of their threads nor one waiting in a thread's queue. It returns them; deleting is yours:

import { AgentService } from '@adonis-agora/agent'

const agent = await app.container.make(AgentService)
for (const actorId of await actorsWithUploads()) {
  const stale = await agent.collectableAttachments({ id: actorId }, {
    olderThan: DateTime.now().minus({ days: 7 }).toJSDate(),
  })
  for (const file of stale) {
    await deleteUpload(file.mediaId) // your storage: the media row and its bytes
  }
}

With attachmentStores.media() the rows live in the agent-attachments collection, one owner per actor (ownerType agent-actor), so actorsWithUploads is a query on the media table and deleteUpload is the media library's own delete.

olderThan is required and has no default — how long a composer may sit open with a file attached is your app's knowledge, and an upload that is merely not sent yet is not garbage. The age cut is passed to the store and applied again on what comes back. Both halves have to be answerable or it refuses (AttachmentInventoryError, status 501) instead of guessing: a store without list, or an agent store without referencedMediaIds, would otherwise make every file look collectable. A message removed by truncateFrom (regenerate, edit-and-resend) stops referencing its files, so they become collectable again.

Both shipped stores implement list: attachmentStores.media() reads the actor's rows in its collection (listByOwner), and attachmentStores.memory() keeps them in process. A store of your own adds:

async list({ actor, stagedBefore, limit }) {
  // only `actor`'s files, newest first, createdAt < stagedBefore when given, at most `limit`
}

Scope it to actor without exception — a store that ignores it turns a file list into a way to read someone else's documents.

Non-public object stores — the download bypassCopy Anchor Link

When the model provider can consume the attachment url natively (most hosted vision models can fetch a public HTTPS URL), the URL passes straight through and nothing downloads it. But when it can't — or when your url points at a non-public object store the provider can't reach (local MinIO in dev, a VPC-only S3 bucket) — the Vercel AI SDK's experimental_download step fetches the bytes and inlines them for the model.

The catch: the AI SDK's default downloader refuses localhost and private hostnames as an SSRF guard, so a staged-against-MinIO attachment dies with AI_DownloadError: URL with hostname localhost is not allowed. The package ships a ready-made replacement, attachmentFetchDownloader, on the @adonis-agora/agent/ai-sdk subpath — a plain-fetch downloader with no hostname policy — which you pass as the model's experimental_download:

import { defineConfig } from '@adonis-agora/agent'
import { aiSdkModel, attachmentFetchDownloader } from '@adonis-agora/agent/ai-sdk'
import { openai } from '@ai-sdk/openai'

export default defineConfig({
  model: () =>
    aiSdkModel(openai('gpt-4o'), {
      // Relax the SDK's SSRF guard so a MinIO/VPC-staged attachment can be fetched.
      experimental_download: attachmentFetchDownloader(),
    }),
})

It mirrors the default's routing — a URL the model supports natively is left to the provider (null), everything else is fetched and inlined as bytes.

The flowCopy Anchor Link

1. Client POSTs a file to /agent/attachments → receives a MessageAttachment (keep its mediaId).
2. Client POSTs to /agent/chat with { message, attachments: [{ mediaId }] }.
3. The server resolves each id through the store, persists the attachments with the user message, and passes each url to the model as a native image/file part.

To index uploaded files into RAG instead of (or as well as) attaching them to a turn, see RAG media ingestion.