Transformers

A transformer turns the content of a stored media into something derived: many files (a video package), one file (an extracted audio track, an optimized image), or no file at all (probed duration/codecs, a blurred image placeholder). Whatever it produces is persisted as a named conversion on the record — the same record.conversions map image presets use — with free-form metadata.

Image conversions answer "same image, different size/format". Transformers answer everything else: transform this content into that. The library ships three — transformers.video(), transformers.probe() and transformers.placeholder() — and the contract is deliberately small so you can write your own. For playing videos end to end (packaging, the authorized stream route, the React player), see Video streaming.

Declaring transformersCopy Anchor Link

Transformers are declared per collection, next to conversions:

import { defineConfig, transformers } from '@adonis-agora/media'

export default defineConfig({
  collections: [
    {
      name: 'videos',
      acceptsMimeTypes: ['video/mp4'],
      transformers: [
        transformers.video(),
        transformers.probe({ eager: true }),
      ],
    },
  ],
})

Preset names and transformer names share the record.conversions namespace, so within a collection each name may appear once — a collision throws TransformerConflictError at boot.

Constructing a transformer in config never loads its engine: the optional mediabunny peer is imported inside the first transform() call, the same laziness processors.sharp() and disks.s3() follow. Install the peer only if you use the built-ins:

pnpm add mediabunny

Typed conversion namesCopy Anchor Link

defineConfig preserves names as literal types, and two utility types extract them:

import type { InferConversions, InferTransformers } from '@adonis-agora/media'

type AppTransformer = InferTransformers<typeof config> // 'video' | 'probe'
type AppConversion = InferConversions<typeof config>   // presets + transformers

Use them to type job payloads and route params against the config instead of restating strings.

Running a transformCopy Anchor Link

Transformations are assumed heavy (the flagship case is a video transcode), so the library does not run them by itself. It ships no queue and depends on none. You have two options:

• eager: true: it runs inside attach, in the same request (below). This is only for cheap transforms.
• Your own background work calls media.library.transform(id, name). With @adonis-agora/durable, that's a step inside a workflow you dispatch after the upload. The step retries on its own, and it survives a deploy or a crashed worker:

import { defineStep } from '@adonis-agora/durable'
import media from '@adonis-agora/media/services/main'

export const transcodeVideo = defineStep(
  'media:package-video',
  async (input: { mediaId: string }) => {
    await media.library.transform(input.mediaId, 'video')
  },
  // A transcode takes minutes: give it room before it counts as stuck.
  { retries: 3, timeoutMs: 30 * 60_000 },
)

import { BaseWorkflow } from '@adonis-agora/durable'
import type { WorkflowCtx } from '@adonis-agora/durable'
import { transcodeVideo } from '#steps/media_steps'
import Lesson from '#models/lesson'

export default class PublishLessonWorkflow extends BaseWorkflow {
  static workflow = { name: 'publish-lesson', version: '1' }

  async run(ctx: WorkflowCtx, input: { lessonId: string; mediaId: string }) {
    await ctx.step(transcodeVideo, { mediaId: input.mediaId })
    await ctx.localStep('publish', async () => {
      await Lesson.query().where('id', input.lessonId).update({ status: 'published' })
    })
  }
}

const lesson = await Lesson.create({ title, status: 'processing' })
const record = await media.library.attach({ /* … */ collection: 'lessons' })
await PublishLessonWorkflow.dispatch(
  { lessonId: lesson.id, mediaId: record.id },
  { runId: `publish-lesson:${record.id}` }, // dispatching the same upload twice is a no-op
)

Any other background runner works the same way: an @adonisjs/queue job, a cron, an ace command. The only call is transform.

transform(id, name) is idempotent: if the conversion already exists it returns the record untouched, so a retried step skips straight through. A mid-flight failure sweeps whatever artifacts were already written and re-throws — no conversion entry is ever persisted for a half-written package, and the retry starts clean. Two concurrent calls both run and produce identical output (the artifact keys are deterministic); serialize with your own lock if the double work matters.

Eager transformersCopy Anchor Link

A transformer with eager: true runs synchronously inside attach and attachExisting — which is what completeUploadToLibrary (the TUS finalize) funnels through, so "TUS drops the bytes wherever you want, transformers derive from them afterwards" needs no extra wiring. An eager failure rolls the new media back whole and keeps whatever a single: true collection was replacing, exactly like eager image conversions.

Reserve eager for cheap transforms (a probe on small files). A remux inside an HTTP upload request is a bad time.

What gets persistedCopy Anchor Link

A transformer's output lands on record.conversions[name]:

// multi-artifact (a video package)
{
  path: 'Course/1/videos/<id>/conversions/video/index.m3u8', // entry artifact
  disk: 'fs',
  prefix: 'Course/1/videos/<id>/conversions/video/',
  files: ['index.m3u8', 'manifest.mpd', 'video/init-1.m4s', 'video/segment-1-1.m4s', ...], // authority for serving + deletion
  meta: { durationSeconds: 41.6, width: 1920, height: 1080, videoCodec: 'avc1.64001f', audioCodec: 'mp4a.40.2', segments: 'cmaf', segmentCount: 22, targetDuration: 4, hlsEntry: 'index.m3u8', dashManifest: 'manifest.mpd' },
}

// metadata-only (probe)
{ meta: { durationSeconds: 41.6, hasVideo: true, ... } }

Deleting the media deletes every listed artifact (batched via deleteMany on capable disks — a video package is hundreds of segments). url()/signedUrl()/deliver() on the conversion resolve the entry artifact; a metadata-only conversion has none and throws ConversionArtifactMissingError — read the record instead.

The video transformerCopy Anchor Link

transformers.video() packages a stored video for adaptive streaming: segments, plus an HLS master playlist and/or a DASH manifest. Serving and playing the result is covered on Video streaming.

The package files upload to the disk 6 at a time. On the first failed upload no new one starts; the error is rethrown once the uploads already running have settled, and the failure sweep then deletes everything written, including files whose upload had started. The persisted files list is sorted.

CMAF or TSCopy Anchor Link

The segments decide which formats can play the package:

CMAF (Common Media Application Format, ISO/IEC 23000-19) is the segment format HLS and DASH share. That is why the default writes both manifests: one copy of the segments, described twice. Each track gets its own playlist (video/, audio/), because DASH players don't play audio and video muxed into one segment.

TS is the original HLS segment format, with audio and video in the same segment. Pick it for players that predate fMP4 HLS (iOS 9 and older, some set-top boxes and smart TVs): transformers.video({ segments: 'ts', formats: ['hls'] }). Asking for DASH over TS throws at boot.

The default engine: mediabunny, remux onlyCopy Anchor Link

The default engine is mediabunny: pure TypeScript, no ffmpeg binary, so it runs in production containers where system ffmpeg doesn't exist. It stream-copies the source's tracks without re-encoding, because encoding requires WebCodecs, which Node does not expose. So by default:

• a source whose codecs the segments can carry (h264/aac, what phones and browsers produce) is remuxed fast, with no quality loss;
• a source it cannot carry fails with HlsSourceUnsupportedError, listing each track's codec and reason. That is definitive for the file: retrying cannot fix it;
• the package has a single rendition at the original resolution. A ladder requires encoding: use the ffmpeg engine.
• the package carries one video and one audio track, the source's primary ones. A second audio language or angle is left out rather than written into files no manifest describes.

mediabunny writes HLS only. For DASH, the library describes the same CMAF segments in an MPD it builds from the HLS playlists (SegmentList with a SegmentTimeline). The segments are untouched.

The engine also handles a trap the naive implementation falls into. AAC audio with priming (an mp4 edit list) starts at a negative timestamp, and a conversion trimmed at t=0 kicks that track off the stream-copy fast path, which is impossible to recover without an encoder. The conversion is trimmed at the source's real first timestamp instead, shifting all tracks equally (A/V sync unchanged) and keeping every track copyable.

Injecting WebCodecsCopy Anchor Link

Node has no built-in WebCodecs, but NAPI-based implementations exist (e.g. @napi-rs/webcodecs, prebuilt — no system binary). Injecting one enables mediabunny's re-encode fallback: a track that cannot be stream-copied gets re-encoded instead of failing.

transformers.video({ webcodecs: () => import('@napi-rs/webcodecs') })

The provided classes are installed onto globalThis (only the missing ones — a native implementation is never overwritten), because that is where the WebCodecs spec says the API lives. resolveWebCodecsSupport(provider?) reports 'native' | 'injected' | 'absent' if you want to branch.

The ffmpeg engine (transcoding, ABR ladders)Copy Anchor Link

If the host has ffmpeg, swap the engine and get real encoding: any source ffmpeg decodes becomes h264/aac, optionally as an adaptive-bitrate ladder.

import { defineConfig, engines, transformers } from '@adonis-agora/media'

export default defineConfig({
  collections: [
    {
      name: 'videos',
      transformers: [
        // 360p / 720p / 1080p; rungs taller than the source are dropped (never upscales)
        transformers.video({ engine: engines.ffmpeg({ ladder: engines.ladders.standard }) }),
        transformers.probe({ engine: engines.ffprobe() }),
      ],
    },
  ],
})

• No ladder: remux (-c copy), same contract as mediabunny. Fast, but TS segments need an h264/aac source; CMAF also carries hevc, vp9 and av1.
• ladder: one rendition per rung, each { height, videoBitrate, audioBitrate? } ('800k', '5M'). Keyframes are forced on segment boundaries so players switch renditions cleanly. Ready-made ladders: engines.ladders.standard (360/720/1080p) and engines.ladders.compact (360/720p).
• Audio: with CMAF, the audio is encoded once and shared by every rung, at audioBitrate (default: the ladder's highest). With TS, each rendition carries its own audio.
• Other options: preset (x264, default 'veryfast'), extraArgs (inserted before the muxer options), ffmpegPath / ffprobePath (default: resolved on PATH).

CMAF packages go through ffmpeg's DASH muxer, which writes the MPD and the HLS master over the same segments; TS packages go through its HLS muxer. It needs ffmpeg 4.3 or newer. The binaries are spawned, never bundled: install ffmpeg in your image (apk add ffmpeg on Alpine). A missing binary throws TransformerRuntimeMissingError; a failed run throws FfmpegFailedError with the exit code and the tail of ffmpeg's stderr.

The metadata probeCopy Anchor Link

transformers.probe() reads a media file's technical metadata — duration, resolution, codecs, sample rate, channel count — and persists it as record.conversions.probe.meta, writing no artifact. It is the reference metadata-only transformer, and it replaces the hand-rolled "download the file, open it, read the duration" service every media app grows eventually:

await media.library.transform(id, 'probe')
const record = await media.library.find(id)
record.conversions.probe?.meta
// { format: 'MP4', mimeType: 'video/mp4; codecs="avc1.42c00d, mp4a.40.2"',
//   durationSeconds: 41.6, width: 1920, height: 1080, hasVideo: true, hasAudio: true, ... }

Image placeholders (blur-up)Copy Anchor Link

transformers.placeholder() gives every image what a page needs to render before the image itself arrives: a tiny WebP preview inlined as a data: URL (a few hundred bytes), the original's dimensions (after EXIF orientation) and its dominant colour. Like the probe it writes no artifact — it all lives in record.conversions.placeholder.meta, so it ships inside the response that already carries the image URL, with no extra request and no client-side decoder.

{
  name: 'avatar',
  acceptsMimeTypes: ['image/png', 'image/jpeg'],
  conversions: [{ name: 'thumb', width: 200, height: 200 }],
  transformers: [transformers.placeholder()],
}

const record = await media.library.find(id)
record.conversions.placeholder?.meta
// { dataUrl: 'data:image/webp;base64,UklGR…', width: 1080, height: 1350, color: '#c81838' }

Render it as a blurred background that the real image covers once loaded:

<img
  src={url}
  width={meta.width}
  height={meta.height}
  style={{
    backgroundImage: `url(${meta.dataUrl})`,
    backgroundSize: 'cover',
    backgroundColor: meta.color,
  }}
/>

The tiny preview is a pixelated 16 px image — the browser's upscale already softens it, and a CSS filter: blur() on a wrapper hides the rest. width/height reserve the box, so nothing shifts when the image lands.

Options: size (long edge of the preview, default 16), quality (WebP quality, default 50), name (default 'placeholder'), eager (default true — a placeholder is only useful if it exists when the image is first rendered, and it costs one small decode — but it reads the whole original, which on a remote disk is a download per attach; set false and transform() from a job if that matters). It needs the sharp peer, loaded on the first transform. A record whose MIME type isn't image/*, or is an SVG (nothing to blur up to, and sharp can't rasterize a dimension-less one), is skipped with meta: {} instead of failing the attach, so declaring it on a mixed collection is safe.

For a blurred image served as a file instead (a background, an og:image teaser), use a conversion preset with blur.

To render it, <MediaImage> from @adonis-agora/media-react paints the preview behind a srcset-ed <img> and drops it on load — next/image-style, from media.library.image(id).

Serving HLS by handCopy Anchor Link

An HLS package (the HLS side of a transformers.video() package — the handler serves the conversion's entry playlist, so it needs 'hls' in formats) cannot be served like a single file: its playlists reference files by paths relative to the storage layout. Served from a URL scheme of your own, they must be rewritten so the player can fetch each reference. HlsDeliveryHandler is the framework-agnostic read path, following the same philosophy as MediaDeliveryHandler — you mount the route, your middleware authorizes, the handler answers "how":

import router from '@adonisjs/core/services/router'
import { HlsDeliveryHandler } from '@adonis-agora/media'
import media from '@adonis-agora/media/services/main'
import { middleware } from '#start/kernel'

const hls = new HlsDeliveryHandler({
  library: media,
  urlForPlaylist: ({ mediaId, file }) =>
    router.makeUrl('videos.hls', { id: mediaId, file }),
})

router
  .get('/videos/:id/hls/:file?', async ({ params, response, auth }) => {
    await authorizeVideoAccess(auth.user, params.id) // ← your rule

    const result = await hls.handle({ mediaId: params.id, file: params.file })
    if (result.kind === 'redirect') return response.redirect(result.url)
    if (result.kind === 'stream') {
      response.header('content-type', result.mimeType)
      return response.stream(result.stream)
    }
    response.header('content-type', result.contentType) // application/vnd.apple.mpegurl
    response.header('cache-control', 'no-store')
    return response.send(result.content)
  })
  .as('videos.hls')
  .use(middleware.auth())

One route serves the whole package:

• playlist requested (master or sub-playlist) → { kind: 'playlist', content }, with every relative reference rewritten: sub-playlists through your urlForPlaylist (so each hop comes back through your auth), media references through urlForSegment — or, by default, a presigned URL straight to the object (segmentTtlSeconds, default 300s), so segment bytes flow from storage/CDN without transiting your app;
• media file requested (a segment, an init section) → { kind: 'redirect' } to a presigned URL, or { kind: 'stream' } when segmentDelivery: 'stream' — for storage that isn't reachable from clients at all.

Requested files are validated against the conversion's persisted artifact list — no caller input ever reaches the disk as a path, so traversal is structurally impossible. References a playlist makes to files the transformer never wrote (or absolute URLs) are left untouched.

The rewriting itself is exported as rewriteHlsPlaylist(content, rewrite) if you need a custom read path: it covers URI lines, URI="…" attributes (#EXT-X-MEDIA audio renditions, #EXT-X-MAP init sections, #EXT-X-I-FRAME-STREAM-INF, #EXT-X-KEY), preserves everything else byte for byte, and accepts an async rewriter.

Writing your own transformerCopy Anchor Link

The contract is one interface:

import type { Transformer, TransformerContext, TransformResult } from '@adonis-agora/media'

export class ExtractAudioTransformer implements Transformer {
  readonly name = 'audio'

  async transform(context: TransformerContext): Promise<TransformResult> {
    const original = await context.getBytes()        // or getStream()
    const { data, durationSeconds } = await extractAacTrack(original) // your engine

    await context.write('audio.m4a', data, { contentType: 'audio/mp4' })
    return { entry: 'audio.m4a', meta: { durationSeconds } }
  }
}

The three typesCopy Anchor Link

Transformer, TransformerContext, TransformerWriteOptions and TransformResult are all exported from the barrel. In full:

interface Transformer {
  readonly name: string          // the conversion name this produces — unique within a collection
  readonly eager?: boolean       // run inside attach/attachExisting instead of on transform()
  transform(context: TransformerContext): Promise<TransformResult>
}

interface TransformerContext {
  record: MediaRecord            // the media being transformed
  disk: Disk                     // the disk holding the original (and receiving the artifacts)
  diskName: string               // its name, as resolvable through the storage manager
  storage: StorageManager        // for reads OUTSIDE the record's disk (auxiliary assets)

  getBytes(): Promise<Uint8Array> // buffer the original — fine for small files
  getStream(): Promise<Readable>  // stream it — what you want for video

  outputPrefix: string           // `<ownerType>/<ownerId>/<collection>/<mediaId>/conversions/<name>/`
  write(
    relativePath: string,
    contents: Uint8Array | Readable,
    options?: TransformerWriteOptions,
  ): Promise<void>
}

interface TransformerWriteOptions {
  contentType?: string
  contentLength?: number         // load-bearing for a Readable on a streaming disk (S3 needs it up front)
}

interface TransformResult {
  entry?: string                 // entry artifact, relative to outputPrefix — omit for metadata-only
  meta?: Record<string, unknown> // persisted verbatim on record.conversions[name].meta (JSON-serializable)
}

Two things are worth naming explicitly. The context, not the transformer, decides where artifacts live: outputPrefix is computed by the library and every write is keyed under it, which is what makes an artifact list the library can trust — and therefore a package it can delete whole. And contentLength is not decoration: a Readable has no knowable length, S3 requires ContentLength up front, so a stream without it gets buffered instead of streamed.

What the context guarantees:

• write(relativePath, contents, options?) is the only artifact path. Every write lands under context.outputPrefix (…/conversions/<name>/), is validated (no .., no absolute paths — TransformerOutputError), and is recorded: the persisted files list comes from what was actually written, never from the transformer's own bookkeeping. A Readable with a contentLength streams to the disk without buffering.
• record, disk/diskName, and storage — the record being transformed, its disk, and the storage façade for reading auxiliary assets (a watermark logo on another disk) via context.storage.disk(name).
• The result names the entry (entry must be one of the written files) and carries meta. A metadata-only transformer writes nothing and returns just { meta }.

Rules of thumb, set by the built-ins:

• import heavy engines inside transform, never at module top level, so configuring the transformer stays free;
• if the runtime lacks a capability you need (an optional peer, WebCodecs), throw TransformerRuntimeMissingError with an install hint;
• accept an injectable engine in your options — it is what makes the orchestration testable without the engine (see FakeTransformer in @adonis-agora/media/testing, which also lets apps test their jobs without ever touching a media engine).

ErrorsCopy Anchor Link

Next stepsCopy Anchor Link

• Video streaming — serving and playing HLS/DASH packages
• Delivery — the single-file read path this extends
• Resumable / TUS — landing the video the transformer will process
• Roadmap — the transformer lineup that isn't built yet, and why