CLI

@adonis-agora/durable ships six ace commands: two long-running workers (durable:work for a store-backed pod, durable:worker for a store-less thin pod), a run lister, a retry, a replay-fixture exporter, and a workflow generator. Except for durable:worker, they all resolve the WorkflowEngine bound by the provider, so they operate on the same engine (and store) your app is configured with.

node ace configure @adonis-agora/durable

configure registers the commands barrel (@adonis-agora/durable/commands) in adonisrc.ts so the commands become available to ace — there's no separate package to install.

durable:work — the worker loop

The long-running worker. On an interval it picks up pending runs, recovers crashed runs, resumes due timers, sweeps execution timeouts, and fires any due scheduled workflows. It stays alive until SIGINT/SIGTERM, then drains in-flight executions so a deploy hands off cleanly.

node ace durable:work
node ace durable:work --interval=500 --drainTimeout=15000

Each tick runs engine.runPending() → engine.recoverIncomplete() → engine.resumeDueTimers() → engine.sweepTimeouts() → fire due schedules → engine.sweepRetention() (retention eviction, self-throttled to once a minute) → engine.sweepStalled() (the stalled-run pager, same throttle, a no-op without listeners). Run this as a dedicated process (or several) when you split web from worker; in a single-process app the in-process dispatcher already runs workflows, so this is optional but recommended for recovery and timers.

durable:worker — the store-less worker

The worker for a role: 'tenant' pod: it serves step bodies and never touches a state store. It boots the container-bound WorkerRuntime, registers every handler under app/steps, advertises its descriptor and heartbeat so the control plane can route to it, and drains on SIGINT/SIGTERM.

node ace durable:worker

It takes no flags — everything comes from config/durable.ts (partition, namespace, transport, stepsPath). Running it on a store-backed role logs a warning and points you at durable:work; see Thin workers for the full topology.

durable:runs — list runs

Lists recent runs, optionally filtered by status and workflow, reading through engine.listRuns.

node ace durable:runs
node ace durable:runs --status=failed
node ace durable:runs --workflow=checkout --limit=100

Every row also carries liveness signals — how long ago the run was updated, how many times recovery has picked it up, and the age of its oldest pending remote step. That matters because suspended alone cannot tell a run mid-step apart from a run whose dispatch was lost.

--stale is the query that separates them. It narrows the listing to running/suspended runs whose oldest pending remote step is older than the threshold — the stranded signature — and prints the durable:retry hint for each one:

# What has been sitting on a remote step for over an hour?
node ace durable:runs --stale=1h

Set the threshold above your longest legitimate step, or a slow-but-healthy run shows up as stranded.

For the listing to show runs across processes, the engine must be backed by a persistent store (config/durable.ts) — the default in-memory store only sees this process's runs.

durable:retry — re-enqueue a run

Re-enqueues a run for a worker to (re-)execute. It calls engine.requeue(runId): the run goes back to pending, any stale lease is cleared, and a worker resumes it (replaying its checkpoints, re-attempting the failed step).

node ace durable:retry checkout:1234

Run durable:work (or any worker) to pick it up afterward. This is the terminal equivalent of the dashboard's retry button — handy for replaying a dead run once you've shipped a fix.

durable:export — capture a replay fixture

Exports a run's replayable history — the run row plus its full checkpoint timeline — as JSON, for the replay-CI loop. Commit the file and assert it with assertReplayable(register, parseRunHistory(json)): a code change that renames, reorders or removes a step at a position the history already recorded then fails the build instead of corrupting an in-flight run on deploy.

node ace durable:export ord-42                                       # JSON to stdout
node ace durable:export ord-42 --out tests/fixtures/checkout.json    # or to a file

An unknown run id exits 1 with an error. In code, the same capture is captureHistory(engine, runId) from @adonis-agora/durable/testing.

make:workflow — scaffold a workflow class

Scaffolds a BaseWorkflow subclass under app/workflows/, the parallel to @adonisjs/queue's make:job. The generated class is auto-registered on the engine at boot — see app/workflows & make:workflow.

node ace make:workflow order

Building your own

The pieces these commands are made of are exported from @adonis-agora/durable, so a custom worker process or admin script composes them instead of re-implementing the loop:

For example, an ops endpoint that reports stranded runs without shelling out to ace:

import { inject } from '@adonisjs/core'
import type { HttpContext } from '@adonisjs/core/http'
import {
  DEFAULT_STALE_MS,
  WorkflowEngine,
  attachLiveness,
  filterStale,
  listRuns,
} from '@adonis-agora/durable'

@inject()
export default class OpsController {
  constructor(private engine: WorkflowEngine) {}

  async strandedRuns({ response }: HttpContext) {
    const runs = await listRuns(this.engine, { statuses: ['running', 'suspended'], limit: 200 })
    const stale = filterStale(await attachLiveness(this.engine, runs), DEFAULT_STALE_MS)

    return response.ok({ stranded: stale.map(({ run }) => run.id) })
  }
}