Agora
State stores

Overview

Where durable state lives. A config-driven StateStore with a Lucid driver that runs on Postgres, MySQL, or SQLite, an in-memory store for tests, and a migration shipped with @adonis-agora/durable.

A state store persists workflow runs and step checkpoints — the source of truth for both durability and the dashboard. It's a pluggable StateStore driver, independent of the transport, so you can mix any store with any transport.

Stores are config-driven drivers: list the ones you use under stores in config/durable.ts, built with the stores factory, and pick the active one by name with store. The lucid driver lazily imports @adonisjs/lucid only when selected.

Drivers

StoreDatabaseSchema
in-memory (the default)For tests and local dev. Not durable.
stores.lucid(...)Postgres · MySQL · SQLiteshipped migrationThe production driver, on @adonisjs/lucid.

The default — when config/durable.ts sets no store — is an in-memory store: zero setup, but state is lost on restart, so it's only for development and tests. For anything durable, select the Lucid store.

config/durable.ts
import { defineConfig, stores, transports } from '@adonis-agora/durable'

export default defineConfig({
  store: 'lucid',
  stores: {
    lucid: stores.lucid(),
  },
  transport: 'memory',
  transports: { memory: transports.memory() },
})

The interface

A StateStore persists runs and checkpoints and exposes the primitives the engine needs for durability:

  • Runs & checkpointscreateRun, updateRun, updateRunIf, getRun, deleteRun, saveCheckpoint, getCheckpoint, listCheckpoints. createRun must reject a duplicate id (throw — a SQL primary-key violation is exactly right) rather than overwrite: the engine treats the throw as "someone else already started this id", re-reads the existing run and returns its state, which is what makes start / signalWithStart / racing scheduler ticks converge on one run instead of forking or clobbering the winner.
  • Recovery & leasinglistIncompleteRuns, listPendingRuns(limit) (so workers poll for pending runs), listDueTimers(nowMs, namespace?, limit?) (the limit caps a due-timer batch per poll; a store may ignore it), and the atomic tryLockRun / renewRunLock / releaseRunLock that make one-instance-per-run leasing work. releaseRunLock(runId, owner?) is owner-fenced: when owner is given, release only if that owner still holds the lease (a conditional write, atomic like renewRunLock) — so a stale executor whose lease was taken over can't wipe the new owner's lease and open the door to a third concurrent executor. Omitting owner is the operator-style unconditional clear (requeue's stale-lease reset); a store implemented against the older single-argument signature simply ignores owner and degrades to the previous unfenced behavior.
  • SignalsputSignalWaiter / takeSignalWaiter / listSignalWaiters / removeSignalWaiter, plus bufferSignal / takeBufferedSignal for the signal machinery.
  • EventsbufferEvent / listBufferedEvents / removeBufferedEvent, so a published event that matched no live waiter is still consumed by a later ctx.waitForEvent.
  • QueryinglistRuns(query) for the dashboard and CLI. RunQuery carries the status/workflow/tag/namespace/attribute filters plus exact-match origin and the epoch-ms range pushdowns createdBefore / createdAfter / updatedBefore / wakeBefore — the last two are what let the retention/stalled sweeps and the blocked-run poll ask the store for candidate rows instead of fetching everything and filtering in process. Paging rides page (1-based, default 1) and size — the same offset shape @adonis-agora/filter takes, so every Agora listing is paged alike. Resolve the pair with the exported runPageWindow(query) helper rather than by hand: it returns the { limit, offset } your adapter spends, and it is what the conformance kit asserts against (no size means no bound — a store must not invent a default window).

Eight members are optional, and the engine works without every one of them:

Optional memberWhat you lose by omitting it
ensureSchema()Boot-time provisioning. A store with nothing to provision (in-memory) simply has no schema to create.
transaction(work)ctx.transaction — the exactly-once "business write and checkpoint commit together" primitive — throws on a store without it.
recordStepHeartbeat(...)Step heartbeats are discarded, so a long remote step shows no liveness in the dashboard or durable:runs.
getLatestCheckpointByName(...)A fast path only. The engine falls back to listCheckpoints plus an in-JS filter.
listCheckpointsByNamePrefix(...)Same — a fast path, with the same fallback.
listOrphanedRuns(nowMs, limit, namespace?)The bounded orphan query (running + free/expired lease) the periodic recovery pass prefers. Without it the engine falls back to listIncompleteRuns and filters in process — correct, but on a busy fleet every worker fetches (and lock-probes) every healthy running run once per tick. Strongly recommended.
listSignalWaitersByRunIds(runIds)The targeted per-page waiter lookup (run_id IN (...), indexed). Without it the dashboard's waiting column and the engine's child resolution fall back to listSignalWaiters('') — a full-table scan per page load. Pair the implementation with an index on run_id.
runValueFacets(axis, query, opts?)Native picker enumeration (distinct values + counts per filter axis). Callers fall back to counting a bounded listRuns scan in-process — same shape, bounded approximation on the non-column axes.

Everything else is required, including updateRunIf.

updateRunIf — the compare-and-set write

updateRunIf(
  runId: string,
  expectedStatuses: RunStatus[],
  patch: Partial<WorkflowRun>,
): Promise<boolean>

Apply patch only if the run's currently persisted status is one of expectedStatuses at write time. It returns true when the write landed and false when the predicate did not match. A non-match is a normal outcome, never a throw.

The predicate must be atomic with the write

For a SQL store the predicate belongs in the UPDATE … WHERE clause. For an in-process store it must be a synchronous check with no await between reading the status and mutating the row. A read-then-write in application code is a TOCTOU race, not a substitute: two racing callers can both observe a matching status before either writes.

Why it exists

A workflow turn computes its outcome from a run snapshot loaded when the turn began — and a turn can run for minutes across remote steps. Meanwhile the run's persisted status can change through a completely different path: an operator cancels it, the execution-timeout sweep marks it cancelled, a poison run is dead-lettered, or ctx.all in failFast mode cancels a sibling.

With a plain updateRun, the turn's now-stale terminal write lands last and resurrects a terminal run: a cancelled run silently reports completed, its output is written, its parent is notified, and a run.completed event fires. updateRunIf moves the "is this run still eligible?" test into the same atomic write, so the loser of the race gets false, emits nothing, and reports the run's true current status instead.

The engine leans on it in five places — the execution-timeout sweep, all three terminal/suspend transitions of a settling turn, and parking a run as blocked — and the conformance kit covers it in three cases, including the two-sequential-writes race directly.

Here is the Lucid driver's implementation, as the shape to copy:

const affected = await trx
  .from(DURABLE_TABLES.runs)
  .where('id', runId)
  .whereIn('status', expectedStatuses)
  .update(row)

return rowsAffected(affected) === 1

Writing a custom store

Implement every required member above, then run the shared conformance contract against it — that is the definition of "correct", not the prose here:

tests/my_store.spec.ts
import { runStateStoreContract } from '@adonis-agora/durable/testing/conformance'
import { MyStateStore } from '../src/my_state_store.js'

runStateStoreContract('MyStateStore', async () => {
  const store = new MyStateStore(/* connection */)
  await store.ensureSchema?.()

  return {
    store,
    cleanup: async () => {
      await store.close()
    },
  }
})

The factory runs once per test case, so each case gets a fresh store. Two escape hatches exist for drivers that genuinely cannot satisfy a slice of the contract: supportsAsyncTransaction: false (a synchronous driver) and supportsTagFilter: false. If the backing infrastructure is not available — no Docker for a testcontainers store — throw StateStoreUnavailableError from the factory and the suite skips instead of failing.

The conformance kit lives on the @adonis-agora/durable/testing/conformance subpath rather than @adonis-agora/durable/testing, because it generates the suite with vitest's describe/it. See Testing.

Encrypting payloads at rest

CodecStateStore decorates any store and runs run/step payloads through a PayloadCodec — encoded on write, decoded on read — so inputs and outputs are never stored in the clear. Searchable metadata (id, status, workflow, tags, namespace, timestamps) is left untouched, so the dashboard, queries and recovery keep working.

Know what the default coverage leaves in the clear: step events (everything step code passes to ctx.log), the structured error messages, and heartbeat progress. A deployment described as "encrypted payloads" still stores whatever its steps log as plaintext. Opt into new CodecStateStore(inner, codec, { coverage: 'extended' }) to cover those too — but only on a fresh table, or with a codec whose decode recognizes its own ciphertext and passes other values through, because rows written before the switch hold those fields unencoded. Search attributes always stay clear (they must remain queryable). The wrapper also forwards recordStepHeartbeat now (it used to silently drop persisted step liveness). And remember the console renders everything decoded — pair the codec with the dashboard config's redact hooks to keep PII out of the operator's browser.

config/durable.ts
import { createCipheriv, createDecipheriv, randomBytes } from 'node:crypto'
import { CodecStateStore, LucidStateStore, defineConfig, type PayloadCodec } from '@adonis-agora/durable'
import db from '@adonisjs/lucid/services/db'
import env from '#start/env'

const key = Buffer.from(env.get('DURABLE_PAYLOAD_KEY'), 'base64') // 32 bytes

const aesCodec: PayloadCodec = {
  encode(value) {
    const iv = randomBytes(12)
    const cipher = createCipheriv('aes-256-gcm', key, iv)
    const body = Buffer.concat([cipher.update(JSON.stringify(value), 'utf8'), cipher.final()])
    return {
      iv: iv.toString('base64'),
      tag: cipher.getAuthTag().toString('base64'),
      body: body.toString('base64'),
    }
  },
  decode(value) {
    const { iv, tag, body } = value as { iv: string; tag: string; body: string }
    const decipher = createDecipheriv('aes-256-gcm', key, Buffer.from(iv, 'base64'))
    decipher.setAuthTag(Buffer.from(tag, 'base64'))
    const plain = Buffer.concat([decipher.update(Buffer.from(body, 'base64')), decipher.final()])
    return JSON.parse(plain.toString('utf8'))
  },
}

export default defineConfig({
  store: 'encrypted',
  stores: {
    encrypted: async () => new CodecStateStore(new LucidStateStore(db), aesCodec),
  },
})

encode and decode are synchronous and decode must be the exact inverse of encode — wrap an async KMS behind a local cache yourself. Beyond run and checkpoint payloads the decorator also encodes buffered signal and event payloads.

CodecStateStore does not implement the optional recordStepHeartbeat, so wrapping a store trades persisted step heartbeats for encryption at rest. The live step.started event still fires.

On this page