Overview
Where durable state lives. A config-driven StateStore with a Lucid driver that runs on Postgres, MySQL, or SQLite, an in-memory store for tests, and a migration shipped with @adonis-agora/durable.
A state store persists workflow runs and step checkpoints — the source of truth for both durability and the dashboard. It's a pluggable StateStore driver, independent of the transport, so you can mix any store with any transport.
Stores are config-driven drivers: list the ones you use under stores in config/durable.ts, built with the stores factory, and pick the active one by name with store. The lucid driver lazily imports @adonisjs/lucid only when selected.
Drivers
| Store | Database | Schema | |
|---|---|---|---|
| in-memory (the default) | — | — | For tests and local dev. Not durable. |
stores.lucid(...) | Postgres · MySQL · SQLite | shipped migration | The production driver, on @adonisjs/lucid. |
The default — when config/durable.ts sets no store — is an in-memory store: zero setup, but state is lost on restart, so it's only for development and tests. For anything durable, select the Lucid store.
import { defineConfig, stores, transports } from '@adonis-agora/durable'
export default defineConfig({
store: 'lucid',
stores: {
lucid: stores.lucid(),
},
transport: 'memory',
transports: { memory: transports.memory() },
})The interface
A StateStore persists runs and checkpoints and exposes the primitives the engine needs for durability:
- Runs & checkpoints —
createRun,updateRun,updateRunIf,getRun,deleteRun,saveCheckpoint,getCheckpoint,listCheckpoints.createRunmust reject a duplicate id (throw — a SQL primary-key violation is exactly right) rather than overwrite: the engine treats the throw as "someone else already started this id", re-reads the existing run and returns its state, which is what makesstart/signalWithStart/ racing scheduler ticks converge on one run instead of forking or clobbering the winner. - Recovery & leasing —
listIncompleteRuns,listPendingRuns(limit)(so workers poll forpendingruns),listDueTimers(nowMs, namespace?, limit?)(thelimitcaps a due-timer batch per poll; a store may ignore it), and the atomictryLockRun/renewRunLock/releaseRunLockthat make one-instance-per-run leasing work.releaseRunLock(runId, owner?)is owner-fenced: whenowneris given, release only if that owner still holds the lease (a conditional write, atomic likerenewRunLock) — so a stale executor whose lease was taken over can't wipe the new owner's lease and open the door to a third concurrent executor. Omittingowneris the operator-style unconditional clear (requeue's stale-lease reset); a store implemented against the older single-argument signature simply ignoresownerand degrades to the previous unfenced behavior. - Signals —
putSignalWaiter/takeSignalWaiter/listSignalWaiters/removeSignalWaiter, plusbufferSignal/takeBufferedSignalfor the signal machinery. - Events —
bufferEvent/listBufferedEvents/removeBufferedEvent, so a published event that matched no live waiter is still consumed by a laterctx.waitForEvent. - Querying —
listRuns(query)for the dashboard and CLI.RunQuerycarries the status/workflow/tag/namespace/attribute filters plus exact-matchoriginand the epoch-ms range pushdownscreatedBefore/createdAfter/updatedBefore/wakeBefore— the last two are what let the retention/stalled sweeps and the blocked-run poll ask the store for candidate rows instead of fetching everything and filtering in process. Paging ridespage(1-based, default1) andsize— the same offset shape@adonis-agora/filtertakes, so every Agora listing is paged alike. Resolve the pair with the exportedrunPageWindow(query)helper rather than by hand: it returns the{ limit, offset }your adapter spends, and it is what the conformance kit asserts against (nosizemeans no bound — a store must not invent a default window).
Eight members are optional, and the engine works without every one of them:
| Optional member | What you lose by omitting it |
|---|---|
ensureSchema() | Boot-time provisioning. A store with nothing to provision (in-memory) simply has no schema to create. |
transaction(work) | ctx.transaction — the exactly-once "business write and checkpoint commit together" primitive — throws on a store without it. |
recordStepHeartbeat(...) | Step heartbeats are discarded, so a long remote step shows no liveness in the dashboard or durable:runs. |
getLatestCheckpointByName(...) | A fast path only. The engine falls back to listCheckpoints plus an in-JS filter. |
listCheckpointsByNamePrefix(...) | Same — a fast path, with the same fallback. |
listOrphanedRuns(nowMs, limit, namespace?) | The bounded orphan query (running + free/expired lease) the periodic recovery pass prefers. Without it the engine falls back to listIncompleteRuns and filters in process — correct, but on a busy fleet every worker fetches (and lock-probes) every healthy running run once per tick. Strongly recommended. |
listSignalWaitersByRunIds(runIds) | The targeted per-page waiter lookup (run_id IN (...), indexed). Without it the dashboard's waiting column and the engine's child resolution fall back to listSignalWaiters('') — a full-table scan per page load. Pair the implementation with an index on run_id. |
runValueFacets(axis, query, opts?) | Native picker enumeration (distinct values + counts per filter axis). Callers fall back to counting a bounded listRuns scan in-process — same shape, bounded approximation on the non-column axes. |
Everything else is required, including updateRunIf.
updateRunIf — the compare-and-set write
updateRunIf(
runId: string,
expectedStatuses: RunStatus[],
patch: Partial<WorkflowRun>,
): Promise<boolean>Apply patch only if the run's currently persisted status is one of expectedStatuses at write time. It returns true when the write landed and false when the predicate did not match. A non-match is a normal outcome, never a throw.
The predicate must be atomic with the write
For a SQL store the predicate belongs in the UPDATE … WHERE clause. For an in-process store it must be a synchronous check with no await between reading the status and mutating the row. A read-then-write in application code is a TOCTOU race, not a substitute: two racing callers can both observe a matching status before either writes.
Why it exists
A workflow turn computes its outcome from a run snapshot loaded when the turn began — and a turn can run for minutes across remote steps. Meanwhile the run's persisted status can change through a completely different path: an operator cancels it, the execution-timeout sweep marks it cancelled, a poison run is dead-lettered, or ctx.all in failFast mode cancels a sibling.
With a plain updateRun, the turn's now-stale terminal write lands last and resurrects a terminal run: a cancelled run silently reports completed, its output is written, its parent is notified, and a run.completed event fires. updateRunIf moves the "is this run still eligible?" test into the same atomic write, so the loser of the race gets false, emits nothing, and reports the run's true current status instead.
The engine leans on it in five places — the execution-timeout sweep, all three terminal/suspend transitions of a settling turn, and parking a run as blocked — and the conformance kit covers it in three cases, including the two-sequential-writes race directly.
Here is the Lucid driver's implementation, as the shape to copy:
const affected = await trx
.from(DURABLE_TABLES.runs)
.where('id', runId)
.whereIn('status', expectedStatuses)
.update(row)
return rowsAffected(affected) === 1Writing a custom store
Implement every required member above, then run the shared conformance contract against it — that is the definition of "correct", not the prose here:
import { runStateStoreContract } from '@adonis-agora/durable/testing/conformance'
import { MyStateStore } from '../src/my_state_store.js'
runStateStoreContract('MyStateStore', async () => {
const store = new MyStateStore(/* connection */)
await store.ensureSchema?.()
return {
store,
cleanup: async () => {
await store.close()
},
}
})The factory runs once per test case, so each case gets a fresh store. Two escape hatches exist for drivers that genuinely cannot satisfy a slice of the contract: supportsAsyncTransaction: false (a synchronous driver) and supportsTagFilter: false. If the backing infrastructure is not available — no Docker for a testcontainers store — throw StateStoreUnavailableError from the factory and the suite skips instead of failing.
The conformance kit lives on the @adonis-agora/durable/testing/conformance subpath rather than @adonis-agora/durable/testing, because it generates the suite with vitest's describe/it. See Testing.
Encrypting payloads at rest
CodecStateStore decorates any store and runs run/step payloads through a PayloadCodec — encoded on write, decoded on read — so inputs and outputs are never stored in the clear. Searchable metadata (id, status, workflow, tags, namespace, timestamps) is left untouched, so the dashboard, queries and recovery keep working.
Know what the default coverage leaves in the clear: step events (everything step code passes to ctx.log), the structured error messages, and heartbeat progress. A deployment described as "encrypted payloads" still stores whatever its steps log as plaintext. Opt into new CodecStateStore(inner, codec, { coverage: 'extended' }) to cover those too — but only on a fresh table, or with a codec whose decode recognizes its own ciphertext and passes other values through, because rows written before the switch hold those fields unencoded. Search attributes always stay clear (they must remain queryable). The wrapper also forwards recordStepHeartbeat now (it used to silently drop persisted step liveness). And remember the console renders everything decoded — pair the codec with the dashboard config's redact hooks to keep PII out of the operator's browser.
import { createCipheriv, createDecipheriv, randomBytes } from 'node:crypto'
import { CodecStateStore, LucidStateStore, defineConfig, type PayloadCodec } from '@adonis-agora/durable'
import db from '@adonisjs/lucid/services/db'
import env from '#start/env'
const key = Buffer.from(env.get('DURABLE_PAYLOAD_KEY'), 'base64') // 32 bytes
const aesCodec: PayloadCodec = {
encode(value) {
const iv = randomBytes(12)
const cipher = createCipheriv('aes-256-gcm', key, iv)
const body = Buffer.concat([cipher.update(JSON.stringify(value), 'utf8'), cipher.final()])
return {
iv: iv.toString('base64'),
tag: cipher.getAuthTag().toString('base64'),
body: body.toString('base64'),
}
},
decode(value) {
const { iv, tag, body } = value as { iv: string; tag: string; body: string }
const decipher = createDecipheriv('aes-256-gcm', key, Buffer.from(iv, 'base64'))
decipher.setAuthTag(Buffer.from(tag, 'base64'))
const plain = Buffer.concat([decipher.update(Buffer.from(body, 'base64')), decipher.final()])
return JSON.parse(plain.toString('utf8'))
},
}
export default defineConfig({
store: 'encrypted',
stores: {
encrypted: async () => new CodecStateStore(new LucidStateStore(db), aesCodec),
},
})encode and decode are synchronous and decode must be the exact inverse of encode — wrap an async KMS behind a local cache yourself. Beyond run and checkpoint payloads the decorator also encodes buffered signal and event payloads.
CodecStateStore does not implement the optional recordStepHeartbeat, so wrapping a store trades persisted step heartbeats for encryption at rest. The live step.started event still fires.
Control plane
The cross-instance broadcast channel for lifecycle events and cancellation — separate from the point-to-point task transport. Omit it and the engine is local-only; pick controlPlanes.redis to fan out across every replica over Redis pub/sub, interoperable with a NestJS fleet.
Lucid
The Lucid StateStore driver — persist runs and checkpoints to Postgres, MySQL, or SQLite through @adonisjs/lucid. The migration ships with @adonis-agora/durable; select the store with stores.lucid().