Run retention & pruning

Every run — completed, failed, cancelled, or dead — stays in the store forever by default. That's the right default for a system whose whole point is auditable history: you can always trace a run back through its checkpoints. But left alone indefinitely, durable_workflow_runs (and its child tables) only grows, and the timer poller's per-tick status scans get linearly slower as it does. The retention module option hard-prunes old terminal history on an interval so the tables stay bounded, while you keep control over exactly what "old" means, per status.

Declaring retention: the module's retention option

DurableModule.forRoot({
  store,
  transport,
  retention: {
    sweepInterval: '1m',
    batchSize: 1_000,
    policies: [
      { statuses: ['completed', 'cancelled'], maxAge: '14d', maxCount: 200 },
      { statuses: ['failed'], maxAge: '90d' }, // keep failures longer for debugging
    ],
  },
});

DurableRetentionOptions has:

• policies — the retention rules, one per (disjoint) status group. Required; omit retention entirely to keep everything forever (the default).
• sweepInterval — how often the prune sweep runs. A number is ms; a string is an ms-style duration ('1m', '5m' — note 'm' is minutes). 0 runs the sweep once, on boot only. Defaults to 60000 (1 minute).
• batchSize — max runs hard-deleted per batch, per policy, looped until a batch comes back short (so a big backlog drains over several batches instead of one giant delete). Defaults to 1000.

Retention only runs on a driving operator instance — a dashboard/ dispatch-only instance (drive: false) or a thin worker (no store) never prunes, so you don't need to worry about two instances racing each other's sweeps. It also requires a store adapter that implements pruneTerminalRuns — the MikroORM adapter does; an adapter that omits it logs a warning and disables retention rather than failing boot.

RetentionPolicy: what gets pruned vs. kept

Within a policy's statuses, a run is kept only if it satisfies every bound you set on that policy, and pruned the moment it violates any:

• maxAge — prune runs whose updatedAt (≈ when they reached their terminal status) is older than now - maxAge. A number is ms; a string is a duration ('7d', '2w', '90m' — again, 'm' is minutes; use '30d' for roughly a month).
• maxCount — keep only the maxCount most-recent (by updatedAt) runs in the status set; prune everything past that.

Set one or both — with both, the most restrictive wins: a run is pruned once it's either past maxCount or older than maxAge, whichever bites first. Boot-time validation rejects a misconfigured policy before it can silently do the wrong thing:

• every policy must set at least one of maxAge/maxCount;
• statuses may only name terminal statuses — completed, failed, cancelled, dead (TERMINAL_RUN_STATUSES); pruning a live status would race the engine and delete work in progress;
• a status may appear in only one policy — the sets across all policies must be disjoint, so "most-recent N" is unambiguous per status;
• a malformed duration string ('7days', '1 month') throws at boot instead of quietly never pruning.

Statuses you don't name in any policy are never pruned — e.g. the example above prunes completed/cancelled at 14 days (capped at 200 rows) and failed at 90 days, but never touches dead runs at all, so a poison-pill run stays inspectable indefinitely unless you add a policy for it.

What pruning removes

A prune sweep hard-deletes — not archives, not soft-deletes — the run row plus its checkpoints, signal waiters, and normalized search-attribute rows, in one transaction per batch, mirroring what deleteRun does for a single run. Once pruned, a run is gone from getRun/listRuns/the dashboard — there's no undo. Pruning is evaluated per run, independent of parent/child relationships: unlike engine.deleteRun (which recursively cascades to a run's children), pruneTerminalRuns only ever deletes runs matched by a policy's own statuses/maxAge/maxCount — a pruned parent's still-young child run (or vice versa) is left alone on its own terminal-status clock.

Dashboard implications

A pruned run's history simply isn't there anymore — you can't drill into a completed run from three weeks ago if your policy caps completed at 14 days. This is deliberate: retention trades long-tail auditability for a bounded table (and a bounded poller scan), so size the policy around how long you actually need to debug or audit a given outcome, not around "just in case."

Operational guidance

Because maxAge/maxCount are set per status set, you don't have to apply the same retention to every outcome. A reasonable starting point:

• completed/cancelled — the "nothing to see here" outcomes. A couple of weeks ('14d') plus a row cap (maxCount) is usually enough to answer "did this run, and when" without the table growing forever.
• failed/dead — the outcomes you actually investigate. Keep these longer ('90d' or more, or a much higher maxCount) since a failure or dead-lettered run is exactly the history you'll want when debugging weeks later, and pruning it early loses the evidence.

Start conservative (longer maxAge, higher maxCount) and tighten once you've confirmed nothing depends on the older history — pruning is irreversible, and there's no cost to leaving retention off entirely until table growth actually becomes a problem.