Agora
Concepts

Tenancy

The two isolation axes — namespace on the engine and partition on the transport — what each one actually partitions, and the boundary that keeps a store-less pod to its own runs.

Tenancy answers one question: whose runs are these, and which pool executes them. It is independent of how many processes you run — Topologies covers that. This page is what the isolation axes mean once you reach for them.

There are exactly two, they take the same kind of value, and they are not interchangeable:

AxisSet onPartitions
namespacethe engine, in config/durable.tsthe store: which runs this engine polls, recovers and resumes. Also prefixes the transport's keys.
partitionthe transport (and a tenant pod's config)the queues: the <name>@<partition> routing token a dispatch targets and a worker serves.

A useful shorthand: namespace is what a control plane polls; partition is what a worker subscribes to.

namespace — the store poll scope

config/durable.ts
import { defineConfig, stores, transports } from '@adonis-agora/durable'

export default defineConfig({
  namespace: 'blue',      // this engine's pool
  transport: 'bullmq',
  transports: { bullmq: transports.bullmq({ connection: process.env.REDIS_URL }) },
  store: 'lucid',
  stores: { lucid: stores.lucid({ connection: 'pg' }) },
})

It isolates two things at once:

  • The store. Every run this engine creates is stamped with its namespace, and the poll paths — driving pending runs, recovering incomplete ones, resuming due timers, waking blocked runs — only act on runs carrying it. resume of a run from another namespace throws NamespaceMismatch and releases the lock rather than replaying someone else's history.
  • The transport. A non-default namespace segments the key prefix: durable becomes durable-<namespace> across every queue and key the pool uses.

Both follow one default-is-bare rule: 'default' (the default value) leaves every name byte-identical to a single-pool deployment. Only a real, non-default namespace prefixes anything.

What an unset namespace means depends on the role. On a standalone engine (the default role) it means 'default': the provider gives the engine namespace: 'default', so it drives, recovers and resumes only the default pool's runs. That includes legacy rows written before runs carried a namespace: the Lucid schema heal adds the column as NOT NULL DEFAULT 'default', and every shipped store reads a missing (NULL) value back as 'default'. An empty string is the default pool too, as it is for the transports: an engine configured with namespace: '' stamps and polls 'default', the stores write '' as 'default', and a legacy row stored with '' reads back as 'default' and matches every 'default' filter. On a control-plane it makes the engine the operator: it scopes nothing, so the store's poll filters no-op, it drives runs of every namespace, and its transports stay on the bare prefix. "Seeing everyone" is the absence of a namespace on a control plane, not a wildcard value. There is no standalone operator: a process that must drive every pool is a control-plane.

Behaviour change. A standalone engine without a namespace used to be the operator, driving every pool's runs on a shared store. It now stays in 'default'. If you relied on that, move the process to role: 'control-plane'.

The engine's namespace and a run's namespace are not the same kind of value. A control plane's may be undefined (the operator). A stored run's never is: every store normalizes it to 'default' on write, matching the SQL column default. So "a control plane with no namespace" and "a run with no namespace" mean different things, and only the first one means all.

partition — the queue suffix

Where namespace decides which runs a control plane owns, partition decides which pool of workers executes them. It is set on the transport, and both sides must agree:

// Dispatch side — this engine's steps route to `<name>@acme`
transports.bullmq({ connection: process.env.REDIS_URL, partition: 'acme' })

// Worker side — a store-less pod declares the tenant it serves
export default defineConfig({
  role: 'tenant',
  transport: 'bullmq',
  transports: { bullmq: transports.bullmq({ connection: process.env.REDIS_URL }) },
  partition: 'acme',
})

Every routing token becomes <name>@<partition>, so two pools serving the same step name never cross. A worker on no partition serves the bare token — which is the same queue every un-partitioned process in the deployment is on.

On the dispatch side the suffix comes from the run's namespace, not from the engine doing the dispatching (a step's own partition still wins when it names one). That is what lets a single operator serve many pools: it drives every namespace's runs, and each one's steps still land on that namespace's queues rather than being pulled onto the operator's bare tokens.

Inside a thin worker's own workflow turn, ctx.step(name, input, { group }) overrides the token for one step; omitted, it inherits the worker's partition, so a workflow and the steps it dispatches stay in the same pool by default. The same suffix is what a Python worker matches with run_redis_worker(worker, partition=...).

The wire between control plane and tenant

A store-less pod holds no database credentials. Everything it would otherwise read from or write to the store crosses the transport as plain JSON, in the same shapes the NestJS and Python SDKs use:

  • StartRunMessage on <prefix>-start-run: { tenant, workflow, input, runId?, tags?, searchAttributes? }. The control plane turns it into a durable run.
  • RunRequest / RunReply on <prefix>-run-request and <prefix>-run-reply, correlated by requestId: every read and control verb (getRunDetail, listRuns, cancel, retry, continue, retryWithInput, …) proxied to the control plane and answered on the reply.
  • TenantEvent: lifecycle events re-published per tenant, so a tenant's dashboard can tail its own runs live.

Which verbs each role exposes is covered in Reading and steering runs.

The isolation boundary

Every read, control action and run-start a store-less pod performs is a wire request answered by the control plane's RunRequestResponder. That responder — not the client — is the boundary, and it enforces:

  • listRuns, runFacets and runValueFacets are namespace-forced to the tenant it derived. The client's namespace is discarded, never merely validated, so a tenant cannot list, count or read the tag vocabulary of another's runs. waitingFor answers only for the run ids the tenant owns.
  • Every runId-bearing verb (getRun, getCheckpoints, getRunChildren, getRunDetail, getFanMembers, getSearchAttributes, signal, cancel, redispatch, retry, continue, retryWithInput) loads the run first and rejects with cross-tenant when run.namespace isn't the derived tenant — anti-IDOR, before anything is read or mutated.
  • workerHealth is group-scoped to the tenant's own @<tenant> queues; the operator's bare groups and every other tenant's are dropped.
  • start forces the run's namespace to the derived tenant, so a run a tenant starts is owned by, and later only reachable by, that tenant.
  • Unknown verbs are rejected rather than silently ignored.

The guarantee therefore holds even against a misbehaving client.

It is only as strong as the claim it reads. The tenant on a wire request is asserted by the caller. Pair this with layered tenant authentication: network/prefix segmentation, plus a signed token the control plane derives the tenant from instead of trusting the body.

Read scoping: scopeReads

The responder fences what a store-less pod can reach. A process that does hold the store can be fenced too: with scopeReads: true and a namespace in effect (an explicit namespace, or the implicit 'default' of a standalone engine), the provider hands the engine a view of the store whose run reads (getRun, listRuns, the counts and facets, and the recovery, dispatch and timer scans) only see that namespace's runs. Writes are not fenced. It is off by default, because a control plane's own operator screens should see everything, and it has no effect on a store without scoping support (the Lucid and in-memory stores have it). See Scoped reads.

Which namespace a new run gets

  • A top-level start is stamped with the starting engine's namespace, unless the call passes opts.namespace explicitly.
  • A run started by a store-less pod is stamped with the tenant the responder verified — the pod cannot choose.
  • A retry (retryWithInput) re-stamps the new run with the original run's namespace, so a fix-and-replay of a tenant's run stays that tenant's instead of falling back to the engine's own.
  • A child run inherits the parent run's namespace, not the namespace of whichever engine happens to execute that parent. This matters precisely because an operator legitimately executes every namespace's parents: without the rule, a recovery-resumed parent's child would fall back to the operator's own pool instead of the tenant's.

Sharing one store and one broker

Because the two axes are independent, one Redis/Postgres pair can host several non-interchangeable pools:

  • Per-tenant worker pools, one control plane. Keep every run in the control plane's namespace and route execution with partition. The control plane polls and recovers everything; each tenant's workers only ever see <name>@<their tenant>. This is the shape the cluster docs describe.
  • Fully separate pools. Give each pool its own namespace. Runs, queues and keys are all prefixed, so a dev cluster and a developer's laptop can share infrastructure without either driving the other's runs. Each namespaced engine drives only its own; an operator (a control-plane with no namespace) alongside them drives all of them, which is what lets one deployed control plane orchestrate work that executes on many pools.

The two combine, and they answer different questions. Reach for partition when the runs are the same system's but the execution must be isolated. Reach for namespace when the runs themselves belong to different deployments.

Pitfalls

  • A worker on no partition claims the shared pool's work. An un-partitioned worker is on the bare <name> tokens — the exact queues every un-partitioned process in the deployment is on. A local process left running against a shared broker is not "another consumer"; it competes for, and steals, the deployed fleet's tasks. Give it a real partition.
  • A run stamped into a namespace nothing polls sits pending forever. Passing opts.namespace (or starting from a tenant pod) routes the run to a pool that must actually have someone polling it — either an engine on that namespace, or an operator control plane. Between namespaced engines there is no fallback owner, so a namespace with no operator above it and no engine of its own is a dead end.
  • A stale local worker with an old build can misroute a run even when partitioned. A process left running after a step or workflow rename can pick up a run with code that no longer matches its history — a NonDeterminismError or a hung run. Kill stale processes; a distinct partition is not protection against a stale build.
  • listRuns's namespace is enforced server-side. A tenant cannot widen its query by passing a different namespace — the responder overwrites it unconditionally. Don't build a client-side filter and call it isolation; there isn't one to bypass in the first place.

Next steps

  • Topologies — the deployment shapes these axes ride on.
  • Roles & config — the role-discriminated config, the runGateway surface, and layered tenant authentication.
  • Thin workers — the store-less runtime that serves a partition.

On this page

Partner

Your product as the official integration for Durable, right here.

Become a partner →