Agora
Concepts

Tenancy

The two isolation axes — namespace on the engine and partition on the transport — what each one actually partitions, and the boundary that keeps a store-less pod to its own runs.

Tenancy answers one question: whose runs are these, and which pool executes them. It is independent of how many processes you run — Topologies covers that. This page is what the isolation axes mean once you reach for them.

There are exactly two, they take the same kind of value, and they are not interchangeable:

AxisSet onPartitions
namespacethe engine, in config/durable.tsthe store: which runs this engine polls, recovers and resumes. Also prefixes the transport's keys.
partitionthe transport (and a tenant pod's config)the queues: the <name>@<partition> routing token a dispatch targets and a worker serves.

A useful shorthand: namespace is what a control plane polls; partition is what a worker subscribes to.

namespace — the store poll scope

config/durable.ts
export default defineConfig({
  namespace: 'blue',      // this engine's pool
  transport: 'bullmq',
  transports: { bullmq: transports.bullmq({ connection: process.env.REDIS_URL }) },
  store: 'lucid',
  stores: { lucid: stores.lucid({ connection: 'pg' }) },
})

It isolates two things at once:

  • The store. Every run this engine creates is stamped with its namespace, and the poll paths — driving pending runs, recovering incomplete ones, resuming due timers, waking blocked runs — only act on runs carrying it. resume of a run from another namespace throws NamespaceMismatch and releases the lock rather than replaying someone else's history.
  • The transport. A non-default namespace segments the key prefix: durable becomes durable-<namespace> across every queue and key the pool uses.

Both follow one default-is-bare rule: 'default' (the default value) leaves every name byte-identical to a single-pool deployment. Only a real, non-default namespace prefixes anything.

Leaving namespace unset makes the engine an operator. It then scopes nothing: the store's poll filters no-op, so it drives, recovers and resumes runs of every namespace, and its transports stay on the bare prefix. "Seeing everyone" is the absence of a namespace, not a wildcard value — 'default' is an ordinary namespace like any other, and an engine set to it sees only its own runs.

The engine's namespace and a run's namespace are not the same kind of value. The engine's may be undefined (the operator). A stored run's never is: every store normalizes it to 'default' on write, matching the SQL column default. So "an engine with no namespace" and "a run with no namespace" mean different things, and only the first one means all.

partition — the queue suffix

Where namespace decides which runs a control plane owns, partition decides which pool of workers executes them. It is set on the transport, and both sides must agree:

// Dispatch side — this engine's steps route to `<name>@acme`
transports.bullmq({ connection: process.env.REDIS_URL, partition: 'acme' })

// Worker side — a store-less pod declares the tenant it serves
export default defineConfig({
  role: 'tenant',
  transport: 'bullmq',
  transports: { bullmq: transports.bullmq({ connection: process.env.REDIS_URL }) },
  partition: 'acme',
})

Every routing token becomes <name>@<partition>, so two pools serving the same step name never cross. A worker on no partition serves the bare token — which is the same queue every un-partitioned process in the deployment is on.

On the dispatch side the suffix comes from the run's namespace, not from the engine doing the dispatching (a step's own partition still wins when it names one). That is what lets a single operator serve many pools: it drives every namespace's runs, and each one's steps still land on that namespace's queues rather than being pulled onto the operator's bare tokens.

Inside a thin worker's own workflow turn, ctx.step(name, input, { group }) overrides the token for one step; omitted, it inherits the worker's partition, so a workflow and the steps it dispatches stay in the same pool by default. The same suffix is what a Python worker matches with run_redis_worker(worker, partition=...).

The isolation boundary

A store-less pod holds no database credentials, so every read, control action and run-start it performs is a wire request answered by the control plane's RunRequestResponder. That responder — not the client — is the boundary, and it enforces:

  • listRuns is namespace-forced to the tenant it derived. The client's namespace is discarded, never merely validated, so a tenant cannot widen its query into another's.
  • Every runId-bearing verb (getRun, getCheckpoints, getSearchAttributes, signal, cancel, redispatch) loads the run first and rejects with cross-tenant when run.namespace isn't the derived tenant — anti-IDOR, before anything mutates.
  • workerHealth is group-scoped to the tenant's own @<tenant> queues; the operator's bare groups and every other tenant's are dropped.
  • start forces the run's namespace to the derived tenant, so a run a tenant starts is owned by, and later only reachable by, that tenant.
  • Unknown verbs are rejected rather than silently ignored.

The guarantee therefore holds even against a misbehaving client.

It is only as strong as the claim it reads. The tenant on a wire request is asserted by the caller. Pair this with layered tenant authentication: network/prefix segmentation, plus a signed token the control plane derives the tenant from instead of trusting the body.

Which namespace a new run gets

  • A top-level start is stamped with the starting engine's namespace, unless the call passes opts.namespace explicitly.
  • A run started by a store-less pod is stamped with the tenant the responder verified — the pod cannot choose.
  • A retry (retryWithInput) re-stamps the new run with the original run's namespace, so a fix-and-replay of a tenant's run stays that tenant's instead of falling back to the engine's own.
  • A child run inherits the parent run's namespace, not the namespace of whichever engine happens to execute that parent. This matters precisely because an operator legitimately executes every namespace's parents: without the rule, a recovery-resumed parent's child would fall back to the operator's own pool instead of the tenant's.

Sharing one store and one broker

Because the two axes are independent, one Redis/Postgres pair can host several non-interchangeable pools:

  • Per-tenant worker pools, one control plane. Keep every run in the control plane's namespace and route execution with partition. The control plane polls and recovers everything; each tenant's workers only ever see <name>@<their tenant>. This is the shape the cluster docs describe.
  • Fully separate pools. Give each pool its own namespace. Runs, queues and keys are all prefixed, so a dev cluster and a developer's laptop can share infrastructure without either driving the other's runs. Each namespaced engine drives only its own; an operator alongside them drives all of them, which is what lets one deployed control plane orchestrate work that executes on many pools.

The two combine, and they answer different questions. Reach for partition when the runs are the same system's but the execution must be isolated. Reach for namespace when the runs themselves belong to different deployments.

Pitfalls

  • A worker on no partition claims the shared pool's work. An un-partitioned worker is on the bare <name> tokens — the exact queues every un-partitioned process in the deployment is on. A local process left running against a shared broker is not "another consumer"; it competes for, and steals, the deployed fleet's tasks. Give it a real partition.
  • A run stamped into a namespace nothing polls sits pending forever. Passing opts.namespace (or starting from a tenant pod) routes the run to a pool that must actually have someone polling it — either an engine on that namespace, or an operator. Between namespaced engines there is no fallback owner, so a namespace with no operator above it and no engine of its own is a dead end.
  • A stale local worker with an old build can misroute a run even when partitioned. A process left running after a step or workflow rename can pick up a run with code that no longer matches its history — a NonDeterminismError or a hung run. Kill stale processes; a distinct partition is not protection against a stale build.
  • listRuns's namespace is enforced server-side. A tenant cannot widen its query by passing a different namespace — the responder overwrites it unconditionally. Don't build a client-side filter and call it isolation; there isn't one to bypass in the first place.

Next steps

  • Topologies — the deployment shapes these axes ride on.
  • Roles & config — the role-discriminated config, the runGateway surface, and layered tenant authentication.
  • Thin workers — the store-less runtime that serves a partition.

On this page