Handshake & negotiation

In a split or polyglot fleet, workers come and go and may run different versions of the SDK — an Adonis worker, a Python worker, an older NestJS worker. The handshake layer keeps that fleet honest: every worker advertises what it can do, the control plane negotiates compatibility before dispatching, and a run that no live worker can execute parks as blocked rather than hanging silently.

The worker descriptorCopy Anchor Link

Every worker publishes a WorkerDescriptor — the single source of truth for routing, compatibility, and observability:

interface WorkerDescriptor {
  instanceId: string
  runtime: 'node' | 'python'
  sdk: { name: string; version: string }
  protocol: { version: number; range: [number, number] }  // wire-protocol majors it speaks
  capabilities: string[]   // named features: 'saga', 'signals', 'priority', 'child-workflows', ...
  workflows: string[]      // registered handler names → routing
  steps: string[]
  registrations?: WorkflowRegistration[]  // what this worker can EXECUTE — see below
  partition?: string
  namespace?: string
  startedAt: number
}

It is advertised in two tiers to stay cheap at steady state and rich on change (ETag-style):

• Heartbeat (every ~10s, EX 35): a compact { ts, status, descriptorHash }.
• Full descriptor: published on startup and whenever it changes, to ${P}-worker-descriptor:<token>:<instance>. The control plane re-reads the full descriptor only when the descriptorHash changes.

Negotiation — three outcomesCopy Anchor Link

The control plane builds a descriptor of its own (it keeps it in memory; it does not publish it) and negotiates it against each worker's: the negotiated session is the highest common protocol major plus the capability intersection. The negotiation is pure (negotiate(local, remote, { required }), exported from @adonis-agora/durable), and each worker lands in one of three states:

Only the protocol range can rule a worker out on its own. A worker that speaks protocol 2 while the control plane speaks 1 is never counted as a live target, so if it is the only worker on a queue, the run parks instead of handing it a task.

Capability-aware routingCopy Anchor Link

A workflow or step can declare the capabilities it requires:

// A step that relies on saga compensation and search attributes
export const settlePayment = defineStep(
  'settle-payment',
  async (input: { orderId: number }) => { /* ... */ },
  { requires: ['saga', 'search-attributes'] },
)

A workflow served by another process declares them on its registration — engine.registerRemote(name, version, { group, executor, requires: ['saga'] }) — and a per-call ctx.step(step, input, { requires }) overrides/augments the step's own list.

Before each dispatch the engine reads the live descriptors published on the routing token it is about to dispatch to, through the primary transport's listWorkerDescriptors (bullmq, db, eventEmitter, and queue with a worker-key store implement it). It dispatches when at least one live worker on that token both advertises every required capability and is protocol-compatible. If none is, the run does not hang and does not dead-letter — it parks as blocked:

Two limits follow from the guard working per routing token:

• It engages only when a descriptor exists. A token with no published descriptor (no worker yet, a worker on the NoopWorkerRegistry, or an SDK that predates descriptors) is dispatched to as before, and the task waits in the queue for a consumer. requires is enforced only against workers that describe themselves.
• It decides whether to dispatch, not which worker takes the task. Every worker on a token consumes the same queue, so an incapable worker sharing a token with a capable one can still pick the task up. Keep the workers on one token homogeneous, or put capability-specific work on its own step name or partition.

Announced registrationsCopy Anchor Link

workflows: string[] says which names a worker routes; it does not say which version it runs, which queue it consumes, or which package declared it. The descriptor's optional registrations carries that — what the worker can execute:

interface WorkflowRegistration {
  name: string
  version?: string   // absent = the announcer did not state one
  group?: string     // the routing token (queue) this worker consumes for it
  requires?: string[]
  origin?: string    // the package that declared the workflow
  onError?: { code?: string; workflow: string }[]  // failure routes — see below
}

The rule is announce only what you can run. A worker publishes a registration for a body it holds and a queue it consumes; it never publishes one for a workflow it merely knows how to route to (an engine's registerRemote). So an entry means a live process said "I can run this", not "somebody, somewhere, might". The control plane's own descriptor announces nothing — it is a router, not an executor.

On a store-less Adonis worker, WorkerRuntime.registerWorkflow(name, body, meta) announces a registration for every body it holds, with group set to the routing token it consumes; workflowAnnouncement(MyWorkflow) builds meta (version, origin, onError) from a class's static workflow. A name advertised with registerWorkflowName (routed elsewhere, not run here) gets no registration. A Python worker fills the same axes with @worker.workflow("pipeline", version="2", origin="…").

LivenessCopy Anchor Link

A registration rides the descriptor key, which is written with the worker-heartbeat TTL (EX 35) and refreshed by the same ~10s beat. A worker that dies stops refreshing, the key expires, and its registrations disappear with it — there is no expiry bookkeeping in the engine, and no way for an announcement to outlive its worker beyond one TTL window.

Folding the fleet's claimsCopy Anchor Link

aggregateAnnouncements(descriptors) (exported from @adonis-agora/durable) folds live descriptors into one AnnouncedWorkflow entry per name@version: its groups, origins, requires, onError route sets, runtimes and instances. It is pure — it holds and caches nothing — so you feed it what a transport reads back, e.g. the bullmq transport's listWorkerDescriptors(token):

import { aggregateAnnouncements } from '@adonis-agora/durable'

const entries = aggregateAnnouncements(await transport.listWorkerDescriptors('pipeline'))
// [{ evidence: 'declared', key: 'pipeline@2', name: 'pipeline', version: '2', groups: ['pipeline'],
//    origins: ['flip-python-db'], requires: [['saga']], onError: [], runtimes: ['python'],
//    instances: ['py-1', 'py-2'], disagreements: [] }]

The same instance publishing its descriptor under several tokens folds to one announcer — instances counts workers, not keys.

DisagreementCopy Anchor Link

Two live workers can announce the same name@version with different groups, origins, capability demands or onError routes. The aggregate reports every distinct claim and picks none of them:

{ key: 'pipeline@2', groups: ['pipeline', 'pipeline-v2'],
  disagreements: [{ axis: 'group', values: ['pipeline', 'pipeline-v2'] }] }

Two origins for one name@version is a name collision between packages; two requires sets is two code versions under one version tag; two groups means a caller cannot know which queue to dispatch to. Silence is not a claim — a worker that announced no origin does not disagree with one that did. A capability demand is compared as a set, so ['saga', 'signals'] and ['signals', 'saga'] agree.

The field the engine acts onCopy Anchor Link

onError is the one announced field the engine uses: for a workflow it serves by convention (a live queue of that name, no local registration), the failure routes come from what the live workers announce. If they announce different route sets (a rollout in progress), the failure is not routed and the engine emits run.unrouted. See Routing a run's failure — remote workflows.

Cross-SDKCopy Anchor Link

Every SDK already publishes workflows: string[], and a bare name not covered by a registration is still announced as an unversioned, group-less claim — so a worker that has not adopted registrations is listed rather than invisible, and a worker that describes some workflows richly and others by name alone is reported in full. The aggregate never invents a version, group or origin nobody stated.

What can I call?Copy Anchor Link

engine.workflowDirectory() does both live reads for you, across the whole fleet: every descriptor (what workers declare) and every worker heartbeat (what is demonstrably consuming a queue right now). engine.announcedWorkflows() is its workflows list.

const dir = await engine.workflowDirectory()
// { supported: true,
//   workflows: [
//     { evidence: 'declared', key: 'report@2', name: 'report', version: '2', groups: ['report@acme'], ... },
//     { evidence: 'observed', key: 'legacy@acme', name: 'legacy@acme', groups: ['legacy@acme'], ... },
//   ],
//   otherPartitions: [{ partition: 'globex', groups: ['billing@globex'], instances: ['g-1'] }],
//   detail: 'Every workflow a live worker in partition "acme" can execute, read just now. ...' }

• An observed entry is a live queue that no descriptor explains: a worker on an SDK too old to describe itself. It has no version, origin or runtime, because nobody stated one. A queue a descriptor declares as a step is left out, since a step cannot be called from outside a run.
• supported: false means no transport in this process can read the fleet. The list is empty because nothing was asked, not because nothing runs. Today only the bullmq transport reads the whole fleet (readAllWorkerDescriptors() / readAllWorkerHeartbeats()); queue, db and eventEmitter answer per-token reads (listWorkerDescriptors) for the routing guard, but not the directory.
• otherPartitions names workers that are live on a partition this engine does not route to. A worker on the wrong partition otherwise looks exactly like no worker at all. A control plane with no namespace (the operator) serves every partition and reports none.
• detail is one sentence saying which of these cases you are in, written for a human.

So an empty list has distinct meanings, and the fields tell them apart:

Each call reads the advertisement keyspace once; nothing is added to the poll loop, and no process keeps the fleet's registrations in memory. There is deliberately no equivalent directory for steps: a step is identified by its position in one run's history and only ctx.step inside a replaying body can call it, so a picker offering steps would offer something no entry point can start.

To start a run on the version a worker declares, register it with engine.remote(name, { group, version }) and pin the start with engine.start(name, input, id, { version }). A pinned start throws when that version is not registered, and never falls back to the newest. Convention routing (calling an unregistered name that matches a live queue) cannot honour a pin, because nothing proves the version exists.

Loud, structured failuresCopy Anchor Link

When negotiation rejects a worker or a run parks, durable emits structured diagnostics — protocol.incompatible / capability.unavailable — carrying both descriptors and the precise delta, not a bare boolean. Those events flow to the dashboard health panel and to Telescope, and are alertable.

Backward compatibilityCopy Anchor Link

A worker that advertises no descriptor (an older SDK, or a pre-handshake aviary worker) is treated as legacy v1 with the v1 capability baseline, and assumed compatible. Existing workers keep flowing untouched; rich negotiation lights up as SDKs adopt it.

The control plane's own descriptor speaks the current single major. Widening its band (to talk to workers on an older or newer major) is the engine constructor option controlPlaneProtocol; it is not a config/durable.ts field.

The current wire protocol is v1. The whole point of the handshake is that a future v2 breaking change is detectable — an old worker and a new control plane discover the mismatch and refuse to corrupt each other, instead of failing in confusing ways at runtime.

Cross-SDK contractCopy Anchor Link

The descriptor, heartbeat, and negotiation wire are pinned as golden JSON fixtures that the Adonis, NestJS, and Python SDKs each produce and parse byte-identically in a conformance test. That is what keeps polyglot negotiation from rotting — see Cross-ecosystem interop.