Telescope

Already running @dudousxd/nestjs-telescope? @dudousxd/nestjs-durable-telescope adds a watcher that records workflow runs and steps as Telescope entries, so durable flows appear next to your requests, queries and jobs.

pnpm add @dudousxd/nestjs-durable-telescope

import { TelescopeModule } from '@dudousxd/nestjs-telescope';
import { DurableTelescopeWatcher } from '@dudousxd/nestjs-durable-telescope';

TelescopeModule.forRoot({
  watchers: [new DurableTelescopeWatcher()],
});

The watcher resolves the engine from the (global) durable providers and records one entry per lifecycle event (run.started, step.completed, run.completed, run.failed, …), tagged workflow:<name>, run:<id>, kind:<step-kind>, and failed on failures — so you can filter and group durable activity in the Telescope UI.

It also trace-groups a run: every entry for a run carries the same traceId, derived from the run id, so a whole workflow reads as a single trace on Telescope's Traces page. Because the id is a pure function of the run rather than a span held in memory, this survives everything a durable workflow does — a run that suspends for a week and resumes on another pod is still one trace, and a worker in a different process derives the same id from the same run id.

Correlating what a workflow did

Recording lifecycle events tells you a step failed. It does not tell you what the step was doing when it failed — the queries it issued, the outbound calls it made, the exception it threw all get recorded by Telescope's other watchers, and without a batch open around the execution they land outside every trace.

So the watcher also opens a Telescope batch around the execution itself — one for each workflow turn, one for each step handler body — through the durable engine's useDurableExecution seam. The scope is open while the body runs, which is the only arrangement that catches an exception thrown from inside a step.

Two consequences worth knowing:

• Batch granularity follows the execution shape. With an in-process transport (event emitter, in-memory) a step handler runs inside the turn that dispatched it and its result resumes the next turn on that same async path, so the whole run is one batch — it is genuinely one causal chain. With a queue-backed transport (BullMQ, SQS, a thin worker) each turn and each step is a separate entry point in a separate process, so each gets its own batch. The traceId is the invariant that holds either way.
• Batches are recorded with origin: 'queue'. Telescope's BatchOrigin union has no durable/workflow member; 'queue' is the closest, and accurate — durable work reaches an executor by being dispatched over a transport.

If your app runs no OpenTelemetry SDK — most don't — pass durableTraceContext() so entries recorded by the other watchers during a turn or step pick up the run's trace id too. @opentelemetry/api on its own propagates nothing, so without it you get correlated batches and null trace ids:

import { TelescopeModule } from '@dudousxd/nestjs-telescope';
import { durableTelescopeExtension, durableTraceContext } from '@dudousxd/nestjs-durable-telescope';

TelescopeModule.forRoot({
  extensions: [durableTelescopeExtension()],
  traceContext: durableTraceContext(),
});

An app that does run a full OTel SDK should keep passing OtelTraceContextProvider (optionally as durableTraceContext(new OtelTraceContextProvider()), which consults it first): the scope's spans already hang off the run's trace, so both agree.

Workflows dashboard

The watcher gives you per-event entries; the extension adds a dedicated health view on top of them. durableTelescopeExtension() registers a durable entry type — labeled Workflows in the nav — and a durable.workflows dashboard, so durable flows get their own at-a-glance page inside Telescope instead of only living in the entry stream. Register it under extensions (this requires a @dudousxd/nestjs-telescope version that supports the extensions option):

import { TelescopeModule } from '@dudousxd/nestjs-telescope';
import { durableTelescopeExtension } from '@dudousxd/nestjs-durable-telescope';

TelescopeModule.forRoot({
  extensions: [
    durableTelescopeExtension({
      runHref: '/durable/runs/{runId}',
      recentFailuresWindowMs: 24 * 60 * 60 * 1000, // default: only show failures from the last 24h
    }),
  ],
});

The extension bundles the DurableTelescopeWatcher, so you still get the per-event entries described above — don't also pass the watcher under watchers when you use the extension. It adds the durable entry type, the durable.workflows dashboard, and ten data providers.

Both options are optional:

• runHref — a URL template (default '/durable/runs/{runId}') used to deep-link a failed run out to the durable dashboard; {runId} is substituted per row.
• recentFailuresWindowMs — how far back the "Stuck runs" table looks (default 24h); a healthy system then shows an empty table instead of surfacing days-old failures as a live incident. Pass 0 to disable the window and show all failed/dead runs.

The dashboard is laid out in four sections:

• Health — Success rate (gauge), Duration p95, Backlog (pending-run count), and Throughput (completed runs/hour, with a trend spark).
• Needs attention — Top failing workflows (bar of the workflows producing the most failures), Stuck runs (recent failed/dead runs as a table, deep-linked via runHref, bounded by recentFailuresWindowMs), and Worker health (every worker group with its queue depth and live-worker count, STARVED groups — backlog with zero live workers — sorted first).
• Workers — one row per live worker, flattened from every group's heartbeats: concurrency mode (fixed/adaptive) and limit, in-flight saturation, RAM %/CPU %, throughput, p95 latency, and the adaptive controller's last limit change (grow/shrink/ram_ceiling/…). A worker from an older SDK with no reported status still lists, with — in the measured columns.
• Trends — Runs over time (stacked done/failed), Duration distribution (a histogram with p50/p95/p99 markers), and Runs by state (a donut of running/pending/cancelling/completed/ failed/dead counts).

Data providers

Ten providers back the panels above, reading from three different sources:

• Telescope's own captured entries (recent-history, bounded by Telescope's prune window): durable.timeseries (success rate / failed count / top failures), durable.duration (p50/p95/p99
  • histogram), durable.runsOverTime, durable.successRate, durable.throughput.
• The live durable store (listRuns, exact and unbounded by the prune window): durable.state (a single status count, e.g. Backlog), durable.recentFailures (the Stuck runs table), durable.stateBreakdown (the Runs-by-state donut).
• The live engine/worker layer (WorkflowEngine.workerHealth() — empty when the transport can't introspect it; only the BullMQ transport currently implements groupHealth): durable.workerHealth (per-group backlog + live-worker rows; query it with { metric: 'starvedCount' } for a single stat — the count of groups with backlog and zero live workers) and durable.workerStatus (per-worker WorkerStatus — the same live adaptive-concurrency snapshot the dashboard surfaces).

Telescope is the health and visibility view, and deep-links out: per-run actions (retry, cancel) and the workflow graph stay in the durable dashboard. For the extensions mechanism itself, see Telescope's concepts/extensions page in the @dudousxd/nestjs-telescope docs.