Storage

Storage is pluggable to the core. The default store is a reference implementation of a public contract, not a privileged internal — so swapping it for your own database, Redis, or an in-memory store for tests is a one-line config change, and the API, dashboard, and pruner behave identically against any of them.

The StorageProvider SPI

Every store implements the same interface:

interface StorageProvider {
  store(entries: Entry[]): Promise<void>;          // batch write
  update(id: string, patch: Partial<Entry>): Promise<void>; // request entry completes
  find(id: string): Promise<EntryWithBatch | null>;
  get(query: EntryQuery): Promise<Page<Entry>>;    // filter by type/tag/batch/family/time
  batch(batchId: string): Promise<Entry[]>;        // the correlation view
  tags(prefix?: string): Promise<TagCount[]>;
  prune(olderThan: Date, keepLast?: number): Promise<number>;
  pruneScoped?(input: PruneScope): Promise<number>; // optional: per-type retention
  pruneScopedBatch?(input: BoundedPruneScope): Promise<BoundedPruneResult>; // optional: bounded deletes
  tryAcquireLease?(key, owner, ttlMs, nowMs): Promise<boolean>; // optional: cross-process prune lock
  releaseLease?(key, owner): Promise<void>;
  clear(): Promise<void>;
}

Reads are keyset-paginated newest-first (createdAt DESC, id DESC), so paging resumes correctly even after older entries are pruned out from under the cursor.

Want to write your own? The custom storage recipe implements every method end to end.

The zero-config SQLite default

Out of the box, core persists to an embedded SQLite store (better-sqlite3). No connection string, no migration, no table setup — it works on the first boot. It's the right default for local development and single-process apps.

It is per-process, though: each replica keeps its own SQLite file, so in a multi-instance deployment the dashboard only sees entries from the pod that served the request. When you scale out, move to a shared store.

Adapters

Self-healing schema (MikroORM adapter)

The MikroOrmStorageProvider persists Telescope entries through MikroORM into your existing MySQL or SQLite — with no migration and no manual table setup. Two design choices make that safe:

• A dedicated, scoped connection. The provider opens its own MikroORM that knows only TelescopeEntry. This matters for two reasons. First, the schema diff can only ever touch telescope_entries — it is structurally incapable of altering your other tables. Second, it avoids a self-capture loop: if storage writes ran on the host connection (which has the query watcher's loggerFactory wired), every INSERT INTO telescope_entries would be captured as a query, recorded, flushed, and re-captured.
• Additive schema sync at boot. On init() the provider runs ensureDatabase() + schema.update({ safe: true }) — it creates telescope_entries if missing and adds any missing columns, and never drops a table or column. Set ensureSchema: false to opt out (e.g. when you provision the table via your own migration).

Want Telescope on a separate database (recommended for high write volume)? Pass explicit connection options instead of borrowing the host MikroORM. See the storage packages for both wirings.

Retention and pruning

There's no unbounded growth. The core pruner runs on a schedule and calls prune() on whatever store you've configured, driven by the prune config:

TelescopeModule.forRoot({ prune: { after: '24h' } });

The Redis adapter has no per-key TTL — pruning is explicit through this same path, so retention behaves the same regardless of backend.

Per-type retention

Different entry types have different value over time: a request is noise after an hour, but an exception is worth a week. prune.perType overrides the global cutoff for specific types — everything else keeps using after:

TelescopeModule.forRoot({
  prune: {
    after: '5m',            // global: requests, queries, cache ops…
    intervalMs: 60_000,
    perType: { exception: '7d' }, // exceptions live a week
  },
});

Each cycle the pruner runs one bulk delete for every non-overridden type at the global cutoff (type NOT IN (...)), plus one delete per overridden type at its own cutoff (type = ?). Per-type durations are validated at boot exactly like after — a bad value is a startup error, not a silent skip. Omitting perType reproduces the prior single-cutoff behaviour byte-for-byte.

Bounded batched deletes

A retention delete is DELETE FROM telescope_entries WHERE created_at < ? AND type NOT IN (...). On a large table that predicate matches nearly every row, so the planner correctly picks a full scan — and that one statement holds row locks for as long as the scan takes. Measured on a shared MySQL instance: 755k rows / 1.3 GB, 21 concurrent prune deletes, the oldest 63 minutes in. Everything else writing to that database queued behind them, and the pruner still was not meeting its own 24h window.

So the pruner does not issue one unbounded delete. It issues a loop of short, individually-committed deletes, oldest first, until the scope is drained or the cycle budget runs out:

TelescopeModule.forRoot({
  prune: {
    after: '24h',
    intervalMs: 300_000,
    batchSize: 1_000,          // rows per DELETE (default 1000)
    maxBatchesPerCycle: 50,    // hard ceiling per scope per cycle (default 50)
    batchPauseMs: 50,          // pause between batches (default 50)
  },
});

The same rows are deleted; the difference is that locks are released between batches, so a fleet of pruners and every co-tenant of the database get windows even while a badly-behind table drains.

• batchSize is a lock-duration knob, not a throughput knob. Bigger batches amortise round-trips but lengthen each statement's lock — the thing being fixed.
• maxBatchesPerCycle stops a badly-behind table from turning one tick into an hour-long loop, which would be the original failure spelled differently. A backlog drains over several cycles instead of monopolising one; hitting the ceiling logs a warning.
• batchPauseMs is only ever paid between batches, so a store that drains in one batch — the healthy steady state — never waits. It exists so a tight delete loop cannot starve a small instance's IOPS budget.

Deletes are oldest-first, which is what makes a partial cycle still lower the age of the oldest surviving entry — the number retention is actually judged on.

Pruning once per fleet, not once per pod

The pruner refuses to start a cycle while one is already running in that process. It cannot see other replicas — so eight pods sharing one store still run eight cycles, deleting the same rows eight times over.

prune.lock closes that. By default, when the configured provider supports it, Telescope takes a lease row in the database it already writes to:

// nothing to configure — this is the default on every in-repo adapter
TelescopeModule.forRoot({ prune: { after: '24h' } });

One replica wins the lease and prunes; the others skip the tick and try again next time. The lease carries a TTL (prune.lockTtlMs, default max(intervalMs * 3, 60s)) so a pod that is killed mid-prune costs at most one TTL of fleet-wide silence instead of blocking retention forever. Set prune: { lock: false } to opt out.

The lock is advisory. Two concurrent prunes are not wrong — they delete the same rows and one wins; the only cost is waste. So the pruner deliberately fails open: if the lock mechanism itself breaks, it warns once and prunes anyway, because a broken lock must never turn into "retention silently stopped and the table grew without bound".

Supplying your own lock

If you already run a primitive with cross-process exclusion — a job engine's singleton mutex, a Redis lock, a Postgres advisory lock — pass an implementation of TelescopePruneLock instead:

import type {
  TelescopePruneLock,
  TelescopePruneLockRequest,
  TelescopePruneLockResult,
} from '@dudousxd/nestjs-telescope';
import { pruneLockAcquired, pruneLockHeld, pruneLockUnavailable } from '@dudousxd/nestjs-telescope';

class SingletonMutexPruneLock implements TelescopePruneLock {
  constructor(private readonly engine: MyJobEngine) {}

  async acquire({ key, owner, ttlMs }: TelescopePruneLockRequest): Promise<TelescopePruneLockResult> {
    try {
      const started = await this.engine.tryStartSingleton(key, { ttlMs, owner });
      if (!started) return pruneLockHeld('another pod holds the singleton');
      return pruneLockAcquired({
        key,
        owner,
        expiresAtMs: Date.now() + ttlMs,
        release: () => this.engine.finish(key, owner).catch(() => undefined),
      });
    } catch (error) {
      return pruneLockUnavailable(String(error));
    }
  }
}

TelescopeModule.forRootAsync({
  inject: [MyJobEngine],
  useFactory: (engine: MyJobEngine) => ({
    prune: { after: '24h', lock: new SingletonMutexPruneLock(engine) },
  }),
});

The whole contract:

1. acquire must not throw — return { acquired: false, reason: 'unavailable' } instead. (The pruner catches a throw and treats it as unavailable, but do not rely on the backstop.)
2. acquire must be atomic across processes for a given key: at most one caller sees acquired: true while a lease is live.
3. A lease must expire on its own after roughly ttlMs even if release is never called. A SIGKILLed holder must not wedge the fleet.
4. Re-acquiring with the same owner while that owner still holds the lease should succeed (refresh), so a restart with a stable identity is not locked out by its own previous lease.
5. lease.release() must be idempotent, must not throw, and must not release a lease that has since been granted to a different owner.
6. Precision is not required. Clock skew, an expiry that fires while the holder is still working, two simultaneous winners — all acceptable. The lock is advisory; getting it wrong costs a duplicated delete, never a wrong result. Do not build consensus machinery.
7. acquire must return promptly and must not block waiting for the lock. "Held" means "skip this tick", not "queue up" — waiting would rebuild the pile-up this exists to remove.

reason matters: 'held' (somebody else has it) makes the pruner stand down silently, 'unavailable' (the mechanism is broken) makes it prune anyway with a warning. If you cannot tell the two apart, return 'unavailable'.

owner is <instanceId>#<pid>, stable for the life of the process and distinct between replicas. key is telescope:prune — one fleet-wide name, because every replica pruning the same store is exactly the set that should contend.

Archiving before prune

Pruning deletes. When a type is worth keeping but not in the live store, hand its doomed entries to an archive.sink before the pruner deletes them — export to S3, a data lake, cold storage:

TelescopeModule.forRoot({
  prune: { after: '5m', perType: { exception: '7d' } },
  archive: {
    types: ['exception'],
    sink: async (entries) => { /* persist the batch durably */ },
    batchSize: 500,        // entries per sink call (default 500)
  },
});

The contract is archive, then delete, per type, per cycle:

• Only listed types are archived; every other type prunes normally.
• For each archived type the pruner fetches entries older than that type's cutoff and streams them to sink in batchSize chunks. It deletes the type only after the sink resolves.
• If the sink throws, that type is not deleted this cycle — the doomed entries survive to be retried next cycle. The error is logged (once per cycle) and the rest of the prune continues. A failing sink can never crash the host or stall the pruner.
• Work is bounded: at most maxBatchesPerCycle batches per type per cycle (default 10); any backlog is picked up next tick.

See the Archiving exceptions to S3 recipe for a copy-pasteable sink.