Structured Logging

Pipeline observability for Generative DOM — matcher selection, partial signals, token lifecycle, diff operations, DOM render, pool tracking, and cleanup.

Generative DOM emits a stream of typed, structured log events through every phase of the streaming pipeline. When something renders the wrong way, the log tells you exactly which matcher was tried, which one won, which tokens were created or pruned, and how the renderer mutated the DOM.

Quick start

import {
  GenerativeDom,
  BufferLogger,
  filterByPlugin,
  summarizeEvents,
} from "@generative-dom/core";

const md = new GenerativeDom({
  container: document.getElementById("output")!,
  plugins: [markdownHeading(), markdownBase()],
  log: {
    level: "debug",
    sink: "buffer",
    bufferSize: 1000,
    plugins: ["markdown-heading"], // optional: only log events from this plugin
  },
});

md.push("## Title\n\nBody text.");
md.flush();

// Inspect the captured events post-mortem:
const headingEvents = filterByPlugin(md.getLogs(), "markdown-heading");
console.log(summarizeEvents(headingEvents));

log is optional. When omitted, Generative DOM uses a NoopLogger and incurs zero overhead — each emit helper short-circuits on a single boolean check before allocating the event object.

The four logger implementations

Logger	Use it when…
NoopLogger (default)	You don't need logs. Truly free.
ConsoleLogger	You want colored, human-readable output while developing.
BufferLogger	You want to inspect events programmatically (getLogs()).
CallbackLogger	You want to forward events to your own sink (Sentry, OpenTelemetry, etc.).

createLogger(config?) builds the right one based on config.sink.

Event taxonomy

Every event is a LogEvent — a discriminated union. Each variant carries a phase tag and a typed type discriminant so consumers can route on either.

Phase	Events
buffer	(reserved for future buffer-level events)
match-block	matcher-tried, matcher-matched, matcher-skipped, matcher-partial
match-inline	matcher-tried, matcher-matched, matcher-skipped
parse-inline	matcher-matched (escape/entity), inline-filter-applied
tokenize	token-created, token-updated
diff	diff-ops-built, diff-op-applied
render	render-start, render-token, render-error, render-cleanup
cleanup	render-cleanup
prune	token-pruned
scheduler	(reserved)

Why a specific matcher won

When you see the wrong element render, the first question is "which matcher fired at this position?" Filter for phase === 'match-block' and look at the sequence:

[match-block:markdown-code]      matcher-tried     pos=0
[match-block:markdown-heading]   matcher-tried     pos=0
[match-block:markdown-code]      matcher-skipped   reason=descriptor-mismatch
[match-block:markdown-heading]   matcher-matched   type=markdown-heading consumed=5

You see the priority-ordered attempt list, which matchers were fast-rejected by matchDescriptor, and which one finally produced the token.

Partial vs full matches

A plugin can return { status: 'partial', minCharsNeeded: N } to tell the tokenizer "I might match, but I need more bytes." The stream emits matcher-partial so you can see who held the buffer open and why the tokenizer stopped:

[match-block:markdown-code]  matcher-tried     pos=0
[match-block:markdown-code]  matcher-partial   pos=0

When more data arrives and the matcher finally succeeds, you'll see the matcher-matched event in the next render cycle.

Token lifecycle

For every token produced, you get token-created with the plugin name, type, and (optionally) the raw markdown. The last token of a non-final tokenize pass is marked pending and emits token-updated so you can watch tokens transition from pending → complete as streaming progresses.

When maxLiveTokens is exceeded, the pruner emits token-pruned with the batch size.

Diff operations

diff-ops-built summarizes the full batch (insert/update/remove count) for a render cycle. diff-op-applied is emitted per-operation as the renderer walks the diff, with the new and previous token attached.

Render, pool, and cleanup

• render-token — plugin render() returned a DOM node. Carries the tag name and whether the element was pooled.
• render-error — a plugin render() threw. The pipeline surfaces a structured <span class="generative-dom-error"> element; the log records the underlying error and offending token.
• pool-acquire / pool-release — element pool accounting. reused: true on acquire means the element came from the pool rather than being freshly created.
• render-cleanup — a plugin's cleanup() ran on an element being replaced or removed.

Configuration reference

interface LogConfig {
  /** Minimum severity. Events below this are dropped. Default: 'warn'. */
  level: "trace" | "debug" | "info" | "warn" | "error";

  /** Optional allow-list of phases. */
  phases?: PipelinePhase[];

  /** Optional allow-list of plugins (matches `event.plugin`). */
  plugins?: string[];

  /** When true, attaches `raw` markdown / inner `content` to events. Off by default. */
  includeTokens?: boolean;

  /** When true, attaches DOM element info to render events. Off by default. */
  includeDOM?: boolean;

  /** Output destination. Default: 'console'. */
  sink?: "console" | "callback" | "buffer";

  /** For sink='callback': receive each event. */
  onLog?: (event: LogEvent) => void;

  /** For sink='callback': 'event' (object) | 'json' (string). Default: 'event'. */
  format?: "event" | "json";

  /** For sink='buffer': max events retained (circular). Default: 1000. */
  bufferSize?: number;
}

The cost of the isEnabled(level) check is a single LEVEL_RANK lookup — a few CPU cycles, no allocations.

Buffer helpers

When sink: 'buffer', retrieve events with md.getLogs() and use the helpers in @generative-dom/core to slice them:

Helper	Purpose
filterByPhase(events, phase)	Only match-block events, etc.
filterByPlugin(events, name)	Only events from markdown-heading.
filterByType(events, type)	Only matcher-matched events, etc.
filterByLevel(events, level)	Only error events, etc.
groupByPhase(events)	Map<PipelinePhase, LogEvent[]>.
groupByPlugin(events)	Map<string, LogEvent[]>.
summarizeEvents(events)	One-line stats string.

Formatters

formatEvent(event) and formatEventPlain(event) produce single-line, human-readable output. formatEventVerbose(event) is multi-line with all fields. formatEventJson(event) produces a single-line JSON string with stable field ordering. formatEventsNdjson(events) emits NDJSON for log aggregation pipelines.

Timestamps in formatter output are UTC (HH:MM:SS.mmm in HH:MM:SS.mmm of the day) so log output is consistent across timezones.

Public API on GenerativeDom

const md = new GenerativeDom({ container, plugins, log });

// Get the underlying logger to bind a child context.
const childLogger = md.getLogger().child({ requestId: "abc" });

// Retrieve buffered events (BufferLogger only; returns [] otherwise).
const events = md.getLogs();
const headingEvents = filterByPlugin(events, "markdown-heading");
console.log(summarizeEvents(headingEvents));

// Clear the buffer.
md.clearLogs();

Performance characteristics

Scenario	Overhead
No logger configured	Zero — no allocation, no function call.
level: 'error', no errors happen	Single LEVEL_RANK comparison per emit site.
level: 'debug', busy render	One LogEvent object allocation per event (~24 fields).

The release build of @generative-dom/core is unchanged in size when logging is unused. The 4 logger classes and the emit helpers tree-shake when no imports reference them.

Recipes

"Why did this heading render as plain text?"

const events = md.getLogs();
const tries = filterByPhase(events, "match-block").filter(
  (e) => e.type === "matcher-tried" || e.type === "matcher-matched",
);
console.log(formatEventsNdjson(tries));

You see every plugin tried at every position and which one finally matched.

"What escaped/escapes did the parser handle?"

const events = md.getLogs();
const specials = events.filter(
  (e) => e.phase === "parse-inline" && e.plugin === "escape",
);

"Which plugin recycled the most elements?"

const events = md.getLogs();
const grouped = groupByPlugin(events.filter((e) => e.type === "pool-release"));
for (const [plugin, evs] of grouped) {
  console.log(`${plugin}: ${evs.length} releases`);
}

Forwarding to Sentry / OpenTelemetry

const md = new GenerativeDom({
  container,
  plugins,
  log: {
    level: "warn",
    sink: "callback",
    format: "json",
    onLog: (jsonOrEvent) => {
      const event =
        typeof jsonOrEvent === "string" ? JSON.parse(jsonOrEvent) : jsonOrEvent;
      if (event.level === "error") {
        Sentry.captureMessage(`[generative-dom] ${event.type}`, {
          extra: event,
        });
      }
    },
  },
});

See also

• Generative DOM Class API — getLogger, getLogs, clearLogs methods
• Types — LogEvent, LogConfig, Logger type definitions
• Streamability Contract — using logging to diagnose streaming bugs
• Architecture — the pipeline diagram with logging hooks