Skip to main content

Telemetry reference

exd telemetry is the structured artifact stream produced by every flag evaluation, the contract between SDKs and downstream consumers, and the agent- and human-facing analysis surface that turns those records into answers.

This tree is the normative telemetry reference — source-code doc comments and the testplan cross-reference its specific anchors. The test plan at testplan/telemetry/ inverts it case-for-case.

The telemetry surface is designed for two consumer classes simultaneously — humans operating the system and agents reasoning over it. Every machine surface (CLI output, capabilities manifest, diagnostic vocabulary, provenance) is structured for direct agent consumption without a natural-language adapter.


Surface

Wire contracts (what SDKs and sinks exchange)

  • evaluation-record — the EvaluationRecord schema. The single normative artifact every evaluation produces.
  • sinks — the sink contract between SDK and consumer. The reference sinks shipped with exd-client.
  • provenance — the mandatory JSON envelope every --format json CLI output wraps a result in.
  • warehouse-contract — the flag_evaluations table contract for Rung 4+ deployments (Snowflake / BigQuery / Databricks / Redshift / DuckDB).

Analysis surface (what exd telemetry exposes)

  • query-catalog — the built-in telemetry.* query catalog: every analytical query, its parameters, its result shape.
  • diagnostics — every T-code the analysis surface can emit, with conditions and remedies.
  • capabilities — the agent-discoverable capabilities manifest emitted by exd telemetry capabilities.
  • thresholds — configurable detection thresholds (SRM significance level, dead-flag retention, …).

Operator surface (what humans configure)

  • agent-policies (Phase 2; deferred-but-planned) — declarative policies governing agent-driven manifest writes.

Vocabulary

  • glossary — defined terms used throughout this reference.

The user-facing CLI surface (exd telemetry summary / srm / …) is documented in reference/cli/exd/telemetry/.


Status

This reference corresponds to telemetry schema version 1. The schema version is independent of the manifest schema version; bumping one does not require bumping the other. Compatibility rules for the telemetry schema are described in evaluation-record § Stability and evolution.

Implementation status

  • Phase 1 (shipped): SDK record-emission with reference sinks, built-in telemetry.* queries, the canned exd telemetry subcommands wrapping them, the diagnostics + capabilities + provenance + thresholds surfaces.
  • Phase 2 (deferred-but-planned): the agent-policy surface — agent-policies/ directory in the manifest, kind = agent token, E040 lint code, push-time policy enforcement. Spec text is preserved as written; the corresponding manifest / server changes have not been made.
  • Deferred indefinitely (no current plan to land): the user-defined query catalog at queries/<name>.toml (parts of query-catalog and capabilities); threshold configuration at queries/thresholds.toml in the manifest tree (most of thresholds, the queries/thresholds.toml@<sha> form in provenance); the Q001Q010 and Q050Q054 lint codes. Built-in queries cover the analysis use case; CLI --thresholds <path> plus built-in defaults cover threshold tuning. Affected sections carry deferred-indefinitely banners.

Adoption ladder

The telemetry surface is designed to be useful at every operational sophistication level. The rungs run from "no telemetry" (Rung 0) through local file output (Rung 1), an existing observability stack (Rung 2), object-store archive (Rung 3), warehouse-native analysis (Rung 4), and streaming fan-out with auto-rollback (Rung 5). This reference defines the contracts that make every rung reachable without changing application call sites.

RungDestinationReference sink
0(no telemetry)
1Local fileFileSink
2OTel collectorOtlpSink
3Object store (S3 / GCS / Azure Blob), ParquetObjectStoreSink
4Warehouse-native tablewarehouse contract
5Streaming bus + auto-rollback(composed from above + your platform)