AI Governance & Security Vendor Wiki

5 min read

Datadog LLM Observability

Researched 2026-06-28. Primary category: llm-observability.

One-liner — The LLM/agent observability module inside Datadog’s broader monitoring platform — buy it where you already run APM and logs.

What it does — End-to-end tracing of LLM and agent chains (root-cause errors and hallucinations), operational metrics (latency, token usage, cost), and out-of-the-box quality + safety evaluations (topic relevance, toxicity) plus sensitive-data scanning. Adds prompt/response clustering, datasets, experiments, a testing playground, offline + online evals, and human review/annotation — all stitched into Datadog APM and the rest of the Datadog platform. GA since 2024-06-26.

Where it sits in the stack — The llm-observability layer of the model/prompt tier. Mostly dev/ops visibility, but its built-in sensitive-data scanning and safety evals help detect sensitive-data leakage in prompts/outputs. Not an inline firewall — complements ai-runtime-security and ai-gateway.

Deployment & architecture — SaaS only (Datadog’s hosted platform); SDK/auto-instrumentation for OpenAI, Anthropic, Gemini, Vertex AI, Bedrock, LangChain, CrewAI, Pydantic, litellm, Strands Agents, plus an MCP server. Its key differentiator is native correlation with existing Datadog APM traces, infra metrics, logs, and security signals.

Positioning & differentiators — The “single pane of glass” play: if you already run Datadog, LLM Observability is a module rather than a new vendor. Versus AI-native specialists (langfuse, arize-phoenix, braintrust, langsmith) it trades some depth/agility for unified correlation, enterprise procurement, and no extra vendor. SaaS-only (no self-host) is the main constraint for data-sensitive shops. (Datadog is also a strategic investor in arize-phoenix.)

Security operations: Cloud SIEM and Bits AI Security Analyst (added 2026-10-23). Datadog also sells security products on the same platform. These are what survey respondents meant when they put “Datadog” under SIEM/Logging and “Datadog Bits Security Analyst” under AI SOC.

  • Cloud SIEM runs threat detection and incident response on Datadog log management. It ships 800+ out-of-the-box detection rules and uses the same 1,000+ integrations. It is listed in Gartner Peer Insights’ SIEM market. No 2025 SIEM Magic Quadrant placement was found (not ruled out).
  • Bits AI Security Analyst became generally available on 2026-03-23. It is an agent inside Cloud SIEM that triages and investigates alerts end to end (gathers evidence, correlates signals, decides on escalation) and returns explained verdicts. It draws on the infrastructure, identity, EDR and APM telemetry already in Datadog. This puts it in ai-soc-analysts as a platform module: it only investigates Cloud SIEM signals. Standalone AI SOC tools such as dropzone-ai or prophet-security work on top of any SIEM. Datadog’s figures (investigations “as little as 30 seconds”, MTTR cut by more than 90%, “up to 98%” faster) are vendor claims.
  • Who it fits: a fund that already sends most logs to Datadog for ops. Cloud SIEM avoids running a second log pipeline. Anti-fit: a firm whose SIEM is microsoft-sentinel or splunk and staying there, because Bits AI Security Analyst does not run on other SIEMs. Data handling is the same as for Datadog generally: security logs leave your boundary into Datadog’s SaaS.

Ownership, funding & M&A — Public company, NASDAQ: DDOG (IPO 2019); founded 2010, HQ New York. LLM Observability is an organically built module, not an acquisition. Confidence: high.

CTO / hedge-fund lens — Day-1 and often the path of least resistance for a fund that already standardizes on Datadog — no new vendor review, unified ops, billing, and SOC integration. The catch: SaaS-only means prompts/outputs (potential MNPI) leave your boundary into Datadog, which a compliance team must bless; the built-in sensitive-data scanning helps but does not eliminate that. If you are not already a Datadog shop, an AI-native or self-hostable tool may fit better. Not a model-risk/SR 11-7 governance system itself — pair with ai-governance-platform.

Competitors / alternatives — langfuse, langsmith, arize-phoenix, braintrust, helicone, comet.

Open questions / to verify

  • Cloud SIEM pricing model (ingest vs. indexed vs. per-signal) and whether Bits AI Security Analyst is priced separately.
  • Gartner SIEM MQ / Forrester placement for Datadog Cloud SIEM.
  • Any self-hosted/in-region data-residency option for regulated customers (appears SaaS-only).
  • Pricing model for LLM Observability relative to core Datadog ingestion.

Sources

History

  • [2026-06-28] Stub created from seed registry.
  • [2026-06-28] Researched; corrected ownership independent→public (NASDAQ: DDOG), confidence low→high; established LLM Observability as a GA module (2024-06) of the broader platform, SaaS-only, set hedge_fund_fit high (incumbency). No M&A.
  • [2026-10-23] Cross-listed in siem-soc (Cloud SIEM) and ai-soc-analysts (Bits AI Security Analyst, GA 2026-03-23) after Oct-2026 CTO survey write-ins (“Datadog” under SIEM/Logging, being implemented; “Datadog Bits Security Analyst” under AI SOC, 1 in production). Added frontmatter sources: and per-category market_position. Primary category unchanged (llm-observability).

Graph View

Backlinks