What are the best AI agent observability platforms in 2026?

Last updated: September 2026

Agent observability is the practice of recording and understanding what AI agents do in production: which model calls they make, which tools they invoke, which systems those tools reach, and under whose authority. The label now covers two different jobs, and most roundups of AI observability platforms only describe one of them.

The first job is engineering telemetry. Platforms like LangSmith, Datadog, Arize, and Fiddler AI capture traces of every model call and tool invocation, score outputs with evaluations, and help engineers debug why an agent failed. Think of it as APM for agents: spans, latency, token counts, and quality metrics.

The second job is audit. When an agent acts on a production system, security and compliance teams need a record of which agent ran, under whose identity, which tool it called, and against which data. That record comes from the layer that brokers the agent's access, such as the gateway logs of an AI Control Plane, because instrumentation the application emits about itself is not an audit trail.

An enterprise that only buys the first kind still cannot answer which agent called which tool as which identity. This roundup compares five platforms enterprises evaluate for agent observability in 2026 (LangSmith, Datadog, Arize, Fiddler AI, and Speakeasy) and is explicit about which job each one does.

Why "agent observability" means two different jobs

The phrase gets used for both jobs because both produce a record of agent behavior. The records answer different questions for different readers.

Job one: model and trace APM

Trace-side platforms instrument the agent application itself. They capture a span for every model call and tool invocation, assemble spans into traces and sessions, and layer evaluations on top: LLM-as-judge scoring, drift detection, hallucination and faithfulness metrics, and experiments that compare prompt or model changes against production data. The reader is an AI engineering team, and the question is "why did the agent do that, and is it getting better or worse?"

This is where LangSmith, Datadog, Arize, and Fiddler AI compete, mostly on top of OpenTelemetry instrumentation.

Job two: tool-call and identity audit

Audit-side observability records what agents were allowed to do and what they actually did, from a position the agent cannot skip. A gateway on the path between agents and the systems they reach logs every Model Context Protocol (MCP) tool call with the identity that made it, resolved against the identity provider rather than self-reported, along with the policy decision and the data scope involved. The reader is a CISO, CIO, or compliance team, and the question is "who did what, through which tool, against which system?"

The two records are not interchangeable:

• Trace telemetry is emitted voluntarily by instrumented code. An agent that is misconfigured, unapproved, or malicious emits nothing.
• Trace metadata carries whatever user ID the application chose to attach. An audit record needs identity the platform verified.
• Traces live in an engineering tool. Audit evidence needs to be immutable, attributable, and exportable to the SIEM the security team already runs.

Willow, an agent identity and access vendor, makes the same point from its own corner of the market: its roundup of agent observability platforms places identity-linked audit beside the tracing tools as a separate, unfilled layer. We reach the same conclusion from the gateway side.

How we evaluated AI agent observability platforms

We compared each platform on the capabilities the two jobs actually require, checked against public product pages and documentation as of September 2026:

• Tracing depth: spans, traces, sessions, and the debugging workflow around them.
• Evaluation: scoring quality in development and production, from LLM-as-judge to drift and hallucination metrics.
• Instrumentation: OpenTelemetry support and framework coverage, so telemetry isn't locked to one orchestration library.
• Identity and audit: whether tool calls are attributed to a verified identity from the IdP, and whether the record stands up as audit evidence.
• Position: whether the platform observes from beside the traffic or sits on the request path where it can also enforce.

How the platforms compare on features

A ✅ means the capability is native and documented, ⚠️ means it is partial or conditional, and ❌ means it is absent. The profiles below explain each row.

1. LangSmith: agent tracing and evals from the LangChain team

LangSmith is a framework-agnostic platform for observing, evaluating, and deploying agents, built by the LangChain team. It traces full conversations and agent runs step by step, tracks cost, latency, and errors through dashboards and alerts, and runs LLM-as-judge, code-based, and multi-turn evaluators on production traces, with annotation queues for human review.

Its most distinctive feature in 2026 is LangSmith Engine, an agent that works from production traces: it clusters recurring failures into prioritized issues, diagnoses the root cause against your traces and code, and proposes prompt or code fixes, opening a GitHub pull request for review when a repository is connected. Instrumentation runs through its own SDKs or OpenTelemetry, and the integration with LangChain and LangGraph is the tightest available.

Where it fits: AI engineering teams debugging and iterating on agents, particularly teams already building on LangChain or LangGraph.

Where it stops: LangSmith observes the application from inside it. User attribution is whatever metadata the application attaches, not identity verified against your IdP, and there is no gateway position from which to record or block a tool call the instrumented code didn't report.

2. Datadog: agent observability inside an existing APM estate

Datadog Agent Observability extends its monitoring platform to agents: end-to-end LLM tracing, datasets and experiments, offline and online evaluations, human review, and production monitoring in one workflow. Its AI Agent Monitoring maps each agent's decision path (inputs, tool invocations, calls to other agents, and outputs) in an interactive graph, and the AI Agents Console, announced in preview in June 2025, extends visibility to third-party agents.

Datadog's structural advantage is correlation. Agent traces land beside the APM, infrastructure, and log telemetry the organization already ships to Datadog, so an engineer can move from a failing LLM span to the service and infrastructure signals underneath it without switching tools. Instrumentation covers Python, Node.js, and Java SDKs plus OpenTelemetry and an HTTP API, with support for OpenAI, Anthropic, Gemini, LangChain, CrewAI, Bedrock, and other frameworks.

Where it fits: engineering organizations already running Datadog that want agent telemetry in the same pane as everything else.

Where it stops: Datadog observes agent behavior; it is not on the path between an agent and the tools it calls, does not broker or log MCP access, and does not bind agent actions to IdP-verified identity.

3. Arize AX and Phoenix: OpenTelemetry-native tracing and evaluation

Arize ships two related products. Phoenix is its open-source tracing and evaluation platform, which runs locally or self-hosted, and Arize AX is the managed platform built on the same foundations, adding agent trajectory visualizations, online evaluations, dashboards, monitors, and human annotation workflows. Both are built on OpenTelemetry plus OpenInference, the AI semantic-convention standard Arize created and maintains, so traces aren't locked to a proprietary format.

Arize has leaned into the standards story hardest of the four: in August 2026 it added native normalization of OpenTelemetry GenAI semantic conventions, so platforms that already emit gen_ai.* spans can send telemetry straight to AX, and it has proposed donating its OpenInference instrumentation libraries to the OpenTelemetry project.

Where it fits: evaluation-heavy teams that want span-, trace-, and session-level evals, and teams that want an open-source, self-hostable starting point in Phoenix.

Where it stops: the same boundary as the other trace platforms. Arize sees what instrumented applications report, so it has no independent record of tool calls, no gateway logs, and no IdP-bound identity on agent actions.

4. Fiddler AI: enterprise agentic observability and model risk

Fiddler AI comes at agent observability from the enterprise-risk side. Its agentic observability traces hierarchically from application to session to agent to trace to span, runs root cause analysis to pinpoint the failing span, and applies 100+ quality metrics through its in-environment Centor Models, which score inputs and outputs without external API calls. It supports OpenTelemetry and frameworks including LangGraph, Amazon Bedrock, AWS Strands Agents, and Google ADK, and its guardrails product scores safety, faithfulness, and PII as a low-latency API.

Fiddler repositioned as the "AI Control Plane for Enterprise Agents" alongside its January 2026 Series C, which invites a comparison we've already written in full: see Speakeasy vs Fiddler AI. The short version is that Fiddler observes and scores from beside the traffic, and its guardrails return scores that the application's own code acts on, while its RBAC governs access to the Fiddler platform rather than which agents can reach which tools.

Where it fits: ML and AI engineering teams debugging multi-agent systems, and model risk management functions in regulated industries that need evaluation evidence.

Where it stops: Fiddler's audit trail is behavior-centric, built around agent and model quality rather than who was allowed to call which tool. It ships no MCP gateway, server registry, or credential management.

What about Langfuse and the other tracing platforms?

The trace-APM side of this market is crowded, and one more name deserves a mention. Langfuse is the most widely adopted open-source option: an MIT-licensed AI engineering platform with hierarchical tracing, evaluations, prompt management, and cost dashboards, OpenTelemetry-native, and free to self-host with all product features included. Teams that want open source with a large community shortlist it beside Phoenix. Evaluation-first platforms like Braintrust also show up in most roundups of the category.

All of them do the first job, and none of them changes the analysis of the second.

5. Speakeasy: the tool-call and identity audit layer

Speakeasy is an AI Control Plane, and its observability is the audit kind. Its MCP gateway sits on the path between agents and the systems they reach, so agent observability comes from the gateway's own logs rather than from instrumentation the application volunteers: every session, tool call, and token is recorded as it happens and attributed to a named person, team, and agent, with identity resolved against your IdP rather than self-reported by the application.

That position produces the record the trace platforms can't:

• Tool-call audit: one searchable trail of who asked what, when, through which MCP tool, against which system, including calls from agents nobody instrumented.
• IdP-bound identity: every action maps to a verified employee or service identity, so attribution survives an audit rather than depending on app-supplied metadata.
• Policy in the same record: blocks, approvals, and budget events sit in the trail beside the calls they governed, because the same gateway enforces them.
• Evidence workflows: the trail exports to the SIEM, BI tools, and warehouses the security and finance teams already run, and the platform holds SOC 2 Type II and ISO 27001, per our security page.

Where it fits: CISO, CIO, and platform teams that need to answer which agent called which tool as which identity, enforce policy on that path, and hand auditors evidence rather than instrumentation.

Where it stops: Speakeasy does not do span-level tracing, evaluations, or drift and hallucination scoring, and it does not sell OpenTelemetry pipelines. Teams that need to debug agent quality should run a trace platform from the first half of this list, and the two records complement each other rather than compete.

Do you need OpenTelemetry for agent observability?

For the tracing job, OpenTelemetry is the default transport, and every trace platform here either accepts or emits it. Its GenAI semantic conventions define the vocabulary for agent spans (operations like invoke_agent and execute_tool), but they remain a moving target: the project moved all GenAI conventions into a dedicated semantic-conventions-genai repository in June 2026, and as of September 2026 the conventions are marked as in development, with no stable release to pin against.

For the audit job, OpenTelemetry is the wrong tool regardless of maturity. It is instrumentation an application emits about itself, which makes it excellent evidence of what instrumented code did and no evidence at all of what uninstrumented, misconfigured, or hostile agents did. Audit records have to come from the layer that brokers access, which is why MCP gateway logs, not spans, are the security team's source of truth.

How to choose an AI agent observability platform

Start from the job, because the platforms cluster cleanly:

• Debugging and improving agents. If the question is why an agent failed and whether the next version is better, choose a trace platform. LangSmith fits LangChain-centric teams, Datadog fits organizations already standardized on its APM, Arize fits evaluation-heavy teams and open-source adopters via Phoenix, and Fiddler fits enterprises that want observability tied to model risk management.
• Auditing and governing agents. If the question is which agent called which tool as which identity, and whether policy was enforced on that call, choose an audit layer on the request path. That is the job Speakeasy is built for.
• Both. Most enterprises running agents in production end up needing both, and the architectures don't conflict: traces flow to the evaluation platform while the gateway writes the audit trail. Neither substitutes for the other.

Summary: when to use each platform

The bottom line

Every platform here is strong at its own job. LangSmith, Datadog, Arize, and Fiddler AI trace and evaluate what agents do, and an engineering team improving agent quality should pick from among them. What none of them records is the audit half of agent observability: tool calls bound to verified identity, captured at the layer the agent cannot route around.

Speakeasy covers that half. The gateway logs every MCP tool call with the person, team, and agent behind it, enforces policy in the same place it records, and exports the evidence to the systems your security team already runs. If the unanswered question in your agent program is who did what, through which tool, against which system, that is the AI Control Plane we've built.

Questions about this roundup, or think we've got something wrong? Talk to our team.