Updated: 2026-01-07 By: Ari Heljakka
Short answer
The observability your agent needs is a function of how agentic it is. A single-turn classifier with no tools needs very little; a multi-turn planner with a dozen tools, memory, and a retrieval layer needs a lot. Most platform comparisons skip this step and produce a winners list, which is why teams over-buy or under-buy. A better procedure is to enumerate the complexity dimensions that drive observability needs, map your agent to a point in that space, and then match a platform archetype to it. This post walks the five dimensions, the three archetypes, and the placements that hold up in production.
Key facts
- Definition: Agentic complexity is the set of properties (tool count, multi-turn state, planner-executor split, retrieval depth, autonomy level) that determine how much observability infrastructure an agent requires to be operable.
- When to use: Before selecting an observability or evaluation platform, map your agent to the complexity dimensions; the map decides the archetype, the archetype narrows the platform shortlist.
- Limitations: Complexity changes as the agent evolves; a stack that fits today may stop fitting in two quarters. Re-map quarterly.
- Example: A single-turn extraction pipeline needs request-level traces and a deterministic scorer; a multi-turn planner with eight tools needs session traces, tool-result instrumentation, semantic clustering, and a managed evaluator layer.
Key takeaways
- Observability requirements scale with complexity dimensions, not with traffic volume alone.
- Five dimensions matter most: tool count, multi-turn state depth, planner-executor split, retrieval depth, and autonomy level.
- Three platform archetypes cover most of the space: trace-only, eval-first, and unified evaluation-and-observability. Each maps to a complexity range.
- Picking by archetype prevents both over-buying (a unified stack for a single-turn classifier) and under-buying (a trace viewer for an autonomous multi-tool planner).
- The map is a living artifact. Re-run it as the agent grows new tools, memory, or autonomy.
Definition
Agentic complexity is the structural property of an agent that determines how hard it is to understand its behavior. The more steps an agent takes, the more tools it touches, the more state it carries, the harder it is to diagnose a single failure without specialized instrumentation.
Observability requirements scale with this complexity in a predictable way. A function-call wrapper around a model needs little more than request logs. A planner that decomposes goals into tool calls, persists state across turns, and re-plans on failure needs full session traces, tool-result instrumentation, semantic clustering, and scored signals on multiple orthogonal dimensions. The middle of the spectrum is where platform selection becomes decisive.
When this matters
The complexity map matters most before any platform decision. Pick from a vendor shortlist without mapping complexity first and you will either over-buy (paying for capabilities you do not yet use) or under-buy (paying for a viewer when you need a gate). Re-do the map every time the agent gains a new structural property: a memory layer, a new tool, a planner step, an autonomy increase.
It also matters during incident reviews. A failure that the current stack cannot diagnose is almost always a sign that the agent's complexity has outgrown the stack's archetype. The fix is rarely a deeper trace search; it is a stack that matches the new complexity profile.
How it works: the five complexity dimensions
The dimensions below are orthogonal. A given agent has a position on each, and the union of the positions decides the archetype.
Dimension 1, tool count
Zero tools: a model with prompt-only output. Few tools: a function-calling agent with one or two integrations. Many tools: an agent with a tool catalog of ten or more, where the choice of which tool to call is itself a non-trivial decision.
More tools mean more failure modes (selection errors, argument errors, response-handling errors) and more spans per trace. Tool-result instrumentation becomes critical past a handful of tools.
Dimension 2, multi-turn state
None: each request is independent. Short: a few turns within a session. Long: persistent memory across sessions, with retrievable history.
Multi-turn state makes failures non-local. The bug at turn 7 is usually rooted at turn 2. Session-level trace reconstruction is mandatory the moment the agent carries state between calls.
Dimension 3, planner-executor split
Unified: one model call handles both planning and execution. Split: a planner emits a structured plan that an executor (often a different prompt or model) carries out, with re-plans on failure.
A planner-executor split multiplies the spans per request and changes the meaning of "trace." The trace is now a tree, not a list, and the relevant unit for evaluation is the plan, not the final response.
Dimension 4, retrieval depth
None: no retrieval. Shallow: one retrieval call per request. Deep: multi-hop retrieval with reranking, query rewriting, and conditional fan-out.
Deep retrieval introduces a class of silent failures (empty results that look successful, off-topic chunks that contaminate the answer) and demands instrumentation that captures retrieval as a first-class event, not as an opaque tool call.
Dimension 5, autonomy level
Low: every action is gated by a user turn. Medium: the agent acts for several steps before checking in. High: the agent runs unattended for many steps, possibly across hours, with delegated authority.
Higher autonomy compresses the human's window to catch errors. The observability stack must therefore catch them: continuous evaluators on the running session, alerts on score drift, kill switches on per-dimension floors.
Platform archetypes
Three archetypes cover most of the spectrum. Each is shaped for a complexity range; stretching one outside its range works for a while and stops scaling.
Archetype A, trace-only
Capture full traces, search and slice them, attach annotations. Scoring is not native; the team brings its own evaluator or scripts. This archetype is well-suited to agents at the low end of every dimension: few tools, short state, unified planner, no or shallow retrieval, low autonomy.
When it stops scaling: as tool count, state depth, or autonomy grow, the lack of scored signals forces the team to build a parallel evaluation system. The traces are good; the operational signal is missing.
Archetype B, evaluation-first
Versioned objectives, calibration datasets, managed judges, per-dimension scoring normalized to 0 to 1, gating in CI/CD. Trace capture is present but typically lighter than archetype A. This archetype fits agents in the middle and upper-middle of the complexity space: medium tool count, multi-turn state, often a planner-executor split, shallow-to-deep retrieval.
When it stops scaling: at very high trace volume with deep retrieval and many tools, the trace fidelity needs to grow, and a dedicated tracing layer often joins the stack.
Archetype C, unified evaluation-and-observability
Combines high-fidelity tracing with versioned evaluators, semantic clustering, and a gate-and-monitor loop. This archetype fits agents at the high end: many tools, deep state, planner-executor split, deep retrieval, medium-to-high autonomy.
When it stops scaling: rarely, in raw capability terms; more often, in cost or operational fit, where parts of the stack are unbundled (open-source tracing layer plus a managed evaluator) to control spend.
Mapping complexity to archetype
| Complexity profile | Fitting archetype |
|---|---|
| Low across all five dimensions | Trace-only |
| Mid on at least two dimensions | Evaluation-first |
| High on tools, state, or retrieval; medium autonomy | Evaluation-first plus tracing layer |
| High on autonomy or planner-executor split | Unified evaluation-and-observability |
The map is approximate. The point is to pick the archetype, then narrow the platform shortlist to that archetype, then evaluate platforms within it on the selection criteria that matter operationally (cost, sampling, evaluator depth, on-call ergonomics).
Example: two agents, two archetypes
Agent X: a single-turn extraction pipeline. It takes a document, runs one model call, returns a structured object. No tools, no state, no planner split, no retrieval, no autonomy.
- Complexity: low across every dimension.
- Archetype: trace-only.
- Stack: an open-source tracing layer with a deterministic schema validator as the score. A handful of golden examples in CI/CD; alerts on schema-failure rate.
Agent Y: a multi-turn support agent with eight tools, persistent memory, a planner that emits a structured plan executed by a separate prompt, deep retrieval with reranking, and medium autonomy (it executes up to five tool calls between user turns).
- Complexity: high on tools, state, planner split, and retrieval; medium on autonomy.
- Archetype: evaluation-first plus tracing layer (or unified).
- Stack: high-fidelity tracing with tool-result instrumentation; versioned evaluators on faithfulness, tool-call quality, and policy adherence; managed judge calibrated against a labeled support-trace dataset; sampling biased toward novel queries; CI/CD gate on per-dimension floors; alerts on score drift in production.
Agent X with a unified stack would be over-bought; Agent Y with a trace-only stack would be under-bought and would land in a quarter-long firefighting cycle.
Limitations
- Complexity drifts. A new tool, a memory layer, or an autonomy increase moves the agent on the map. Re-run the mapping when structural properties change.
- The five dimensions are not exhaustive. Some teams need to add a sixth (regulatory exposure, real-time constraints, multi-agent coordination). Adapt the map; do not skip it.
- Archetypes blur. Several products span two archetypes; the boundary is set by the data model, not by the marketing.
- Volume and cost are independent dimensions. A low-complexity agent with very high volume can still demand archetype B for cost reasons (sampling discipline). Do not confuse complexity with traffic.
- The map is not the platform decision. It narrows the shortlist. The final decision still depends on operational criteria (cost-per-trace, sampling, evaluator depth, on-call ergonomics).
Evidence and sources
- OpenTelemetry GenAI semantic conventions, https://opentelemetry.io/docs/specs/semconv/gen-ai/, for the trace-and-span model that anchors every archetype.
- Anthropic, Building agents with Claude, https://docs.anthropic.com/en/docs/agents-and-tools, for the planner-executor and tool-use patterns that drive complexity dimensions.
- OpenAI, Function calling and tool use, https://platform.openai.com/docs/guides/function-calling, for the tool catalog and structured-response shapes referenced in dimension 1.
FAQ
How do I measure complexity quickly? Score each of the five dimensions on a 0 to 2 scale (none, some, a lot). Sum and read the band: 0 to 3 is low, 4 to 6 is medium, 7 to 10 is high. Crude, but enough to pick an archetype.
Should I always pick the most capable archetype? No. Over-buying produces stacks that are heavy to operate and produce signals nobody acts on. Pick to fit, and re-map when the agent grows.
Can I mix archetypes? Often, yes. A common composite is open-source tracing (archetype A's strength) plus a managed evaluator (archetype B's strength) for cost-sensitive but quality-critical workloads.
Where does autonomy fit in the cost model? Higher autonomy increases the cost of being wrong, which raises the bar on evaluator depth and alerting. It does not necessarily increase trace volume; it increases the value of each trace's evaluation.
What is the most common mis-match? Adopting a trace-only stack for an agent with multi-turn state and deep retrieval. Failures live at the session level; a trace search is not enough; the team rebuilds the missing evaluation system in scripts.