Building AI Agents that Don't Break

An interactive field guide to the agents that actually survive contact with production, why they fail, how to build them, and how to prove they work.

Chapter 01 · Why AI Agents?

The Dream Being Sold

AI agents are being pitched as systems that don't just answer questions, but take actions, call tools, update systems, follow workflows, and push work forward. That's the dream.

Teams are betting on agents because they can save a lot of time. Agents can triage a support ticket, investigate an issue, draft a change, write and fix your code, query your database, or trigger the next step in an ops workflow.

But most agent projects still don't survive contact with production. An MIT / Project NANDA report found that around 95% of gen-AI initiatives aren't producing measurable business impact.

Not because the idea is bad, but because reliability is hard. Tool calls fail. Context is messy. Costs and latency spike. Behavior changes across model updates. Debugging "why it did that" is often painful.

95% of gen AI initiatives aren't producing measurable business impact.

Chapter 02 · The Current State

The Current State of Agents

An agent is a software system capable of making decisions, reasoning, and taking actions to perform complex tasks with reduced human intervention. At a very basic level, it looks like this:

In LLM Environment out action feedback

When you're actually building an agent, think in terms of agent design, or flow. Many agent systems are variations of the same idea: the agent receives context, reasons about what to do next, and produces a response or action.

Depending on the design, this may involve keeping track of state or memory, planning over multiple steps, or interacting with external systems via tool calls, but none of these are strictly required. Some agents are purely reactive; others layer on structure as complexity grows.

Practical agents add layers: orchestration, tool reliability, verification, guardrails, observability. Hover any node to see its role.

Router LLM: The decision-maker. Looks at context and routes to the right sub-agent.

Do You Really Need an Agent?

An agent is simply a system that can observe context, decide, and act on its environment. That doesn't automatically make it the right abstraction. The real question is whether adding autonomy actually helps or just adds complexity.

Agent-style autonomy is useful when a task requires ongoing decision-making across multiple steps, involves interaction with external systems, and has clear signals of success or failure.

If a fixed workflow plus one or two model calls is enough, the goal is hard to evaluate, or actions are high-risk without strong guardrails, adding an agent usually makes the system harder, not easier.

When in doubt, start with a tool-using assistant, instrument it, and only graduate to an agent once you consistently see real multi-step decision-making emerge in practice.

Chapter 03 · Real-World Use

How Companies Use Them for Business Value

Across companies, agents create business value in a few recurring categories:

• Investigative & Analytical Workflows: Agents answer complex questions that require multiple steps, querying data, aggregating results, checking assumptions, explaining conclusions.
• Triage & Decision Support: In support, security, and incident response, agents classify issues, gather context, suggest next steps, and escalate when needed.
• Controlled Automation Behind Feature Flags: Some teams let agents take actions, updating records, triggering workflows, drafting changes, but only behind strong guardrails.
• Internal Productivity for Engineering & Operations: Agents investigate issues, draft code changes, run diagnostics, or coordinate multi-step ops tasks.

What these cases share is restraint. The agents that deliver value today are not "general workers", they're specialists.

Problems with Agents

After reading this, you're probably excited about AI agents. That excitement makes sense, but most break down quickly in practice.

Long-Term Planning is Hard

LLMs are good at short-range reasoning but struggle to carry a coherent plan across many steps. Agents lose track of goals, repeat actions, or loop.

Massive Solution Space

At every step, there are many plausible actions. Because they're non-deterministic, agents may choose a different path each time, and outcomes become hard to reproduce.

State & Memory are Unreliable

Context windows are limited; summaries lose detail. The agent's internal view drifts from reality, leading to repeated work or confidently wrong actions.

Fragile to Tooling Failures

The same task can succeed once and fail the next. When tool calls are malformed or return unexpected outputs, agents rarely recover gracefully.

Chapter 05 · Frameworks

Major Agent Frameworks You Should Know

You technically don't need a framework. A simple loop around a model call, some state, and a few tool calls goes surprisingly far. Frameworks exist because once agents grow, the same problems keep appearing.

LangGraph: Agents as graphs / state machines. Built around the idea that agent behavior should be explicit.

Chapter 07 · Reliability

The Agent Testing Pyramid

A pattern developed by Rogerio Chaves (CTO, LangWatch). Inspired by the traditional software testing pyramid, adapted to the realities of AI agents. All three layers are necessary, skipping one creates blind spots.

• PEAK: Simulations
• MIDDLE: Evals & Optimization
• FOUNDATION: Unit Tests

Simulations

End-to-end behavioral tests. Simulated users drive multi-turn flows; a judge evaluates whether the agent completes the task as a user experiences it.

Chapter 08 · Observability & Evals

Agent Observability and Evals

Once your agent runs in the real world, the question is no longer "can it respond?" It's "can I trust it, every single time, under messy real-world conditions?"

Why it Matters

Evals measure:

• Output quality: Is the agent's response correct, complete, grounded, and useful?
• Behavior: Did it call the right tools at the right time? Ask clarifying questions? Respect constraints?
• Systems tradeoffs: Acceptable quality within latency, tool reliability, and cost budgets?

Chapter 09 · Simulations

Scenarios: Simulated Reality

Unit tests and per-call evals aren't enough. You need to test how the agent behaves across whole conversations. Scenarios simulate users driving multi-turn flows, while a judge verifies real-world outcomes.