How to Architect an AI Agent That Survives Production

Your agent demo chains six tool calls flawlessly and ends before anything breaks. Ship the same loop to production and it retries a failing action for 40 minutes, or "helpfully" emails a customer nobody asked it to contact. Here are six architecture patterns that separate agents that survive production from agents that only survive demos.

The gap demos don't show

The core distinction between demo-ware and production-ware: production systems spend most of their design budget constraining the autonomy that demos exist to celebrate. Not eliminating it — constraining it, on purpose, at specific joints. An agent is a system where the model chooses the next action, which tool, which arguments, whether to continue, rather than executing a fixed pipeline. That choice-making is both the value and the risk, so the design question is never "agent or not" but where the choosing happens and what contains it.

Pattern 1: the bounded loop

The observe-decide-act-repeat primitive only belongs in production wrapped in explicit budgets on every axis it can run away on. One real configuration: max_steps = 12 actions per task, max_tokens = 150,000 as a spend ceiling, max_wall_clock = 300 seconds, a cap of 3 consecutive tool errors before surrender, and max_repeats = 2 to catch a loop retrying the identical tool call forever.

A calibration note from production use: most useful task classes converge at a surprisingly low step count, 5-15 steps. If an agent regularly needs 40 steps, that's not an autonomy problem — it's a decomposition problem.

Patterns 2 and 3: workflow skeleton, and tool tiers

The highest-value structural decision is a deterministic workflow skeleton with agentic stages inside it. The skeleton supplies what models are bad at — guaranteed ordering, stage-scoped tool access, deterministic gates between stages. The agentic stages supply what fixed pipelines are bad at — handling the variety inside one step.

Every tool must be tiered by consequence, and the tiering has to live in the tool executor, not the prompt. Four tiers: Tier 0 — Observe (read, search, fetch, list); Tier 1 — Reversible acts (draft, stage, sandbox creation); Tier 2 — Consequential acts (send, merge, deploy-to-staging, requiring a deterministic precondition check); Tier 3 — Irreversible (payments, deletions, production deploys), which require a human gate or stay out of the agent's reach entirely.

Patterns 4 and 5: checkpoint/resume, and a capped critic loop

Production tasks span minutes to hours, hit flaky tools, and get interrupted by restarts. The fix is externalizing the agent's full state — plan, completed steps, tool results, pending intent — into a durable store at every step boundary, making the loop stateless and resumable.

For the critic loop — a second model pass reviewing the first's output against a rubric — the sharp caveat is capping it at one round. Generate, critique, revise converges in one iteration on most real tasks; further rounds produce oscillation, where the revision un-fixes what the previous round fixed, at linear cost.

Pattern 6: human gates that don't kill throughput

Gate at consequence boundaries, not step boundaries. One approval covering a complete proposed action set — "send these 3 emails, file these 2 tickets — approve?" — beats five sequential micro-approvals, both for throughput and for how much attention a reviewer can actually give.

Architecture alone doesn't ship trust — verification does

Generative systems fail differently from ordinary software: they don't crash, they produce something plausible and occasionally wrong, with total confidence. Every automated claim needs a structured evidence gate — a URL, a fetch timestamp, the specific check performed, a verdict — rather than a vibe.

On the evals side, one team's largest text pipeline runs about 3,900 test cases on every change, with an expected failure count of exactly zero. Where an LLM judges cases with no single right answer, each pair gets judged twice with the sides swapped; only a verdict that survives the swap is recorded, and roughly 11% do not.

Recurring anti-patterns worth naming

A few shapes reliably predict production pain. The self-delegating swarm — agents spawning agents — multiplies every failure mode by the fan-out while mostly delivering coordination overhead; a workflow skeleton with parallel stages captures the same parallelism without the anarchy. The god-context loop appends every observation to one ever-growing context until the model drowns in its own history, which is exactly what checkpointed state with summarized memory exists to prevent. Prompt-enforced safety — any instruction shaped like "the model is told not to..." presented as a control — isn't one; tiering enforced in the executor is. And the demo metric, "it completed the task in our tests" with no denominator, hides the numbers that actually matter: completion rate, intervention rate, and cost per completed task, measured on real traffic.

One test worth asking before reaching for an agent at all: could a senior engineer flowchart the task completely? If the path is genuinely fixed, build the flowchart — it will beat an agent on every production metric. Agents earn their place when the path varies per instance enough that enumerating branches up front is infeasible.

What to build first

If you're already running agents in production, start with the cheapest fix: add a bounded loop with repeat detection to your existing agent, before tackling tool tiering or checkpointing. Adding step budgets and a surrender protocol at design time costs far less than discovering their absence during an incident.