Agent Reliability Engineering: Field Notes on Building Predictable, Steerable Agents

Claude Code, Codex, and similar assistants help you run your daily tasks. You feed them the inputs and they work through a series of steps: pull data, scrub it, draft, check, format — and out comes a report. It works. You do it every day.

The obvious next move: schedule it. Run it every day, unattended at 6 AM, and read the output over coffee.

That’s when the trouble starts.

Running an AI workflow once, interactively, is a good start. Running unattended, every day, is a system. And the math is unforgiving. If your workflow has 10 steps and each succeeds 95% of the time, the whole chain succeeds 0.95^10 ≈ 60% of the time: two mornings out of five, you’re reading a broken report. At 90% per step, you’re at 35%. Or worse, a plausible one that’s silently wrong, which you won’t catch over coffee.

Traditional software typically fails hard and deterministically: same input, same crash. Agents fail softly and stochastically: same input, different output. The worst failure is the confident paragraph citing a source that doesn’t say that, or doesn’t even exist.

Engineers are trained to build reliable systems from unreliable components, using patterns like redundancy and auto-failover. We decompose problems, write contracts, deploy validation gates, run audits. Stochastic LLMs add a few new wrinkles.

TL;DR: Decompose your complex tasks into simple tasks that you can easily evaluate, optimize, and correct on the fly. If all the individual steps and the end-to-end-logical flow are verified, then by induction the whole task is verified.

What is reliability from the business perspective? It means consistently delivering the intended result within defined limits for correctness, completeness, timeliness, cost, and authorized behavior, in a transparent and auditable manner. For a daily report, that means accurate, sufficiently complete analysis delivered on time, within budget, to the right recipients. When those requirements cannot be met, the system must detect the problem and follow a defined recovery or escalation path. Measure both incorrect outputs delivered and missing correct outputs: an agent that confidently ships incorrect information is unreliable, and so is one that doesn’t ship when it should, and so is one that ships with missing information it should have picked up.

What follows are field notes from building agent pipelines that run unattended and produce output you can trust enough to act on, even in a high-stakes, regulated context.

Observability: tracing and run manifests. Every run logs a comprehensive persistent trace with a manifest of exactly what it ran on: the starting state (input files, hashes, as-of dates), prompt versions (ideally tied to an immutable prompt registry like Langfuse), model string, tool and skill calls with versions, token counts, latency, and exit status per step. This lets you do error analysis, reproduce a final output exactly, and satisfy an auditor. The logged manifest and trace ship with the final output as first-class deliverables. A report you can’t reconstruct is a report you can’t defend.
Context engineering: Keep It Simple, Stupid. A series of small prompts is more predictable, verifiable, and steerable than one mega-prompt. Small prompts reduce and control what’s in context, avoiding context rot, and each prompt is easier to reason about, test, and evaluate in isolation. Stay under 50% of the context window during runs; degradation sets in well before the advertised window fills. High context usage is a signal to decompose.

Decomposing large prompts into multiple small prompts takes several forms, in increasing order of isolation:
- Skills: An orchestrator skill that invokes other skills in sequence, potentially with a deep hierarchy.
- Subagents: Tasks that can run in their own isolated context and return a summary to the caller without polluting the parent’s context. They add complexity, but provide isolation, scoped tools, model selection, and parallelism. Claude Code’s dynamic workflows let you define fan-out, pipelines, and other deterministic topologies in a script.
- An external orchestrator: Each prompt runs in a fully independent session — e.g., a Python script where each step processes artifacts from the previous step and writes new ones, invoking claude -p per step.
Idempotent, resumable steps. It’s useful to think of an agent as a flowchart with a series of steps with control branches and loops. Some agent frameworks like LangGraph make this explicit. Or if you are using skills, each step could be a skill, with control flow orchestrated by a higher-level skill. Each step should be idempotent: running it three times in succession produces the same output as running it once. Each step reads the previous step’s artifacts and writes its own. Retries do not duplicate records, send the same message twice, or create any additional effects. If checks pass, the step is atomically marked completed. A completed step can then be skipped on rerun, which makes the pipeline resumable from the last successful step after a failure. With artifacts checkpointed in a data store, retries are per step — much cheaper than rerunning the pipeline.
Contracts: every step has clearly defined structured outputs. Each step gets a fixed deliverables list with explicit cardinality: “deliver exactly three artifacts,” and “deliver only the specified outputs; do not create extra documents.” Allow unknown/unavailable as legal values: a schema that forces a value into every field rewards the LLM for guessing.
- JSON Schema output contracts: additionalProperties: false, required fields, length caps, character-class patterns, and closed enums for classifications — enforced by a deterministic harness-side validator, never by trusting the model to police itself.
- Standardized templates: fixed file structures, fixed workbook tab layouts, fixed report page schemas, fixed QC-report formats, and quantified minimums (≥30 pages, ≥25 charts) paired with an actual-count verification block, so “did it deliver?” is a mechanical check rather than a judgment call.
Grounding: bronze, silver, gold. Borrow the medallion architecture from data engineering and apply it to agent grounding:
1. Bronze: immutable raw sources. Stamp everything with a create date and an as-of date so freshness is decidable. Set max-age thresholds per source class, and state the policy up front: does a stale source fail the gate, or flag the output with a warning?
2. Silver: cleaned and merged intermediates. Where multiple bronze sources cover the same fact, rank them and take the best available. Use consistent, greppable nomenclature for unavailable and estimated values so gaps are searchable, not silent.
3. Gold: the final output. Every claim traces back to a named bronze source reference that can be re-fetched, or at minimum cited, e.g., “Wikipedia, retrieved 2026-08-30.” A factual claim with no bronze ancestor is a problem, and the audit pass should treat it as a possible hallucination to be verified or removed.
Memory: Karpathy wiki methodology. A simple approach to agent memory is to use a wiki with a specified structure as the silver store, with links back to the bronze canonical sources. Andrej Karpathy published an LLM Wiki repo with a methodology for auto-maintaining a structured wiki. Read sources, extract the relevant facts into the wiki, then build the final product from the wiki.

There are several agent memory modules and SaaS services. They can be convenient but don’t relieve you of the need to think carefully about how you structure the data for your workflows. A wiki is a simple place to start and will work well when the whole wiki fits easily within the LLM’s context window. As the amount of data grows, you will need to think harder about structuring agent memory to always give the right context to the LLM, which may involve different types of memory and database engines like SQL, vector store, graph database, and simple tools and skills to let the LLM extract exactly what it needs when it needs it.
Deterministic code over LLM-as-computer. Whenever possible, use a tool, a Python or JS script, a shell script, or a spreadsheet artifact instead of a stochastic LLM-as-computer prompt. Sorting, counting, arithmetic, date math, joins, format conversion — anything with one right answer belongs in code that produces it every time. Reserve the model for work that actually needs semantic understanding and judgment. Agents should not do their own bookkeeping. An extraction agent judges what is relevant; a tool or script creates and stores a well-structured record.
Hard gates, enforced by contracts and deterministic code. Sanity-check before each step (are prerequisites in place?) and after (does the output look clean?). Before a writing step, for instance: is silver data present, recent, and clean? Render and inspect final artifacts — open the PDF, count the pages, check that the charts rendered. Enforce hard budgets (output length, tokens, wall clock) and cap retries, e.g., at three attempts, so no gate becomes an infinite loop. In more complex topologies, add loop detection for orchestrator/subagent ping-pong, where agents bounce work back and forth without making progress.
Soft gates: LLM-as-judge and critic-optimizer loops. Hard gates catch what code can check — schemas parse, counts match, budgets hold. Soft gates catch qualitative issues only an inspection with semantic understanding can catch. Does the output follow the expected style? Is it free of internal contradictions? Does it leave an obvious question unanswered? Does every statement of fact link to a bronze source? Would a skilled reader find it trustworthy?
- Rubric, not vibes. The judge scores against an explicit rubric with named criteria: groundedness, coverage, coherence, style conformance. 20 questions, not “rate this”. Make the “soft” gate as hard as possible. Prefer binary pass/fail per criterion over scalar scores; binary judgments are consistent across runs and easier to calibrate against human labels.
- Reward the agent for correctly saying “I don’t know” — label fields unknown/unavailable when the source data isn’t there. Never reward guessing. An eval that penalizes honest abstention trains your pipeline to make stuff up.
- Structured verdicts. The judge emits a parseable data structure: JSON with issues found, severity, location, and a recommended action for each. That structure can then feed into an optimizer step, which fixes the flagged issues and re-evaluates.
- Critic-optimizer loop with a floor and a ceiling. Iterate judge → fix → rejudge until hard checks pass and soft checks clear a minimum rubric score — but with a bounded iteration count (e.g., three passes), after which remaining issues go into the exception report rather than another lap. An unbounded quality loop is a potential infinite loop and an infinite bill.
- Judges are cheap; use many narrow parallel ones. A groundedness judge, a style judge, and a contradiction judge — each with a small focused prompt on a smaller model — beat one omnibus judge, for the same context-rot reasons that small steps beat big prompts. Claim-by-claim verification against bronze sources deserves its own pass.
- Separate the judge’s context. The judge runs in its own session with the rubric, the output, and the sources — not the full generation history. A judge that saw the drafting process inherits its assumptions; a fresh-context judge reads the artifact more like your reader will. A different model is even better, as models tend to favor their own handiwork.
- Evaluate the judge itself. A judge is a model component like any other: validate it against a labeled set of known-good and known-bad outputs, track its agreement rate with human reviewers, and rerun that calibration when the underlying model version changes. Evaluating the judge can be hard, but an uncalibrated judge that passes everything is worse than no judge — it’s false assurance with a paper trail.
- Gaps and exceptions are a first-class deliverable at every complex step: what couldn’t be sourced, what was estimated, what conflicts were found. A pipeline that reports its own holes is reliable; one that silently papers over them is not.
Evals: unit tests for every step. Every hard and soft gate maps naturally to an eval you can run as a suite in a tool like promptfoo or Langfuse. An eval suite lets you pick the cheapest model that consistently gets the job done. When the model version, or anything else, changes, rerun the suite and see what regressed.
- Score each step against its contract. Hard checks validate the output schema; soft checks apply the rubric from the soft-gates section above, emitting structured verdicts.
- Measure reliability as pass^k, not pass@k. Pass@k asks “did it succeed at least once in k tries?” — the demo metric. Pass^k asks “did it succeed all k times?” — the production metric. Ten steps that each pass 90% of trials give a pipeline that passes 35% of the time. Run each eval N ≥ 5 times and report the worst case; set per-step thresholds based on the pipeline length you need. A 10-step pipeline targeting 95% end-to-end needs each step at roughly 99.5%.
- Generate evals from observed failures and edge cases. Follow Hamel Husain and Shreya Shankar’s error-analysis loop: sample production traces, label the failures, cluster them, write an eval per cluster using the hardest cases. Your eval suite should be a fossil record of everything that has actually gone wrong.
- Feedback discipline (Mitchell Hashimoto). Every incident and every edge case produces a permanent artifact: a new eval, a new gate, a validator rule, a line in the agent’s instruction file — engineered so the agent never makes that mistake again. Gates accrete, so they need a governance process: who adds them, where they live, how they’re versioned, and, periodically, which ones a stronger model has made obsolete.
Good evals enable auto-improvement. As a general principle, the best tasks to give an AI are the ones that are easiest to verify. When the agent can check its own work via an unambiguous and immediate signal — run tests, validate the schema, count the pages — it can self-correct. Verification asymmetry is key: when generation is hard and checking and correcting are easy, put the checking in the loop and let the model iterate against it.

We can extend this paradigm from runtime course correction to prompt optimization. In March 2026, Karpathy released an autoresearch repo that hands the ML research loop itself to an agent. Applied to prompts, the loop is:
1. Examine a prompt or task and consider ways to improve it.
2. Modify the prompt and run it against test inputs and evals, scoring the result with a rubric.
3. If the result improved, keep it; if it got worse, discard it.
4. Go to step 1 and iterate. Look at the highest-performing prompts found so far and consider ways to improve them with new ideas and combinations of existing ideas.
This optimization pattern transfers to any pipeline with three ingredients:
1. A scalar, trusted metric: your eval suite’s pass^k score, a rubric score, a latency number. It must be cheap to compute and hard to game.
2. A bounded search space: the agent edits one prompt, one skill, one validator; everything else is frozen.
3. A keep-or-revert loop: each change is scored against the metric; improvements are committed, regressions are discarded, and every experiment is logged to avoid repeating it.
The runtime and dev-time evals become the objective function for automated task optimization — the same artifact serving defense and offense. Now your agent can be self-improving.

Use train/validate/test set discipline and do not overoptimize, or results may not generalize outside the test set. Look at the top-performing prompts and use them to help write prompts that make obvious sense and cover all the bases, then retest. The optimizer will optimize exactly what you measure, which is generally too specific and gameable. Easy verification makes the loop possible; common sense makes it safe.
Orchestration: fixed-shape workflows. At the orchestration level, use numbered, fixed-order workflows with named phases and typed input contracts — entity + YYYY-MM, batch ID + NAV pack — so every run is repeatable in shape. The agent gets freedom within a step, not over the sequence of steps. When run 47 and run 48 follow the same numbered phases with the same typed inputs, diffs between them are meaningful, failures are attributable, and “where did it break?” has a one-word answer.

There are smart, highly autonomous patterns like ReAct but they are harder to reason about and steer. There is a tradeoff between maximally creative and resourceful agents, and predictable, steerable, reliable agents. Save maximum autonomy for rare cases when it’s needed.
Dedicated audit pass at the end. Before anything ships, run a final audit: layered QC combining the unit gates already passed per step with comprehensive integration checks across the whole deliverable. All components present, all counts met, all failure modes checked — missing source links, orphaned claims, stale data, unresolved exception, repetition or contradiction across sections as opposed to within sections. This is the last line of defense. If any item fails, do not deliver. No report beats a wrong report. Consider automated reports and dashboards of metrics and trends.
Human in the loop. Three maturity stages:
- Assistant: the human drives, the model executes one step at a time, everything is reviewed and corrected in place.
- Supervised agent: the pipeline runs end-to-end but pauses at approval gates after key steps, presenting a summary of what was done and what needs elevated review — primary sources that conflict, low-confidence claims, exceptions the critic couldn’t resolve.
- Hands-free with escalation: the pipeline runs unattended and inverts the interaction — instead of the human checking in on the agent, the agent reaches out to the human when a gate trips.
The human’s role naturally diminishes as the agent matures. But there must always be a boundary where the agent detects it has encountered something outside the design envelope and brings in a human for the edge case. Promotion between maturity stages must be earned, not assumed: as pass^k history accumulates per step, dial human involvement down.

Design for the reviewer at every stage. Surface the exception list and the diff against the prior run, not a long trace to read from scratch. A reviewer who must find the problems themselves is a bottleneck; a reviewer handed exactly the five items needing judgment is a control.

Escalation should arrive where the human already works: a Slack or Teams message with the exception summary, the relevant diff, and approve/reject/correct actions inline — not a log entry waiting to be discovered, and not an email to a folder nobody checks. Route by severity: soft-check warnings post to a channel for async review; hard-gate failures stop and notify someone for immediate intervention; anything high-stakes blocks until a named human approves. The right workflow tools must match SLAs and escalation paths and support the desired resolution latency.

When a human intervenes, the correction should be as cheap as possible: fix the input or override the judgment, then rerun from that step — which the checkpointed, resumable architecture gives you for free. And capture every reviewer verdict — approved, rejected, corrected, and why — as labeled data. Human corrections are the highest-quality eval inputs you can get, and they justify the next promotion toward hands-free.

Further considerations

Parallelize everything. Use async/await, skills and workflows to run tasks in parallel whenever dependencies allow.
Beware of prompt injection. Initial searches should run in an agent with limited capabilities. Downloaded data should be treated as hostile and subject to a scan and sanitization process before going downstream.
Least privilege and sandboxing. Scoped credentials per step, read-only by default, write actions gated, execution in a container with egress allowlists. A correctly behaving agent with excessive permissions is an insider threat when it gets a bad input.
Treat prompts as code. For higher-maturity deployments, prompts should be in CI and/or a prompt repo like Langfuse, which makes it easy for non-devs to iterate on prompts. When a model updates, you can eval current and previous prompt versions to catch new problems and regressions.
Tool design. There is a tradeoff between fewer, wider tools, which allow more efficiency and creativity, and least privilege. Write tool errors for the model to recover from (what went wrong and what to try next), and truncate and structure tool outputs before they enter context.
Four sources of metrics on how well the agent is working.
1. Runtime evals and gate trips, tracked over time.
2. Usage and growth: if people use it, it’s probably useful.
3. Vibes: what people tell you about how it works in the field. Provide easy inline human evals via thumbs-up/down and surveys.
4. Controlled experiments and benchmarks: run the agent end-to-end on real-world out-of-sample tasks and score the results by human or automated means. A good harness for A/B testing, letting humans evaluate and label alternative outputs and traces, can be worth its weight in gold.
Deployment, change control, and rollback. Version prompts, model configurations, tools, and validators together as a release. Require regression evals before promotion, test consequential changes in parallel-testing mode, and roll them out gradually while monitoring quality, latency, and cost. Keep a known-good release ready to restore, with explicit rollback triggers.
Software supply-chain security. Pull from a curated internal mirror (Artifactory, Nexus, GitHub Packages) or verified sources (Chainguard or Docker Official images, PyPI/npm Trusted Publishers with provenance attestations) rather than raw public registries; use minimum-age flags at a minimum (pnpm minimumReleaseAge, uv --exclude-newer, Renovate cooldowns) so a freshly hijacked release never reaches your build; pin dependencies to hashes in a lockfile and GitHub Actions to commit SHAs.

Concluding remarks

The need for good engineering doesn’t go away. Engineering moves to:

Evals appropriate to the task. These are the most important part of the process, covering development-time optimization, runtime gating, and critic-optimnizer loops.
Problem abstraction and decomposition. Breaking down the problem into tractable chunks of deterministic tools and prompts.
Context engineering. LLM-friendly memory structures appropriate to the task to give the LLM the info it needs when it needs it.

You will never get determinism from an LLM, but if you can verify and correct each step and the end-to-end task, you can make agents sufficiently trustworthy and reliable within a defined operational envelope. Set the floor, then raise it. Go forth and make reliable agents!

Agent Reliability Engineering: Field Notes on Building Predictable, Steerable Agents

Further considerations

Concluding remarks

Further reading

Reliability from unreliable components

Observability and prompt registries

Context engineering and decomposition

Contracts and structured output

Idempotency, checkpointing, durable execution

Grounding, provenance, hallucination

Agent memory

Deterministic code and verification asymmetry

LLM-as-judge and critic-optimizer loops

Evals

Automated prompt optimization and Goodhart

Human in the loop and model governance

Security: prompt injection, least privilege, supply chain

Tool design

Books

Courses