2025 was the year of the agent demo. 2026 is the year teams learned what it costs to keep one running. An agent that books a meeting flawlessly on stage will happily spend $4 in tokens looping over a malformed calendar API at 2 a.m. — and nobody notices until the bill arrives.

We have shipped a handful of agentic features now, killed two, and kept three. Here is what separated the survivors.

Scope the tools, not the model

The single biggest reliability win is giving the agent fewer, sharper tools. A model with 30 loosely-defined tools spends most of its reasoning deciding which to call and gets it wrong often. A model with 5 tools, each with a tight schema and a one-line description of when not to use it, is dramatically more predictable.

Treat every tool like a public API: validate inputs, return structured errors the model can recover from, and never let a tool fail silently. The agent is only as good as the worst error message it gets back.

Hard budgets, not soft hopes

Every agent run gets three ceilings before it starts: a step budget (max tool calls), a token budget, and a wall-clock budget. When any one trips, the run halts and hands off to a fallback — a simpler deterministic path or a human queue. This one change turned our worst-case cost from "unbounded" to "predictable", which is the difference between a feature you can launch and one you can't.

The question that kills most agent projects isn't "can it do the task?" — it's "what happens on the 1% of runs where it can't?" — our post-mortem on the first agent we shipped

Evaluate the trajectory, not just the answer

A correct final answer can hide a terrible path: six wasted tool calls, a hallucinated parameter that happened not to matter, a retry loop that got lucky. If you only score the final output, you are blind to the failure modes that will eventually bite you.

We log every run as a replayable trace and score the trajectory: did it pick the right tool first, did it stop when it should have, did it stay in budget? These trajectory evals catch regressions that output-only evals miss entirely.

Keep a human in the loop where it's cheap to be wrong slowly and expensive to be wrong fast

Not every step needs approval — that defeats the point. But the irreversible ones do: sending money, emailing a customer, deleting data. We gate exactly those actions behind a confirmation and let everything else run free. The art is drawing that line in the right place, and it's a product decision as much as an engineering one.

The shape of an agent that ships

  • A small set of well-typed tools with explicit "don't use when" guidance.
  • Step, token, and time budgets with a defined fallback when they trip.
  • Full run traces, replayable, with trajectory-level evals in CI.
  • Human confirmation on irreversible actions only.
  • A dashboard that shows cost-per-successful-task, not just success rate.

None of this is glamorous. It's the boring infrastructure that turns a clever demo into something you can actually put in front of a million users — and it's exactly what we build, hands-on, in the AIOps cohort.