For two years the reflex was: reach for the biggest model, wrap it in a prompt, ship. It worked, and it was wildly expensive. In 2026 the smartest teams have flipped the default — they reach for the smallest model that clears the bar, and only escalate when it doesn't.

Why small models got good enough

Two things changed. First, the small open-weight models (3B–12B parameters) got genuinely capable at the narrow tasks that make up most production workloads — classification, extraction, routing, short rewrites, structured output. Second, fine-tuning and distillation got cheap and reliable enough that you can teach a small model your specific task in an afternoon.

The result: for a huge class of jobs, a tuned small model now matches a frontier model's quality at a fraction of the latency and 1/20th of the cost.

The 80/20 of a real workload

Look at the actual distribution of requests hitting your AI feature. In almost every product we've measured, ~80% are easy and repetitive, and ~20% genuinely need a big model's reasoning. Sending all 100% to a frontier model means paying frontier prices for the easy 80% — pure waste.

The model is not the product. The product is the routing, the evals, and the fallback. The model is a swappable part.

How to actually route

A routing layer is simpler than it sounds. The pattern we teach:

  • Default small. Every request hits the small model first.
  • Confidence gate. If the small model's output fails a cheap check (low confidence, schema violation, refusal), escalate.
  • Escalate to big. The hard 20% goes to the frontier model.
  • Log the split. Track what fraction escalates — it tells you where to fine-tune next.

Over time you push the escalation rate down by teaching the small model the patterns it keeps failing on. Every percentage point you move from "big" to "small" drops cost and latency at the same time.

What the small model can't do (yet)

Multi-step reasoning, long-context synthesis, and genuinely novel problems still belong to the big models. Don't fight that. The skill is knowing the boundary and building the seam where work crosses it cleanly.

The takeaway

"Which model is best?" is the wrong question for a builder. "What's the cheapest model that passes my eval for this request?" is the right one — and answering it, request by request, is where the real engineering (and the real savings) live.