"Context windows are huge now, so just stuff everything in and skip retrieval." We hear this a lot. It's wrong, and the bills (and the quality regressions) prove it.

Bigger context windows didn't kill retrieval-augmented generation. They changed what good looks like. The new name for the discipline is context engineering: deliberately deciding what information the model sees on each request, how it's ordered, and — crucially — what gets left out.

Why "just stuff it all in" fails

Three reasons. First, cost: a million-token prompt on every request is real money and real latency. Second, the lost-in-the-middle problem: models reliably attend to the start and end of a long context and skim the middle, so burying the key fact in paragraph 400 means it might as well not be there. Third, distraction: irrelevant context actively degrades answers. More is not more.

What context engineering actually involves

  • Retrieve the few genuinely relevant chunks — good retrieval still matters, arguably more.
  • Rank and place them where the model will actually use them (the ends, not the middle).
  • Compress what you can — summaries, structured extracts — instead of raw dumps.
  • Decide what to omit. The hardest and highest-leverage choice.
Prompt engineering was about the instruction. Context engineering is about everything else the model sees — and that's usually where the answer is won or lost.

This is a product job as much as an engineering one

What counts as "relevant context" for a support bot vs. a coding assistant vs. a financial analyst is a product decision rooted in understanding the user's task. The PM who can specify "the model should see the last 3 user messages, the account tier, and the 2 most similar past tickets — and nothing else" is doing the most valuable work in the building.

How to know it's working

The same way you know anything in AI is working: evals. Hold the model constant and vary the context assembly; measure faithfulness and answer quality. You'll often find that removing context improves results — which is the clearest sign the discipline is real.

The takeaway

Don't ask "how do I fit more in the window?" Ask "what is the minimum the model needs to see to get this right?" That question — and the retrieval, ranking, and omission it implies — is context engineering, and it's the core skill of building useful AI products in 2026.