Article · Blog
The agent didn't make a mistake. It answered exactly what you asked.
Most problems with AI agents in production aren't hallucination or model limitations. It's that the engineer doesn't realize they're asking the wrong question. The model delivers what it received. The problem happens before the call.
The agent didn't make a mistake. It answered exactly what you asked.
In a project I maintained for a few months, I had an agent that classified user intents. The task seemed simple: receive the message, identify what the user wanted, route to the right flow.
In tests, it worked well. In production, it started misclassifying cases that seemed obvious. The first hypothesis was hallucination. The second was high temperature. We adjusted parameters, swapped the prompt, added examples. The errors kept coming.
It took a few days to understand what was happening.
The agent wasn't making mistakes. It was answering with precision exactly what we had configured it to see.
The problem that looks like a model issue almost never is
When an LLM delivers a bad result in production, the natural reaction is to adjust the model: swap the system prompt, add more few-shot examples, change the base model, lower the temperature.
That fixes things sometimes. But it fixes the symptom.
The real problem, in most cases I've seen, lives in three places: what goes into the context, the order it goes in, and what was left out because it seemed irrelevant.
In the classifier case, the agent received the user's message but didn't receive the session history. From the outside, that seemed reasonable: the current message contained the intent. No need for history.
Except users were typing things like "I want to cancel." Cancel what? Without the prior context, any answer was a guess. The agent guessed. Sometimes it got it right. Sometimes it didn't.
The solution wasn't to swap the model. It was to change what reached it.
Context isn't just what you pass. It's what you decide not to pass.
This is the part most people ignore.
When you build an agent's context, you're making active decisions about what to include. Message history, current session state, user data, product metadata. And for everything you include, there's something you leave out, usually because it seems redundant, sensitive, or too expensive in tokens.
The problem is that the model has no opinion about what's missing. It responds with what it has. If what it has is incomplete, the response will be incomplete. But it will sound confident.
That's the trap. The model doesn't say "I don't have enough information to answer this." It answers anyway, with its usual fluency, and you'll only find out it was wrong when the user complains or when you look at the logs.
The illusion of the perfect prompt
For a long time, I treated context engineering as prompt engineering. In my head, they were the same thing.
They're not.
The prompt is what you tell the model. Context is the environment where it operates. You can have a flawless prompt and a broken context. The result will be bad in a way the prompt will never fix.
A practical example: in a support agent I worked on, we had a well-written system prompt, with a defined tone of voice, clear escalation instructions, and examples of good responses. The problem was that the agent didn't receive the user's current plan. So when someone asked about a limit or a feature, the agent would make up a generic answer based on what the model knew about the product, not the actual state of that user's account.
Swapping the prompt wasn't going to fix that. The data simply wasn't there.
What I learned to do differently
The first change was to treat context as a schema, not a string.
Before, I was building context as text: a series of concatenated messages, maybe with some data inserted in between. It seemed to work. But it was fragile, because any structural change could break how the model interpreted it.
Now I think of context as a contract. What fields exist, what each one represents, what the priority order is when the context starts getting too large to fit in the window. That forces an explicit decision about what's essential and what's disposable.
The second change was to start logging the full context that reached the model on every production call. Not just the prompt. Not just the response. The whole package.
This sounds obvious, but most systems I've seen in production don't do it. And without it, when something goes wrong, you're debugging in the dark. You see the bad output but you don't know exactly what went in.
The third change was to stop relying on manual prompt tests to validate production behavior. The model can answer correctly across ten manual variations and fail on the eleventh, which only shows up with real user data. Evaluating LLM output at scale needs automation, and automation needs structure.
The most expensive mistake I've made in this area
I built a pipeline where the agent received context from an external tool via function calling. The tool returned a JSON with user data. The agent used that data to personalize the response.
The problem: in some cases, the tool returned a partially filled JSON when the data wasn't available. Null fields, empty arrays, blank strings.
The agent interpreted this as if the data existed but was empty, not as the absence of information. And then it made things up. Not through random hallucination, but because the contract I had implicitly established said "these fields always exist." When they genuinely didn't, the model filled them in with whatever seemed reasonable.
The fix was simple: pass the state explicitly. If the data isn't available, say so in the context. "Field X is not available for this user" is fundamentally different from passing field X with a null value.
The model understands absence when you articulate it. It can't infer absence from silence.
What nobody talks about at scale
In testing, you control everything. You set up the scenario, choose the data, know exactly what's going in.
In production, context is generated dynamically. Different users, different states, combinations you never tested. The variability of real context is orders of magnitude larger than any set of manual tests.
And here's the point that changes everything: your agent's quality in production is bounded by the quality of your context pipeline, not by the model you chose.
You can use the most expensive model on the market. If the context reaching it is inconsistent, incomplete, or poorly structured, the result will be inconsistent, incomplete, or poorly structured.
Switching from GPT-4 to Claude doesn't fix bad context. Fixing bad context fixes bad context.
The agent that breaks in production is almost never wrong. It's answering with precision what it received. The problem is in what you decided to pass to it, and in what you thought you didn't need to pass.
Before adjusting any prompt, it's worth looking at the full context that came in on the last call that went wrong. Most of the time, the answer is right there.