Real-World Perspectives: What AI Agents Actually Look Like Outside of Demos
By Marcel Weigand, Senior AI Solutions Architect at Merantix Momentum
There is a version of AI agents that lives in keynote slides, and there is a version that lives in production. They are not the same system, and pretending they are is one of the more expensive mistakes I see enterprises make right now. In the demo version, an agent takes a natural-language request, reasons across a few tools, and returns a clean answer. In the production version, the same agent has to deal with malformed inputs, half-broken upstream pipelines, permissioned data it cannot see, tools that time out, and an auditor who wants to know exactly why it said what it said at 03:14 on a Tuesday.
Everything I want to share here comes from that second world. The gap between the demo and production is real, wide, and recurring. But contrary to popular belief, the model is rarely the bottleneck: when a client project stalls, it is almost never because the LLM isn't smart enough, but because the system architecture is misaligned, engineering culture breaks under non-determinism, or evaluation loops are absent in production. At this stage of maturity, swapping one frontier model for another fixes none of that. Most of the work of shipping an agent is organizational and architectural, not algorithmic.
1. Architecture: Are agents the right system at all?
The first architectural question to answer is whether you should build an agent at all. Engineering discipline starts by evaluating if an agent setup is truly necessary rather than defaulting to one. A large share of requests framed as "we need an AI agent" turn out to be standard workflows with a single fuzzy step. Extracting fields from an invoice or classifying a support ticket does not need an agent. These are linear pipelines where an LLM handles unstructured text, and traditional software handles the rest.
An agent is the right architecture when the path itself is variable, meaning the system genuinely has to decide at runtime what to do next, which tools to call, in what order, based on what it just learned. That's a much smaller set of problems than the current discourse suggests, and misclassifying a workflow as an agent carries real costs: more latency, more spend, harder testing, harder debugging, and less predictability, which matters when the user is a paying customer or a regulator.
A useful way to figure out which situation you're in is to look at four questions about the process.
- Is the input structured data, or messy natural language?
- Is the sequence of steps fixed, or does the system have to decide it based on the situation?
- Is it hardcoded which external systems get called, or does the system have to choose?
- And what's the cost of being wrong, cheap and recoverable, or expensive and irreversible?
The more these lean toward "variable" and "expensive," the more an agent starts to earn its price. Working through them honestly puts a given problem further to the left than the initial ask suggests, more often than not.
If the conclusion is genuinely that you need an agent, the instinct to build one that does everything is still probably wrong. Small agents with tight scopes are easier to evaluate, debug, secure and trust. Before you build anything, consider one key scenario: what is the worst thing this agent could plausibly do? Because smaller agents have more definable and limited failure modes, identifying this worst-case scenario directly shapes your tool boundaries, moving you away from fragile, capability-first prototypes and toward reliable, failure-first systems you can safely deploy in a regulated environment.
2. Culture & Non-Determinism: Non-determinism breaks your engineering culture
Even with the correct architecture in place, the next compounding challenge hits: your engineering team is rarely prepared for how non-deterministic systems behave. Software teams have spent thirty years building habits around deterministic systems. Write the code, write the test, green build, ship. Agents don't behave that way. The same prompt can produce different outputs. A tool call that succeeded twice can fail the third time in a way that's hard to reproduce. Traditional CI and incident response assume a level of reproducibility that simply isn't there.
That forces a cultural shift teams often underestimate. You need engineers comfortable reasoning in distributions rather than assertions, product managers who define success as "acceptable behaviour across a population of interactions," and leadership that accepts a probabilistic quality bar without treating every anomaly as a P1 incident.
This cultural shift directly feeds the final layer of complexity: evaluation. Once you are shipping distributions instead of assertions, the question of whether the system is good enough cannot be solved statically during development. Instead, it requires a continuous loop in production.
3. Evaluation: The gap you cannot close during development
Evaluation has an underlying epistemic problem: you cannot think of all the edge cases during development, because you don't yet know what real users will do. Test cases cover the situations you can imagine, which are biased toward the ones the system already handles well.
Synthetic test sets help, but only so far. One workshop with a domain expert to capture the underlying business rules and a small set of real examples, then generate a much larger synthetic test set from those. The agent runs against it, outputs checked in three layers: deterministic assertions for anything with a clear right answer, an LLM-as-judge for fuzzier cases, and only the genuinely uncertain outputs routed to the domain expert. Everything else the system evaluates itself.
The two loops, and where the domain expert fits into each, look like this:
The three orange nodes are the same person. The goal is to keep their involvement targeted, consequential at each point but never continuous.
The right half of the diagram is what stops the offline evaluation from going stale. Real users will always find cases nobody thought to write down. That is why you need the second loop running continuously in production: reference-free checks on live traffic, covering groundedness, safety and self-consistency, that require no golden answer to run, plus a small stream of flagged and sampled traces that get reviewed and fed back into the golden set. The next release runs against the improved set. Without that loop, the offline evaluation quietly drifts away from what actually happens with real users.
Making any of this work requires proper observability: trace-level visibility into every prompt, retrieval, tool call and retry, and the ability to replay a full session end to end. Teams that treat this as optional spend months guessing what went wrong and cannot close the loop. If the ratio of outputs the system can self-evaluate is not improving week on week, the loop is not working.
The boring parts
The pattern is fairly consistent. The gap between the demo and production shows up as an architectural problem, a cultural problem, and an evaluation problem, roughly in that order, each compounding the next.
The teams winning with agents right now aren't the ones with the flashiest capabilities. They are the ones who have made peace with the boring parts: choosing the right shape of system, accepting that a non-deterministic system needs a non-deterministic engineering culture, and running the evaluation loop continuously rather than treating it as something you do once before launch.
Engineering discipline is what ships agents. The model is the easy part. The data, the guardrails, the evaluation loop and the organisational readiness are where the real work lives, and where the value actually gets captured.
That's the perspective from production. It's less exciting than the demo, and it's also the version that pays.
Subscribe to the Merantix Momentum Newsletter now.
More articles
.png)
From bottleneck to breakthrough: Why AI is crucial for progress in healthcare and pharma
Real-World Perspectives: What AI Agents Actually Look Like Outside of Demos
By Marcel Weigand, Senior AI Solutions Architect at Merantix Momentum
There is a version of AI agents that lives in keynote slides, and there is a version that lives in production. They are not the same system, and pretending they are is one of the more expensive mistakes I see enterprises make right now. In the demo version, an agent takes a natural-language request, reasons across a few tools, and returns a clean answer. In the production version, the same agent has to deal with malformed inputs, half-broken upstream pipelines, permissioned data it cannot see, tools that time out, and an auditor who wants to know exactly why it said what it said at 03:14 on a Tuesday.
Everything I want to share here comes from that second world. The gap between the demo and production is real, wide, and recurring. But contrary to popular belief, the model is rarely the bottleneck: when a client project stalls, it is almost never because the LLM isn't smart enough, but because the system architecture is misaligned, engineering culture breaks under non-determinism, or evaluation loops are absent in production. At this stage of maturity, swapping one frontier model for another fixes none of that. Most of the work of shipping an agent is organizational and architectural, not algorithmic.
1. Architecture: Are agents the right system at all?
The first architectural question to answer is whether you should build an agent at all. Engineering discipline starts by evaluating if an agent setup is truly necessary rather than defaulting to one. A large share of requests framed as "we need an AI agent" turn out to be standard workflows with a single fuzzy step. Extracting fields from an invoice or classifying a support ticket does not need an agent. These are linear pipelines where an LLM handles unstructured text, and traditional software handles the rest.
An agent is the right architecture when the path itself is variable, meaning the system genuinely has to decide at runtime what to do next, which tools to call, in what order, based on what it just learned. That's a much smaller set of problems than the current discourse suggests, and misclassifying a workflow as an agent carries real costs: more latency, more spend, harder testing, harder debugging, and less predictability, which matters when the user is a paying customer or a regulator.
A useful way to figure out which situation you're in is to look at four questions about the process.
- Is the input structured data, or messy natural language?
- Is the sequence of steps fixed, or does the system have to decide it based on the situation?
- Is it hardcoded which external systems get called, or does the system have to choose?
- And what's the cost of being wrong, cheap and recoverable, or expensive and irreversible?
The more these lean toward "variable" and "expensive," the more an agent starts to earn its price. Working through them honestly puts a given problem further to the left than the initial ask suggests, more often than not.
If the conclusion is genuinely that you need an agent, the instinct to build one that does everything is still probably wrong. Small agents with tight scopes are easier to evaluate, debug, secure and trust. Before you build anything, consider one key scenario: what is the worst thing this agent could plausibly do? Because smaller agents have more definable and limited failure modes, identifying this worst-case scenario directly shapes your tool boundaries, moving you away from fragile, capability-first prototypes and toward reliable, failure-first systems you can safely deploy in a regulated environment.
2. Culture & Non-Determinism: Non-determinism breaks your engineering culture
Even with the correct architecture in place, the next compounding challenge hits: your engineering team is rarely prepared for how non-deterministic systems behave. Software teams have spent thirty years building habits around deterministic systems. Write the code, write the test, green build, ship. Agents don't behave that way. The same prompt can produce different outputs. A tool call that succeeded twice can fail the third time in a way that's hard to reproduce. Traditional CI and incident response assume a level of reproducibility that simply isn't there.
That forces a cultural shift teams often underestimate. You need engineers comfortable reasoning in distributions rather than assertions, product managers who define success as "acceptable behaviour across a population of interactions," and leadership that accepts a probabilistic quality bar without treating every anomaly as a P1 incident.
This cultural shift directly feeds the final layer of complexity: evaluation. Once you are shipping distributions instead of assertions, the question of whether the system is good enough cannot be solved statically during development. Instead, it requires a continuous loop in production.
3. Evaluation: The gap you cannot close during development
Evaluation has an underlying epistemic problem: you cannot think of all the edge cases during development, because you don't yet know what real users will do. Test cases cover the situations you can imagine, which are biased toward the ones the system already handles well.
Synthetic test sets help, but only so far. One workshop with a domain expert to capture the underlying business rules and a small set of real examples, then generate a much larger synthetic test set from those. The agent runs against it, outputs checked in three layers: deterministic assertions for anything with a clear right answer, an LLM-as-judge for fuzzier cases, and only the genuinely uncertain outputs routed to the domain expert. Everything else the system evaluates itself.
The two loops, and where the domain expert fits into each, look like this:
The three orange nodes are the same person. The goal is to keep their involvement targeted, consequential at each point but never continuous.
The right half of the diagram is what stops the offline evaluation from going stale. Real users will always find cases nobody thought to write down. That is why you need the second loop running continuously in production: reference-free checks on live traffic, covering groundedness, safety and self-consistency, that require no golden answer to run, plus a small stream of flagged and sampled traces that get reviewed and fed back into the golden set. The next release runs against the improved set. Without that loop, the offline evaluation quietly drifts away from what actually happens with real users.
Making any of this work requires proper observability: trace-level visibility into every prompt, retrieval, tool call and retry, and the ability to replay a full session end to end. Teams that treat this as optional spend months guessing what went wrong and cannot close the loop. If the ratio of outputs the system can self-evaluate is not improving week on week, the loop is not working.
The boring parts
The pattern is fairly consistent. The gap between the demo and production shows up as an architectural problem, a cultural problem, and an evaluation problem, roughly in that order, each compounding the next.
The teams winning with agents right now aren't the ones with the flashiest capabilities. They are the ones who have made peace with the boring parts: choosing the right shape of system, accepting that a non-deterministic system needs a non-deterministic engineering culture, and running the evaluation loop continuously rather than treating it as something you do once before launch.
Engineering discipline is what ships agents. The model is the easy part. The data, the guardrails, the evaluation loop and the organisational readiness are where the real work lives, and where the value actually gets captured.
That's the perspective from production. It's less exciting than the demo, and it's also the version that pays.
.png)
.png)
