From Black Box to Glass Box: Engineering Control into Production AI Agents

Even top models solve the same task reliably only half the time. Eight proven patterns make your AI agents production-ready.
by
Samuel Pujade-Renaud

AI agents have reached such an advanced level of understanding and reasoning that we could be tempted to blindly give them full control of our automation systems. Agent SDKs have matured to the point that, with a wide list of tools and huge prompts, you can set up a pipeline in a day. That speed comes with a high risk: hand the agents the keys, and you inherit their variance.

Think of a modern airliner. The pilot flies, but a flight computer keeps the aircraft inside a safe envelope. Agentic systems need the same split, with the LLM as the pilot and the code as the envelope. Without it, two problems show up:

  • Across repeated runs, reliability drops. On τ²-bench [1], a benchmark simulating multi-turn customer-service tasks with tool use, Claude Opus 4.5 succeeds on all four repeated attempts of the same retail task (pass^4) only about half the time (51.8%, leaderboard accessed October 2026) [2].
  • Multi-agent systems fail in predictable ways. One study sorted the failures into 14 different categories and 3 main clusters: (i) system design issues, (ii) inter-agent misalignment, and (iii) task verification [3]. Most are engineering problems, not model problems.

‍

To solve these failures, the code must own the workflow, the inputs, the verification and the uncertainty, while the LLM must own the reasoning.

This discipline is sometimes called harness engineering. The model does the flying, the harness is everything around it: the control flow, the tools, the checks and the budgets. As a Machine Learning Engineer at Merantix Momentum, I move agentic systems out of the sandbox and into real-world production. Here are eight harness patterns we use to engineer control, reliability and safety into them.

To show how these patterns work in practice, we’ll follow an illustrative example: a compliance agent hat checks hardware components against regional safety and environmental standards. Simply handing an LLM the product specs and a database of laws would be risky: one component could be marked compliant on one run and non-compliant on another, each time with a confident justification. Here is how we can engineer control into that pipeline.

Workflow over autonomy: deterministic orchestration

First, ask whether you know the workflow path in advance. If you don’t (open-ended research, debugging…), an LLM orchestrator that plans and executes its own next steps shines, but that flexibility costs you predictability and makes errors harder to catch. If you can draw the flowchart before the run starts, letting the model rediscover it on every run adds variance. Instead, keep transitions between stages deterministic and give the agent bounded freedom within each one. Two designs fit this approach:

  • A Directed Acyclic Graph (DAG) workflow allows us to deterministically set the control flow using static nodes and edges. It works great with predictable or linear pipelines like a standard Retrieval Augmented Generation (RAG) pipeline.
  • A finite-state machine (FSM) enables cycles through states given certain conditions, allowing iteration, error correction and retry loops for semi-autonomous agents.
Figure 1: A Finite-State Machine (FSM) architecture for a Regulatory Compliance Agent

In the Compliance Agent, an FSM prevents the LLM from going off-task. We define rigid states: Extract_Materials -> Retrieve_Regulations -> Assess_Compliance. The LLM reasons and may iterate within a state, but the deterministic code decides transitions based on validated fields in its outputs.

Literature and benchmark tasks demonstrate how moving from a pure stochastic ReAct (Reason + Act) [4] loop to a Finite-State Machine architecture significantly increases success rate while cutting cost over real-world workflows. The StateFlow framework [5] showed a 28% higher success rate on Decision Making and 13% improvement in task success for Code/Data execution, with respectively 3x and 5x lower cost. Another benefit of fixed stages is having each of them testable and debuggable on its own, making the system much easier to maintain.

Constrain the Action Space: The agent decides what, code decides where

Setting static stages and transitions gives us control over the environment, but it doesn’t guarantee that the action conditions are correctly met by the agent. Actions trigger transitions based on the LLM’s structured output, which might be wrongly formatted or go beyond the expected action space. An LLM filling tool parameters itself could propose an invalid value or, worse, a valid but wrong value (e.g., “Region: UK” instead of “Region: EU”) causing the system to fail silently.

The code must be in control of what the LLM is allowed to use. In the Compliance Agent, when the model flags a component, we don't let it freely type out the region or the severity level. Instead, we implement specific protections:

  • Parameter Binding: Pass parameters (Region: EU) directly from the request. Never let the model generate a value the system already knows.
  • Least privilage tool scoping: Each agent is granted only the smallest possible set of tools that it needs in its current state.
  • Closed-set selection: Declare the verdict as an enum ((COMPLIANT, NON_COMPLIANT, NEEDS_REVIEW)) and enforce it with structured outputs, so an out-of-set value can’t be generated.

‍

These design patterns ensure fewer silent errors, a smaller attack surface and failures directly linked to a reasoning problem rather than a wiring problem.

Context Engineering: give relevant information to your agent

Context windows have grown dramatically, and even low-priced models like Claude Haiku 5.5 can handle 1M input tokens. This leads to a dangerous temptation: dumping everything in and letting the model find the needle in the haystack. But the advertised window is not the usable one. On NoLiMa [6], most of the 12 long-context models tested fall below half their short-context score at just 32K tokens. Even GPT-4o drops from 99.3% to 69.7%. RULER [7] finds the same gap between claimed and effective context length.

In the Compliance Agent, checking for restricted substances doesn’t mean feeding it all of EU law. We couple it with a retrieval tool built for legal text instead. This principle can be applied to other tool calls like web search. Instead of returning a full 50k-token page content, the tool only outputs windows centered around specific product characteristics (e.g., "lead content in solder") chosen by the agent as keywords. The result is fewer tokens used per call, less noise for the model to misread, and reproducible extractions.

Bounded Autonomy: budget and safety net

As mentioned, multi-turn interactions have a risk of decreasing agent performance. Besides hallucinations and getting lost in long contexts, an LLM is prone to overthinking by prioritizing internal thought over real-world actions. On the other hand, low-overthinking trajectories are estimated to yield ~30% better performance at 43% less compute [8].

If a supplier's datasheet is missing, the unbounded Compliance Agent might burn through its context window repeatedly searching for the same absent information. A concrete goal is to reduce the number of reasoning turns per agent using strict boundaries:

  • Turn budget with last-turn notice: Limit the number of reasoning turns used by the agent, and add a last-turn notice to ensure the agent outputs a complete result before the hard stop.
  • Budget-aware tool use: Set a call limit per tool and show the remaining budget each round, pushing the agent to optimize its strategy [9].
  • Finalizer Agent: Acts as a safety net. If our compliance agent runs out of turns looking for a missing datasheet, the Finalizer steps in, aggregates the existing evidence, and safely exits the workflow.

‍

Apart from capping overthinking and lowering costs, these guardrails provide control over latency and spend per run, while avoiding empty-handed ends.

Structure the reasoning: Evidence before answers

A common assumption is that LLM reasoning is a black box: you can adjust the prompt, but not the thinking itself. One lever is often overlooked as a way to directly impact the reasoning: output schemas.

Output format and completeness are crucial for agentic reasoning. We can force the Compliance Agent’s output schema to provide the evidence: it must cite the exact regulatory clause (e.g., Article 4, paragraph 1) and the matching material spec before the conclusion. Models generate text left to right, so the order of fields in your schema is the order in which the model reasons.

Adding this structured list of found facts into the output schema forces the LLM to look for evidence within its context. As a highly positive side effect, it produces an instant audit trail for human reviewers. On ELI5 benchmark, even the best models lack full citation support 50% of the time [10]. If an agent cites a source or tool it never actually queried during the run, the system can instantly flag it as fabricated evidence.

Verify outside the model: Trust, but check in code

LLMs favour their own outputs when grading them [11], and self-critique can degrade performance Even using a secondary model doesn’t prevent it from reproducing the same reasoning mistake or data gap. Always prefer sound external verification, like ground-truth validation against reference data, deterministic repair, or schema validators.

In the Compliance Agent, every violation outputs a code to be looked up in the official risk taxonomy. If the LLM hallucinates a non-existent code, the deterministic code validator catches it. One corrective retry is triggered with valid nearest-neighbor options listed. If the check itself can’t run (loading or timeout error), the code wasn’t validated and it must be treated as “unverified”, separate from both pass and fail. If the corrected code is still not valid, the hallucinated code is flagged with a fallback value before it ever reaches downstream systems unnoticed.

Use Variance as a Signal: Consensus and calibrated uncertainty

Run-to-run variance is usually treated as noise. It's actually information.

The first way to make use of this variance is majority voting as a “Wisdom of the Crowds”. By generating multiple samples from parallel runs and aggregating the results, you can increase reliability and reduce hallucinations. However, this multiplies costs and does not guarantee high accuracy if a systematic flaw exists.

A cheaper alternative redundancy and reconciliation. In the Compliance Agent, two assessment agents run in parallel on the same component instead of relying on one agent’s reasoning. Each agent writes its own tool queries, so they take different paths. One searches for “lead content limits”, the other for “solder exemptions”, and they might reach different conclusions. When they disagree, a reconciler agent steps in, reasons over the combined evidence from both and makes a fresh classification. Unlike self-grading, the reconciler gets more evidence, not a second opinion.

Figure 2: Redundancy and reconciliation architecture

If the reconciler can't settle the disagreement, the answer is never dropped. The component still receives a complete assessment, flagged for human review with reasons attached: "agents disagreed on the lead exemption", "datasheet missing". In this design, disagreement isn’t a failure. It is a mechanism to catch a broader search that tends to produce a more robust final answer. It also provides a stronger reliability signal than self-reported confidence scores, as LLMs tend to lean towards overconfidence [11]. Even if an agreement doesn’t prove that the answer is correct, using a different prompt, retrieval strategy or model for each agent makes it less likely that both make the same mistake.

Operate it like Software: Observability & evaluation

Every control in this article produces information: a failed validation, a budget hit, a disagreement between agents, a fact without a source. In a black box, these signals disappear into logs nobody reads. Treated as engineering data, they become a clear view of how your system really behaves. That means applying the discipline of ordinary software engineering:

  • Tracing: Record a span for every agent and tool call, tracking inputs, outputs, tokens, turns used, and whether the budget was hit.
  • Per-run cost accounting: Count LLM tokens plus external API fees, including for runs that failed. Report unknown prices as unknown, not as zero.
  • Reliability evaluation: Measure pass^k over repeated runs rather than single-shot accuracy.
  • Deterministic tests: Mock the model, use fixtures for external providers, and write unit tests for all components. Assert at startup that every input the system accepts has what it needs downstream (e.g., regulation index for every supported region).
  • Versioning: Pin the model snapshot, version prompts and schemas, and log every version in each trace.

‍

Finally, evaluation is also a manual process. Check your traces on failing or unreliable examples to see exactly where the workflow failed. Treat metrics as signals, but don’t skip the manual review, even if it takes time. This is where you’ll learn the most.

Control is about understanding failure, not avoiding it

The patterns that we detailed in this article won't make your model smarter, but they will let you trust it. They turn a black box into a glass box: honest, bounded in cost and time, explicit about its evidence and open about its uncertainties.

When designing a new pipeline from scratch at Merantix Momentum, these are eight questions we ask to ensure production-readiness:

  • Does the control flow live in the code or in the prompt?
  • Does the model generate any value the system already knows?
  • Do your tools return targeted extracts rather than whole documents?
  • Are there budgets, do your agents know about them, and what happens when they run out?
  • Can every claim in the output be sourced from a fact?
  • Is every verifiable output checked against ground truth in code?
  • When agents disagree or confidence is low, is the result flagged or quietly resolved?
  • Can you trace any single run end to end and say what it cost?

‍

The harness you build around the model is what makes the difference between an impressive MVP that only works on happy paths, and a production-level agentic system that is robust, efficient, and maintainable. Controlling your agent doesn’t mean limiting it. It’s about making every answer explainable and visibly wrong when it’s wrong. 



Sources

1. Barres et al., 2025, “τ²-Bench: Evaluating Conversational Agents in a Dual-Control Environment,” arXiv:2506.07982

2. Sierra Research, τ²-bench leaderboard, https://taubench.com/leaderboard (accessed October 2026)

3. Cemri et al., 2025, “Why Do Multi-Agent LLM Systems Fail?”, arXiv:2503.13657

4. Yao et al., 2022, ReAct, arXiv:2210.03629

5. Wu et al., 2024, StateFlow, arXiv:2403.11322

6. Modarressi et al., 2025, NoLiMa: Long-Context Evaluation Beyond Literal Matching, arXiv:2502.05167

7. Hsieh et al., 2024, RULER: What's the Real Context Size of Your Long-Context Language Models?, arXiv:2404.06654

8. Cuadron et al., 2025, “The Danger of Overthinking,” arXiv:2502.08235

9. Liu et al., 2025, “Budget-Aware Tool Use Enables Effective Agent Scaling,” arXiv:2511.17006

10. Xiong et al., 2023, “Can LLMs Express Their Uncertainty?”, arXiv:2306.13063

11. Panickssery, Bowman & Feng, 2024, “LLM Evaluators Recognize and Favor Their Own Generations,” arXiv:2404.13076

‍

Subscribe to the Merantix Momentum Newsletter now.

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.

More articles

The latest industry news, interviews, technologies and resources.

Turning AI challenges into real effects – together

How we can transform complex challenges in healthcare and pharmaceuticals into real impact through responsible AI, strong collaboration, and digital leadership.

"AI in Market Access: Do We Have What It Takes?"

Interview with Menasheh Fogel - On AI in Market Access, Organizational Challenges, and the Future of Data-Driven Pricing and Reimbursement Strategies.

From bottleneck to breakthrough: Why AI is crucial for progress in healthcare and pharma

Rising costs, a shortage of specialists and a flood of data are pushing the healthcare system and the pharmaceutical industry to their limits. Why artificial intelligence is now the key to taking care, research and development to a new level.

What We've Learned from Building AI Copilots

From the Copilot demo to real changes in how we work: What we’ve learned from real-world experience

From data chaos to breakthroughs

An expert interview with Dr. Stephan Hegge, VP of Corporate Strategy at HotSpot Therapeutics and Dr. Thomas Wollmann, CTO at Merantix Momentum

From Black Box to Glass Box: Engineering Control into Production AI Agents

AI agents have reached such an advanced level of understanding and reasoning that we could be tempted to blindly give them full control of our automation systems. Agent SDKs have matured to the point that, with a wide list of tools and huge prompts, you can set up a pipeline in a day. That speed comes with a high risk: hand the agents the keys, and you inherit their variance.

Think of a modern airliner. The pilot flies, but a flight computer keeps the aircraft inside a safe envelope. Agentic systems need the same split, with the LLM as the pilot and the code as the envelope. Without it, two problems show up:

  • Across repeated runs, reliability drops. On τ²-bench [1], a benchmark simulating multi-turn customer-service tasks with tool use, Claude Opus 4.5 succeeds on all four repeated attempts of the same retail task (pass^4) only about half the time (51.8%, leaderboard accessed October 2026) [2].
  • Multi-agent systems fail in predictable ways. One study sorted the failures into 14 different categories and 3 main clusters: (i) system design issues, (ii) inter-agent misalignment, and (iii) task verification [3]. Most are engineering problems, not model problems.

‍

To solve these failures, the code must own the workflow, the inputs, the verification and the uncertainty, while the LLM must own the reasoning.

This discipline is sometimes called harness engineering. The model does the flying, the harness is everything around it: the control flow, the tools, the checks and the budgets. As a Machine Learning Engineer at Merantix Momentum, I move agentic systems out of the sandbox and into real-world production. Here are eight harness patterns we use to engineer control, reliability and safety into them.

To show how these patterns work in practice, we’ll follow an illustrative example: a compliance agent hat checks hardware components against regional safety and environmental standards. Simply handing an LLM the product specs and a database of laws would be risky: one component could be marked compliant on one run and non-compliant on another, each time with a confident justification. Here is how we can engineer control into that pipeline.

Workflow over autonomy: deterministic orchestration

First, ask whether you know the workflow path in advance. If you don’t (open-ended research, debugging…), an LLM orchestrator that plans and executes its own next steps shines, but that flexibility costs you predictability and makes errors harder to catch. If you can draw the flowchart before the run starts, letting the model rediscover it on every run adds variance. Instead, keep transitions between stages deterministic and give the agent bounded freedom within each one. Two designs fit this approach:

  • A Directed Acyclic Graph (DAG) workflow allows us to deterministically set the control flow using static nodes and edges. It works great with predictable or linear pipelines like a standard Retrieval Augmented Generation (RAG) pipeline.
  • A finite-state machine (FSM) enables cycles through states given certain conditions, allowing iteration, error correction and retry loops for semi-autonomous agents.
Figure 1: A Finite-State Machine (FSM) architecture for a Regulatory Compliance Agent

In the Compliance Agent, an FSM prevents the LLM from going off-task. We define rigid states: Extract_Materials -> Retrieve_Regulations -> Assess_Compliance. The LLM reasons and may iterate within a state, but the deterministic code decides transitions based on validated fields in its outputs.

Literature and benchmark tasks demonstrate how moving from a pure stochastic ReAct (Reason + Act) [4] loop to a Finite-State Machine architecture significantly increases success rate while cutting cost over real-world workflows. The StateFlow framework [5] showed a 28% higher success rate on Decision Making and 13% improvement in task success for Code/Data execution, with respectively 3x and 5x lower cost. Another benefit of fixed stages is having each of them testable and debuggable on its own, making the system much easier to maintain.

Constrain the Action Space: The agent decides what, code decides where

Setting static stages and transitions gives us control over the environment, but it doesn’t guarantee that the action conditions are correctly met by the agent. Actions trigger transitions based on the LLM’s structured output, which might be wrongly formatted or go beyond the expected action space. An LLM filling tool parameters itself could propose an invalid value or, worse, a valid but wrong value (e.g., “Region: UK” instead of “Region: EU”) causing the system to fail silently.

The code must be in control of what the LLM is allowed to use. In the Compliance Agent, when the model flags a component, we don't let it freely type out the region or the severity level. Instead, we implement specific protections:

  • Parameter Binding: Pass parameters (Region: EU) directly from the request. Never let the model generate a value the system already knows.
  • Least privilage tool scoping: Each agent is granted only the smallest possible set of tools that it needs in its current state.
  • Closed-set selection: Declare the verdict as an enum ((COMPLIANT, NON_COMPLIANT, NEEDS_REVIEW)) and enforce it with structured outputs, so an out-of-set value can’t be generated.

‍

These design patterns ensure fewer silent errors, a smaller attack surface and failures directly linked to a reasoning problem rather than a wiring problem.

Context Engineering: give relevant information to your agent

Context windows have grown dramatically, and even low-priced models like Claude Haiku 5.5 can handle 1M input tokens. This leads to a dangerous temptation: dumping everything in and letting the model find the needle in the haystack. But the advertised window is not the usable one. On NoLiMa [6], most of the 12 long-context models tested fall below half their short-context score at just 32K tokens. Even GPT-4o drops from 99.3% to 69.7%. RULER [7] finds the same gap between claimed and effective context length.

In the Compliance Agent, checking for restricted substances doesn’t mean feeding it all of EU law. We couple it with a retrieval tool built for legal text instead. This principle can be applied to other tool calls like web search. Instead of returning a full 50k-token page content, the tool only outputs windows centered around specific product characteristics (e.g., "lead content in solder") chosen by the agent as keywords. The result is fewer tokens used per call, less noise for the model to misread, and reproducible extractions.

Bounded Autonomy: budget and safety net

As mentioned, multi-turn interactions have a risk of decreasing agent performance. Besides hallucinations and getting lost in long contexts, an LLM is prone to overthinking by prioritizing internal thought over real-world actions. On the other hand, low-overthinking trajectories are estimated to yield ~30% better performance at 43% less compute [8].

If a supplier's datasheet is missing, the unbounded Compliance Agent might burn through its context window repeatedly searching for the same absent information. A concrete goal is to reduce the number of reasoning turns per agent using strict boundaries:

  • Turn budget with last-turn notice: Limit the number of reasoning turns used by the agent, and add a last-turn notice to ensure the agent outputs a complete result before the hard stop.
  • Budget-aware tool use: Set a call limit per tool and show the remaining budget each round, pushing the agent to optimize its strategy [9].
  • Finalizer Agent: Acts as a safety net. If our compliance agent runs out of turns looking for a missing datasheet, the Finalizer steps in, aggregates the existing evidence, and safely exits the workflow.

‍

Apart from capping overthinking and lowering costs, these guardrails provide control over latency and spend per run, while avoiding empty-handed ends.

Structure the reasoning: Evidence before answers

A common assumption is that LLM reasoning is a black box: you can adjust the prompt, but not the thinking itself. One lever is often overlooked as a way to directly impact the reasoning: output schemas.

Output format and completeness are crucial for agentic reasoning. We can force the Compliance Agent’s output schema to provide the evidence: it must cite the exact regulatory clause (e.g., Article 4, paragraph 1) and the matching material spec before the conclusion. Models generate text left to right, so the order of fields in your schema is the order in which the model reasons.

Adding this structured list of found facts into the output schema forces the LLM to look for evidence within its context. As a highly positive side effect, it produces an instant audit trail for human reviewers. On ELI5 benchmark, even the best models lack full citation support 50% of the time [10]. If an agent cites a source or tool it never actually queried during the run, the system can instantly flag it as fabricated evidence.

Verify outside the model: Trust, but check in code

LLMs favour their own outputs when grading them [11], and self-critique can degrade performance Even using a secondary model doesn’t prevent it from reproducing the same reasoning mistake or data gap. Always prefer sound external verification, like ground-truth validation against reference data, deterministic repair, or schema validators.

In the Compliance Agent, every violation outputs a code to be looked up in the official risk taxonomy. If the LLM hallucinates a non-existent code, the deterministic code validator catches it. One corrective retry is triggered with valid nearest-neighbor options listed. If the check itself can’t run (loading or timeout error), the code wasn’t validated and it must be treated as “unverified”, separate from both pass and fail. If the corrected code is still not valid, the hallucinated code is flagged with a fallback value before it ever reaches downstream systems unnoticed.

Use Variance as a Signal: Consensus and calibrated uncertainty

Run-to-run variance is usually treated as noise. It's actually information.

The first way to make use of this variance is majority voting as a “Wisdom of the Crowds”. By generating multiple samples from parallel runs and aggregating the results, you can increase reliability and reduce hallucinations. However, this multiplies costs and does not guarantee high accuracy if a systematic flaw exists.

A cheaper alternative redundancy and reconciliation. In the Compliance Agent, two assessment agents run in parallel on the same component instead of relying on one agent’s reasoning. Each agent writes its own tool queries, so they take different paths. One searches for “lead content limits”, the other for “solder exemptions”, and they might reach different conclusions. When they disagree, a reconciler agent steps in, reasons over the combined evidence from both and makes a fresh classification. Unlike self-grading, the reconciler gets more evidence, not a second opinion.

Figure 2: Redundancy and reconciliation architecture

If the reconciler can't settle the disagreement, the answer is never dropped. The component still receives a complete assessment, flagged for human review with reasons attached: "agents disagreed on the lead exemption", "datasheet missing". In this design, disagreement isn’t a failure. It is a mechanism to catch a broader search that tends to produce a more robust final answer. It also provides a stronger reliability signal than self-reported confidence scores, as LLMs tend to lean towards overconfidence [11]. Even if an agreement doesn’t prove that the answer is correct, using a different prompt, retrieval strategy or model for each agent makes it less likely that both make the same mistake.

Operate it like Software: Observability & evaluation

Every control in this article produces information: a failed validation, a budget hit, a disagreement between agents, a fact without a source. In a black box, these signals disappear into logs nobody reads. Treated as engineering data, they become a clear view of how your system really behaves. That means applying the discipline of ordinary software engineering:

  • Tracing: Record a span for every agent and tool call, tracking inputs, outputs, tokens, turns used, and whether the budget was hit.
  • Per-run cost accounting: Count LLM tokens plus external API fees, including for runs that failed. Report unknown prices as unknown, not as zero.
  • Reliability evaluation: Measure pass^k over repeated runs rather than single-shot accuracy.
  • Deterministic tests: Mock the model, use fixtures for external providers, and write unit tests for all components. Assert at startup that every input the system accepts has what it needs downstream (e.g., regulation index for every supported region).
  • Versioning: Pin the model snapshot, version prompts and schemas, and log every version in each trace.

‍

Finally, evaluation is also a manual process. Check your traces on failing or unreliable examples to see exactly where the workflow failed. Treat metrics as signals, but don’t skip the manual review, even if it takes time. This is where you’ll learn the most.

Control is about understanding failure, not avoiding it

The patterns that we detailed in this article won't make your model smarter, but they will let you trust it. They turn a black box into a glass box: honest, bounded in cost and time, explicit about its evidence and open about its uncertainties.

When designing a new pipeline from scratch at Merantix Momentum, these are eight questions we ask to ensure production-readiness:

  • Does the control flow live in the code or in the prompt?
  • Does the model generate any value the system already knows?
  • Do your tools return targeted extracts rather than whole documents?
  • Are there budgets, do your agents know about them, and what happens when they run out?
  • Can every claim in the output be sourced from a fact?
  • Is every verifiable output checked against ground truth in code?
  • When agents disagree or confidence is low, is the result flagged or quietly resolved?
  • Can you trace any single run end to end and say what it cost?

‍

The harness you build around the model is what makes the difference between an impressive MVP that only works on happy paths, and a production-level agentic system that is robust, efficient, and maintainable. Controlling your agent doesn’t mean limiting it. It’s about making every answer explainable and visibly wrong when it’s wrong. 



Sources

1. Barres et al., 2025, “τ²-Bench: Evaluating Conversational Agents in a Dual-Control Environment,” arXiv:2506.07982

2. Sierra Research, τ²-bench leaderboard, https://taubench.com/leaderboard (accessed October 2026)

3. Cemri et al., 2025, “Why Do Multi-Agent LLM Systems Fail?”, arXiv:2503.13657

4. Yao et al., 2022, ReAct, arXiv:2210.03629

5. Wu et al., 2024, StateFlow, arXiv:2403.11322

6. Modarressi et al., 2025, NoLiMa: Long-Context Evaluation Beyond Literal Matching, arXiv:2502.05167

7. Hsieh et al., 2024, RULER: What's the Real Context Size of Your Long-Context Language Models?, arXiv:2404.06654

8. Cuadron et al., 2025, “The Danger of Overthinking,” arXiv:2502.08235

9. Liu et al., 2025, “Budget-Aware Tool Use Enables Effective Agent Scaling,” arXiv:2511.17006

10. Xiong et al., 2023, “Can LLMs Express Their Uncertainty?”, arXiv:2306.13063

11. Panickssery, Bowman & Feng, 2024, “LLM Evaluators Recognize and Favor Their Own Generations,” arXiv:2404.13076

‍

Oops! Something has gone wrong.
Oops! Something has gone wrong.
Oops! Something has gone wrong.
Oops! Something has gone wrong.
Oops! Something has gone wrong.

Discover more whitepapers

Data-driven to the drug of tomorrow

Opportunities and barriers of AI in a GxP world.

The AI Canvas: Our tool for project evaluation

Discover the AI Canvas!

Artificial Intelligence for Private Equity Portfolios

Increase in value for your entire portfolio

Data-driven to the drug of tomorrow

Opportunities and barriers of AI in a GxP world.

Artificial Intelligence for Private Equity Portfolios

Increase in value for your entire portfolio

The AI Canvas: Our tool for project evaluation

Discover the AI Canvas!

Subscribe to the Merantix Momentum Newsletter now.

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.