V
Vikrant Bhalodia
Guest
AI agents are becoming remarkably good at making decisions. Give one a business objective, connect it to a few APIs, and it can decide which tools to call, what information to retrieve, how to process the results, and what to do next. For developers, this removes much of the conditional logic that traditionally had to be written into applications.
But there is a problem with giving an AI agent too much freedom. Every decision has a cost, and that cost is not always visible in the final output. An agent might complete a task correctly while consuming unnecessary tokens, repeating database queries, triggering redundant API calls, or spending several seconds deciding something that a simple rule could have handled immediately.
The industry spends considerable time discussing what AI agents can do independently. We spend less time asking how many decisions they should actually be allowed to make. That distinction matters because greater autonomy does not automatically produce better software.
Sometimes the smartest engineering decision is to stop the agent from deciding.
Consider an AI agent designed to process customer support requests. It receives a message, identifies the customer's issue, retrieves account information, checks the company's policies, and recommends a response. In a fully autonomous setup, the agent might decide which database to query, whether another search is necessary, which model to use, and whether it has enough information to respond.
That flexibility sounds useful, especially when customer requests vary widely. Yet many support workflows contain predictable steps. Account verification, order lookup, refund eligibility checks, and ticket creation usually follow established business rules. Allowing a language model to reconsider these steps during every request introduces decision-making overhead without necessarily improving the outcome.
A more controlled architecture could use deterministic application logic for account verification and policy enforcement while reserving model reasoning for interpreting ambiguous customer requests. The agent still contributes where language understanding matters, but the application does not ask it to rediscover the same workflow every time.
This distinction is central to agent design. A task can involve many operations without requiring an independent AI decision at every stage. Developers should separate work that genuinely benefits from reasoning from work that is already governed by reliable rules.
An AI agent typically operates through a repeating cycle. It examines the current task, selects an action, calls a tool, evaluates the result, and decides what to do next. Depending on the architecture, each cycle may require another language model request.
This is useful when the task is unpredictable. It becomes expensive when the agent repeatedly evaluates choices that could have been resolved earlier.
Imagine a reporting agent asked to summarize quarterly sales performance. It first decides to retrieve revenue data, then decides to retrieve regional data, then checks customer segments, and finally requests historical comparisons. If each decision requires another model call, the workflow accumulates inference costs and response delays.
The expense is not limited to the number of calls. As the workflow progresses, the agent may send earlier messages, tool outputs, and intermediate results back to the model. Context can grow with every iteration, increasing input-token consumption even when the next decision is relatively simple.
Nilesh Bhandarwar’s HackerNoon article on building AI agents and their workflow architecture explains how repeated reasoning and tool execution form the basis of many agent systems. That same architecture makes it important to distinguish necessary reasoning from avoidable repetition.
A practical alternative is to retrieve known data requirements through a predefined workflow and use the model to interpret the combined results. This reduces the number of decisions without eliminating the agent's ability to analyze complex information.
Model pricing receives most of the attention when teams estimate AI-agent operating expenses. Token consumption is easy to measure, and providers publish pricing that makes basic calculations straightforward. Yet inference is only one part of the cost.
Each agent decision may trigger additional infrastructure activity. A database query consumes compute resources, an external API may charge per request, and a document search may require retrieval and reranking operations. Some workflows also create temporary files, execute code, or call other agents.
Consider two approaches to the same reporting task. The first uses a fixed workflow to retrieve three datasets and asks a model to produce a summary. The second gives an agent access to several tools and allows it to determine the retrieval sequence independently.
The autonomous approach might retrieve the same datasets, but it could also repeat searches, inspect unnecessary tables, or call tools that provide overlapping information. Even if both approaches produce equally useful reports, their operating costs may differ substantially.
Here is a hypothetical comparison for one completed task:
These figures are illustrative, not measured benchmark results. Actual costs depend on models, infrastructure, tool behavior, and workload.
In this example, the autonomous workflow costs more than five times as much to complete the same task. At 100,000 executions, the difference would be approximately $14,000, assuming the illustrative per-task costs remain constant.
The important metric is not the price of one model call. It is the cost of completing a successful business task, including everything the agent does along the way.
Latency is another consequence of excessive agent autonomy. A conventional application can execute independent operations concurrently when their dependencies are known. An agent that decides what to do after every result may turn those operations into a sequential chain.
Suppose an application needs customer information, order history, and payment status. If those requests are independent, the application can issue them concurrently and process the responses together. An agent might instead retrieve customer information, inspect it, decide to request order history, and only then determine that payment information is necessary.
Each reasoning step introduces waiting time. The final response depends not only on the speed of the underlying APIs but also on how quickly the model selects its next action.
This matters for interactive applications where users expect timely responses. A few additional seconds may be tolerable for complex research, but the same delay can make customer support tools, internal dashboards, and transactional applications frustrating to use.
Developers should identify operations that can be planned ahead of time and reserve sequential agent decisions for cases where the next action genuinely depends on an unknown result. Reducing unnecessary reasoning loops can improve response times without changing the model itself.
Every decision point introduces another opportunity for the agent to choose an unsuitable action. It might select the wrong tool, misinterpret a response, retry an operation unnecessarily, or continue working after enough information has already been collected.
These failures are particularly troublesome when agents interact with systems that modify business data. Reading a customer record is relatively easy to recover from. Issuing a refund, changing account permissions, or updating an inventory quantity can have consequences that require investigation and corrective action.
An agent that retries a failed read operation may simply consume extra resources. An agent that retries a payment operation without checking whether the first request succeeded could create a duplicate transaction.
This is why business-critical actions should rely on application-enforced permissions, idempotency controls, and explicit approval rules rather than model judgment alone. The model may recommend an action, but the surrounding system must determine whether that action is permitted and safe to execute.
Siddharth Chauhan’s HackerNoon article, I Taught an AI Agent How to Undo Its Mistakes, explores the engineering challenge of handling actions that need compensation after something goes wrong. The underlying lesson is that recovery needs to be designed into the workflow rather than left to an agent's next guess.
Giving an agent more choices increases the importance of defining which actions it can take, what conditions must be satisfied, and when execution must stop.
The decision-making overhead becomes more complicated when one agent delegates work to another. A planning agent might ask a research agent to gather information, send the results to an analysis agent, and then ask a review agent to validate the final answer.
Such arrangements can be useful when tasks require genuinely different capabilities. But dividing a straightforward workflow among several agents can introduce repeated context processing, additional model calls, communication delays, and disagreements about what should happen next.
For example, a research agent may decide that more information is needed while the planning agent considers the task complete. A review agent might reject an answer and send it back for another attempt. Without clear stopping conditions, the system can spend considerable resources revisiting decisions without making meaningful progress.
A better design gives each agent a specific responsibility, a limited set of tools, and a clear definition of completion. Delegation should occur because the task benefits from a separate capability, not simply because the architecture supports multiple agents.
The number of agents is not a measure of system quality. A smaller workflow with predictable responsibilities may be easier to maintain, test, and operate than a collection of agents continuously negotiating their next actions.
One useful way to control agent complexity is to classify decisions according to how much uncertainty they contain.
Some decisions are deterministic. If a customer is not authenticated, the application should not expose account information. If a transaction exceeds an approved limit, it should require authorization. These decisions belong in application logic and policy enforcement systems.
Other decisions involve structured uncertainty. A support request might need to be classified into billing, technical assistance, or account management. A small classifier or a narrowly scoped model call may be sufficient.
A third category requires broader reasoning. Investigating an unfamiliar software failure, interpreting conflicting documents, or planning a complex migration may justify a more capable agent that can select tools and revise its approach.
The goal is to use the least complex decision mechanism that reliably solves the problem.
The challenge is not simply choosing between autonomous agents and traditional application logic. It is identifying which decisions require model reasoning and which should remain governed by predictable rules. This connects with the broader engineering principle of setting strict system boundaries, where defining the problem, limiting system responsibilities, and maintaining human oversight become more important than giving AI unrestricted control. For production systems, these boundaries help developers reduce unnecessary reasoning steps while keeping critical business operations predictable.
This approach does not eliminate autonomy. It places autonomy where its ability to handle uncertainty provides a measurable benefit.
Teams already set limits on API requests, database connections, and infrastructure consumption. AI agents need similar boundaries for their decision-making activity.
A decision budget defines how much work an agent can perform before it must finish, request assistance, or stop safely. The budget can include model calls, tool requests, retries, execution time, and monetary cost.
For example, an internal reporting agent might be allowed six model calls, ten tool requests, and one retry per failed operation. If it reaches those limits without completing the task, the workflow should return a controlled failure or request human review rather than continuing indefinitely.
The limits should reflect the complexity and risk of the task. A research agent investigating an unfamiliar problem may require more steps than an agent retrieving a standard account summary.
A simplified policy might look like this:
This example illustrates admission checks rather than a complete runtime. A production system also needs independent retry counters, deadline enforcement, cancellation handling, and checks before costly or irreversible operations.
Budgets should not become arbitrary restrictions that prevent agents from completing legitimate tasks. Teams need to measure normal workloads, identify expensive outliers, and adjust limits based on observed outcomes.
A busy agent is not necessarily a productive agent. High tool-call counts, lengthy execution traces, and repeated planning steps may indicate unnecessary work rather than greater capability.
Developers should evaluate agents using task completion rates, cost per successful task, response-time distributions, unnecessary tool calls, and the frequency of human intervention. These measurements provide a more meaningful picture than simply counting how many actions an agent performs.
It is also useful to compare autonomous and controlled workflows against the same evaluation dataset. If a highly autonomous agent costs substantially more but produces similar results, the additional freedom may not be justified.
For tasks where autonomy does improve outcomes, the improvement should be measurable. An agent that resolves complex support cases without escalation may justify extra reasoning steps if it reduces overall handling time and maintains acceptable accuracy.
Observability plays an important role here. Teams need traces that connect model calls, tool executions, retries, and final outcomes. Without that visibility, unnecessary decisions can remain hidden inside workflows that appear successful from the outside.
AI agents are useful because they can handle situations that are difficult to express through fixed application rules. They can interpret ambiguous requests, choose among unfamiliar information sources, and adjust their approach when new evidence appears.
Those capabilities are valuable, but they should not become the default mechanism for every software operation. A system that asks a language model to repeatedly decide predictable tasks may be replacing straightforward engineering with unnecessary reasoning overhead.
The better approach is to define where autonomy creates value and where conventional software should remain in control. Known business rules can stay deterministic, predictable data retrieval can follow planned workflows, and uncertain problems can be handled by agents operating within clear boundaries.
As AI agents become part of production software, engineering teams will need to evaluate more than whether an agent can complete a task. They will need to understand how many decisions it makes, what those decisions cost, and whether the same result could have been achieved with a simpler design.
The goal is not to build agents that make the most decisions. It is to build systems that make the right decisions with as little unnecessary work as possible.
But there is a problem with giving an AI agent too much freedom. Every decision has a cost, and that cost is not always visible in the final output. An agent might complete a task correctly while consuming unnecessary tokens, repeating database queries, triggering redundant API calls, or spending several seconds deciding something that a simple rule could have handled immediately.
The industry spends considerable time discussing what AI agents can do independently. We spend less time asking how many decisions they should actually be allowed to make. That distinction matters because greater autonomy does not automatically produce better software.
Sometimes the smartest engineering decision is to stop the agent from deciding.
The Difference Between Making a Decision and Completing a Task
Consider an AI agent designed to process customer support requests. It receives a message, identifies the customer's issue, retrieves account information, checks the company's policies, and recommends a response. In a fully autonomous setup, the agent might decide which database to query, whether another search is necessary, which model to use, and whether it has enough information to respond.
That flexibility sounds useful, especially when customer requests vary widely. Yet many support workflows contain predictable steps. Account verification, order lookup, refund eligibility checks, and ticket creation usually follow established business rules. Allowing a language model to reconsider these steps during every request introduces decision-making overhead without necessarily improving the outcome.
A more controlled architecture could use deterministic application logic for account verification and policy enforcement while reserving model reasoning for interpreting ambiguous customer requests. The agent still contributes where language understanding matters, but the application does not ask it to rediscover the same workflow every time.
This distinction is central to agent design. A task can involve many operations without requiring an independent AI decision at every stage. Developers should separate work that genuinely benefits from reasoning from work that is already governed by reliable rules.
Every Additional Decision Creates Another Model Call
An AI agent typically operates through a repeating cycle. It examines the current task, selects an action, calls a tool, evaluates the result, and decides what to do next. Depending on the architecture, each cycle may require another language model request.
This is useful when the task is unpredictable. It becomes expensive when the agent repeatedly evaluates choices that could have been resolved earlier.
Imagine a reporting agent asked to summarize quarterly sales performance. It first decides to retrieve revenue data, then decides to retrieve regional data, then checks customer segments, and finally requests historical comparisons. If each decision requires another model call, the workflow accumulates inference costs and response delays.
The expense is not limited to the number of calls. As the workflow progresses, the agent may send earlier messages, tool outputs, and intermediate results back to the model. Context can grow with every iteration, increasing input-token consumption even when the next decision is relatively simple.
Nilesh Bhandarwar’s HackerNoon article on building AI agents and their workflow architecture explains how repeated reasoning and tool execution form the basis of many agent systems. That same architecture makes it important to distinguish necessary reasoning from avoidable repetition.
A practical alternative is to retrieve known data requirements through a predefined workflow and use the model to interpret the combined results. This reduces the number of decisions without eliminating the agent's ability to analyze complex information.
The Cost Is Bigger Than Token Usage
Model pricing receives most of the attention when teams estimate AI-agent operating expenses. Token consumption is easy to measure, and providers publish pricing that makes basic calculations straightforward. Yet inference is only one part of the cost.
Each agent decision may trigger additional infrastructure activity. A database query consumes compute resources, an external API may charge per request, and a document search may require retrieval and reranking operations. Some workflows also create temporary files, execute code, or call other agents.
Consider two approaches to the same reporting task. The first uses a fixed workflow to retrieve three datasets and asks a model to produce a summary. The second gives an agent access to several tools and allows it to determine the retrieval sequence independently.
The autonomous approach might retrieve the same datasets, but it could also repeat searches, inspect unnecessary tables, or call tools that provide overlapping information. Even if both approaches produce equally useful reports, their operating costs may differ substantially.
Here is a hypothetical comparison for one completed task:
| Measurement | Controlled workflow | Highly autonomous agent |
|---|---|---|
| Model calls | 2 | 9 |
| Database queries | 3 | 14 |
| External API requests | 2 | 7 |
| Total execution time | 4 seconds | 19 seconds |
| Estimated operating cost | $0.03 | $0.17 |
These figures are illustrative, not measured benchmark results. Actual costs depend on models, infrastructure, tool behavior, and workload.
In this example, the autonomous workflow costs more than five times as much to complete the same task. At 100,000 executions, the difference would be approximately $14,000, assuming the illustrative per-task costs remain constant.
The important metric is not the price of one model call. It is the cost of completing a successful business task, including everything the agent does along the way.
Too Many Decisions Also Make Agents Slower
Latency is another consequence of excessive agent autonomy. A conventional application can execute independent operations concurrently when their dependencies are known. An agent that decides what to do after every result may turn those operations into a sequential chain.
Suppose an application needs customer information, order history, and payment status. If those requests are independent, the application can issue them concurrently and process the responses together. An agent might instead retrieve customer information, inspect it, decide to request order history, and only then determine that payment information is necessary.
Each reasoning step introduces waiting time. The final response depends not only on the speed of the underlying APIs but also on how quickly the model selects its next action.
This matters for interactive applications where users expect timely responses. A few additional seconds may be tolerable for complex research, but the same delay can make customer support tools, internal dashboards, and transactional applications frustrating to use.
Developers should identify operations that can be planned ahead of time and reserve sequential agent decisions for cases where the next action genuinely depends on an unknown result. Reducing unnecessary reasoning loops can improve response times without changing the model itself.
More Freedom Means More Ways to Fail
Every decision point introduces another opportunity for the agent to choose an unsuitable action. It might select the wrong tool, misinterpret a response, retry an operation unnecessarily, or continue working after enough information has already been collected.
These failures are particularly troublesome when agents interact with systems that modify business data. Reading a customer record is relatively easy to recover from. Issuing a refund, changing account permissions, or updating an inventory quantity can have consequences that require investigation and corrective action.
An agent that retries a failed read operation may simply consume extra resources. An agent that retries a payment operation without checking whether the first request succeeded could create a duplicate transaction.
This is why business-critical actions should rely on application-enforced permissions, idempotency controls, and explicit approval rules rather than model judgment alone. The model may recommend an action, but the surrounding system must determine whether that action is permitted and safe to execute.
Siddharth Chauhan’s HackerNoon article, I Taught an AI Agent How to Undo Its Mistakes, explores the engineering challenge of handling actions that need compensation after something goes wrong. The underlying lesson is that recovery needs to be designed into the workflow rather than left to an agent's next guess.
Giving an agent more choices increases the importance of defining which actions it can take, what conditions must be satisfied, and when execution must stop.
Why Multi-Agent Systems Can Multiply the Problem
The decision-making overhead becomes more complicated when one agent delegates work to another. A planning agent might ask a research agent to gather information, send the results to an analysis agent, and then ask a review agent to validate the final answer.
Such arrangements can be useful when tasks require genuinely different capabilities. But dividing a straightforward workflow among several agents can introduce repeated context processing, additional model calls, communication delays, and disagreements about what should happen next.
For example, a research agent may decide that more information is needed while the planning agent considers the task complete. A review agent might reject an answer and send it back for another attempt. Without clear stopping conditions, the system can spend considerable resources revisiting decisions without making meaningful progress.
A better design gives each agent a specific responsibility, a limited set of tools, and a clear definition of completion. Delegation should occur because the task benefits from a separate capability, not simply because the architecture supports multiple agents.
The number of agents is not a measure of system quality. A smaller workflow with predictable responsibilities may be easier to maintain, test, and operate than a collection of agents continuously negotiating their next actions.
Not Every Decision Needs a Language Model
One useful way to control agent complexity is to classify decisions according to how much uncertainty they contain.
Some decisions are deterministic. If a customer is not authenticated, the application should not expose account information. If a transaction exceeds an approved limit, it should require authorization. These decisions belong in application logic and policy enforcement systems.
Other decisions involve structured uncertainty. A support request might need to be classified into billing, technical assistance, or account management. A small classifier or a narrowly scoped model call may be sufficient.
A third category requires broader reasoning. Investigating an unfamiliar software failure, interpreting conflicting documents, or planning a complex migration may justify a more capable agent that can select tools and revise its approach.
The goal is to use the least complex decision mechanism that reliably solves the problem.
The challenge is not simply choosing between autonomous agents and traditional application logic. It is identifying which decisions require model reasoning and which should remain governed by predictable rules. This connects with the broader engineering principle of setting strict system boundaries, where defining the problem, limiting system responsibilities, and maintaining human oversight become more important than giving AI unrestricted control. For production systems, these boundaries help developers reduce unnecessary reasoning steps while keeping critical business operations predictable.
| Decision type | Suitable approach | Example |
|---|---|---|
| Fixed business rule | Deterministic code | Enforcing a refund limit |
| Structured classification | Classifier or focused model call | Categorizing a support ticket |
| Known sequence of operations | Predefined workflow | Retrieving data for a standard report |
| Uncertain multi-step problem | Bounded AI agent | Investigating an unfamiliar production failure |
| High-impact business action | Policy enforcement and human approval | Authorizing a large financial adjustment |
This approach does not eliminate autonomy. It places autonomy where its ability to handle uncertainty provides a measurable benefit.
How to Put a Decision Budget Around an AI Agent
Teams already set limits on API requests, database connections, and infrastructure consumption. AI agents need similar boundaries for their decision-making activity.
A decision budget defines how much work an agent can perform before it must finish, request assistance, or stop safely. The budget can include model calls, tool requests, retries, execution time, and monetary cost.
For example, an internal reporting agent might be allowed six model calls, ten tool requests, and one retry per failed operation. If it reaches those limits without completing the task, the workflow should return a controlled failure or request human review rather than continuing indefinitely.
The limits should reflect the complexity and risk of the task. A research agent investigating an unfamiliar problem may require more steps than an agent retrieving a standard account summary.
A simplified policy might look like this:
Code:
AGENT_LIMITS = {
"max_model_calls": 6,
"max_tool_calls": 10,
"max_retries_per_tool": 1,
"max_execution_seconds": 30,
"max_cost_usd": 0.25
}
def should_continue(usage):
return (
usage["model_calls"] < AGENT_LIMITS["max_model_calls"]
and usage["tool_calls"] < AGENT_LIMITS["max_tool_calls"]
and usage["elapsed_seconds"] < AGENT_LIMITS["max_execution_seconds"]
and usage["estimated_cost_usd"] < AGENT_LIMITS["max_cost_usd"]
)This example illustrates admission checks rather than a complete runtime. A production system also needs independent retry counters, deadline enforcement, cancellation handling, and checks before costly or irreversible operations.
Budgets should not become arbitrary restrictions that prevent agents from completing legitimate tasks. Teams need to measure normal workloads, identify expensive outliers, and adjust limits based on observed outcomes.
Measure Useful Decisions, Not Just Agent Activity
A busy agent is not necessarily a productive agent. High tool-call counts, lengthy execution traces, and repeated planning steps may indicate unnecessary work rather than greater capability.
Developers should evaluate agents using task completion rates, cost per successful task, response-time distributions, unnecessary tool calls, and the frequency of human intervention. These measurements provide a more meaningful picture than simply counting how many actions an agent performs.
It is also useful to compare autonomous and controlled workflows against the same evaluation dataset. If a highly autonomous agent costs substantially more but produces similar results, the additional freedom may not be justified.
For tasks where autonomy does improve outcomes, the improvement should be measurable. An agent that resolves complex support cases without escalation may justify extra reasoning steps if it reduces overall handling time and maintains acceptable accuracy.
Observability plays an important role here. Teams need traces that connect model calls, tool executions, retries, and final outcomes. Without that visibility, unnecessary decisions can remain hidden inside workflows that appear successful from the outside.
The Best Agent Is Not Always the Most Autonomous One
AI agents are useful because they can handle situations that are difficult to express through fixed application rules. They can interpret ambiguous requests, choose among unfamiliar information sources, and adjust their approach when new evidence appears.
Those capabilities are valuable, but they should not become the default mechanism for every software operation. A system that asks a language model to repeatedly decide predictable tasks may be replacing straightforward engineering with unnecessary reasoning overhead.
The better approach is to define where autonomy creates value and where conventional software should remain in control. Known business rules can stay deterministic, predictable data retrieval can follow planned workflows, and uncertain problems can be handled by agents operating within clear boundaries.
As AI agents become part of production software, engineering teams will need to evaluate more than whether an agent can complete a task. They will need to understand how many decisions it makes, what those decisions cost, and whether the same result could have been achieved with a simpler design.
The goal is not to build agents that make the most decisions. It is to build systems that make the right decisions with as little unnecessary work as possible.