At scale, the cost of running AI agents is the cost of one completed task multiplied by the number of tasks, not the per-token price on a provider's pricing page. The cost per task depends on how many model calls the agent makes, how much context each call re-sends, which model handles each step, how often steps fail and retry, tool and retrieval fees, and human review time. Surprises usually come from loops, long contexts and retries. This guide gives you the formula, the cost drivers, the levers, the budget caps and a first-month checklist.
An AI agent runs a language model in a loop: it plans a step, calls a tool, reads the result and decides whether to continue (for the difference from a chatbot, see AI agents for business). A token is the unit of text that providers meter and bill; the context is everything the model sees in one call. Build cost and one agent's monthly cost are in how much an AI agent costs a business, so this guide has no price ranges.
What Makes Up the Cost of One AI Agent Task?
One task costs the sum of four lines: model calls, tools and retrieval, human review and a share of fixed cost. Divide by the success rate, because a failed task still cost money.
| What you compute | Formula | What each symbol means |
|---|---|---|
| Model calls | C_calls = Σ_k ( U_k × P_in + K_k × P_cached + O_k × P_out ) | Per call k: regular input, cache-billed input and output tokens, each at the price of the model that ran the step. |
| Cost per attempt | C_attempt = C_calls + C_tools + ( p_review × m × L ) + ( F / N ) | C_tools: tool, search, retrieval, speech and messaging charges. p_review: share of tasks a person reviews. m: review minutes. L: cost of a reviewer's minute. F: fixed monthly cost. N: tasks per month. |
| Cost per resolved task | C_resolved = C_attempt / s | s: share of attempts that end resolved, by a definition agreed in advance. |
The monthly bill is N times C_attempt. The review line never appears on a provider's invoice, so estimates that start from a pricing page forget it, and a small drop in s raises the cost of every resolved task.
Why Does an Agent's Cost Grow Faster Than Its Number of Steps?
Because in a standard agent loop every call re-sends everything before it: instructions, tool definitions, the request and all earlier results.
An illustrative scenario, in symbols only: S is the fixed input of every call (system prompt, tool definitions, task) and d is what each step adds to the history (model output and tool result). Step one's input is S, step two's is S + d, step three's is S + 2d. The growing part is d times 0 + 1 + 2 + ... up to the last step.
| Steps | Fixed part | Growing part | Total input |
|---|---|---|---|
| 4 | 4 × S | 6 × d | 4 × S + 6 × d |
| 8 | 8 × S | 28 × d | 8 × S + 28 × d |
| 16 | 16 × S | 120 × d | 16 × S + 120 × d |
From 4 steps to 8, the fixed part doubles but the growing part rises from 6d to 28d, more than fourfold, and a 16-step loop carries 120d. So average cost per step misleads. Each sentence the model writes in one step is billed as output, then again as input at every later step. Trimming and caching, below, change this curve.
Which Cost Drivers Matter at Volume, and How Do You Measure Each?
Eight drivers worth measuring:
| Driver | Why it grows | How to measure | How to reduce |
|---|---|---|---|
| Model calls per task | Loops, replanning, tool errors, steps that could be code. | Calls per task: median and tail. | Step caps, code instead of a model. |
| Context per call | Re-sent history, long tool results, bloated prompts. Hebrew often takes more tokens than English; the gap varies by model. | Input tokens per step, regular versus cached. | Trimming, focused retrieval, caching. |
| Output length | Verbose answers. With many providers, thinking tokens also bill as output (confirm with yours). | Output and thinking tokens per step. | Short structured output, minimal reasoning on easy steps. |
| Model choice | A strong model on every step. | Calls and tokens by model. | Routing. |
| Retries and failures | Invalid output, timeouts, rate limits. Each retry pays again for the whole context. | Attempts per step, s, failure causes. | Validation, capped retries with backoff. |
| Tools, retrieval, channels | Search, vector stores, OCR, speech and messaging bill by usage, from separate vendors. | Vendor charges per task. | Caching results, skipping needless calls. |
| Human review | Minutes of a person per reviewed task; the queue grows with volume. | p_review, minutes, how often reviewers change the output. | Checks in code first, evidence beside the change. |
| Fixed cost | Infrastructure, monitoring, evaluation runs (which call models too), maintenance. | F per month, F / N per task. | Shared infrastructure, targeted evaluation runs. |
How Do You Cut Agent Costs Without Hurting Quality? Eight Levers in Their Usual Order of Impact
This is the common order, not a law, so measure first. In a routed pipeline the strong model runs only where needed:
- Code for rule steps. Validation, lookups by ID, sums and routing on a fixed field need no model. Trade-off: rules need upkeep and break on messy input.
- Model routing. A small, fast model handles classification, extraction and dispatch; a strong model gets only the steps where the small one fails in testing, with escalation when it is unsure. Trade-off: wrong routing quietly erodes quality. Model prices differ and change often; the Grok 4.6 analysis shows how a new model changes the bill.
- Context discipline. A short system prompt, only the tools a step needs (tool definitions are tokens), a few relevant passages instead of whole documents, summarized history, tool results cut to needed fields. Trade-off: trimming can delete the one detail the model needed, so test on real cases.
- Prefix caching. Each call in an agent loop starts like the previous one, so many providers bill that part at a reduced rate under their own terms (minimum length, cache lifetime, sometimes a write charge). Put stable content first. Trade-off: a small prefix change, such as a timestamp on top, breaks the match; check the hit rate in your logs.
- Structured outputs. A fixed schema cuts parsing failures, retries and output tokens, which usually cost more per token than input. Trade-off: a rigid schema can force a wrong field, so allow "unknown" and escalation.
- Batching and async. Nightly summaries and document backlogs do not need answers in seconds, and many providers price batch work lower for slower turnaround (ask for current terms). Trade-off: delay.
- Early exit and step caps. Stop when done or at the cap, then hand a state summary to a person instead of retrying. Trade-off: hard tasks get cut off, so count them and tune the cap.
- Evaluation. Before swapping a model, trimming context or adding a cache, rerun the same cases and compare cost per resolved task, not per call, as in the guide to evaluating AI agents. Trade-off: evaluation costs money, but less than a production regression.
What Does a Specific Model Cost per Million Tokens?
This article prints no token prices on purpose: they change often, and a number in an article goes stale. Find Grok 4.6's current input and output pricing on its model page, other models on the models index, and confirm with the provider's own price list. On a price card, read input versus output price, cached input price, any long-context tier and separate charges for built-in tools such as search.
An Illustrative Scenario: One Process, Two Designs
An illustrative scenario: synthetic, with a volume assumption and symbolic amounts only, no prices or measured results.
An inbound-request agent handles a thousand tasks a day (an input assumption): classify, fetch the customer record, draft a reply, check it against policy.
Four calls to one model, history re-sent each time. Input of 4 × S plus 6 × d at P_in, output at P_out.
A small model classifies, code fetches the record, a strong model drafts once. Its context is S_lean only: the record, the relevant policy and the request. Rules check the result.
Design B sends the strong model S_lean instead of 4 × S + 6 × d, so it is likely cheaper per attempt even after adding the small model's call. It is cheaper per resolved task only if s does not fall and p_review does not rise: if the small model misclassifies and reviewers correct more, review can swallow the saving. Decide on a shared case set, and choose B only if its C_resolved is lower.
How Do You Stop an Agent From Running Without Limits? Budgets, Caps and Loop Breakers
Enforce budgets in server code, not by asking the model to be thrifty: a model stuck in a loop will not stop itself. Before launch, have these in place:
- A token budget per task: a hard ceiling across all steps. On reaching it, stop and hand over with a state summary.
- Caps on steps and tool calls, per tool too, so one failing tool cannot feed a loop.
- A loop breaker and capped retries: stop when the same call with the same parameters, or the same error, repeats; retry with backoff and an idempotency key so a retry never acts twice.
- A daily budget per workflow and customer: a soft alert first, a hard stop only where pausing is safe. Critical workflows get a fallback path.
- A drift alert on cost per resolved task by workflow: a prompt edit, a model swap or a change in input can raise it without anything breaking.
- Per-step logging of task ID, workflow, customer, model, token counts and tool charges: numbers and IDs, never request content or personal data.
Test caps and breakers too, including cases built to trigger a loop, as in the guide to evaluating AI agents.
What Does Human Review Cost, and Why Is It Often the Largest Line?
Human review costs p_review × m × L per task and is often the largest line, because a minute of a person usually costs more than one model call. Check with your own numbers: if a reviewed task takes several minutes and an automated one a handful of calls, even a small reviewed share can outweigh every other line.
- Remove what code can verify: exact matches, missing fields, duplicates.
- Route by action risk and system confidence, not every task.
- Cut the minutes: source evidence beside the proposed change, no window switching. See human-in-the-loop approval queues.
- Measure minutes per task by reviewer and type, and how often reviewers change the output: almost never suggests rules could approve; almost always means the agent pays for work done twice.
How Do You Estimate the Cost Before Launch?
Measure real tasks and multiply by volume, instead of multiplying a token price by a guess at conversation counts:
- Sample real tasks. As a working practice, 50 to 100 representative tasks is a reasonable start (guidance, not a statistic), with edge cases, messy input and long documents.
- Run them through the intended design with per-step logging: steps, tokens, tool charges, retries, outcome, review minutes.
- Compute the median and the tail, not only the average: budget on the average, alert on the tail.
- Multiply by volume per scenario, with a safety margin that fits the sample size, and set a daily ceiling with a defined stop.
| Scenario | What changes | What to check |
|---|---|---|
| Expected | The normal task mix and volume. | Cost per resolved task against the revenue or saving the process brings. |
| Busy day | High volume, long documents, provider rate limits. | Maximum daily cost, queue times, retries caused by rate limits. |
| Failure storm | A target system or model provider returning errors: retries and stuck agents. | Whether caps and breakers stop the spend, and what happens to stopped tasks. |
Also ask your provider about volume or committed-use agreements, caching and batch terms, rate limits at your peak, notice of price changes and model retirement, and data handling. These are questions, not promises of what you will get.
When Should You Own the Pipeline, and When Is Optimizing Not Worth It?
Owning the pipeline pays off above a certain volume, a ready-made platform below it. A fee per task, conversation or resolution grows in a straight line with volume; building swaps part of it for a fixed cost plus a lower variable cost. Break-even is the fixed monthly cost of owning divided by the gap between the platform's fee per task and your variable cost per task. For control, data and vendor dependence, see custom automation versus off-the-shelf tools.
At DevShift, when we scope a custom AI agent, we agree acceptance checks that include usage cost. Once the process is defined, a quote can separate setup from ongoing costs with explicit usage assumptions, and the cost of human review is part of that picture. More on custom AI development for business.
Skip the optimization when:
- Volume is low: a ready tool and a tracking sheet beat your own metering.
- The process still changes: stabilize it first.
- People or a tool vendor, not tokens, drive the bill: trimming prompts will not move it.
- An error costs far more than the saving: do not downgrade a sensitive step without evaluation.
What Should You Measure in the First Month, and What Do You Do Now?
In the first month in production, measure these instead of settling for the invoice total:
- Cost per resolved task, by workflow, daily.
- Model calls per task (median and tail) and tokens per step: regular input, cached input and output.
- Retries and failures by step and error type, and s with a written definition of "resolved".
- Tool and channel charges per task, reconciled to vendor invoices.
- Reviewed share of tasks, minutes per review and how often reviewers change the output.
- How often caps and breakers fired, and what happened to stopped tasks.
- Cost drift after each prompt or model change on the same evaluation set, plus the week's three most expensive tasks, read by a person.
And what to do now:
- Pick one workflow and write down what counts as "resolved" in it.
- Turn on per-step logging before launch, not after.
- Run a real sample and compute C_resolved and the share of each line.
- Change one lever and measure again on the same sample.
If you want help mapping the process and defining the usage assumptions for a first estimate, a short call with us is a good starting point.
Frequently asked questions
Why does an AI agent cost more than a regular chatbot?
Because an agent makes several model calls for one task, and each call re-sends the context accumulated so far, while a simple chatbot usually answers in one call. An agent also calls tools that are billed separately, retries when a step fails and sometimes needs human review. So an agent's cost per task depends on its number of steps and the size of its context, not only on the per-token price.
How do I estimate what an AI agent will cost to run before launch?
Run a sample of real tasks through the intended design with per-step logging of tokens, tool charges, retries and review minutes, then multiply by expected volume. As a working practice, 50 to 100 representative tasks is a reasonable starting sample. Compute the median and the tail, build scenarios for a normal day, a busy day and a failure storm, and add a safety margin that fits the sample size.
What is the biggest cost driver when running agents at volume?
There is no single fixed answer, but it is usually one of three: context re-sent at every step, retries and loops, or human review time. The per-token price itself is often the smallest surprise of the three. To know what is true for you, compute how much of the cost per task comes from each line of the formula, and only then choose what to improve.
Can a cheaper model do the job without hurting quality?
Often yes for simple steps such as classification, field extraction and routing, and less so for steps that need judgment. The safe approach is routing: a small model for easy steps, a strong model for hard ones, and escalation when the small model is unsure. Validate on the same evaluation set and compare cost per resolved task, because a cheap model that raises retries or human review can end up costing more.
How do I prevent a runaway agent loop from inflating the bill?
Enforce limits in server code: a token budget per task, a maximum number of steps, a loop breaker that stops a repeated call to the same tool with the same parameters, and retries with backoff. When a limit trips, hand the task to a person with a state summary. Add a daily budget and an alert on drift in cost per task, and include loop cases in your evaluation suite before launch.
Does prompt caching really save money?
Yes, when the provider supports it and the start of your prompt is identical across calls, because that part of the input is billed at a reduced rate. In an agent loop this is often true, but the terms vary by provider: minimum length, how long the cache lives and sometimes a charge for writing to it. A small change to the prefix breaks the saving, so measure the hit rate in your logs instead of assuming it.
How should I price a product that runs AI agents for customers?
Start from cost per resolved task for each workflow and each customer, and price so the margin survives your customers in the expensive tail of the distribution, not just the average. Choose a billing unit that resembles your cost unit, such as a resolved task or credits, and add a fair-use allowance. Provider prices change, so define in advance how you will update your own pricing.
Where do I check the current per-million-token price of a model like Grok 4.6?
On the model page on this site and on the provider's official price list. Per-token prices change often, so do not rely on a figure printed in an article. On a price card, read input versus output price, the cached input price, any higher tier for very long context if one exists, and separate charges for built-in tools such as search and code execution.


