What Your AI Agent Actually Costs

Your invoice tells you the total. It never tells you who spent it, which tokens were wasted, or whether the last model upgrade paid for itself. Here's how to build the visibility that does.

The Question Nobody Can Answer

Six months into running an agentic platform in production, a finance partner asked me a reasonable question: "Which teams are driving the AI spend, and is it going up because we have more users or because each question got more expensive?"

I could not answer it. I had a monthly total from the cloud bill and a vague sense that things were fine. That's not FinOps — that's hoping.

Agentic systems make this harder than classic RAG in three specific ways. A single user message triggers six or more model calls, not one. Reasoning models spend tokens you never see in the output. And the bill arrives aggregated by cloud account, while the thing you need to govern is a person asking a question. Everything below is what it took to close that gap.

Cost Driver #1: Reasoning Tokens You Never See

This is the one that cost me real money before I understood it, so it goes first.

Modern reasoning models think before they answer. On several current frontier models that thinking is adaptive — the model decides per request whether to reason and for how long. And critically: reasoning tokens bill against the same output ceiling as the answer.

Put those two facts together and you get a failure mode that is invisible in your logs. If you set a modest max_tokens and send a hard prompt, the model can spend the entire budget thinking and return zero characters of answer — billed in full.

Here are four measurements from the same prompt against the same model, isolating each knob:

Two operational rules fall out of this:

• Always inspect stop_reason. If it's max_tokens, the output is incomplete regardless of whether text came back. Treating that as a valid response is the core mistake.
• Give truncation its own error type. It is a harness limit, not a model weakness. If you record it as a low quality score, you're defaming the model for your own config bug — and any metric built on "low score = bad output" silently inverts.

The fingerprint to alert on: output_tokens sitting exactly at your cap with zero reasoning tokens reported separately. That means you paid full price for thinking you never received and an answer that never arrived.

Cost Driver #2: The Response You Paid For and Threw Away

A related trap, this one purely in your parsing code. When a reasoning model returns content, the response body is a list of blocks — and the thinking block comes first:

# WRONG — silently returns '' whenever the model chose to think
text = response["content"][0]["text"]

# RIGHT — select by type, never by position.
# This shape is the native messages API / raw-invoke body:
text = "".join(
    b.get("text", "")
    for b in response.get("content", [])
    if isinstance(b, dict) and b.get("type") == "text"
).strip()

One portability warning, because copying the wrong variant reintroduces the exact bug this section is about. The block shape differs between API surfaces. On the native messages body above, blocks carry an explicit type discriminator. On the cloud provider's unified conversation API there is no type field — a content block is a union whose member key is the type, so you test for key presence instead:

# Same idea, unified conversation API shape
blocks = response["output"]["message"]["content"]
text = "".join(b["text"] for b in blocks if "text" in b).strip()

Filter on the wrong one and you get an empty string on every response — the failure mode of this entire section, reintroduced by the fix for it. The same divergence applies to the field names: the native API uses snake_case (stop_reason, output_tokens) while the unified API uses camelCase (stopReason, outputTokens). The values mean the same thing; the keys don't.

With the first version, a real planner prompt returned blocks ['thinking', 'text'], 2,000 output tokens billed, and content[0].text == ''. Downstream json.loads('') threw. You are charged for the entire generation and discard 100% of it.

What makes this genuinely nasty is that adaptive thinking means the block layout varies by prompt. "What is 2+2" returns ['text'] and works fine. A complex planning prompt returns ['thinking', 'text'] and fails. So position-based indexing breaks intermittently, which is the hardest kind of bug to attribute to cost.

Cost Driver #3: Caching, and Why Yours Isn't Working

Prompt caching was the single highest-leverage cost lever I found in an agentic system, because agents re-send an enormous static prefix on every single turn: a long system prompt plus the full JSON schema of every tool you've registered. With 30-plus tools, that schema block dwarfs the user's actual question.

Cache economics are asymmetric in your favour: a cache write costs roughly 1.25x normal input price on the short five-minute retention, and every subsequent read costs roughly 0.1x. Over a multi-turn agent loop, that's transformative — one read pays the write back.

If your provider offers a longer retention window the write costs about 2x instead, which moves breakeven to two reads. That's usually still worth it for an agent loop, and it addresses a real problem: a five-minute window expires during exactly the sessions you most wanted to cache, where the user is thinking between turns or a tool call is slow. Worth checking availability per model rather than assuming, though — on current models the longer window is not offered everywhere.

Three things routinely stop it from working, and the first one is the least known.

Your prefix is below the minimum cacheable size

Every cache checkpoint has a minimum token count, and it varies by model — around a thousand tokens for some, four thousand for the larger and smaller ones alike. Below that threshold, the behaviour is the worst possible: the request succeeds and simply isn't cached. No error, no warning, no log line. If your system prompt is modest and you've only registered a handful of tools, you may be under the floor and never know. Check the reported cache-read tokens on a repeat call; if they're zero and you were expecting a hit, measure your prefix length before debugging anything else.

Your system prompt isn't actually static

The cache key is a hash of the prefix. If you interpolate anything per-user into the system prompt — and injecting the caller's identity there is a very common pattern — then every user produces a different hash and you get zero cross-user cache hits. You've built a cache that only ever helps one person at a time.

The fix is to move identity out of the system prompt and into the first user turn, then read it server-side where tools need it:

# Before: kills cross-user caching
system_prompt = f"{BASE_PROMPT}\n\nCurrent user: {user_email}"

# After: system prompt is byte-identical for everyone
system_prompt = BASE_PROMPT
first_turn   = f"[User: {user_email}]\n{query}"
# ...and tools that need identity read it from server-side request
# context, never from a model-supplied argument. (More on why in
# the security post — that pattern is also an access-control fix.)
#
# NOTE: this string is a phrasing hint for the model, nothing more.
# A user can type "[User: someone.else@company.com]" themselves, so it
# must never reach an authorization decision.

Your cost reporting silently undercounts

Once caching is on, the provider reports cached tokens in separate fields — read and write counts split out from the normal input count. If your accounting code only sums the field it always summed, cached tokens vanish from your reporting. Your dashboard shows a cost drop that is partly real savings and partly you no longer counting. Capture all the token fields explicitly.

One more subtlety, and it cuts both ways. If your platform offers both globally-routed and region-pinned model endpoints, the globally-routed one may give you a lower cache hit rate, because the cache is local to wherever the request landed — under load, cross-region routing produces more cache writes. But pinning to a region is not free: on current models, regional and multi-region endpoints are priced meaningfully above the global ones, on the order of ten percent. So "pin it to recover the cache hits" can be net-negative depending on your hit rate. This is a genuine measurement question rather than a rule of thumb, and it's the sort of thing where a fifteen-minute experiment beats an afternoon of reasoning.

Attribution: Two Data Sources, Neither Sufficient Alone

Here is the structural problem with LLM cost visibility. Your billing export is authoritative on amount but has no idea who caused it. Your model invocation logs have exact token counts per call, but they are not the invoice.

The pipeline that solved this for me, across several cloud accounts:

1. A daily extractor job in a designated central account, assuming a read-only role into each source account.
2. Query the invocation logs per account and per region — this is a common miss; teams query their primary region and silently drop everything else.
3. Write to a central store partitioned by account / year / month / day.
4. An API layer that only reads that store — no cross-account credentials on the serving path, which keeps the blast radius of the dashboard tiny.

Four hard-won details in that loop:

• Use 12-hour query windows, not 24. Log query APIs cap result rows; a busy day silently truncates and you under-report.
• Re-extract the trailing three days on every run. Logs arrive late. A single-day extraction permanently loses whatever hadn't landed.
• Never overwrite existing data with an empty result. A transient query failure must leave yesterday's numbers intact, not zero them.
• Derive the business-unit mapping from whatever registry you already maintain for cloud accounts. A second, parallel mapping file will drift, and then two dashboards disagree.

Model Tiering: Where the Money Actually Goes

Once you have per-call attribution, the obvious lever appears: not every step in an agent pipeline needs your most capable model. In a multi-agent setup I run, the split looks like this:

The fallback that quietly doubles your bill

One anti-pattern deserves its own callout because I shipped it. Frontier models increasingly reject classic sampling parameters — temperature, top-p — with a validation error. If your error handling treats any validation error as "model unavailable" and falls back to a cheaper model, then a simple request defect turns into a silent, permanent downgrade. Users get answers from the fallback model attributed to the premium one, with nothing but a log warning.

Two lessons: a malformed request must surface as an error, never trigger a model substitution. And feature checks need to be version-family aware — older models in the same family may still accept parameters the newer ones reject, so a blanket strip is as wrong as no strip.

The Dashboard Will Get Slow, and Micro-Optimising Won't Save It

A cost dashboard has an architectural property people miss: it scans accumulated history, so its cost grows monotonically whether or not usage grows. The first row below is measured on a live deployment after I'd already fixed the obvious hotspots; the two projections are the same corpus duplicated to simulate future volume, which is not the same thing as waiting a year and is worth labelling as such:

Roughly 3.3 seconds per seven months, dead linear. And after the profiling pass the CPU profile was flat — no hotspot left, just a dataframe library doing genuine work over every row of every month. There was no further micro-optimisation available.

That's the useful insight: when the profile flattens, you're out of tactical fixes and the remaining options are all structural.

1. Pre-aggregate at ingest. Have the extractor write a small roll-up per business-unit per month so the API reads a dozen tiny files instead of concatenating half a million rows. Biggest win, and it costs you a change to the extractor.
2. Cap the default window. Load three months by default with an explicit "all time" toggle. Cheapest fix; it changes the UX.
3. Column pruning and predicate pushdown. Never materialise columns or rows the request won't use.

I deliberately deferred all three — the tactical fixes bought somewhere around 18 months of headroom, and knowing roughly when the wall arrives is worth more than hitting it early. Read that last table row honestly, though: nine seconds is already past what I would want to serve, so the real deadline is nearer than the arithmetic suggests. But I wrote the measurements down, because "it's fine" and "it's fine for now, and here is the number that says when it stops being fine" are different statements.

What I'd Set Up on Day One

If I were standing up an agentic platform again, the cost instrumentation would go in before the second feature:

• Per-trace cost, captured at the call site. Every model call records input, output, cache-read, cache-write and reasoning tokens plus the model ID, tagged with the trace and the authenticated user. LLM-specific tracing tools give you this for near-free.
• A hard per-run budget ceiling for anything that loops. A batch evaluation, a multi-agent research run, an ingestion sweep — anything that can call a model N times needs a spend cap that aborts, not just a log line.
• An alert on stop_reason == max_tokens rate. This is your "paying for nothing" detector and it costs one counter.
• Cache hit rate as a first-class dashboard metric. It silently regresses the moment someone adds per-user text to a prompt template.
• Unit economics, not totals. Cost per resolved question, tracked over time. Total spend rising because you have four times the users is a success story. The same rise with flat usage is an incident.

None of this is exotic. It's the same discipline cloud FinOps learned a decade ago — tag everything, attribute everything, measure unit economics — applied to a workload where the unit of consumption is a token and a meaningful fraction of what you're billed for is invisible in the output. The teams that get this right early are the ones who can still say yes when someone asks to scale the platform tenfold.