AI can overthink. Are you paying for it?

For a business, this can be an important question. Answering it might mean reading a supplier confirmation and comparing two dates. It might involve two conflicting documents, a revised agreement, perhaps an exception clause.

An important business decision is not necessarily a difficult task. It can be quite straightforward, with a clear set of rules. Yet when you configure an LLM to automate the task, or part of it, turning the reasoning setting up feels safer. If the decision matters, surely more thinking is better?

Sometimes it is. But studies have found that additional reasoning can be redundant, and in the worst case it degrades the answer. On usage-based API billing, you still pay for those extra reasoning tokens.

The business question is simple: does extra reasoning make the workflow more reliable, and is the improvement worth paying for?

There's more than meets the eye

LLMs read and write tokens: words, parts of words and punctuation. You pay for what you feed the model and for what it writes back, and output costs more per token. Reasoning models can use additional tokens while working through a problem, even when the user only sees a short answer. We often call this "thinking", because the model generates tokens almost as if it is reasoning about the question and possible answers before it settles on one.

OpenAI and Google both bill reasoning tokens as output tokens, even when their content is not shown to the user. A short answer can have a much larger token bill behind it.^{[2] [3]}

That can be money well spent. Working through interacting constraints, or comparing possible solutions, may need substantial reasoning.

What overthinking actually looks like

Researchers working through the Anthropic Fellows Program tested a deceptively simple question: you have one apple and one orange, how many pieces of fruit do you have? They added irrelevant information about the probability of each fruit being a particular variety.

The answer is two. Yet two of the tested models got less accurate as they reasoned for longer. They turned irrelevant information into part of a calculation that did not need it.^[1]

Overthinking can also leave the answer unchanged. Researchers at Google DeepMind and the University of Michigan found models repeatedly checking answers or exploring unnecessary alternatives on simple tasks. The extra work added little useful progress.^[4]

In a separate study, researchers tested a method for stopping redundant reasoning. On one grade-school maths benchmark, it cut generated tokens by 35.9%, while accuracy moved by an insignificant amount: 94.2% to 94.5%.^[5]

Qwen3-8B on GSM8K. Source: Sun et al., ACL 2026

These studies were not designed to measure anyone's API bill, and none of them lists a price difference. The fruit test was built to expose a failure mode of LLM reasoning, and the stopping method needs control over intermediate reasoning that most closed APIs do not offer. Nor do these experiments speak for every reasoning model.

Still, there is a useful takeaway: check whether the extra reasoning leads to overthinking, then decide whether it is worth keeping.

Fix the right problem

Take the supplier workflow. If the confirmation states the delivery date clearly, a modest model call may be enough to extract it. Ordinary code can compare it with the deadline. A genuine conflict between the confirmation and the contract is a different kind of problem, and may justify deeper analysis.

If the latest confirmation is missing, reasoning cannot establish what the supplier actually promised. The next step is to retrieve the document or ask for clarification. Turning on reasoning does not magically produce the missing information.

And if a wrong date could disrupt production, the workflow needs proper verification. That might mean checking extracted fields against the source and requiring human approval, a "Human at the Helm", for consequential actions. More reasoning by the same model should not be treated as independent verification.

Before changing the model setting, work out what is actually failing.

Measure the cost of a usable result

Tokens do not measure business value. They are a usage meter. Use them to measure what it costs to reach a result the end user or stakeholder can rely on.

This is why we spend most of our time understanding the problem first. It gets you a long way toward avoiding these pitfalls: it defines what success means, which mistakes are costly, and which situations your test needs to cover.

Once you understand the problem, you design an experiment. Build a dataset to test on. Set up a few configurations: different context and inputs, models of different sizes and costs from different providers, reasoning on and off. From the configurations that pass, pick the simplest and cheapest. Small savings per call compound at high volume. If your problem is low volume and high complexity, the more expensive configuration may well be worth it.

And do not mistake a brief answer for brief hidden reasoning. Check what reasoning controls your provider offers. A hard output limit can cut the response off mid-reasoning, before any answer is produced. Other models let you adjust the reasoning effort itself, which is worth comparing against the default.^{[2] [3]}

Profit × Planet: what we can and cannot claim

When redundant computation drops, cost drops, and so does resource use. At AmPhi Labs we care about both profit and planet, and LLM inference has an environmental cost too. Part of our job is to avoid wasting tokens, because wasted tokens also mean unnecessary emissions. Our rule: meet the business need, then trim the fat.

The financial savings of using fewer tokens are much easier to measure than the environmental ones. Tokens are not directly tied to a measure of energy or carbon. Hardware, model architecture, serving conditions and electricity supply all matter. Google's own measurement accounts for idle capacity, supporting machines and data centre overhead as well as active computation.^[6]

A 30% cut in tokens does not prove a 30% cut in emissions.

So we report what we can measure: quality, cost, latency and token use. Any carbon claim needs its own evidence, either direct measurement or a transparent estimate with a defined scope.

So, when is extra thinking worth the cost?

Extra reasoning can prevent mistakes whose consequences far outweigh its cost. If it resolves a difficult supplier dispute or avoids expensive rework, pay for it. If it only makes a straightforward answer slower and pricier, don't.

Sources

1. [1]Gema et al., Inverse Scaling in Test-Time Compute. TMLR, December 2025. Research conducted as part of the Anthropic Fellows Program with collaborators at other institutions. The fruit example reports declining accuracy for Claude Opus 4 and DeepSeek R1 as reasoning length increases.
2. [2]OpenAI, Reasoning models. API documentation on reasoning tokens and billing.
3. [3]Google, Gemini thinking and API pricing. Thinking controls, output limits and billed thinking tokens.
4. [4]Zhang et al., Do LLMs Really Need 10+ Thoughts for "Find the Time 1000 Days Later"? Towards Structural Understanding of LLM Overthinking. ACL 2026. Google DeepMind and University of Michigan; analysis includes Qwen3 and distilled DeepSeek-R1 models.
5. [5]Sun et al., Stop When Enough: Adaptive Early-Stopping for Chain-of-Thought Reasoning. ACL 2026. Table 1: Qwen3-8B on GSM8K, 2.62M to 1.68M generated tokens (−35.9%); accuracy 94.24% to 94.54%. Some other model and benchmark combinations show small accuracy declines. See Limitations for generation-control requirements.
6. [6]Google Cloud, How much energy does Google's AI use? We did the math. Provider approach to the operational footprint of serving AI.