Test-time compute is the work an already-trained model performs for one request while producing, checking, or revising its answer. Test-time scaling means giving the model more of that request-time work, or allocating it differently, without retraining the model. The extra budget can extend one line of reasoning, generate alternative answers, or use an observation to try again. Those choices are often bundled under ‘let it think longer,’ but they do not buy the same thing.
That distinction is now practical because current OpenAI and Anthropic APIs expose reasoning-effort controls while Google offers an Extended Thinking model for complex live-agent tasks, turning “let it think” into a quality, latency, and cost decision for each request.
The useful question is not simply, “How much thinking should I buy?” It is: where should the extra compute go? Test-time scaling allocates it to depth, width, or feedback. Depth extends one attempt. Width creates separate attempts. Feedback evaluates work and changes the next attempt. Each repairs a different failure, and none guarantees correctness.
What test-time compute actually is
Start with an ordinary request. Training is already over, and the model begins with learned weights that stay fixed while it answers. At each step, the model processes the request and the tokens generated so far to choose the next token. That ordinary generation already consumes test-time compute: computation performed after training for one particular input.
The term covers more than the model's hidden reasoning tokens. It includes the work the model and its surrounding system perform to produce, inspect, and improve this one answer. Continuing the generation adds more token-level operations to the current history. Starting another generation creates a separate candidate history. Running a test or verifier creates an observation that a later attempt can use. Revising with that observation performs another generation under changed conditions.
Test-time scaling is the deliberate decision to increase or reallocate that per-request work relative to a baseline answer. The extra work can matter because it changes the transient information available to a later choice: a longer prefix, a genuinely different candidate, or new evidence about an error. If the added work does not change that state or evidence, it may increase latency and cost without improving the answer.
This is not training or fine-tuning. Training changes reusable model weights for future requests. Test-time compute leaves those weights alone and is paid again for each request. It is also not synonymous with selecting a larger model or producing a longer visible answer. A system may spend substantial compute on candidates, checks, and revisions that never appear in the final response.
We will follow one realistic coding task through all three choices. An agent must repair invoice_total, a function that sums prices, applies tax, and returns integer cents. Its contract requires exact decimal arithmetic, an 8 percent tax applied after aggregation, round-half-up once to the nearest cent, then conversion to integer cents. The visible invoice contains items priced at $12.49, $7.25, and $3.10. Hidden tests vary the inputs to expose half-cent rounding, multiple-item accumulation, zero tax, and binary floating-point drift.
First we need to see why one uninterrupted generation can fail. Then we can spend the same request-time budget on a deeper path, a wider candidate set, or a feedback loop and see what actually changes.
Before scaling, identify the one-shot failure
An autoregressive language model generates a response one token at a time. At every position it receives the request plus the tokens already written, then assigns probabilities to possible next tokens. Once a token is chosen, that token becomes part of the state used to predict everything that follows.
This left-to-right process matters because an early choice can narrow the rest of the answer. Suppose our coding agent begins with the assumption that converting dollars to cents is just int(total * 100). It can write an elegant explanation and many consistent lines of code after that. The later tokens are coherent with the earlier assumption, but coherence does not repair the assumption.
A one-shot attempt is one uninterrupted generation from request to answer. It may spend many internal reasoning tokens, but it has no separate alternative to compare with and no external result that changes its course. Its success depends on the model entering a useful line of reasoning and staying there.
For this lesson, a useful accounting unit is a generated reasoning token, a candidate response, a verifier evaluation, or a tool call such as a test run. These units have different hardware costs, so we will not pretend they are perfectly interchangeable. The important constraint is that the system has a finite request-time budget and must decide how to allocate it.
There are three places to allocate that budget:
Depth: continue or enrich the same reasoning trajectory.
Width: sample several separate trajectories from the same starting request.
Feedback: inspect work, then condition the next attempt on what the inspection revealed.
The distinction is about information flow. Extra depth sees the same request and its own growing prefix. Width gives each attempt the request but not the siblings' work. Feedback adds a new observation, such as “the half-cent test failed,” before the next generation. Width is logical branching; whether those branches run simultaneously is a separate serving decision.
That gives us a diagnostic before we spend anything. If the model merely stops before completing a sound plan, buy depth. If different plausible starts lead to different outcomes, buy width. If a checker can reveal what went wrong, buy feedback. A high effort setting may combine these internally, but an application designer still needs this mental model to understand latency, cost, and failure.
Depth gives one trajectory room to finish
The simplest scaling strategy is to let one attempt run longer. The model can decompose the task, inspect edge cases, and carry intermediate facts farther before committing to the final response. In our invoice task, a deeper trajectory must preserve a specific dependency: represent the prices exactly, sum them to $22.84, apply the 8 percent tax to obtain $24.6672, round that total once to $24.67, then return 2467 cents.
The benefit is continuity. Every later step sees the reasoning accumulated earlier. Here, the final integer conversion only makes sense after the representation and rounding policy are settled. Once an earlier step truncates a fraction of a cent, no later step can reconstruct it. Proofs, long calculations, and code changes with dependent invariants have the same shape.
But depth is not the same as recovery. If the trajectory commits early to binary floating point and treats a failing cent as harmless noise, more tokens can reinforce the wrong frame. The model may repeatedly check arithmetic inside a mistaken representation. It is thinking longer, but within a shrinking part of the solution space.
That gives us the first useful rule: depth helps when a sound path is unfinished, not merely when a path is wrong. A model that often reaches the correct approach and then runs out of room is a good candidate for higher effort. A model that confidently commits to a bad representation needs an alternative or new evidence, not an even longer monologue.
