Reasoning Models

Some chat apps show a short message such as "Thinking" for a few seconds, or even a minute, before the answer appears. Other models start answering at once. The models that pause are usually called reasoning models.

The name describes something the model does at inference time. Inference is the moment a finished model is used: you send a request and it produces a reply. So "reasoning" here does not name a new kind of machine. It is a name for what the model does in the seconds after you press send.

This lesson shows what the model writes during that pause, where the step sits inside the model, how it ended up in everyday products, and what it costs. After this lesson you can explain the difference between a reasoning model and a standard one, and say when the extra thinking is worth paying for.

1. One question and everything that came back

Start with a concrete case. Someone asks a reasoning model this:

Four of us need to be at the airport by six. Cheapest way?

The question looks short, but it hides several smaller questions. There are at least two ways to travel. Each has a price that depends on the number of people. There is also a deadline, so a cheap option that arrives late is not an answer.

Here is everything the model produced, in order:

[thinking]
Cab: one fare, four people, door to door.
Train: four tickets, and the shuttle at the far end.
The 5:10 gets in at 5:48, so that clears six.
The cab is quicker. For four of us it is the more expensive of the two.
So: train, unless the 5:10 is cancelled.

[answer]
Take the 5:10 and the shuttle. You will be at the terminal by 5:48,
and it is cheaper for four.

The reply has two parts. The first part, marked [thinking] here, is not the answer. It is what the model wrote on the way to the answer. The second part is the answer itself.

Look at what the thinking part does, line by line:

1. It lists the first option, the cab, and what it involves: one fare shared by four people, with no changes.
2. It lists the second option, the train, and notices a hidden extra: a shuttle is needed at the far end.
3. It checks the deadline. The 5:10 train arrives at 5:48, which is before six.
4. It compares the two. The cab is faster, but it costs more for this group.
5. It reaches a conclusion and adds a condition: the train, unless that train is cancelled.

Only after these five lines does the model write the short reply that the person reads. The reply is short because the sorting out has already happened.

Chat apps usually show the two parts differently. The thinking is often collapsed, greyed out, or shown as a summary, and the final answer is shown as the normal reply.

2. The model is the same, with one step added

A reasoning model is still a large language model. It still reads tokens and predicts the next token, one at a time. Nothing about that basic machine has changed. What changed is the order of what it writes.

flowchart LR
    subgraph LLM["A large language model"]
        direction LR
        A["Your prompt"] --> B["Thinking step<br/>the working,<br/>written out first"]
        B --> C["Final output"]
    end

The diagram shows three boxes inside one model. Your prompt goes in. The model first writes its working. Then it writes the final output. The thinking step happens inside the model's turn, before anything is handed back as the answer.

The reason this helps comes from how a model produces text. A model writes one token at a time, and every token it has already written becomes part of what it reads when choosing the next one. So the working is not decoration. Once the model has written "The 5:10 gets in at 5:48", that fact is on the page in front of it, and the final answer can build on it.

Without a thinking step, the first word of the reply is also the first word of the answer. The model has to commit to "Take the train" or "Take a cab" before it has compared anything. With a thinking step, the comparison is written down first, and the answer is chosen with the comparison in view.

3. People work the same way

Nobody answers a hard question with the first word that comes to mind. Asked about the airport trip, a person would pause, list the options, check the timetable, compare the prices, and then speak. For a harder problem, the person would reach for paper.

A reasoning model was given room to do the same thing. The thinking step is the model's scratch paper.

The comparison has a limit. A person's scratch paper is a true record of how that person reached the answer. A model's written working is text it generated. It often reflects how the answer was reached, but it is not a guaranteed record of every calculation that happened inside the network. So the working is useful for checking an answer, and it is not proof that the answer is right.

4. How the thinking step became a product feature

The thinking step was not part of the first chat products. It arrived in three stages.

In the first stage, the idea was a guess. In the second stage, it was tested in a fair way: the questions were identical, and the only difference was whether the model answered at once or worked first. In the third stage, the results decided the matter. Answers after working were better, especially on problems with several steps, such as math and logic puzzles.

At first, users had to ask for the working themselves by adding a line such as "think step by step" to the prompt. This technique is called chain-of-thought prompting, because the model writes a chain of small thoughts that lead to the answer. Later, model makers trained models to do the step on their own and gave it its own place in the reply. A model that does this by default is what products now label a reasoning model or a thinking model.

5. What the thinking step costs

The working is made of tokens, and the model has to generate every one of them. This has two consequences.

Time. The model writes the working before the answer, so the answer starts later. This is the pause you see in the app. A harder question usually produces longer working and a longer wait.

Money. The tokens in the thinking step are usually billed as output tokens, even when the app hides them or shows only a summary. Output tokens are the more expensive kind, as the lesson on tokens explained.

A worked case with made-up numbers. Suppose the airport question produces 60 tokens of working and 25 tokens of final answer.

without a thinking step:   25 output tokens
with a thinking step:      60 + 25 = 85 output tokens

The reply the person reads is the same length in both cases. The request with thinking uses more than three times the output tokens. On a hard problem, the working can be thousands of tokens long while the answer is one paragraph.

The working also takes space. While the model is writing it, the working sits in the context window together with the prompt, so very long working leaves less room for everything else.

6. When the thinking step helps

The step helps when the question needs several smaller steps that depend on each other. It adds little when the answer is a single fact or a short rewrite.

Because of this, many products let the user or developer choose. At the time of writing, common forms are a switch that turns thinking on or off, a setting for how much effort the model should spend, and separate model names for reasoning and non-reasoning versions. The exact options differ between providers and change often, so check the provider's current documentation.

The criteria for choosing are the ones from the previous section. Turn the step on when a wrong answer is costly and the problem has several steps. Leave it off when the task is simple, when the reply must come quickly, or when the request is repeated so many times that the extra output tokens add up.

Interview points

• A reasoning model is a large language model that generates its working first and its final answer second. The architecture and the token-by-token generation are the same as in a standard model.
• The thinking step happens at inference time, inside the model's turn, between receiving the prompt and returning the final output.
• The step works because each generated token becomes input for the next one. Intermediate results that are written down can be used when the answer is produced.
• The idea started as a prompting technique called chain-of-thought prompting, where the user asked the model to think step by step. Tests on the same questions with and without the working showed better results with it, so model makers built the step into the models.
• Reasoning tokens are usually billed as output tokens, even when the product shows only a summary of the thinking. They also take space in the context window while the reply is being generated.
• The trade-off is higher latency and higher cost in exchange for better answers on multi-step problems. Simple lookups and short rewrites gain little.
• The visible working is generated text. It helps a reader check the answer, but it is not a guaranteed record of the model's internal computation.
• Products expose the step as an on/off switch, an effort level, or a separate model, and the right setting depends on task difficulty, the cost of a wrong answer, and the time and token budget.

Summary

• "Reasoning model" names what a model does at inference time, after the prompt arrives.
• The reply has two parts: the working, then the final answer.
• The model is the same kind of machine as before, with a thinking step placed before the output.
• The step entered products because answers written after the working were better in tests.
• Thinking takes extra time and extra output tokens.
• The step helps most on problems with several dependent steps.