Reasoning Models and Thinking Tokens Explained

Some LLMs start writing their answer at once. A reasoning model writes pages of working, like a scratchpad, and only then gives you its response.

Those extra thinking tokens cost time and memory, but they can help with maths, code, and problems with several dependent steps.

They’re wasted on many everyday prompts. Asking a reasoning model to spend five minutes choosing a sandwich filling is a waste of resources.

What chain-of-thought means

Chain-of-thought is the intermediate text a model produces while working towards an answer. It can include calculations, plans, checks, and corrections.

A standard chat model can sometimes be nudged with a prompt like “work through this step by step”. Worked examples can also teach it the kind of reasoning you want.

The generated text gives the model something like scratch paper. Each new token becomes part of the context, so later steps can refer back to earlier calculations.

That doesn’t make every step true. A model can write a confident-looking mistake and build on top of it.

What makes a reasoning model different?

Reasoning models are trained to spend more effort before answering. Reinforcement learning can reward correct answers to problems that are easy to check, like maths questions or code tests.

During training, the model learns behaviours like:

• breaking a problem into smaller parts
• checking an earlier result
• trying another route after a failure
• spending more tokens on harder questions

How o1 and DeepSeek changed the game

For years the recipe was: bigger model, more data, better answers. OpenAI’s o1, released in September 2024, added a new axis. Give the model more time to think at inference, and it solves harder problems.

o1 proved the concept but kept the weights and details closed.

DeepSeek-R1, released January 2025, proved you could get there openly and cheaply. The R1-Zero experiment applied large-scale reinforcement learning directly to a base model with no human-written reasoning examples needed. The model learned to verify itself, backtrack, and spend more tokens on harder questions, all because those behaviours produced more correct answers on checkable tasks like maths and code. It was trained with reinforcement learning and rule-based rewards, without starting from a supervised set of polished reasoning examples. It developed long working traces and self-correction, but the text could be messy, repetitive, and mixed across languages. DeepSeek-R1 added curated examples and more training stages to make the output easier to use. DeepSeek also released smaller distilled models trained from R1’s output.

DeepSeekMath had already introduced the training method behind this — GRPO, or Group Relative Policy Optimization — which made reinforcement learning for reasoning affordable enough to scale. R1-Zero was the proof that it worked on a real model without supervised fine-tuning.

The open weights changed the world (kind of). Suddenly anyone could download a reasoning model, distill it into something smaller, inspect the training method, and build on it. The question shifted from “can only frontier labs build reasoning models?” to “how cheaply can reasoning be reproduced and deployed?”

By 2025 and 2026, reasoning was everywhere. Google shipped Gemini 2.5 as a “thinking model”. Anthropic added extended thinking to Claude. OpenAI’s GPT-5 exposed reasoning effort controls, including a minimal setting. Chain-of-thought went from a prompting trick to a model-design paradigm.

Test-time compute

Test-time compute is the computer work spent after you send a prompt. It’s also called inference-time compute.

There are several ways to spend more:

• generate a longer reasoning chain
• produce several candidate answers and pick one
• try different paths and compare them
• write an answer, criticise it, and revise it
• call tools like search, Python, or a calculator

A smaller model given enough useful working time can sometimes beat a larger model that answers right away. This works best on problems where extra steps can correct a mistake.

More work doesn’t guarantee a better answer. If the model lacks the needed knowledge, another thousand tokens just produces a longer guess.

The dot experiment: what the model writes isn’t what it thinks

Here’s a strange result. A 2024 paper called “Let’s Think Dot by Dot” found that a model restricted to outputting only dots — ........ — as filler could still solve certain tasks better than with no filler at all. The dots carry no meaning. But each dot is another token, and each token means another forward pass through the model. That extra computation is enough to help.

This suggests the visible chain-of-thought may not be the real reasoning. The words you read are one thing. What happens inside the model’s hidden states is another.

The Jacobian lens work from Anthropic’s interpretability team makes this concrete. They found that models form internal representations of intermediate concepts, like “spider” before “8” in a question about spider legs, that surface in hidden activations before the answer, and sometimes never appear in the visible chain-of-thought at all. When researchers swap the internal “spider” direction for “ant”, the model’s answer shifts from 8 to 6. The model was thinking internally. It just didn’t write it down.

What you see in a reasoning trace isn’t a transcript. It’s a computational scratchpad, and the real work may be happening in a representational space that doesn’t map cleanly to words. Seeing spider in the hidden states before the answer is evidence the model computed the intermediate. But it didn’t need to say it out loud to get the right answer.

This connects to a newer trend: compressed reasoning styles. Some leaked traces and fine-tuning experiments have shown models reasoning in terse, grammar-stripped fragments — sometimes called “grug speak” or “caveman style”. Strip away the prose, keep the decisive steps:

check edge case. reject. try direct. verify. done.

The reasoning quality holds up with far fewer tokens. If the model doesn’t need fluent sentences to think, why pay for them? The dots prove extra computation helps regardless of what the tokens say. The Jacobian lens proves the words may not be the point.

The cost on a home machine

Thinking tokens are generated like any other output tokens. If your computer produces 10 tokens per second:

Thinking length	Waiting time at 10 tok/s
500 tokens	about 50 seconds
1,000 tokens	about 1 minute 40 seconds
5,000 tokens	about 8 minutes 20 seconds
10,000 tokens	about 16 minutes 40 seconds

The final answer may contain only 200 tokens after all that work.

Long reasoning also expands the context cache. On a memory-tight GPU, a huge thinking trace can eat into the room left for your prompt, or push part of the cache into slower memory.

For local use, token speed changes which reasoning model is practical. A model that scores higher on a benchmark may be worthless to you if every question takes an hour.

Which reasoning models fit at home?

Most modern models have reasoning modes:

Model class	Rough four-bit weight size	Hardware idea
7B to 8B	4 to 6 GB	Entry GPU, recent laptop or CPU
14B	8 to 10 GB	12 GB GPU or more
32B	18 to 22 GB	24 GB GPU with sensible context
70B	38 to 45 GB	Two GPUs, 48 GB card or CPU offloading

Sizes vary with the quantization recipe and metadata. Leave memory for the KV cache and runtime. The quantization guide explains why a Q4 file isn’t always exactly four bits per weight.

A 30B Q4 reasoning model is a solid target for a 24 GB card like the RTX 3090, something like Qwen 3.5 27B. Smaller 9B and 14B versions run faster but may make more reasoning errors.

When to use one

A reasoning model earns its wait on tasks like:

• debugging code with several possible causes
• maths problems and proofs
• planning a change across many files
• logic puzzles and constraint problems
• scientific questions that need a worked calculation

Use a normal instruct model for:

• short factual questions
• rewriting a paragraph
• casual chat
• summaries
• brainstorming where many quick ideas help
• creative prose

The overthinking problem

Reasoning models sometimes keep going after they’ve got enough information. They may recheck an easy result, change a correct answer, or circle the same idea.

Longer thinking can therefore reduce accuracy on easy tasks. It also makes the answer slower and burns more electricity.

If your app supports a thinking budget or effort setting, start low. Raise it for hard prompts instead of maxing it out every time. Some models accept controls in their chat template; others use a token limit or a /think setting. Follow the model card because these controls aren’t standard across runtimes.

A strict output limit can cut a reasoning trace off before the answer. Make sure the token budget leaves room for both the working and the final response.

Better prompts for reasoning models

Give the model the problem, constraints, and desired answer format. Don’t pad the prompt with demands to be brilliant or to think forever.

A solid coding prompt might be:

This Python function sometimes returns duplicate IDs.
Find the cause, show a failing input, then provide the smallest fix.
Do not change the public return type.

The model now knows what to inspect and how to finish. If tools are available, ask it to run a test or calculate the result rather than trusting mental arithmetic.

For an important decision, ask for assumptions and checks in the final answer. Don’t depend on a raw chain-of-thought as proof. The evidence you can trust is a reproducible calculation, source, command, or test.

A sensible first setup

Install one small, quick instruct model and one reasoning model that fits fully in memory.

On a 24 GB GPU, that could mean:

• a fast 7B to 14B instruct model for daily chat
• a 32B Q4 reasoning model for difficult code or maths
• an 8K to 16K context until you know you need more

Run the same five problems through both. Record accuracy, total wait, and whether the answer needed correction. Your own prompts tell you more than a broad leaderboard about which model deserves space on your drive.

Read further

• DeepSeek-R1 paper
• Chain-of-thought prompting paper
• Scaling LLM test-time compute
• Let’s Think Dot by Dot: Hidden Computation in Transformer Language Models
• Verbalizable Representations Form a Global Workspace in Language Models (Jacobian lens)
• LLM quantization without the alphabet soup
• RTX 3090 for local AI inference
• Speculative decoding and MTP explained
• Ollama, llama.cpp or vLLM: which one should you use?