Chain-of-Thought in Detail

Knowledge

You already know Chain-of-Thought (CoT) as "Think step by step." But why does it actually work? And how can you use it systematically instead of just appending a single sentence?

Why CoT Works

LLMs generate token by token -- each new word is based on all previous ones. When an LLM gives a direct answer, it has to execute the entire reasoning process in a single "leap." For complex problems, this leads to errors.

Chain-of-Thought solves this by forcing the model to explicitly articulate intermediate steps. Each written intermediate step becomes part of the context for the next step. The model can literally build on its own work.

Analogy: Imagine you're asked to calculate 347 x 23 in your head. Tough, right? But with paper and intermediate steps (347 x 20 = 6940, 347 x 3 = 1041, sum = 7981), it's trivial. That's exactly what you're doing with CoT -- you're giving the LLM "paper."

Variants of Chain-of-Thought

Zero-Shot CoT: You simply add "Think step by step" or "Let's work through this logically." This works surprisingly well for many tasks.

Few-Shot CoT: You provide examples that demonstrate the desired reasoning process -- not just the answer, but also the intermediate steps.

Example:

Question: A store has 12 apples. 5 are sold, then 8 new ones arrive.
Reasoning:
- Start: 12 apples
- After sale: 12 - 5 = 7 apples
- After delivery: 7 + 8 = 15 apples
Answer: 15 apples

Question: A bus has 23 passengers. At the first stop, 7 get off and 4 get on.
At the second stop, 3 get off and 9 get on. How many are on the bus?

The LLM will apply the same step-by-step approach to the new question.

Understand

Self-Consistency: Multiple Reasoning Paths, One Answer

A single CoT path can be wrong. Self-Consistency solves this: you have the LLM think independently multiple times and take the majority vote.

Here's how it works:

1. Ask the same question multiple times (e.g., 5 times) with higher temperature (0.7-0.9)
2. Each answer follows its own reasoning path
3. Compare the final answers
4. The answer that appears most frequently is likely correct

Example: "Does a kilogram of steel or a kilogram of feathers weigh more?"

• Path 1: "Steel is heavier than feathers... so steel weighs more." -> Steel (wrong)
• Path 2: "A kilogram is a kilogram, regardless of the material." -> Same (correct)
• Path 3: "Both weigh one kilogram. The question is a trick." -> Same (correct)

Majority vote: Same (2 out of 3) -- correct.

When to Use CoT -- and When Not

CoT is particularly useful for:

• Mathematical problems and logic puzzles
• Multi-step decisions
• Tasks that require planning
• Analysis and argumentation

CoT is less useful for:

• Simple factual queries ("What is the capital of France?")
• Creative tasks like poems or stories
• Translations
• Simple classifications

Apply

Build your own prompt chain: break down a complex problem into steps that build on each other.

Reflect

Chain-of-Thought isn't magic -- it's a tool. The art lies in recognizing when you need it and which variant (Zero-Shot CoT, Few-Shot CoT, Self-Consistency) is best suited for your specific problem. In the next section, you'll learn additional advanced techniques that you can combine with CoT.