Chain-of-thought (CoT) prompting is a technique that dramatically improves LLM performance on tasks requiring multi-step reasoning. Instead of asking the model to jump directly to a final answer, you instruct it to "think step by step" or demonstrate the desired reasoning process through examples.
The technique was popularized by research from Google Brain, which showed that simply adding "Let's think step by step" to a prompt could improve accuracy on math word problems from around 18% to 79% with the same model. The improvement comes from forcing the model to make its reasoning explicit, which reduces errors from skipped logical steps.
There are two main variants. Zero-shot chain-of-thought adds a generic instruction like "Think through this step by step" without providing examples. Few-shot chain-of-thought includes one or more worked examples showing the complete reasoning process, which the model then mimics for new inputs.
CoT prompting is especially effective for arithmetic and mathematical reasoning, logical deduction and constraint satisfaction, multi-hop question answering where the answer depends on combining multiple facts, code generation that requires understanding program flow, and any task where the answer depends on a sequence of dependent decisions.
The tradeoff is increased token usage and latency, since the model generates the reasoning steps in addition to the final answer. For production applications, teams often parse the model's output to extract just the final answer while logging the reasoning chain for debugging. Some implementations use a two-pass approach: generate reasoning with CoT, then generate a concise response based on that reasoning.