Teaching Large Language Models To Reason

Summary & Methodology Analysis

Large language models often struggle with complex reasoning tasks such as arithmetic, commonsense, and symbolic reasoning when prompted to output answers directly. To address this, the authors introduce chain-of-thought prompting. Instead of using standard few-shot prompting where models skip straight to the answer, this method augments each exemplar with a manually composed chain of thought for an associated answer, forming input, chain of thought, and output triples. This allows the model to generate a series of intermediate reasoning steps before arriving at the final output.

The paper evaluates this method across multiple models and datasets, including GPT-3, LaMDA, PaLM, UL2 20B, Codex, GSM8K, SVAMP, ASDiv, AQuA, MAWPS, CSQA, StrategyQA, Date Understanding, Sports Understanding, SayCan, and SST-2. Scaling plays a crucial role, as the emergence of chain-of-thought reasoning only occurs at large model scales. For example, prompting PaLM 540B with eight chain-of-thought exemplars achieves state-of-the-art accuracy on the GSM8K benchmark, surpassing finetuned GPT-3 with a verifier. Furthermore, PaLM 540B achieves 75.6% on StrategyQA, outperforming the prior state of the art of 69.4%, and reaches 95.4% on sports understanding, outperforming an unaided sports enthusiast at 84%.

Despite these strong results, several limitations remain. Fully characterizing a model's computations that support an answer remains an open question. Additionally, manual annotation costs for chains of thought could be prohibitive for finetuning. There is also no guarantee of correct reasoning paths, which can lead to both correct and incorrect answers. Finally, because the emergence of chain-of-thought reasoning only happens at large model scales, it is costly to serve in real-world applications.