Improving Language Model Reasoning with Self-Consistency

Summary & Methodology Analysis

When building applications with language models, developers frequently rely on chain-of-thought prompting to tackle multi-step problems. However, standard deployments often use naive greedy decoding, which simply picks the single highest-probability token at each step. This approach is prone to early errors cascading through the rest of the generated reasoning path, leading to frequent failures on complex reasoning benchmarks. The paper addresses this limitation by proposing self-consistency as a replacement for greedy decoding. Instead of taking a single deterministic path, the system samples from the model's decoder to generate a diverse set of distinct reasoning paths. Because a complex problem can often be solved through multiple valid logical routes, generating various chains allows the model to explore different ways of reaching a conclusion.

Once the diverse set of reasoning paths is generated, the approach aggregates the final answers by marginalizing out the sampled reasoning paths. It then selects the most consistent answer in the final answer set via a majority vote or a weighted sum. This strategy treats reasoning like an ensemble method, where the consensus answer across many sampled trials overrides isolated logical slips. The method was evaluated across a wide range of models and datasets, including UL2-20B, GPT-3-175B, LaMDA-137B, PaLM-540B, GSM8K, SVAMP, AQuA, StrategyQA, ARC-challenge, AddSub, MultiArith, ASDiv, CommonsenseQA, ARC-easy, BoolQ, HotpotQA, e-SNLI, ANLI, and RTE. On the PaLM-540B model, self-consistency delivered substantial absolute accuracy gains, including +17.9% on GSM8K, +11.0% on SVAMP, +12.2% on AQuA, +6.4% on StrategyQA, and +3.9% on ARC-challenge.

Despite these performance improvements, the method comes with specific constraints that engineers must weigh against their requirements. First, self-consistency can be applied only to problems where the final answer is from a fixed answer set. Second, because it requires generating multiple reasoning paths per prompt rather than a single greedy generation, self-consistency incurs more computation cost. The paper does not specify the exact dollar cost or latency overhead beyond noting this increase in computation cost.