Back to Feed
Reasoning / Benchmarks & Evals

Improving Multi-Step Question Answering Through Decomposition

Original: Verification Without Sufficiency: Per-Chunk Filtering Fails on Multi-Hop RAG, and Decomposition Repairs It
By Randhir Kumar Published 2026-08-01

Listen to the summary

Uses a voice available on your device

Audio options
On this page

Key Takeaways

  • Traditional methods of checking individual search results fail when a question requires multiple steps of reasoning.
  • The paper introduces a new approach that selects evidence based on an anchor paragraph and verifies remaining information relative to that anchor.
  • Using a language model to break complex questions into smaller sub-questions significantly improves the reliability of evidence selection.
  • Performance on complex multi-hop datasets is currently much lower than on simple single-hop tasks, highlighting a clear area for improvement.

Summary & Methodology Analysis

When computers answer questions using external documents, they often try to verify if each piece of retrieved information is correct before providing an answer. Standard methods often rely on scoring single chunks of text against the original question. The paper shows that this approach is ineffective for multi-hop questions (queries that require combining information from multiple sources) because no single document contains the full answer on its own. This leads to poor performance on challenging datasets compared to single-step tasks.

Interactive System Flowchart

Click diagram to expand and zoom

Cross-Examination & FAQs

A deeper dive clarifying mechanics, constraints, and baseline evaluations.

Q1. What is the main problem identified in this research?

The paper finds that standard methods for checking the accuracy of retrieved information fail for complex questions because they evaluate documents individually rather than as a set.

Q2. What does the author suggest to fix this issue?

The author suggests breaking complex questions into smaller, simpler sub-questions and verifying evidence against these sub-questions instead of the original complex query.

Q3. Does this approach improve accuracy?

Yes, using an automated model to decompose questions captures a notable portion of the potential performance limit for verification tasks.

Q4. What datasets were used to evaluate this performance?

The researchers used several datasets including HotpotQA, 2WikiMultihopQA, MuSiQue, and SQuAD v1.1.

Q5. How does the proposed method select evidence?

The method uses conditional evidence selection by anchoring on a top-ranked paragraph from the first reasoning step and then verifying other chunks as pairs with that anchor.

Q6. What are the limitations of the current decomposer?

The performance is tied to the accuracy of the initial document retrieval, and the current system is limited to only one follow-up question.

Q7. How did existing verification methods perform on multi-hop questions?

Performance was quite low, with entailment scores reaching only 0.643 on HotpotQA, 0.523 on 2WikiMultihopQA, and 0.560 on MuSiQue.

Q8. What specific models were used for decomposition?

The research used Qwen2.5-1.5B-Instruct and Qwen2.5-7B-Instruct models.

Q9. What is the significance of the difference between SQuAD and multi-hop datasets?

The high score of 0.951 on single-hop SQuAD versus lower scores on multi-hop datasets demonstrates that current verification tools are not optimized for complex reasoning.