Extracting Hidden Reasoning from Large Models
Listen to the summary
Uses a voice available on your device
Audio options
On this page 4 sections
Related concepts 4 concepts
Key Takeaways
- EchoCoT achieves up to 66.4% success rates on the OpenThoughts test set using the EchoCoT-LTGO configuration.
- The method demonstrates capability for high fidelity extraction, including recovering 33,463 tokens from a Gemini-2.5 model.
- Optimized extraction trajectories generalize well to unseen datasets like MATH500, with success rates reaching up to 80%.
- The framework provides a scalable way to probe reasoning models without requiring access to their original tokenizer or weights.
Summary & Methodology Analysis
EchoCoT functions by identifying a reasoning replay surface that exists during tool calling interactions within a single request. By iteratively injecting instructions through a scratchpad tool, the framework forces target models to output their normally hidden chain of thought (CoT) sequences. The method utilizes an LLM based optimizer that follows an Inject-Reflect-Distill workflow, where an auxiliary model generates and refines injection strategies by analyzing batch performance and accumulating experience. This approach allows the framework to operate as a black box attack without requiring internal access to the target model parameters.
Interactive System Flowchart
Cross-Examination & FAQs
A deeper dive clarifying mechanics, constraints, and baseline evaluations.
Q1. What is the core problem addressed by this paper?
The paper investigates whether hidden chain of thought traces can be extracted from black box large reasoning models through API interactions.
Q2. What is EchoCoT?
EchoCoT is a multi-step attack framework that forces models to reveal hidden reasoning through targeted instruction injections.
Q3. Why is this research important for developers?
It highlights potential security risks regarding the recovery of proprietary reasoning assets from models deployed behind APIs.
Q4. How well does the EchoCoT-LTGO perform on standard benchmarks?
On the OpenThoughts test set, it achieves 22.8% to 46.1% ASR at 99 and 30.8% to 66.4% ASR at 90 across three target models.
Q5. What models were tested in the study?
The researchers launched attacks against Gemini-2.5, DeepSeek-V4-Flash, and open-source models from DeepSeek, Qwen, and GLM.
Q6. Does the model scale to long reasoning traces?
Yes, on Gemini-2.5, it successfully extracted 33,463 tokens from a 32,948-token target.
Q7. Can the extraction method transfer to new datasets?
Yes, optimized trajectories achieve up to 80% ASR at 90 across benchmarks including MATH500, JEEBench, and LiveCodeBench.
Q8. What are the limitations regarding evaluation of proprietary models?
Because ground truth chain of thoughts and original tokenizers are unavailable for frontier proprietary models, the quantitative fidelity evaluation is only a proxy estimate.
Q9. Did the authors assess the consistency of their extraction results?
The paper does not quantify run-to-run variability due to API and computational budget constraints.