Back to Feed
Agents / Benchmarks & Evals

Improving Information Retrieval via Agent Collaboration

Original: SearchOS-V1: Towards Robust Open-Domain Information-Seeking Agent Collaboration

Listen to the summary

Uses a voice available on your device

Audio options
On this page 4 sections
Related concepts 4 concepts

Key Takeaways

  • Addresses repetitive search loops and budget waste in long-horizon tasks through state externalization.
  • Implements a Search-Oriented Context Management system to track execution progress via evidence graphs and coverage maps.
  • Achieved 80.3 item-level F1 on the WideSearch benchmark.
  • Achieved 76.5 set F1 on the GISA benchmark.
  • Employs a middleware harness to intercept model and tool interactions for real-time stall detection.

Summary & Methodology Analysis

SearchOS-V1 manages information-seeking tasks by treating them as relational schema completion problems with grounded citations. The core mechanism is Search-Oriented Context Management (SOCM), which offloads execution state into four primary persistent artifacts: a Frontier Task, an Evidence Graph, a Coverage Map, and Failure Memory. This structural approach allows the agent to maintain context over long-horizon sessions where standard state tracking often fails, preventing common issues like repetitive search loops and inefficient resource utilization. A pipeline-parallel orchestration mechanism manages the task queue, dispatching sub-tasks to resolve specific coverage gaps identified in the map. The system also includes a middleware harness that acts as a control layer, intercepting calls between the model and external tools. This layer enforces budgets and performs automated stall detection during the inference process, ensuring that the agents do not dwell on unhelpful search paths. The framework utilizes a hierarchical skill system to provide the agents with reusable strategy access, which helps standardize how agents interact with the search tools. Performance evaluations demonstrate the efficacy of this orchestration, yielding an item-level F1 of 80.3 on WideSearch and a set F1 of 76.5 on the GISA benchmark. However, the system faces notable limitations in its current implementation. The authors note that the framework lacks an automated process for synthesizing new search agent skills from raw data sources. Furthermore, they acknowledge that current prompt-level safeguards are insufficient for the secure governance of heterogeneous post-trained agents, representing a significant gap in the reliability of deployments involving diverse models like GLM-5 or Qwen3.5-35B-A3B.

Interactive System Flowchart

Click diagram to expand and zoom

Cross-Examination & FAQs

A deeper dive clarifying mechanics, constraints, and baseline evaluations.

Q1. What is the primary goal of SearchOS?

The goal is to improve the reliability and efficiency of information-seeking agents during long-horizon tasks.

Q2. How does SearchOS handle task progress?

It externalizes the execution state into shared, persistent artifacts like evidence graphs and coverage maps.

Q3. Does this system improve search accuracy?

Yes, it achieved an 80.3 item-level F1 on the WideSearch benchmark and 76.5 set F1 on the GISA benchmark.

Q4. What is the role of the Search Tool Middleware Harness?

It intercepts interactions between the model and tools to handle context preparation, evidence extraction, budget enforcement, and stall detection.

Q5. What models and datasets were used in the research?

The researchers used the WideSearch and GISA benchmarks, along with models such as GLM-5 and Qwen3.5-35B-A3B.

Q6. How does the system manage sub-task dispatching?

It uses a pipeline-parallel orchestration mechanism that targets unresolved coverage gaps continuously.

Q7. Does the system automatically update its own search skills?

No, the paper notes that it does not currently automate the synthesis of large-scale search agent skills from data sources.

Q8. Are there known security limitations for this agent framework?

Yes, the authors acknowledge that current prompt-level safeguards are insufficient for governing heterogeneous post-trained agents.

Q9. How does this compare to baselines like ReAct or Plan-and-Solve?

The paper includes those as baselines but does not provide specific performance comparisons against them in the provided facts beyond the final benchmark scores.