Benchmarking Robots for Finding and Following
Listen to the summary
Uses a voice available on your device
Audio options
On this page 4 sections
Related concepts 5 concepts
Key Takeaways
- Existing benchmarks only test following, whereas UESF-Bench requires agents to first locate a target before following it.
- The SeekFollow-VLA model uses a task-driven router to balance seeking and following, achieving a task success rate of 0.35 in single-person settings.
- In multi-person environments, the proposed router approach reached a task success rate of 0.20, outperforming single-head baselines.
- The simulation environment integrates 1.43 million samples generated via Habitat 3.0.
Summary & Methodology Analysis
The research addresses the limitation of current embodied benchmarks that assume targets are always visible. To solve this, the authors developed UESF-Bench, a framework that frames navigation as a partially observable sequential decision problem where an agent must first explore to find a language-described target. The architecture uses a Qwen3-4B backbone, a type of large language model used here to process sequence data, fed by DINO-V3 and SigLIP, which are computer vision models used for visual feature extraction. These visual inputs are combined with language instructions to provide the model with context for its environment.
Interactive System Flowchart
Cross-Examination & FAQs
A deeper dive clarifying mechanics, constraints, and baseline evaluations.
Q1. What is the primary problem this paper addresses?
Current embodied benchmarks assume targets are already visible, failing to account for the need to first search for a target before following it.
Q2. What is UESF-Bench?
It is a new benchmark and probing framework designed to test unified embodied seeking and following tasks.
Q3. Did the model perform well?
Yes, it significantly outperformed single-head baselines in both single-person and multi-person environments.
Q4. What architecture does the SeekFollow-VLA model use?
It uses a dual-head architecture featuring a Task-Driven Router that dynamically balances seeking and following behavior based on implicit stage inference.
Q5. Which models and tools were used for encoding observations?
The model uses DINO-V3 and SigLIP to encode visual observation history, combined with a Qwen3-4B backbone.
Q6. What training signals were used for the router mechanism?
The model was trained using waypoint prediction loss and an auxiliary stage-aware cross-entropy loss.
Q7. What is the specific task success rate (TSR) improvement in the multi-person setting?
The task-driven router approach achieved a TSR of 0.20 compared to 0.04 for other baselines.
Q8. Are there any known drawbacks to this approach?
The paper notes that the task-driven dual-head model exhibits a higher collision rate compared to other variants.
Q9. What datasets were utilized in the simulation environment?
The environment relies on Habitat 3.0 and uses assets including SMPL-X, ATLAS, HM3D, and MP3D.