Xiaomi-Robotics-1: Scaling Robot Action Models

Summary & Methodology Analysis

Robotics research faces a significant data bottleneck due to the slow, costly, and non-scalable nature of traditional real-robot teleoperation. This paper addresses this by building a foundational vision-language-action (VLA) model at scale and developing a scalable auto-labeling pipeline to leverage massive, diverse datasets. Key novel methods include a two-stage training recipe, comprising pre-training on over 100k hours of UMI trajectory data and post-training on 10k hours of cross-embodiment data. An auto-labeling pipeline uses a pre-trained Vision-Language Model (VLM) to caption trajectory clips with scene state transition descriptions, and a cross-embodiment alignment technique maps descriptive state-transition understanding into imperative task instructions.

Xiaomi-Robotics-1 employs a Mixture-of-Transformers architecture, coupling a pre-trained Vision-Language Model (VLM) with a Diffusion Transformer (DiT). This architecture uses Choice Policies to generate candidate action chunks and associated scores within the VLM. It also utilizes Flow-matching to generate action chunks conditioned on robot state and the VLM's KV cache. The training process benefits from amortized training, achieved through pack-and-process optimization of VLM/DiT forward passes.

Xiaomi-Robotics-1 achieved a 57.4% success rate on the RoboCasa365 benchmark and an average score of 20.07 on the RoboDojo benchmark. In low-data downstream fine-tuning, the model achieved a 75% average success rate on four new tasks. However, a limitation noted is that the model performs lower on the memory dimension of the RoboDojo benchmark compared to models that explicitly model memory. The authors also observed early termination due to overfitting in data-scaling experiments.