Using Temporal Distance for Robot Training

Summary & Methodology Analysis

RynnValue addresses the scaling bottleneck in robotic learning caused by reliance on task-internal anchors like human preferences. The authors trained the model on a heterogeneous dataset spanning over 7,000 hours and 3M instruction-conditioned trajectory clips. By deriving temporal-distance labels directly from timestamps and applying a completion cutoff to each trajectory, the model learns to predict both absolute time to completion and relative displacement between observations. To prevent the model from learning shortcuts, the team implemented random temporal sampling and order shuffling, ensuring the model decouples observation sequence position from actual temporal distance. This system uses value-isolation attention, a mechanism that restricts queries to specific language-visual contexts to avoid value extrapolation, and potential-based shaping to convert predictions into dense rewards for policy learning.

The architecture leverages RynnBrain as its core visual-language backbone. This integration allows the model to handle diverse robotic inputs, including real-world, simulation, and egocentric data. By predicting progress through temporal distance, RynnValue creates a scalable reward function that generalizes better across various robot embodiments than methods restricted to narrow task-specific anchors. The framework demonstrates significant empirical gains, with RynnValue-8B yielding a Kendall’s τa of 0.675 on the RBM-EVAL-OOD suite, surpassing preference-supervised models at 0.655 and progress-only baselines at 0.292. In practical testing, the application of potential-based shaping boosted real-world policy success from 52.5% to 72.5% and offline RL success from 63.8% to 82.5% compared to the strongest existing baselines.

Despite these performance improvements, the model faces specific technical constraints. Current estimations are limited to short windows of observations, which restricts the model utility for long-horizon tasks. Additionally, the visual models may struggle with precision-sensitive operations where subtle physical states, such as grasp stability or exact placement, are difficult to distinguish visually. The paper does not specify the exact latency or memory overhead for inference, but notes that these visual ambiguities remain a bottleneck for high-precision scenarios.