Predicting Robot Failures Before Touching Objects

Summary & Methodology Analysis

ContactGuard functions by intercepting the robot's control loop to evaluate the consequences of planned action sequences. The architecture relies on a latent world model, a predictive system that learns internal state representations, trained using a JEPA-style architecture with ViT-Tiny encoders. This model is trained on unlabelled trajectories via next-latent regression, which forces the system to predict future latent states from current ones, using SIGReg regularisation to stabilise the learning process. By analysing these predicted states, the system can determine if a planned movement will result in a failure before the hardware commits to the contact event.

The inference pipeline involves scanning the robot's policy for imminent gripper closures. Once an action chunk is identified, the system anchors its prediction just before the contact event and rolls out the frozen latent world model autoregressively to estimate the resulting state. A lightweight logistic-regression probe, which is a simple binary classifier, then evaluates this future latent representation. If the probe detects a failure condition, the execution is aborted. This modular approach allows the monitor to remain separate from the primary control policy while operating on the same input streams.

While effective at detecting failures, the system has clear operational constraints. The monitor is limited to identifying potential failures that occur within the immediate next action chunk, meaning it cannot look arbitrarily far into the future. Furthermore, once an abort signal is triggered, ContactGuard does not provide a recovery mechanism or a way to complete the task autonomously. These limitations mean the system currently serves as a safety-critical monitoring layer rather than a complete replacement for task-planning logic.