Back to Feed
Robotics / Multimodal

Helping Underwater Robots Navigate When Sensors Fail

Original: AquaJEPA: Action-Conditioned Multimodal Predictive Representations for Underwater Robot Dynamics
By Alan-Barsag Gazzaev, Alexey Gavrilov, Sergey Muravyov Published 2026-07-31

Listen to the summary

Uses a voice available on your device

Audio options
On this page

Key Takeaways

  • AquaJEPA helps robots understand the physical consequences of their thruster commands, which is essential for control when sensors are unreliable.
  • The system uses a smart fusion method that combines data from cameras, sonar, and internal movement sensors to maintain awareness during sensor failures.
  • In simulation tests, the model successfully reached more goals and maintained lower navigation error rates than existing baseline approaches.
  • The training process includes a deliberate drop in sensor data, which teaches the robot to remain robust and perform well even when it loses specific inputs.

Summary & Methodology Analysis

The researchers designed AquaJEPA to address the problem of unreliable sensing in underwater environments where visibility changes or motion makes navigation difficult. To solve this, they implemented a system that predicts future outcomes based on the robot's current state and intended actions. The model uses separate encoders, which are specialized software modules that translate complex raw data into a digital format, for different sensors like cameras and sonar. These inputs are fed into a fusion module that is aware of data masking, allowing the system to combine information even if some streams are missing.

Interactive System Flowchart

Click diagram to expand and zoom

Cross-Examination & FAQs

A deeper dive clarifying mechanics, constraints, and baseline evaluations.

Q1. What is the main problem this paper solves?

The paper addresses the challenge of robots losing their ability to see or sense accurately underwater due to poor visibility or vehicle movement.

Q2. How does this research help robots perform better?

It provides a system that predicts future movement outcomes based on thruster commands, allowing the robot to continue navigating even if its sensors fail.

Q3. Did the researchers test this on real robots?

No, the research was conducted using the Stonefish simulator to model underwater environments and robot physics.

Q4. What is the role of the action encoder in this model?

The action encoder uses a Gated Recurrent Unit, a specific type of memory-based computer layer, to summarize sequences of thruster commands.

Q5. How does the training process ensure the robot is robust?

The system applies modality dropout, which is the random removal of sensor inputs during training, and a counterfactual action margin to force the model to stay sensitive to action consequences.

Q6. How did AquaJEPA perform compared to other models?

In tests, AquaJEPA reached 74 goals compared to 68 for baseline models and achieved a lower final error rate of 0.906 meters.

Q7. What sensors does the model use to make predictions?

It uses RGB cameras, forward-looking sonar, and proprioception, which is the robot's internal sense of its own movement and position.

Q8. What are the limitations of this study?

The study is limited to one robot model and simulator, does not independently test every architectural component, and uses controlled proxies rather than real-world water conditions.

Q9. What is a counterfactual action margin?

The paper does not specify the exact mathematical definition of the counterfactual action margin, but it notes that it is used during training to improve action sensitivity.