Low Cost Open Source Stereo Capture System
Listen to the summary
Uses a voice available on your device
Audio options
On this page 4 sections
Related concepts 3 concepts
Key Takeaways
- The system costs approximately USD 200 per unit in commodity parts.
- The dataset created with this hardware, Ego-OSCAR-550h, contains 550 hours of egocentric stereo video.
- The pipeline achieves a 96 percent usable-session rate with a residual visual-inertial synchronization drift of 700 microseconds.
- Hardware design utilizes clock bridging and real-time watchdog mechanisms to manage synchronization between the camera, SBC, and IMU.
Summary & Methodology Analysis
The Ego-OSCAR hardware stack relies on commodity components including a Dexcin USB stereo camera, a Radxa Rock 5C single-board computer (SBC), and a TDK InvenSense ICM-20948 6-axis IMU. To manage data ingestion, the system utilizes a Seeed Studio Xiao ESP32-S3 microcontroller that captures Start-of-Exposure pulses from the cameras and merges them with high-frequency IMU streams. The SBC handles video processing via Rockchip's Media Process Platform, enabling hardware-accelerated MJPEG decoding and H.264 encoding. A watchdog mechanism monitors a 1 Hz heartbeat from the SBC to detect and signal system hangs during recording sessions.
Offline processing is required for final data alignment. The system identifies LED flashes in the video stream to map ESP32 interrupts to specific video frame indices. Per-session calibration uses a chessboard to derive pinhole camera intrinsics and radial-tangential distortion coefficients. This methodology supports the construction of the Ego-OSCAR-550h dataset, which provides an alternative to existing platforms like Ego4D, Ego-Exo4D, EPIC-KITCHENS, and Project Aria. While the system is effective for capture, it integrates with external frameworks like VINS-Fusion and RAFT-Stereo for downstream analysis.
Limitations of the current implementation center on data quality and operational control. The device cannot perform real-time rejection of faulty sessions, necessitating post-capture review. Furthermore, the reliance on consumer-grade IMUs introduces sources of pose error. The paper also notes that while the system facilitates data collection, it does not provide ground truth for camera pose and does not demonstrate policy performance gains in specific downstream tasks like those found in UMI, Mobile ALOHA, or other robotics applications.
Interactive System Flowchart
Cross-Examination & FAQs
A deeper dive clarifying mechanics, constraints, and baseline evaluations.
Q1. What is the primary goal of Ego-OSCAR?
The project addresses the need for a low-cost, fully open-source, stereo-inertial capture device that can be freely reproduced for research.
Q2. How much does the hardware system cost?
The complete bill of materials is approximately USD 200.
Q3. What kind of data does this system produce?
It produces egocentric stereo video paired with hardware-synchronized inertial data.
Q4. How is synchronization achieved between the camera and the IMU?
The system uses clock bridging where a microcontroller captures camera Start-of-Exposure pulses and merges them with high-frequency IMU data.
Q5. Does the system provide real-time feedback on session quality?
No, the device cannot reject bad sessions in real time.
Q6. What is the recorded drift for visual-inertial synchronization?
The residual visual-inertial synchronization drift is 700 microseconds.
Q7. Does the paper demonstrate performance gains for robotics policies?
No, the system does not demonstrate policy performance gains.
Q8. Are there limitations regarding ground truth data?
Yes, the system does not provide ground truth for camera pose.
Q9. How does the system handle potential hardware hangs during operation?
A real-time watchdog mechanism on the microcontroller monitors a 1 Hz heartbeat from the SBC and signals error states if the system hangs.