Improving Object Counting Using Depth Data

Summary & Methodology Analysis

The core methodology centers on the Depth-Guided Detector, or DG-Det. This component processes multi-scale RGB features by utilizing the backbone and Feature Pyramid Network structure of GroundingDINO. To incorporate depth, the system injects depth cues into these visual representations using a fusion module that models local depth relations, supplemented by global depth priors injected via Feature-wise Linear Modulation. This architectural integration allows the system to handle complex occlusion scenarios that typically degrade counting precision in standard RGB-based pipelines.

Beyond the primary detector, the system implements an occlusion-aware head to estimate foreground overlap for each object, enabling frame-level detection refinement. The tracking pipeline performs de-duplication by factoring in center displacement, appearance features, and depth differences into the trajectory matching cost. Finally, an occlusion-adaptive temporal voting mechanism filters trajectories to produce the total object count per video, ensuring consistency across frames while mitigating false detections caused by visual similarity.

Evaluation shows strong performance improvements, with the method achieving a 62.01 percent reduction in mean absolute error over baselines on the RGBD-VideoCount dataset. However, practical deployment faces specific constraints regarding input quality. The authors note that pseudo-depth estimation models often fail to preserve fine-grained object boundaries and lack the consistent absolute scale required for optimal precision. Furthermore, physical sensors exhibit significant noise or missing data in scenes involving transparent media like glass doors, where reflections and refractions distort the underlying geometry.