Compressing 3D Vision Tokens for Efficient QA

Summary & Methodology Analysis

Projection based 3D vision language models generate thousands of tokens per scene, creating redundant 3D representations that lead to high computational and memory overhead. To solve this without losing the geometric coherence and semantic information needed for 3D question answering, the paper introduces 3DZip. The pipeline operates in three distinct steps. First, coarse voxelization partitions the 3D space into voxels of size delta to aggregate tokens using mean pooling, removing point level redundancy from multi-view projection. Second, feature diversity guided anchor selection uses a Determinantal Point Process, a probabilistic model of diversity, to select a subset of representative anchors that suppress object level redundancy by maximizing feature space diversity. Third, spatially constrained token merging assigns non anchor tokens to their most feature similar anchor only when the spatial distance is below a threshold tau g, which preserves geometric consistency while discarding isolated, non contributing tokens. For results, 3DZip outperforms existing 3D aware and 2D based baselines on the SQA3D benchmark at various token budgets, achieving 52.8 exact match at 64 tokens compared to 50.2 for DTC. The paper evaluates against numerous models and datasets including LLaVA 3D, Video 3D LLM, SR 3D, SQA3D, OpenEQA, ScanQA, ScanNet, HM3D, LLaMA, DeepSeek, Qwen, FastV, SparseVLM, VisionZip, VisPruner, and DTC. Regarding limitations, object state recognition tasks yielded high scores across all methods, suggesting that such queries are inherently easier and less dependent on complex spatial reasoning. The paper does not specify precise hardware requirements, dollar costs, or exact latency metrics beyond the reported benchmark scores.