Building Smaller Faster Document Retrieval Models
Listen to the summary
Uses a voice available on your device
Audio options
On this page 4 sections
Related concepts 4 concepts
Key Takeaways
- DistilVDR replaces massive multi-billion-parameter systems with an efficient single-vector architecture.
- The HiRes variant retains 86.9 percent of the performance of an 8 B parameter teacher model.
- The system enables a 15.6 times smaller index for one million documents compared to current sub-1 B multi-vector baselines.
- The approach uses a dynamic-tiling strategy to process document images based on their aspect ratio.
Summary & Methodology Analysis
The methodology employs a distillation process, a technique where a small student model is trained to mimic the output of a larger teacher model, to compress visual document retrieval capabilities. A frozen 8 B parameter vision-language teacher generates target embeddings for both queries and documents. Student encoders are then trained using a pointwise cosine alignment loss to regress onto these cached teacher embeddings. By discarding the teacher after training, the deployment phase only requires the compact student encoders, which utilize an asymmetric architecture: a document tower composed of an InternViT-300M visual encoder and a ModernBERT-base text backbone, and a 70 M parameter DistilBERT-base text-only query tower. This document tower incorporates a dynamic-tiling strategy that splits images into a grid of tiles up to a set budget to maintain efficiency.
Interactive System Flowchart
Cross-Examination & FAQs
A deeper dive clarifying mechanics, constraints, and baseline evaluations.
Q1. What is the primary benefit of DistilVDR?
It provides a compact, end-to-end, single-vector document retrieval system that is faster and smaller than existing multi-billion-parameter models while maintaining high retrieval performance.
Q2. Does this model work for both images and text?
Yes, it is designed for visual document retrieval where it handles both document images and query text modalities.
Q3. Can I use this for production systems?
The system is optimized for smaller indices and faster performance, though the paper notes that evaluations are limited to the ViDoRe benchmark and may not reflect performance on in-house enterprise corpora.
Q4. How does the document encoder handle different image sizes?
It uses a dynamic-tiling strategy that splits images into a grid of tiles based on their aspect ratio up to a defined budget.
Q5. How do these student models compare to larger multi-vector baselines?
DistilVDR-HiRes retains 86.9 percent of an 8 B parameter teacher's performance, though variants perform 7-10 points lower in NDCG@5 than the strongest 4-8 B multi-vector models.
Q6. What happens to the large teacher models after training?
The teachers are discarded after training to leave only the smaller student encoders for deployment.
Q7. What specific architectures are used for the encoders?
The document encoder uses an InternViT-300M visual encoder and a ModernBERT-base text backbone, while the query encoder uses a DistilBERT-base text backbone.
Q8. How efficient is the indexing compared to other sub-1 B models?
DistilVDR stores one million documents in an index that is 15.6 times smaller than the strongest sub-1 B multi-vector baseline.
Q9. Are there limitations to the training approach?
Yes, the students are restricted to reproducing the teacher's embeddings and cannot exceed the teacher's performance on the replaced side.