Removing Glass Reflections from Videos

Summary & Methodology Analysis

Videos captured through glass often contain reflections that degrade visual quality and interfere with downstream vision tasks, but video reflection removal remains largely underexplored due to the lack of paired video data, temporally coherent removal models, and dedicated evaluation benchmarks. To solve this, the authors propose a workflow starting by filtering reflection-free videos from Ditto-1M using a Vision-Language Model, a system that processes both images and text, and applying FLUX frame-by-frame with reflection-injection prompts to obtain pseudo reflection videos. A structure-guided reflection renderer based on Wan2.1, a video generation model, reconstructs these pseudo reflection videos from clean transmission videos and lineart conditions. Physics-Grounded Augmentation primitives are applied to reflection lineart corresponding to glass optical parameters such as roughness, thickness, reflectance, partial coverage, and static appearance, and then fused with transmission lineart to render photorealistic reflected videos.

The system then executes Stage I reflection-aware latent adaptation via LoRA fine-tuning, a training method that adapts large models efficiently by updating a small subset of parameters, with residual-derived reflection-intensity supervision. Stage II executes one-step pixel-geometric refinement starting from a noisy latent with reconstruction, SSIM, and reflection-aware depth consistency losses. Full PGA reaches 34.13 dB PSNR, a measure of image reconstruction quality, and 0.9704 SSIM, a metric for structural similarity, representing a gain of +1.78 dB and +0.0061 SSIM over the planar-only setting.

Despite these advancements, limitations remain. Nested reflections from multiple glass layers remain challenging because the restoration target becomes ambiguous, causing the model to either leave slight residual inner-layer reflections or over-remove them. Additionally, the synthesis pipeline models temporally coherent reflections with clip-level controls, but does not explicitly simulate the coupled change between camera motion and reflection geometry, such as viewpoint-dependent reflection parallax.