Improving Video Frame Interpolation with Diffusion

Summary & Methodology Analysis

The SNM-VFI framework addresses the primary weakness of standard Video Frame Interpolation (VFI) techniques, which often rely on simplistic linear motion models that fail in complex scenes. To solve this, the architecture first estimates optical flow using the RAFT model and calculates latent representations via pre-trained video diffusion models like Stable Video Diffusion. By constructing multi-frame, symmetric nonlinear flow-based intermediate frames and associated confidence maps, the system explicitly captures complex motion patterns that linear models typically miss. These flow-based predictions act as a spatial prior, which initializes and guides the iterative denoising process of the diffusion component. This architectural integration ensures that the final video output is both structurally sound and visually detailed. The system ultimately employs a confidence-aware weighting mechanism to fuse the reliable flow-based structural data with the high-quality textures generated by the diffusion process. Performance results show that SNM-VFI excels in motion-heavy benchmarks, recording a PSNR of 22.8125 and LPIPS of 0.1811 on the KITTI dataset for x2 interpolation settings. Despite these gains, the method faces a persistent challenge where traditional metrics like PSNR may not perfectly align with human visual perception, as seen in cases where models like VFIFormer still report higher PSNR values.