Real-time Video Anomaly Detection via State-Space
Listen to the summary
Uses a voice available on your device
Audio options
On this page 4 sections
Related concepts 1 concepts
Key Takeaways
- Achieves frame-level AUC of 67.9% on UCSD Ped2 and 70.2% on CUHK Avenue.
- Delivers low-latency performance of 0.74 ms per frame on UCSD Ped2 and 0.77 ms per frame on CUHK Avenue using an Apple M3 Pro.
- Supports modular backbones including ImageNet-pretrained ResNet-18 or DINOv2 ViT-S/14.
- Implements a causal architecture that avoids the latency overhead of non-causal streaming buffer methods.
Summary & Methodology Analysis
The method processes video by mapping input frames into vector embeddings using a frozen ImageNet-pretrained ResNet-18 backbone. The authors also provide support for a heavier alternative using a frozen DINOv2 ViT-S/14. These embeddings are fed into a causal diagonal state-space core, which maintains a fixed-size state across time. Within each layer, an input-dependent event-boundary gate dynamically modulates the decay of the recurrence based on the current state and input, effectively identifying anomalies by calculating the Euclidean distance between the predicted and actual embedding at each frame.
Interactive System Flowchart
Cross-Examination & FAQs
A deeper dive clarifying mechanics, constraints, and baseline evaluations.
Q1. What is the primary contribution of this paper?
It provides a strictly causal streaming video anomaly detection system that eliminates the need for clip buffering and validates performance on actual edge hardware.
Q2. Does this approach require a GPU to run?
No, the paper validates its throughput and latency directly on an Apple M3 Pro.
Q3. How accurate is the system?
It reaches 67.9% frame-level AUC on the UCSD Ped2 dataset and 70.2% frame-level AUC on the CUHK Avenue dataset.
Q4. What happens if I use the DINOv2 ViT-S/14 model instead of ResNet-18?
The implementation supports it as a stronger, but heavier, alternative to the default ResNet-18 backbone.
Q5. Are the performance metrics final and fully optimized?
No, the results are from an untuned configuration, and the authors note that these figures trail prior non-causal state-space models.
Q6. How is the settling-delay theoretical bound derived?
It is based on a fixed decay, which is a loose upper bound because the implementation uses a time-varying gate once active.
Q7. How was the detection criteria evaluated?
The evaluation uses a simplified frame-overlap approximation rather than the original region- and track-based detection criteria specifications.
Q8. What is the processing speed of the model?
It achieves 0.74 ms per frame on UCSD Ped2 and 0.77 ms per frame on CUHK Avenue, which exceeds 1,300 FPS.
Q9. How does the model calculate anomaly scores?
It calculates the Euclidean distance between the predicted frame embedding and the actual embedding observed at the current timestep.