Certified Training Against Motion Blur

Summary & Methodology Analysis

The researchers address the vulnerability of neural networks to convolutional perturbations like motion blur by introducing a training method that provides formal safety guarantees. The process begins by prepending specific layers to the target neural network to represent the convolutional perturbation as a linear mapping through parameterised kernels. During the training loop, the system computes a batch of perturbed images using pre-computed coefficient and bias tensors, effectively applying standard convolution operations within the model architecture. This setup allows for the direct integration of robustness constraints into the training pipeline. To ensure these constraints are met, the method employs Symbolic Interval Propagation, or SSIP, which serves as a technique to compute tight bounds on the model outputs. In some cases, this is combined with Reversed Symbolic Interval Propagation, or RSIP, to refine these bounds further. These propagated bounds are then used to calculate a robust cross-entropy loss alongside standard training loss, which guides the gradient descent process to update network parameters while maintaining verified robustness. Limitations regarding performance are present, as certified training involves a higher computational cost than standard Adversarial Training because of the required bound propagation steps. Furthermore, deep architectures like ResNet18 suffer a larger drop in standard accuracy during this process compared to shallower models due to strong regularisation. Additionally, the researchers found that RSIP based training was infeasible on the TinyImageNet dataset due to high memory requirements.