Scaling VARMA Models for Efficient Estimation

Summary & Methodology Analysis

The paper addresses the computational bottlenecks of estimating VARMA models, which typically suffer from non-convex likelihoods and evaluation costs that scale linearly with time series length T. To solve this, the authors implement an approximation of the infinite-autoregressive representation by truncating the process at a finite lag m. By reparameterizing the model using partial-autocorrelation, they enforce stationarity and invertibility by construction. This architectural shift allows the algorithm to bypass the standard challenges of standard maximum-likelihood estimation in high-dimensional settings.

From an implementation perspective, the framework creates a set of fixed-size sufficient statistics. This design decouples the loss function evaluation from the series length, effectively providing a stable performance baseline regardless of input size. To optimize the compute load during the calculation of these statistics, the authors utilize Parseval (Fourier) evaluations, which successfully reduce the complexity from O(m squared) to O(m log m). Optimization is then performed via L-BFGS to minimize either a least-squares prediction loss or a covariance-marginalized maximum-a-posteriori loss.

The framework is compatible with various structures including VAR, VARMAX, and component-wise ARMA, providing a versatile tool for high-dimensional time series analysis. Despite these improvements, the method has two primary limitations. First, it does not resolve the underlying identifiability issues inherent to VARMA models. Second, the truncated approximation introduces a bias that is dependent on the chosen truncation length m, which represents a trade-off between computational efficiency and model fidelity.