Optimizing Autoregressive Robotic Policy Deployment

Summary & Methodology Analysis

Autoregressive normalizing flows (a probabilistic model that maps simple distributions to complex data by predicting variables one at a time) typically provide exact likelihoods for robot policy learning. However, they suffer from high computational overhead because the inverse transformation required for action generation is sequential. RoMAN-Flow addresses this by training an initial policy via maximum likelihood estimation and then applying NF-IQL. This post-training step updates the actor using only offline dataset actions, bypassing the need for sequential sampling during the optimization phase. To solve the deployment latency, the paper uses one-step policy distillation, where a student network is trained to approximate the teacher's inverse transformation, enabling parallel action generation in a single forward pass. Performance results demonstrate that RoMAN-Flow achieves a success rate of 81.1 percent on MetaWorld-MT50 and 85 percent on RoboMimic Square-MH using an XL configuration with 685.5M parameters. The research notes that these high-precision tasks require significant model capacity, as smaller actor configurations failed to show performance improvements. The paper does not specify the exact latency improvements in milliseconds, but emphasizes that the primary architectural limitation remains the sequential generation bottleneck of the original autoregressive formulation, which the distillation process is designed to circumvent.