Optimizing Analog Circuits With Strategic Resets

Summary & Methodology Analysis

The paper addresses the sample inefficiency of traditional reinforcement learning and other optimization methods in analog circuit sizing, particularly their lack of generalization across different performance targets and wasted computational resources exploring unpromising parameter regions. The core methodology relies on standard actor-critic reinforcement learning, specifically Soft Actor-Critic, utilizing Graph Attention Network layers and multi-layer perceptrons to process circuit graphs and generate action distributions. Circuit performance objectives are evaluated via black-box simulation using Ngspice. The novel contribution of Lighthouse RL involves maintaining a priority queue of promising but incomplete solutions and a successful set of parameter configurations during an exploration phase. It then systematically selects high-performing parameter configurations discovered during training, termed lighthouses, to serve as strategic reset points for subsequent episodes. Finally, the system transitions to an exploitation phase where training episodes are initialized exclusively from the discovered lighthouse states to accelerate convergence without further expanding the exploration set.

The algorithm architecture processes inputs through Graph Attention Network layers, a type of neural network that processes graph structured data by attending to neighboring nodes, combined with multi-layer perceptrons. These components evaluate circuit graphs and generate action distributions for the Soft Actor-Critic agent, which is a reinforcement learning algorithm that maximizes both expected return and policy entropy for stability. The simulation backend relies on Ngspice for black-box evaluation of circuit performance objectives against designs based on the Skywater SKY130 PDK. During the process, the priority queue tracks incomplete solutions while successful parameter configurations feed into the lighthouse selection mechanism, which initializes subsequent episodes to bypass redundant exploration.

Despite the performance improvements, limitations remain in lighthouse state selection because their diversity currently depends on training dynamics rather than explicit diversity enforcement. The paper does not specify any additional architectural limitations, nor does it provide details on compute infrastructure scale or training time outside of the reported inference steps and success rates. The evaluated models include the two-stage OpAmp and the multistage amplifier, tested across both within distribution evaluations and extrapolation beyond the training distribution.