HRL-Topo: Hierarchical Reinforcement Learning with Large Language Models for Analog Circuit Topology Synthesis

Automatic synthesis of circuit topologies is challenging due to the large combinatorial design space, strict connectivity and component constraints, and costly simulator evaluations. Existing approaches including genetic algorithms, graph- or LLM-based methods often produce invalid or suboptimal netlists and fail to exploit reusable design abstractions. We present HRLTopo, a hierarchical RL framework with both high-level low-level policies to synthesize valid, high-performance circuit topologies. Policies are implemented with LLMs and optimized through RL, where the high-level policy learns to propose semantic subgoals that guide exploration, and the low-level policy learns to generate incident-encoded netlists satisfying constraints while achieving downstream objectives. A structured reward is also proposed to drive this hierarchy. Experiments on power converter topology synthesis show that HRL-Topo produces more valid and higher-quality circuits, improves sample efficiency, and outperforms baselines. Proposed hierarchical RL, incorporating LLM-based goal generation and constraint-aware netlist construction, provides an effective and scalable approach for design automation.