SLIC: Reinforcement Fine-Tuning Small Models to Master Multi-Turn Analog Circuit Optimization

Why Analog Circuit Design Needs a "Small and Specialized" Language Model

Analog integrated circuit design has long been regarded as a "craft" in engineering: engineers must repeatedly trade off among multiple coupled performance metrics and iterate adjustments round after round with the help of simulators. This multi-turn, simulator-driven task characteristic is, in theory, highly suitable for language model intervention. However, the real obstacle is that analog design relies heavily on proprietary process design kits (PDKs) provided by vendors, and hosted large-model workflows mean uploading sensitive data to external services—something most real design scenarios find unacceptable.

According to the SLIC (Small LM Iterating on Circuits) framework published by the research team, a more pragmatic route is: instead of pursuing parameter scale, train a 4B-level small model that can be deployed locally and master closed-loop optimization capability through reinforcement fine-tuning. This approach directly addresses the deployment requirement of "data never leaving local premises," and offers a new solution to the tension between model scale and engineering usability.

PDK (Process Design Kit) is a collection of files provided by foundries to design companies, typically including device models (SPICE models), layout design rules, parasitic extraction rules, and technology files for simulation. It is the bridge connecting circuit design and actual manufacturing processes, and directly determines whether simulation results are credible. Because PDKs contain foundry process secrets and large amounts of proprietary parameters, they are usually bound by strict confidentiality agreements and cannot be freely uploaded to third-party services. This is precisely the fundamental reason why "data never leaving local premises" has become a rigid requirement in analog design—if a model is to participate in real design workflows, it must be able to securely read and invoke PDK data locally.

The Three Technical Pillars of SLIC

The SLIC method design revolves around "how to provide effective learning signals in a sparse, strongly constrained search space," and consists mainly of three parts.

Structured Representation + Discrete Relative Update Actions

The framework represents circuits in a structured form and defines optimization actions as discrete relative updates, rather than letting the model directly generate arbitrary parameter values. This constraint significantly narrows the action space, making every step in multi-turn iteration interpretable and reproducible, and reducing the risk of the model "going off track" in continuous parameter space.

Reinforcement Fine-Tuning (RFT) differs from conventional supervised fine-tuning in that it does not rely on labeled "standard answers"; instead, it lets the model sample actions in an environment, obtain reward signals, and update its policy accordingly. In SLIC, the environment is the circuit simulator, actions are adjustments to circuit parameters, and rewards come from multi-objective performance evaluation. This approach is particularly suited to optimization tasks where "there is no single correct answer, only better or worse," and is also the key training paradigm that enables small models to acquire closed-loop iterative capability.

Domain- and Success-Gated Multi-Objective Reward

The difficulty of analog circuit optimization lies in the mutual constraints among multiple performance metrics; a single reward can easily induce the model to sacrifice one metric for another. SLIC adopts a domain- and success-gated multi-objective reward mechanism, filtering out attempts that fail to satisfy basic domain constraints, and then allocating rewards based on multi-objective achievement. This focuses policy learning on the region that is "both feasible and excellent."

Reward-Guided Rollout Steering (RGRS)

Reward-Guided Rollout Steering (RGRS) is one of the more distinctive components of the framework: it converts each step's simulator feedback into signals used simultaneously for policy learning and online rollout construction. In scenarios with sparse rewards and strong constraints, learning efficiency from the final success signal alone is extremely low; RGRS essentially "navigates" the exploration process with intermediate feedback, improving sample efficiency.

Sparse Reward refers to environments that provide effective feedback only when the task ultimately succeeds, with almost no discrimination among intermediate steps. In circuit optimization, if four metrics still fail to all meet targets after a parameter adjustment, there is often no clear "sense of direction," and the model struggles to judge whether this step is good or bad. Sparse rewards lead to low exploration efficiency and unstable training. The core contribution of RGRS is precisely to transform the simulator's intermediate feedback into dense guiding signals, alleviating the learning difficulties caused by sparse rewards.

Benchmark and Results: Small Models Outperforming Very Large Models

The research team also released an open goal-conditioned benchmark containing 41 amplifier topologies, along with a standardized protocol for in-distribution validation, held-out topology evaluation, and cross-process-node transfer testing. The significance of this benchmark is that it turns "circuit optimization capability" from scattered case studies into a horizontally comparable evaluation target.

Trained using only the open-source 65 nm CMOS predictive technology model (PTM), SLIC-Random achieved 87.1% four-metric in-distribution Pass@1 within eight rounds of dialogue. According to the original data, this result is 48.5 percentage points higher than the 763B-parameter DeepSeek^Ad-V4.1-Flash Think. This comparison shows that in highly structured engineering tasks, domain-aligned training methods may be more effective than simply stacking parameters.

In out-of-distribution scenarios, SLIC-Softmax achieved 72.6% Pass@1 on held-out topologies; the trained policy can also transfer directly to five unseen process nodes without additional fine-tuning. More importantly, this transfer capability extends to a commercial TSMC 65 nm CMOS PDK: SLIC-Random achieved 49.9% Pass@1 on held-out topologies, with fully local inference throughout—proprietary data need not be transmitted to external LLM services.

Pass@1 is a commonly used evaluation metric in code generation and program synthesis, meaning "the probability that the model passes the test with only one generated answer." This article borrows this metric to measure circuit optimization: four-metric Pass@1 refers to the proportion of circuit solutions proposed by the model that simultaneously meet targets on four metrics—gain, bandwidth, power consumption, and phase margin. Compared with Pass@k (which allows sampling k times and taking the best), Pass@1 more strictly reflects the reliability of a single output and is closer to the determinism required in engineering deployment.

Test-Time Scaling: Additional Gains from Multi-Turn Interaction

Another noteworthy feature of SLIC is that performance grows with the number of simulator interactions at test time. The original article shows that under a 32-round budget, held-out topology Pass@1 can rise to 83.2%. This means deployers can flexibly adjust optimization depth according to compute budget, rather than being locked into fixed inference costs. For tasks like circuit design where "a few more rounds of simulation can buy better results," this scalability has direct engineering value.

Some Industry Implications

The value of SLIC lies not only in circuit design itself. It demonstrates a reusable paradigm: in professional engineering domains that are data-sensitive, constraint-dense, and require multi-turn iteration, localized small models combined with domain-customized reinforcement fine-tuning may be closer to deployment conditions than hosted calls to general-purpose large models. Of course, the 49.9% pass rate on held-out topologies with a commercial PDK also suggests that there is still a gap from open-source models to real production lines; the stability of cross-node transfer and performance at more complex circuit scales are directions that warrant further observation.

---

Reference source: SLIC: Reinforcement Fine-Tuning Small LMs for Multi-Turn Analog Circuit Optimization (RSS feed)