Activation Steering with a Feedback Controller

View PDF

Abstract:Controlling the behaviors of large language models (LLM) is fundamental to their safety alignment and reliable deployment. However, existing steering methods are primarily driven by empirical insights and lack theoretical performance guarantees. In this work, we develop a control-theoretic foundation for activation steering by showing that popular steering methods correspond to the proportional (P) controllers, with the steering vector serving as the feedback signal. Building on this finding, we propose Proportional-Integral-Derivative (PID) Steering, a principled framework that leverages the full PID controller for activation steering in LLMs. The proportional (P) term aligns activations with target semantic directions, the integral (I) term accumulates errors to enforce persistent corrections across layers, and the derivative (D) term mitigates overshoot by counteracting rapid activation changes. This closed-loop design yields interpretable error dynamics and connects activation steering to classical stability guarantees in control theory. Moreover, PID Steering is lightweight, modular, and readily integrates with state-of-the-art steering methods. Extensive experiments across multiple LLM families and benchmarks demonstrate that PID Steering consistently outperforms existing approaches, achieving more robust and reliable behavioral control.

Comments: 10 pages in the main text. ICLR2026 Poster

Subjects:

Machine Learning (cs.LG)

Cite as: arXiv:2510.04309 [cs.LG]

(or arXiv:2510.04309v2 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2510.04309

arXiv-issued DOI via DataCite

Submission history

From: Nguyen Viet Dung [view email] [v1] Sun, 5 Oct 2025 18:05:28 UTC (38,145 KB) [v2] Fri, 27 Mar 2026 06:33:50 UTC (21,106 KB)