Activation Steering

Model steering influences model output behavior during inference by directly modifying the activation vectors of intermediate layers, without modifying weights.

Activation steering is an inference-time intervention technique that requires no model retraining: at each token generation step, a scaled direction vector is added to the hidden state of a specific layer (h ← h + α·v), where α controls the intervention strength and v is the target direction.

Recent research on representation engineering and activation steering has shown that certain semantic or stylistic attributes (such as sentiment倾向 and refusal behavior) have approximately linear directions in activation space, and adding or subtracting vectors along that direction can modulate model output.

Unlike fine-tuning, activation steering takes effect immediately, allows the strength to be adjusted sentence by sentence, and does not change the model weights.

The researchers steer the generation process via activation vectors, causing measurable changes in the metaphor density of model outputs.

Knowledge extraction enhancement (KEE) techniques include three categories—activation steering, decoding adjustments, and prompt engineering—which modify the inference process and may reinforce existing predictions or surface answers that are otherwise hard to reach.

Activation steering is a technique that intervenes in the model's internal hidden-layer representations at inference time, with representative methods including contrastive activation addition (CAA) and inference-time intervention (ITI).

Activation-based steering directly intervenes in the model's internal activation directions, whereas prompt-based steering influences indirectly through input text, and the difference in their performance under accumulated multi-turn error is the core of the study's comparison.

The basic operation of activation steering is to find an activation direction vector representing a certain concept and add it to intermediate-layer activations during inference, thereby intervening in the output without changing the weights.

Linear steering relies on the linear representation hypothesis; when the concept distribution itself is curved, shifting activations along a fixed direction deviates from the true concept manifold, leading to control distortion or unnatural intermediate states.

A study published on arXiv (arXiv:2609.38205) investigated the effects of system prompts on internal computations within Transformers.

Activation steering or representation engineering applies intervention vectors directly to the model's internal activation layers, altering the generation direction without modifying the model's weights.

Injecting vectors into the residual stream is a commonly used causal intervention technique in mechanistic interpretability, acting directly on the model's intermediate representations rather than at the vocabulary level.

In language model-driven war games, the model's language output simultaneously determines what an agent attempts to do and what becomes the simulated reality.