VERS: Editing Activation Explanations to Halve Reasoning Model Tokens

Reasoning language models generate chains of thought before giving an answer, but many of these checks do not affect the final result. Natural language autoencoders (NLAs) can explain a model's activations in text and reconstruct the activations from that text, thereby steering the model at the level of individual tokens. This paper investigates whether such editing can further guide how the model reasons. The researchers propose VERS (Verbalize, Edit, Reconstruct, Steer), which converts the explanations edited at "doubt points" along a reasoning trace into a fixed direction and adds this direction at paragraph breaks, thereby guiding the reasoning process.

How VERS Works

The core of VERS is the use of natural language autoencoders (NLAs) to produce textual explanations of model activations and reconstruct them. Specifically, it first verbalizes (generates explanations) at the "doubt points" of a reasoning trace (i.e., positions where the model is uncertain), then edits these explanations, reconstructs the activations through the NLA, and finally extracts the edited activation direction as a fixed vector. At inference time, this direction is added to the activations at each paragraph break, thereby guiding the model to reduce unnecessary reasoning steps.

Natural language autoencoders (NLAs) are a class of models that map a model's internal activation vectors to natural language explanations and can reconstruct the activations in reverse from that explanation. Their training typically uses an "explain-reconstruct" loop objective: given the activations of a certain layer, generate a textual description, then reconstruct the original activations from that text alone, optimizing the fidelity of the explanation with a reconstruction loss. The value of NLAs lies in providing a readable, editable semantic interface for high-dimensional activations that are otherwise difficult to interpret directly. Unlike sparse autoencoders (SAEs), which output feature directions, NLAs output natural language, so the activations can be changed indirectly by modifying the wording in the text, which makes "language-mediated" activation steering possible.

Doubt points refer to positions in a reasoning trace where the model exhibits uncertainty or self-questioning, such as tokens containing expressions like "wait" or "let me double-check." These positions often correspond to steps where the model performs redundant verification or repeated confirmation. VERS chooses to explain and edit at doubt points because these positions concentrate "checking" behavior, and editing their activations is most likely to change the model's decision about whether to continue checking. Extracting the edited activation change at doubt points as a fixed direction and then applying it at paragraph breaks is equivalent to generalizing the "reduce doubt" signal across the reasoning paragraphs of the entire reasoning trace.

Experimental Results: Significant Token Reduction, Slight Accuracy Drop

On the full GSM8K test set, VERS reduces the number of generated tokens by 45.8% while accuracy drops by only 0.82 percentage points (pp). On MATH-500, it saves 31.3% of tokens with a 3.05 pp accuracy drop. When the strength is halved, VERS's trade-off between tokens and accuracy falls between the published methods SEAL and DEER.

SEAL and DEER are two published methods that influence reasoning length by intervening in the model's internal state. SEAL typically learns or constructs a direction that controls the "thinking depth" of reasoning and injects it during generation to regulate the number of tokens; DEER compresses reasoning costs through dynamic early exit or by controlling the number of reasoning steps. The common idea of both is to trade off between token efficiency and accuracy. The key difference of VERS is that its steering direction comes from editing and reconstructing natural language explanations of activations, rather than from directly optimized or hand-set vectors. Therefore, at comparable token savings, its accuracy loss can be compared to measure the competitiveness of the method.

Transferability: The Direction Transfers to Larger Models

By fitting an affine mapping and adjusting the strength for each model, the direction can be transferred to larger Qwen3 models, up to 235B-A22B, saving 30-35% of tokens. This shows that the direction learned by VERS has a certain generality and does not depend on a specific model scale.

An affine mapping refers to a linear transformation plus a bias of the form y = Ax + b, used here to convert the steering direction obtained on a small model into the activation space of a larger model. Because models of different scales have different hidden dimensions and representation geometries, the direction cannot be copied directly; a mapping matrix A and bias b must be fitted from paired activations so that the small-model direction still has a steering effect on the large model after transformation. In addition, the applied strength (scaling coefficient) must be adjusted separately for each target model, because the same direction has different magnitudes of effect on different models. This transfer procedure indicates that the VERS direction captures semantic properties shared across scales, rather than an artifact specific to one model.

Limitations: Noticeable Accuracy Loss on Competition Problems

However, on competition problems, VERS loses 3-7 pp of accuracy across all model scales. In addition, the fixed direction fails to distinguish necessary checks from unnecessary ones. This means that VERS may not be applicable to tasks requiring rigorous reasoning, and its mechanism cannot yet precisely control which reasoning steps are skipped.

Conclusion

VERS demonstrates a new approach to guiding reasoning models by editing activation explanations. It is notably effective at reducing token consumption, and the direction is transferable. However, its accuracy loss on complex reasoning tasks and its inability to distinguish necessary checks suggest that more fine-grained control methods are needed in the future.

One-line Summary

VERS generates a fixed direction by editing activation explanations to guide reasoning, greatly reducing tokens but hurting accuracy on competition problems.

Paragraph Summary

This paper proposes VERS, which uses natural language autoencoders (NLAs) to produce and edit textual explanations of a reasoning model's activations at "doubt points," then reconstructs the activations and extracts them as a fixed direction. This direction is applied at paragraph breaks in the reasoning trace to guide the model to reduce redundant reasoning steps. On GSM8K, tokens are reduced by 45.8% with only a 0.82 pp accuracy drop; on MATH-500, tokens are reduced by 31.3% with a 3.05 pp accuracy drop. When the strength is halved, its trade-off falls between SEAL and DEER. Through an affine mapping, the direction can be transferred to Qwen3 models up to 235B-A22B, saving 30-35% of tokens. However, on competition problems it loses 3-7 pp across all model scales, and the fixed direction cannot distinguish necessary from unnecessary checks, indicating that the method still has limitations on rigorous reasoning tasks.