Linear Probe

A linear probe is an interpretability analysis technique that trains a simple linear classifier on the activations of a certain layer of the model, using classification accuracy to measure whether that layer linearly encodes a certain attribute.

A linear probe is a simple linear classifier trained on model activations, used to detect whether specific behaviors or concepts exist inside the model.

The linear probe transfers to rotation failure detection on PhotoTourism and ScanNet-1500 without retraining.

A linear probe predicting pose failure on VGGT's frozen features achieved an ROC-AUC of 93.5% on unseen categories.

The linear probe is attached to features of VGGT's fusion layers and only needs to learn a single weight vector to output a score for whether an image pair will fail.

Linear concept erasure methods (such as LEACE) can guarantee that no linear classifier can predict the sensitive attribute after erasure, but they cannot prevent MLP probes from recovering the erased information.

The researchers trained linear probes on LLMs of multiple architectures and sizes to distinguish metaphorical from non-metaphorical expressions.

A linear probe classifying the selection arm achieves an accuracy of up to 59.7% (chance level 50%), indicating that the selection behavior leaves detectable traces in the model's representations.

Forward decoding accuracy alone cannot prove that the probed factor is aligned with the dominant geometry of the representation, and the success of a linear probe should not be simply equated with the model having learned a linear and low-dimensional representation.

The paper "Is a Linear Probe Evidence of a Linear Representation?" points out that from an asymptotic perspective, any sufficiently rich feature map—including random feature maps—admits a linear readout, so high probe accuracy is too permissive to prove linear representation.

Linear probe accuracy is significantly affected by feature dimension, sample size, and regularization strength, and high-dimensional random features can also achieve high linear separability.

In the kernel machine case, linear probes can still achieve high accuracy because random feature maps can also linearly separate data at sufficiently high dimensions, but the model has not learned a linear world model and its responses to out-of-distribution data may deviate completely from expectations.

Through linear probing analysis, VocalGrad research found two failure modes in LALMs: representations exist but the LM head does not exploit them, and information gradually becomes non-linearly decodable as it propagates across layers.

If you want a model to learn a new relation through fine-tuning, you should first check whether the corresponding old structure is already linearly encoded in its pretrained representation; for relations with poor linear readability, simply adding fine-tuning samples may yield limited benefit.

The researchers mapped labels that the fine-tuned model answered incorrectly back to the original answers to which they were initially assigned (for example, if the model answered Munich→zebra and zebra corresponds to Asia, this is restored to Munich→Asia), and these restored answers appear more often among the top answers of the linear probe for that subject than the answers the pretrained model itself gave.

Linear probes can be used to determine in advance which knowledge is worth investing fine-tuning in as a diagnostic path.

In experiments across 11 language models and 70 relations, the linear readability of the original relation (linear probe accuracy) strongly predicted generalization performance after fine-tuning, with a Pearson correlation coefficient of 0.87 to 0.92 across relations within the same model.

The study used linear probes to continuously track the degree of alignment at each turn of the conversation, making it possible to observe that the degree of alignment does not rise monotonically but may decay or even flip after several turns.

Through linear probe analysis, customer-goal attribution information becomes highly recoverable in the model's middle layers and is concentrated in specific attention heads.

In single-latent ablation experiments, the proportion of cases where the sign predicted by gradients matched the true sign reached 91%–97%, whereas the direction of change suggested by probes was correct only 17%–35% of the time.