Linear Probing
A neural network interpretability technique in which lightweight linear classifiers are trained on the hidden states of individual model layers to detect whether a given layer's representations encode specific attribute information.
Core Facts
Timeline (last 90 days)
The smooth degradation at 60% sparsity was measured under a linear probing protocol, and the abstract does not provide data on whether performance is equally stable in downstream fine-tuning or zero-shot scenarios.
Through linear probing or a simple classifier, the language identity of each frame can be read out along the language selectivity dimension, without additional annotated data or large-scale fine-tuning.
A linear probe is an interpretability analysis technique that trains a simple linear classifier on the activations of a certain layer of the model, using classification accuracy to measure whether that layer linearly encodes a certain attribute.
All Facts (3)
A linear probe is an interpretability analysis technique that trains a simple linear classifier on the activations of a certain layer of the model, using classification accuracy to measure whether that layer linearly encodes a certain attribute.
75%UnverifiedThe smooth degradation at 60% sparsity was measured under a linear probing protocol, and the abstract does not provide data on whether performance is equally stable in downstream fine-tuning or zero-shot scenarios.
50%UnverifiedThrough linear probing or a simple classifier, the language identity of each frame can be read out along the language selectivity dimension, without additional annotated data or large-scale fine-tuning.
50%