Classification Head
The final linear layer in a neural network used for classification tasks. Its weight matrix has the shape [hidden_size × num_classes], and its output dimension is strictly equal to the number of classes, making it non-trivially extensible once the dimension is fixed.
Core Facts
Timeline (last 90 days)
Training only the classification head (freezing the entire backbone) suits scenarios with a sample size in the hundreds to thousands and with classes semantically close to ImageNet, but the model's ability to adapt to domain shift is almost zero.
The output of a judgment-type decision model is a closed, structured result, which is essentially a classification or regression task involving a single forward pass, with no autoregressive iteration overhead.
A classification head is typically a simple fully connected layer with Softmax activation that maps a high-dimensional feature vector to a probability distribution over each category.
In a hybrid architecture, only the lightweight classification head needs to run at inference time without calling the full large model each time, thereby reducing latency and cost.
All Facts (4)
The output of a judgment-type decision model is a closed, structured result, which is essentially a classification or regression task involving a single forward pass, with no autoregressive iteration overhead.
65%UnverifiedTraining only the classification head (freezing the entire backbone) suits scenarios with a sample size in the hundreds to thousands and with classes semantically close to ImageNet, but the model's ability to adapt to domain shift is almost zero.
50%UnverifiedA classification head is typically a simple fully connected layer with Softmax activation that maps a high-dimensional feature vector to a probability distribution over each category.
50%UnverifiedIn a hybrid architecture, only the lightweight classification head needs to run at inference time without calling the full large model each time, thereby reducing latency and cost.
50%