Verified70% confidenceFactExact time
Model distillation softens the teacher model's softmax outputs via temperature scaling, allowing information from low-probability classes to be transferred to the student model.
4
Sources
70%
Confidence
Long-term
Relevance
8/28/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedHard distillation refers to the student model directly learning the final output labels or generated text of the teacher model, while soft distillation requires the student model to learn the complete probability distribution (logits) output by the teacher model.79% similarVerifiedModel distillation technology was proposed by Geoffrey Hinton's team in 2015, originally used for model compression, by having a student model fit the soft-label outputs of a teacher model.75% similarVerifiedKnowledge distillation transfers the full probability distribution information output by the teacher model via soft labels, producing an implicit data augmentation effect.73% similarUnverifiedModel distillation transfers the knowledge of a large model to a smaller student model, resulting in significant speed improvements but with some capability loss.73% similarVerifiedFlash models typically use knowledge distillation technology, training smaller-parameter, faster-inference student models with flagship models as the teacher model.72% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/811045API
curl https://kongchang.com/api/v1/knowledge/claims/811045MCP
get_claim(id=811045)