Verified75% confidenceFactExact time
Model distillation was first proposed by Hinton et al. in 2015; by having a small model learn the output distribution of a large model, the student model can often retain 80% to 95% of the teacher model's performance.
3
Sources
75%
Confidence
Long-term
Relevance
9/10/2026
First Seen
Sources
Related Entities
Related Claims
VerifiedModel distillation technology was proposed by Geoffrey Hinton's team in 2015, originally used for model compression, by having a student model fit the soft-label outputs of a teacher model.77% similarVerified模型蒸馏是一种将大型教师模型能力迁移到更小学生模型的技术，可能被用于扩散危险能力77% similarVerifiedModel distillation softens the teacher model's softmax outputs via temperature scaling, allowing information from low-probability classes to be transferred to the student model.76% similarUnverifiedDistillation can be viewed as an implicit, data-constrained form of model merging—distilling a student using the averaged outputs of multiple teacher models is effectively performing a soft merge at the level of output distributions.76% similarVerifiedThis experiment adopts trajectory-level distillation, which belongs to the category of behavioral cloning, directly having the student model imitate the external behavioral sequences of the teacher model.75% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/891631API
curl https://kongchang.com/api/v1/knowledge/claims/891631MCP
get_claim(id=891631)