Unverified60% confidenceFactExact time
Knowledge distillation was systematically proposed by Hinton et al. in 2015, with the core idea of using the output probability distribution of a large model to guide the training of a small model.
2
Sources
60%
Confidence
Long-term
Relevance
7/14/2026
First Seen
Sources
Gemini Pro付费用户为何集体失望？五大核心槽点深度解析
Related Claims
VerifiedKnowledge distillation was proposed by Hinton et al. in 2015, originally referring to training a small model using the soft label outputs of a large model.84% similarVerifiedKnowledge Distillation is a technique for transferring the knowledge of a large model to a smaller, lightweight model, capable of retaining most of the performance while reducing the parameter count by tens of times.82% similarUnverifiedThe core idea of knowledge distillation is to have the small model learn the probability distribution (soft labels) of the large model's outputs rather than just hard labels81% similarVerifiedKnowledge Distillation was originally proposed by Hinton et al. in 2015, with the core idea of having a small model learn the soft labels of a large model rather than hard labels.77% similarUnverifiedIn the transfer of LLM reasoning capabilities, knowledge distillation usually refers to training a weak model using chain-of-thought answers generated by a strong model as supervision data, and DeepSeek-R1's distilled series falls into this category.75% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/503643API
curl https://kongchang.com/api/v1/knowledge/claims/503643MCP
get_claim(id=503643)