Verified65% confidenceFactExact time
Hard distillation refers to the student model directly learning the final output labels or generated text of the teacher model, while soft distillation requires the student model to learn the complete probability distribution (logits) output by the teacher model.
3
Sources
65%
Confidence
Long-term
Relevance
8/7/2026
First Seen
Sources
Related Claims
UnverifiedSoft-label distillation is a form of knowledge distillation in which the teacher model outputs a full probability distribution (soft labels) at each position, and the student model learns by minimizing the difference from the teacher distribution (such as KL divergence).80% similarVerifiedModel distillation softens the teacher model's softmax outputs via temperature scaling, allowing information from low-probability classes to be transferred to the student model.79% similarVerifiedDistillation can be viewed as an implicit, data-constrained form of model merging—distilling a student using the averaged outputs of multiple teacher models is effectively performing a soft merge at the level of output distributions.78% similarUnverifiedOutput distillation involves directly using text generated by a teacher model—such as dialogues and reasoning chains—as training data to fine-tune a student model, without requiring access to the teacher's internal parameters.75% similarVerified模型蒸馏是一种将大型教师模型能力迁移到更小学生模型的技术，可能被用于扩散危险能力74% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/704190API
curl https://kongchang.com/api/v1/knowledge/claims/704190MCP
get_claim(id=704190)