Unverified60% confidenceFactExact time
In traditional white-box distillation, the student model can directly access the complete probability distribution (logits) of the teacher model's output layer, whereas commercial large language model APIs typically return only the final generated text or top-k token probabilities.
2
Sources
60%
Confidence
Long-term
Relevance
9/10/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedTraditional distillation usually requires access to the teacher model's full output probabilities (logits), but in black-box API scenarios attackers can only obtain the final text, so it has evolved into "black-box distillation"—instruction fine-tuning on large numbers of input-output pairs.77% similarVerifiedHard distillation refers to the student model directly learning the final output labels or generated text of the teacher model, while soft distillation requires the student model to learn the complete probability distribution (logits) output by the teacher model.73% similarUnverifiedThe endpoint-style policy distillation method uses the difference in log probabilities of the teacher model before and after RL training as the supervision signal, applied to trajectories generated by the student model itself.73% similarVerifiedKnowledge distillation transfers the full probability distribution information output by the teacher model via soft labels, producing an implicit data augmentation effect.68% similarVerifiedThe essence of large language models is learning the conditional probability distribution of what the next word is given the preceding text68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/886034API
curl https://kongchang.com/api/v1/knowledge/claims/886034MCP
get_claim(id=886034)