Unverified50% confidenceFactExact time
On-policy distillation (OPD) is a variant of knowledge distillation in which training data is generated in real time by the student model under its current parameters, and the teacher model provides per-token probability distributions over the self-generated responses as supervision signals.
1
Sources
50%
Confidence
Long-term
Relevance
10/30/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedIn on-policy distillation, the student model first generates trajectories itself, and then the teacher model provides supervision signals for these paths, thereby avoiding the distribution shift between offline data and student behavior.75% similarUnverifiedOn-policy distillation (OPD) has the student model generate its own trajectories, and then the teacher model provides supervision signals on those trajectories.73% similarUnverifiedKnowledge distillation was systematically proposed by Hinton et al. in 2015, with the core idea of using the output probability distribution of a large model to guide the training of a small model.68% similarVerifiedModel Distillation in machine learning refers to training a small student model using the outputs of a large teacher model, in order to compress model size while retaining capability.66% similarUnverifiedIn knowledge distillation, off-policy distillation uses trajectories generated by the teacher model as training samples, and suffers from the distribution shift problem where the state distribution visited by the teacher is inconsistent with the student's own state distribution.66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1114712API
curl https://kongchang.com/api/v1/knowledge/claims/1114712MCP
get_claim(id=1114712)