OPSD
A knowledge distillation variant that does not rely on a fixed offline teacher model; instead, the current policy or an enhanced version of it generates token-level supervision signals in real time during training and feeds them back to the model, which can alleviate the credit assignment problem under sparse rewards.
Timeline (last 90 days)
The paper "Every Token Needs Its Own Teacher: Multi-teacher Adaptive On-policy Self-distillation" points out that existing OPSD methods assume the teacher is fixed, and this assumption has fundamental limitations.
The paper categorizes existing OPSD approaches into two types: one where the teacher is fixed throughout training and has access to the full reference; the other where the teacher is unchanged within each response but adaptively varies across responses in the degree of exposure to partial reference.
Experiments were conducted on two mathematical reasoning benchmarks, AIME and HMMT, and ST-OPSD improves accuracy over the original OPSD and other intervention methods.
The study validated on six visual reasoning benchmarks, and VAG achieves consistent advantages over both GRPO and OPSD methods and sets a new best average performance level.
All Facts (4)
The paper "Every Token Needs Its Own Teacher: Multi-teacher Adaptive On-policy Self-distillation" points out that existing OPSD methods assume the teacher is fixed, and this assumption has fundamental limitations.
50%UnverifiedThe paper categorizes existing OPSD approaches into two types: one where the teacher is fixed throughout training and has access to the full reference; the other where the teacher is unchanged within each response but adaptively varies across responses in the degree of exposure to partial reference.
50%UnverifiedExperiments were conducted on two mathematical reasoning benchmarks, AIME and HMMT, and ST-OPSD improves accuracy over the original OPSD and other intervention methods.
50%UnverifiedThe study validated on six visual reasoning benchmarks, and VAG achieves consistent advantages over both GRPO and OPSD methods and sets a new best average performance level.
50%