Schedule Distillation
A model training technique in which a high-quality decoding trajectory constructed offline in advance (such as an unmasking order) serves as a teacher. The model learns during training to imitate the parallel sampling behavior corresponding to that trajectory, thereby internalizing inference-time schedule search as a model capability.
Timeline (last 90 days)
Research from VILA-Lab proposes dSpark, a framework based on schedule distillation for optimizing the decoding order of diffusion language models.
Schedule distillation requires additional offline construction and distillation costs.
The generalization of schedule distillation to larger models and more tasks still awaits further verification by the community.
All Facts (3)
Research from VILA-Lab proposes dSpark, a framework based on schedule distillation for optimizing the decoding order of diffusion language models.
50%UnverifiedSchedule distillation requires additional offline construction and distillation costs.
50%UnverifiedThe generalization of schedule distillation to larger models and more tasks still awaits further verification by the community.
50%