RCSD
A draft model training framework for speculative decoding that uses implicit preference pairs and a masked loss to directly optimize acceptance length, improving inference acceleration.
Timeline (last 90 days)
RCSD (Reinforcement and Continual Learning for Domain-Specific Speculative Decoding) is a training framework that directly aligns draft model optimization with a system-level objective (maximizing acceptance length).
RCSD uses the acceptance and rejection sequences naturally produced during speculative inference to construct implicit preference pairs as preference signals in reinforcement learning to optimize the draft model.
RCSD introduces a masked loss to avoid penalizing tokens that are aligned with the target model, thereby improving the acceptance rate without sacrificing output accuracy.
Compared with the HASS method, RCSD improves the average acceptance length by 35%.
RCSD achieves a 1.35x end-to-end inference speedup under real workloads, with no degradation in generation quality.
RCSD does not require modifying the target model or the inference architecture and is a pure training-side optimization with low deployment cost.
RCSD's training framework has continual learning characteristics and can adapt to domain-specific data distributions.
RCSD is currently still at the academic research stage, and its code and models have not yet been made public.
All Facts (8)
RCSD (Reinforcement and Continual Learning for Domain-Specific Speculative Decoding) is a training framework that directly aligns draft model optimization with a system-level objective (maximizing acceptance length).
50%UnverifiedRCSD uses the acceptance and rejection sequences naturally produced during speculative inference to construct implicit preference pairs as preference signals in reinforcement learning to optimize the draft model.
50%UnverifiedRCSD introduces a masked loss to avoid penalizing tokens that are aligned with the target model, thereby improving the acceptance rate without sacrificing output accuracy.
50%UnverifiedCompared with the HASS method, RCSD improves the average acceptance length by 35%.
50%UnverifiedRCSD achieves a 1.35x end-to-end inference speedup under real workloads, with no degradation in generation quality.
50%UnverifiedRCSD does not require modifying the target model or the inference architecture and is a pure training-side optimization with low deployment cost.
50%UnverifiedRCSD's training framework has continual learning characteristics and can adapt to domain-specific data distributions.
50%UnverifiedRCSD is currently still at the academic research stage, and its code and models have not yet been made public.
50%