NgramRL
A speculative decoding method for reinforcement learning post-training that builds an n-gram drafter from recent rollout history, accelerating autoregressive rollout generation without requiring an auxiliary model or drafter training.
Timeline (last 90 days)
NgramRL uses recent rollout history to build an n-gram drafter for each prompt, with no auxiliary model, no drafter training, and no drafter state resident on the GPU.
NgramRL introduces an adaptive fan-out prefetcher and a causal budget allocator that predict the verification result (j, b) and fetch multiple candidate continuations in parallel in advance, falling back to sequential lookup when they are not covered.
On two model families and three types of reasoning and structured generation workloads, NgramRL can commit 3.60–7.77 tokens per target-model forward pass in the hot-cache setting.
NgramRL's average acceptance length is up to 1.99 times higher than SuffixDecoding's.
NgramRL's average acceptance length is 7.0 times higher than prompt-lookup decoding's.
On the Spider structured generation task, with a candidate budget of 16, the prefetcher covered 96.6% of Qwen3-8B's verification results.
On the Spider task, NgramRL's measured rollout throughput improves 1.38 times over the synchronous version, approaching the theoretical CPU-GPU overlap limit of 1.40 times.
The 3.60–7.77 acceptance length reported by NgramRL is a steady-state metric obtained under hot-cache conditions; gains are expected to shrink on cold start or when prompt history is sparse.
NgramRL's comparison baselines are concentrated on model-free speculative decoding methods and lack a side-by-side comparison with model-based drafter approaches.
NgramRL's advantage is zero extra VRAM and zero drafter training, at the cost of maintaining a rollout history cache organized by prompt, shifting engineering complexity to the data pipeline side.
All Facts (10)
NgramRL uses recent rollout history to build an n-gram drafter for each prompt, with no auxiliary model, no drafter training, and no drafter state resident on the GPU.
50%UnverifiedNgramRL introduces an adaptive fan-out prefetcher and a causal budget allocator that predict the verification result (j, b) and fetch multiple candidate continuations in parallel in advance, falling back to sequential lookup when they are not covered.
50%UnverifiedOn two model families and three types of reasoning and structured generation workloads, NgramRL can commit 3.60–7.77 tokens per target-model forward pass in the hot-cache setting.
50%UnverifiedNgramRL's average acceptance length is up to 1.99 times higher than SuffixDecoding's.
50%UnverifiedNgramRL's average acceptance length is 7.0 times higher than prompt-lookup decoding's.
50%UnverifiedOn the Spider structured generation task, with a candidate budget of 16, the prefetcher covered 96.6% of Qwen3-8B's verification results.
50%UnverifiedOn the Spider task, NgramRL's measured rollout throughput improves 1.38 times over the synchronous version, approaching the theoretical CPU-GPU overlap limit of 1.40 times.
50%UnverifiedThe 3.60–7.77 acceptance length reported by NgramRL is a steady-state metric obtained under hot-cache conditions; gains are expected to shrink on cold start or when prompt history is sparse.
50%UnverifiedNgramRL's comparison baselines are concentrated on model-free speculative decoding methods and lack a side-by-side comparison with model-based drafter approaches.
50%UnverifiedNgramRL's advantage is zero extra VRAM and zero drafter training, at the cost of maintaining a rollout history cache organized by prompt, shifting engineering complexity to the data pipeline side.
50%