Rejection Sampling
A Monte Carlo sampling method that approximates a target distribution by sampling from a proposal distribution and accepting or rejecting samples with a certain probability. It is commonly used to correct sampling bias.
Timeline (last 90 days)
The losslessness of speculative decoding relies on a modified rejection sampling mechanism: for each token proposed by the draft model, the target model computes its probability and decides acceptance or rejection accordingly; once rejected, resampling is done from the difference between the target model and draft model distributions (after normalization).
Rejection sampling fixes modulo bias as follows: find the largest multiple of n within the random range as the limit, discard and redraw if the random number is greater than or equal to that limit, and each retained value maps to exactly the same number of outcomes.
Anari et al. (2026) proposed using rejection sampling to correct the distributional bias in parallel decoding of masked diffusion models, decomposing the log acceptance probability into a sum of token-level terms.
Reduced variance leads to less clipping, because in rejection sampling an overly volatile estimate of the acceptance probability requires more aggressive clipping to ensure numerical stability, and clipping introduces additional distributional bias.
Speculative decoding is based on a rejection sampling scheme that is mathematically strictly equivalent to original decoding and does not change the final sampling distribution.
Traditional speculative decoding uses strict rejection-sampling-based verification rules, where a draft token is only accepted when it exactly matches the target model's sampled result.
All Facts (6)
The losslessness of speculative decoding relies on a modified rejection sampling mechanism: for each token proposed by the draft model, the target model computes its probability and decides acceptance or rejection accordingly; once rejected, resampling is done from the difference between the target model and draft model distributions (after normalization).
50%UnverifiedRejection sampling fixes modulo bias as follows: find the largest multiple of n within the random range as the limit, discard and redraw if the random number is greater than or equal to that limit, and each retained value maps to exactly the same number of outcomes.
50%UnverifiedAnari et al. (2026) proposed using rejection sampling to correct the distributional bias in parallel decoding of masked diffusion models, decomposing the log acceptance probability into a sum of token-level terms.
50%UnverifiedReduced variance leads to less clipping, because in rejection sampling an overly volatile estimate of the acceptance probability requires more aggressive clipping to ensure numerical stability, and clipping introduces additional distributional bias.
50%UnverifiedSpeculative decoding is based on a rejection sampling scheme that is mathematically strictly equivalent to original decoding and does not change the final sampling distribution.
50%UnverifiedTraditional speculative decoding uses strict rejection-sampling-based verification rules, where a draft token is only accepted when it exactly matches the target model's sampled result.
50%