Sampling Strategy
The method by which an LLM selects a specific token from the probability distribution over the next token, such as greedy decoding, temperature sampling, top-k and top-p.
Timeline (last 90 days)
A deterministic sampler inflates every score, so sampling should use the variation that exists in actual delivery.
When the production cost of candidates is low and the filtering cost is also low, the optimal strategy is to rapidly generate many answers rather than striving to get one answer right.
All Facts (2)
A deterministic sampler inflates every score, so sampling should use the variation that exists in actual delivery.
50%UnverifiedWhen the production cost of candidates is low and the filtering cost is also low, the optimal strategy is to rapidly generate many answers rather than striving to get one answer right.
50%