top_p
A hyperparameter on the large-model invocation side that controls the probability threshold of the candidate token set through nucleus sampling, affecting the diversity and stability of outputs.
Timeline (last 90 days)
When a model's sampling parameters (such as temperature and top_p) cause the output format to deviate from the training distribution, vLLM's parser may fail to match boundaries correctly, resulting in truncated parameters or format errors.
When temperature, top_p and top_k are used in combination, their effects are hard to predict, and this is the main source of output style drift for the same prompt under different parameters.
top_p is clamped on Flash: in thinking mode anything below 0.95 is treated as 0.95, and in non-thinking mode it is fixed at 1.0.
The first of the four blueprint layers is tightening model constraints, recommending temperature ≤0.7, with strict data extraction and code generation recommended at close to 0.0–0.2.
Common hyperparameters include temperature (controls output randomness, the lower the more deterministic), max tokens (limits generation length), top_p, and so on.
temperature, top_p, and max_tokens are common across most models, but some reasoning models do not accept temperature, or have special semantics for max_tokens.
Top-p (nucleus sampling) filters out extremely low-probability outputs by truncating the long tail of the probability distribution.
All Facts (7)
When a model's sampling parameters (such as temperature and top_p) cause the output format to deviate from the training distribution, vLLM's parser may fail to match boundaries correctly, resulting in truncated parameters or format errors.
50%UnverifiedWhen temperature, top_p and top_k are used in combination, their effects are hard to predict, and this is the main source of output style drift for the same prompt under different parameters.
50%Unverifiedtop_p is clamped on Flash: in thinking mode anything below 0.95 is treated as 0.95, and in non-thinking mode it is fixed at 1.0.
50%UnverifiedThe first of the four blueprint layers is tightening model constraints, recommending temperature ≤0.7, with strict data extraction and code generation recommended at close to 0.0–0.2.
50%UnverifiedCommon hyperparameters include temperature (controls output randomness, the lower the more deterministic), max tokens (limits generation length), top_p, and so on.
50%Unverifiedtemperature, top_p, and max_tokens are common across most models, but some reasoning models do not accept temperature, or have special semantics for max_tokens.
50%UnverifiedTop-p (nucleus sampling) filters out extremely low-probability outputs by truncating the long tail of the probability distribution.
50%