Temperature Parameter

When a large language model generates each token, it samples from a probability distribution; the temperature parameter controls the degree of randomness—higher temperature produces more diverse outputs, while lower temperature produces more deterministic ones.

The Softmax formula with temperature parameter T is P(xᵢ) = exp(zᵢ/T) / Σexp(zⱼ/T); when T→0 it degenerates to argmax, and when T→∞ it approaches a uniform distribution

The temperature parameter design in the language model sampling process means that the same prompt may produce completely different outputs across different calls, so retrying itself does not guarantee convergence.

temperature为0时退化为贪婪解码，每步选取概率最高的Token，但仅能保证单次模型版本内的确定性

温度（temperature）参数控制采样过程，温度越高输出越随机多样，温度越低越趋向最高概率的选择

The first stage covers the essential differences between large-model application development and traditional backend development, the boundaries of model capabilities, the role of tokens and context windows, the impact of the temperature parameter on output, API calling and billing logic, and what each mainstream model excels at.

In the original example, with logits of 4.1, 3.4, and 2.3, raising the temperature from 0.5 to 2 lowers the probability of the highest-scoring token from 78.5% to 46.7%.

Low-cost techniques similar to system prompts also include few-shot examples, output format constraints and temperature parameter tuning.

Inference parameters usually include temperature, top-p and maximum generation length; a lower temperature is conducive to code stability.

Even with the LLM temperature set to 0, the same intent may still generate different SQL structures due to differences in prompt wording, context length, and prior tool call results.

An LLM outputs a probability distribution when generating each word, and the higher the temperature the more random the sampling; even if two assistants recall the same information sources, different temperature settings can lead to different wording or even a different order of companies mentioned.

Under a sampling strategy with a temperature parameter greater than 0, even if the input is exactly the same, a large model may produce different token sequences across multiple runs, thereby following different tool-calling paths and reasoning processes.

The temperature parameter T adjusts the sharpness of the logits distribution via softmax(logits/T): the lower T is, the closer it is to greedy decoding, and the higher T is, the closer it is to uniform sampling.

In the context of attention, the temperature parameter controls the sharpness of the softmax: low temperature makes attention concentrate on a few keys (low entropy, possibly overfitting), while high temperature makes the weights more uniform (high entropy, possibly under-focusing).

The temperature parameter T controls the smoothness of the softmax; a higher T value makes the probability distribution smoother and exposes more information about the teacher model's internal decision logic.

A lower temperature parameter makes model output more deterministic and is suitable for structured tasks, while a higher temperature introduces more creativity and is suitable for brainstorming scenarios.

When T=1 the output is a standard softmax distribution; when T>1 the probability distribution becomes more uniform—the higher the temperature, the flatter the output distribution, and the more fine-grained information the student model captures.

In typical practice, the temperature parameter T is commonly set between 3 and 10; after distillation training is complete, the student model reverts to the standard T=1 setting at inference time.

When the probability distribution during the decoding phase fluctuates (such as when the temperature parameter is too high or the context matches the training distribution of a particular language), the model may slip into the semantic space of another language when predicting the next token.

Sampling parameters, especially temperature, significantly affect model output, and are set independently by each tool and almost never exposed to the user.