Verified75% confidenceFactExact time
TPS (Tokens Per Second) is the core metric for measuring the inference speed of large language models, specifically referring to the number of tokens the model outputs per second during the generation phase.
3
Sources
75%
Confidence
Long-term
Relevance
9/5/2026
First Seen
Sources
Related Entities
Related Claims
Unverified对于大语言模型推理，吞吐通常以 token 数为单位衡量，即 tokens/s 是评估 LLM 推理框架性能的核心指标69% similarUnverifiedTime per output token (TPOT) measures the speed at which a model generates tokens one by one during the autoregressive decoding phase.66% similarVerifiedThe token generation process of large language models is serial, based on an autoregressive mechanism, where each token's generation depends on the previous token.65% similarUnverifiedThe formula for local AI inference speed is Token/s ≈ memory bandwidth / data volume to be read per token, and large language models generate each token in an autoregressive manner.61% similarUnverifiedThe MoT (Mixture of Tokens) mechanism allows the model to dynamically allocate computational resources when processing inputs of different modalities.61% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/862243API
curl https://kongchang.com/api/v1/knowledge/claims/862243MCP
get_claim(id=862243)