Autoregression
A generation mechanism used in language models whereby each step predicts the next token conditioned on all previously generated prefix tokens, incrementally constructing a complete sequence. Autoregression is the core mechanism of generative large language models such as GPT.
Core Facts
Modern large language models generate code in an auto-regressive manner, predicting the next token each time by referencing only the existing context.
Large model output generation uses an autoregressive approach to produce tokens one by one, requiring recalculation of attention over all preceding context with each token generated.
Timeline (last 90 days)
Flow-matching action experts (such as π0.5) model action generation as a continuous transformation process from noise to actions, producing smooth action sequences through iterative denoising and suiting high-dimensional continuous control; autoregressive methods (such as π0-FAST) discretize actions into tokens and predict them token by token.
A classifier maps input to one of a finite set of predefined categories with a fixed, closed candidate space; a generative language model predicts the next token from the entire vocabulary at each step, with an open candidate space, building output autoregressively.
Autoregressive generation is the core decoding paradigm of current mainstream large language models (such as the GPT series): generating one token at a time and appending it to the end of the sequence, looping until an end token is encountered.
The formula for local AI inference speed is Token/s ≈ memory bandwidth / data volume to be read per token, and large language models generate each token in an autoregressive manner.
Due to the autoregressive nature, each decode step requires accessing the KV Cache of all previous tokens.
LLMs generate text in an autoregressive manner, producing only one token at a time, and the generation of the next token must wait for the previous token to be completed.
The autoregressive token prediction paradigm is built on correlation rather than causality.
自回归推理时batch size通常为1，每个token生成只做一次矩阵-向量乘法，算术强度极低，导致memory-bound问题，单用户场景下GPU利用率往往不到10%
现代主流LLM采用自回归生成方式，逐个token按顺序生成
自回归推理是memory-bound（内存带宽受限）问题，因为batch size通常为1时算术强度极低，GPU大部分时间在等待数据搬运
2 more timeline events
All Facts (13)
Modern large language models generate code in an auto-regressive manner, predicting the next token each time by referencing only the existing context.
75%VerifiedLarge model output generation uses an autoregressive approach to produce tokens one by one, requiring recalculation of attention over all preceding context with each token generated.
65%UnverifiedLLMs generate text in an autoregressive manner, producing only one token at a time, and the generation of the next token must wait for the previous token to be completed.
70%UnverifiedFlow-matching action experts (such as π0.5) model action generation as a continuous transformation process from noise to actions, producing smooth action sequences through iterative denoising and suiting high-dimensional continuous control; autoregressive methods (such as π0-FAST) discretize actions into tokens and predict them token by token.
50%UnverifiedA classifier maps input to one of a finite set of predefined categories with a fixed, closed candidate space; a generative language model predicts the next token from the entire vocabulary at each step, with an open candidate space, building output autoregressively.
50%UnverifiedAutoregressive generation is the core decoding paradigm of current mainstream large language models (such as the GPT series): generating one token at a time and appending it to the end of the sequence, looping until an end token is encountered.
50%UnverifiedThe formula for local AI inference speed is Token/s ≈ memory bandwidth / data volume to be read per token, and large language models generate each token in an autoregressive manner.
50%UnverifiedDue to the autoregressive nature, each decode step requires accessing the KV Cache of all previous tokens.
50%UnverifiedThe autoregressive token prediction paradigm is built on correlation rather than causality.
50%Unverified现代主流LLM采用自回归生成方式，逐个token按顺序生成
50%Unverified自回归推理时batch size通常为1，每个token生成只做一次矩阵-向量乘法，算术强度极低，导致memory-bound问题，单用户场景下GPU利用率往往不到10%
50%Unverified自回归推理是memory-bound（内存带宽受限）问题，因为batch size通常为1时算术强度极低，GPU大部分时间在等待数据搬运
50%Unverified输出生成是严格串行的自回归过程，每生成一个token都需要一次完整的模型前向传播
50%