Autoregressive Model

The VRAM occupied by KV Cache grows linearly with sequence length.

vLLM has previously primarily served autoregressive large language models.

The denoising process of diffusion models typically requires tens to hundreds of iterations (called sampling steps), each of which requires a complete forward pass through the neural network.

Acceleration solutions such as consistency models and flow matching are narrowing the computational overhead gap of multi-step inference in diffusion models.

The technical root cause of hallucination in large models lies in the autoregressive generation mechanism, where the model predicts only the statistically most likely next token based on the preceding context; researchers divide hallucinations into intrinsic hallucinations and extrinsic hallucinations.

Audio is a time-series signal, and music generation models need to maintain coherence over long time spans; integrating them into a workflow engine primarily designed for images/video requires specialized adaptation in sampling scheduling, memory management, and output format

The root cause of clipping is that diffusion or autoregressive generative models optimize pixel/frame-level likelihood rather than explicit rigid-body constraints, so errors accumulate and amplify over long rollouts.

Generating each token with an autoregressive model requires loading the entire model weights, causing memory bandwidth to become the bottleneck rather than compute capability.

In the high-concurrency scenario (64 requests), an autoregressive model of the same size overtakes NCM in speed.

In the single-request scenario, NCM is 4 times faster than an autoregressive model of the same size on translation tasks and nearly 2 times faster on news summarization tasks.

Real-time interactive world models typically need to adopt streaming inference schemes based on autoregressive or diffusion models, combined with state compression (such as encoding historical frames into compact latent representations) to reduce computational burden.

AI music generation models typically first use a text encoder to understand the prompt, then use a diffusion model or autoregressive model to progressively denoise or predict segment by segment in the audio latent space, and finally a vocoder reconstructs the waveform.

Autoregressive models can only determine one token per forward pass, so inference cannot be parallelized along the time dimension, and latency grows linearly with output length.

Longitudinal EHR data is characterized by events occurring in temporal clusters and being highly correlated with one another, and the strict sequential generation assumption of AR models does not fully align with this characteristic.

BD exhibits a clear quality-efficiency trade-off across different block sizes: small-block configurations retain most of AR's predictive performance with significantly faster inference, while large blocks favor quality.

A new study proposes using Block Diffusion (BD) instead of autoregression (AR) for zero-shot longitudinal EHR forecasting.

Frozen linear probes trained on BD and AR representations perform almost identically across various clinical prediction tasks.

BD remains below AR in most settings of generative rollout.

At comparable quality, Lafa's self-speculative decoding scheme achieves 2.54x the throughput of autoregressive models, outperforming multi-token prediction and discrete diffusion methods.

Evaluation on instruction-following, reasoning, and coding tasks shows that the flow models learned by Lafa are comparable to or even better than autoregressive models in quality.