Concept
Prefill Stage
One of the two stages of Transformer autoregressive generation, referring to the compute-intensive stage that processes the entire input prompt in parallel at once, with performance bounded by compute throughput (TFLOPS).
One of the two stages of Transformer autoregressive generation, referring to the compute-intensive stage that processes the entire input prompt in parallel at once, with performance bounded by compute throughput (TFLOPS).