T4
The NVIDIA Tesla T4 is a cloud inference GPU based on the Turing architecture. It uses passive cooling and is commonly deployed in data centers and cloud services.
Core Facts
Timeline (last 90 days)
The test is based on the Qwen3.8-27B model, using EXL3 4.0bpw quantization, a DFlash2 block drafter (EXL3 4bpw, 7-token draft), greedy decoding, running on 2 T4 GPUs.
The author fixed the thresholds before running the evaluation, among which metric T4 requires that the proportion of "incorrectly judged as covered" not exceed 5%.
The author reproduced it on a T4 with the same model and testing method: at 8 concurrency, a single request decodes at 122–124 tokens/s, and the whole server produces 701–708 tokens/s, likewise about a 5.7-fold difference.
On the same T4, vLLM 0.27.1 took only 5.8 seconds in total for 8 requests after ready; vLLM's startup overhead was about 165 of 171 billed seconds.
Running SGLang 0.5.21 on a single T4 in Colab to load the Qwen2.5-0.5B-Instruct fp16 model, in one complete cold-start measurement, 88% of the billed time was spent before the first user request completed.
On a T4 GPU, Laya takes 33 milliseconds per single question, and batch processing can compress that to 7 milliseconds
作者称CWAA V6在一块T4 GPU上可达到0.38秒/步的训练速度且高度稳定
NVIDIA T4是2018年发布的推理优化GPU，拥有65 TFLOPS的FP16算力和16GB显存
All Facts (7)
On a T4 GPU, Laya takes 33 milliseconds per single question, and batch processing can compress that to 7 milliseconds
65%UnverifiedThe test is based on the Qwen3.8-27B model, using EXL3 4.0bpw quantization, a DFlash2 block drafter (EXL3 4bpw, 7-token draft), greedy decoding, running on 2 T4 GPUs.
50%UnverifiedThe author fixed the thresholds before running the evaluation, among which metric T4 requires that the proportion of "incorrectly judged as covered" not exceed 5%.
50%UnverifiedThe author reproduced it on a T4 with the same model and testing method: at 8 concurrency, a single request decodes at 122–124 tokens/s, and the whole server produces 701–708 tokens/s, likewise about a 5.7-fold difference.
50%UnverifiedRunning SGLang 0.5.21 on a single T4 in Colab to load the Qwen2.5-0.5B-Instruct fp16 model, in one complete cold-start measurement, 88% of the billed time was spent before the first user request completed.
50%Unverified作者称CWAA V6在一块T4 GPU上可达到0.38秒/步的训练速度且高度稳定
50%UnverifiedNVIDIA T4是2018年发布的推理优化GPU，拥有65 TFLOPS的FP16算力和16GB显存
50%