Tesla T4
An NVIDIA data-center GPU inference card based on the Turing architecture. It features 16 GB of GDDR6 memory, a 70 W low-power passive cooling design, and supports INT8/INT4 inference acceleration. It is widely used for cloud-based and on-premises AI inference deployment.
Timeline (last 90 days)
Evaluated with llama.cpp on a Tesla T4, the HumanEval pass@1 retention rate is 96.7% (29/30).
Evaluated with llama.cpp on a Tesla T4, the MMLU subset retention rate is 92.9% (344/502).
Running Qwen3-30B-A3B (18.6GB) on a 16GB Tesla T4, using the MoE cache raised decoding speed from the default 22.6 tokens/s to 43.2 tokens/s.
On the T4, --fit plus a 10% cache reached 23.6 tokens/s with a 75.6% hit rate; --cpu-moe alone was 9.6 tokens/s; --cpu-moe plus a 50 MiB cache was only 3.8 tokens/s.
On the T4 with --cpu-moe plus a 10000 MiB cache, the hit rate was 95.73% for small batches of up to 8 tokens and 59.91% for batches of 9 to 32 tokens.
Concurrent requests amplify the demand for cache slots, causing more frequent LRU evictions; on the T4, four concurrent requests achieved total throughput of 37.0 tokens/s, lower than the 43.2 tokens/s of a single request, with the hit rate dropping from 95.7% to 92.3%.
Early versions of OjasX could already fit a 534M parameter model into the 9.5GB VRAM of a Tesla T4 for training, whereas standard PyTorch would crash at 16.4GB at that time.
A Reddit user built a local AI inference platform with a total of 64GB of VRAM using 4 Tesla T4 GPUs
The Tesla T4 is an NVIDIA data center inference card based on the Turing architecture, with a single-card TDP of 70W and a passive cooling design
The Tesla T4's memory bandwidth is only 300GB/s, compared to 1008GB/s for the RTX 4090
2 more timeline events
All Facts (10)
Evaluated with llama.cpp on a Tesla T4, the HumanEval pass@1 retention rate is 96.7% (29/30).
50%UnverifiedEvaluated with llama.cpp on a Tesla T4, the MMLU subset retention rate is 92.9% (344/502).
50%UnverifiedOn the T4, --fit plus a 10% cache reached 23.6 tokens/s with a 75.6% hit rate; --cpu-moe alone was 9.6 tokens/s; --cpu-moe plus a 50 MiB cache was only 3.8 tokens/s.
50%UnverifiedRunning Qwen3-30B-A3B (18.6GB) on a 16GB Tesla T4, using the MoE cache raised decoding speed from the default 22.6 tokens/s to 43.2 tokens/s.
50%UnverifiedOn the T4 with --cpu-moe plus a 10000 MiB cache, the hit rate was 95.73% for small batches of up to 8 tokens and 59.91% for batches of 9 to 32 tokens.
50%UnverifiedConcurrent requests amplify the demand for cache slots, causing more frequent LRU evictions; on the T4, four concurrent requests achieved total throughput of 37.0 tokens/s, lower than the 43.2 tokens/s of a single request, with the hit rate dropping from 95.7% to 92.3%.
50%UnverifiedEarly versions of OjasX could already fit a 534M parameter model into the 9.5GB VRAM of a Tesla T4 for training, whereas standard PyTorch would crash at 16.4GB at that time.
50%UnverifiedThe Tesla T4's passive cooling relies on an external blower fan, with noise levels that can exceed 55dB, requiring soundproofing treatment in home environments
50%UnverifiedThe Tesla T4's memory bandwidth is only 300GB/s, compared to 1008GB/s for the RTX 4090
50%UnverifiedThe Tesla T4 is an NVIDIA data center inference card based on the Turing architecture, with a single-card TDP of 70W and a passive cooling design
50%