NVIDIA T4

Kaggle is a data science competition platform that also offers a free cloud notebook environment (Kaggle Notebooks), equipped with GPUs (such as NVIDIA Tesla P100 or T4) and limited compute hours (typically around 30 hours per week).

The actual cooling capability of passively cooled cards (such as the T4) depends heavily on chassis airflow, intake temperature and heat from adjacent boards, so the steady-state clocks the same card can sustain may differ markedly across different chassis or rack positions.

The NVIDIA T4 has a TDP of 70W, a boost clock of up to 1590 MHz, and a passive cooling design (no fan on the card, relying on server airflow for cooling).

When reading gpu_log.csv on a T4, one typically sees the same pattern: clocks.sm holds steady at 1590 in the first minute, then gradually drops to 1200–1350 between minutes 3 and 5 and stabilizes.

The frequency-locked experiment shows that with SM frequency fixed at 1,200 MHz and no power limit, median power is 62.0W at 66°C and rises to 66.0W at 78°C, a slope of about 0.26 W/°C, that is, leakage current causes higher power consumption at the same frequency as temperature rises.

The T4's specs are 8.1 TFLOPS for FP32 and 65 TFLOPS for FP16, and the spec sheet does not note the corresponding frequency, temperature conditions or footnotes.

Back-calculating from 2,560 CUDA cores and two operations per clock, 8.1 TFLOPS requires about 1,582 MHz to hold.

The experiment uses an n1-standard-2 VM, a single passively cooled Tesla T4, driver 615.71.09, vLLM 0.29.0, with the model being Gemma 4 E2B at 4-bit weights, single request, greedy decoding, 256 tokens per output.

Across all samples, the correlation coefficient between decode rate and SM frequency is as high as 0.99, and the correlation coefficients with temperature are -0.97 and -0.86.

Even with the frequency locked at 1,200 MHz, 29% of sampling points still hit the 70W power limit once the T4 heated up.

NVIDIA TensorRT documentation explicitly mentions that GPUs like the T4 that are easily limited by the power wall will have lower stable frequencies and worse performance when cooling is poor, and recommends not locking the frequency long-term unless there is a clear deterministic requirement.

In the speculative decoding A/B comparison, cooling the card to 50°C before each server start kept the speedup stable at 1.297x or 1.298x regardless of run order; without pre-cooling, the speedup falls between 1.279x and 1.337x.

On this host, cooling the T4 to 50°C takes less than four minutes, whereas idling with the model running at 72–76°C for 30 minutes does not cool it down.

Google Compute Engine's GPU page repeatedly lists the T4's 65, 130 and 260 TOPS, likewise without conditional explanations; Compute Engine's SLA mainly covers availability failures and does not address this kind of temperature-induced performance degradation.

After locking SM frequency at 1,200 MHz, temperature rose from 65–69°C to 75–79°C, but the decode rate only fell from 104.8 to 104.1 tokens/s, compressing the drift from about 7% to about 0.7%.

On the same Google Cloud single-card T4, Gemma 4 E2B decodes at 115.3 tokens/s when cold at 45°C and drops to 106.8 tokens/s when hot at 77°C, a sustained speed drop of about 7.4%.

Among the experiment's 154,815 sampling points, the T4 reported only two throttle reasons, idle and software power cap, software and hardware thermal throttling never triggered, and the highest temperature reading was 83°C.

The T4's power limit on this VM is locked at 70W, and it runs close to that limit continuously from the first request.

The NVIDIA T4 is based on the Turing architecture, with 16GB GDDR6 VRAM and 320 Tensor Cores per card.

A 70B-parameter model requires approximately 35–40GB of VRAM after Q4 quantization, which a single T4 cannot handle, but dual T4s can basically satisfy.