NVIDIA H100
An enterprise-grade data center GPU introduced by NVIDIA, based on the Hopper architecture. It is widely used for AI training and inference and supports the SXM5 form factor and InfiniBand interconnect.
Core Facts
The NVIDIA H100 single-card FP64 double-precision floating-point performance reaches approximately 34 TFLOPS.
FP8 is an 8-bit floating-point format natively supported by newer architectures such as the NVIDIA H100, which can compress data volume by half compared to FP16/BF16
In October 2022, the Biden administration introduced its first export controls targeting advanced computing chips, restricting the export of high-end GPUs such as the NVIDIA A100 and H100 to China.
Timeline (last 90 days)
Nathan Lambert believes China is in no way inferior to the US in AI talent, and that the real shortcoming is compute; he gives the example that an ordinary OpenAI researcher can call on roughly 5,000 H100s of compute at any time.
GPUs (such as NVIDIA H100/H200) cost tens of thousands of dollars each, and with tight supply and rapid iteration, the hardware depreciation cycle is short.
One on-demand H100 running full-time at $3.99/hour costs about $2913, equivalent to about 832,000 mid-tier batch reviewer calls or 16.6 million small batch calls.
NVIDIA H100/H200 relies on CoWoS to stack-package GPU dies together with HBM memory.
The experimental setup used a single NVIDIA H100 serving via vLLM, with test models Gemma 4 12B and Gemma 4 31B, and input data consisting of 50 million tokens of real public text.
The core upgrade of the H200 over the previous-generation H100 lies in VRAM capacity and bandwidth.
The M5 Ultra's 80-core GPU is the highest consumer-grade Apple Silicon configuration, but its floating-point compute still lags behind data center-grade GPUs such as the NVIDIA H100/H200 by an order of magnitude.
AWS Trainium2 is Amazon's second-generation self-developed machine learning training chip, based on the NeuronCore-v3 architecture, and is positioned as a cost-effective competitive alternative to the NVIDIA H100.
NVIDIA's H100/H200 series chips have long been in short supply.
The MI355X's memory capacity exceeds that of the NVIDIA H100 (80 GB) and H200 (141 GB).
26 more timeline events
All Facts (20)
The NVIDIA H100 single-card FP64 double-precision floating-point performance reaches approximately 34 TFLOPS.
90%VerifiedFP8 is an 8-bit floating-point format natively supported by newer architectures such as the NVIDIA H100, which can compress data volume by half compared to FP16/BF16
75%VerifiedIn October 2022, the Biden administration introduced its first export controls targeting advanced computing chips, restricting the export of high-end GPUs such as the NVIDIA A100 and H100 to China.
75%VerifiedThe NVIDIA H100 comes with 80GB of HBM3 memory
65%Unverified传统GPU如NVIDIA A100/H100在推理阶段面临内存带宽瓶颈，模型权重需要在每次生成Token时从显存反复读取
90%UnverifiedNVIDIA H100的峰值计算吞吐量与显存带宽之间存在巨大差距，大多数GPU程序的瓶颈在于数据搬运而非计算本身
90%UnverifiedNVIDIA B200 GPU基于Blackwell架构，相比H100在训练性能上提升约2.5倍，推理性能提升可达5倍
85%UnverifiedxAI于2024年9月在孟菲斯建成首期COLOSSUS集群，仅用约122天完成10万块NVIDIA H100 GPU的部署
85%UnverifiedBetween 2023 and 2024, the delivery lead time for NVIDIA H100s once reached 36–52 weeks; by the second half of 2025, GPU availability improved significantly as TSMC expanded its CoWoS advanced packaging capacity.
60%UnverifiedFull deployment of DeepSeek-V3.2 typically requires a cluster of multiple high-end GPUs (such as NVIDIA A100/H100)
60%UnverifiedA single NVIDIA H100 card is priced as high as $30,000–$40,000, and global annual supply is limited due to constraints on TSMC's advanced process capacity.
60%UnverifiedNathan Lambert believes China is in no way inferior to the US in AI talent, and that the real shortcoming is compute; he gives the example that an ordinary OpenAI researcher can call on roughly 5,000 H100s of compute at any time.
50%UnverifiedGPUs (such as NVIDIA H100/H200) cost tens of thousands of dollars each, and with tight supply and rapid iteration, the hardware depreciation cycle is short.
50%UnverifiedOne on-demand H100 running full-time at $3.99/hour costs about $2913, equivalent to about 832,000 mid-tier batch reviewer calls or 16.6 million small batch calls.
50%UnverifiedNVIDIA H100/H200 relies on CoWoS to stack-package GPU dies together with HBM memory.
50%UnverifiedThe experimental setup used a single NVIDIA H100 serving via vLLM, with test models Gemma 4 12B and Gemma 4 31B, and input data consisting of 50 million tokens of real public text.
50%UnverifiedThe core upgrade of the H200 over the previous-generation H100 lies in VRAM capacity and bandwidth.
50%UnverifiedThe M5 Ultra's 80-core GPU is the highest consumer-grade Apple Silicon configuration, but its floating-point compute still lags behind data center-grade GPUs such as the NVIDIA H100/H200 by an order of magnitude.
50%UnverifiedAWS Trainium2 is Amazon's second-generation self-developed machine learning training chip, based on the NeuronCore-v3 architecture, and is positioned as a cost-effective competitive alternative to the NVIDIA H100.
50%UnverifiedThe MI355X's memory capacity exceeds that of the NVIDIA H100 (80 GB) and H200 (141 GB).
50%