H200
An NVIDIA GPU based on the Hopper architecture, equipped with HBM3e memory and aimed at high-performance computing and AI training and inference.
Core Facts
The retail price of a single NVIDIA H100/H200 GPU is as high as $30,000–$40,000.
NVIDIA's H100/H200 series GPUs hold more than 80% of the AI training and inference market.
Both the NVIDIA H100/H200 and the AMD MI300X integrate HBM.
Timeline (last 90 days)
The power supply and cooling capacity of hyperscale data centers are scarce resources, and the economic return from freeing up 300 racks of space to deploy H100/H200 clusters is far higher than keeping A100s.
The experiment selected four frontier coding agents—Claude Opus, Claude Sonnet, Codex, and Gemini CLI—each allocated eight H200 GPUs and 12 hours of runtime.
The team initially tried to run the full training pipeline on T4 GPUs, but was defeated by out-of-memory errors every time, and eventually moved to H100, H200 and TPU.
SM90 is the streaming multiprocessor codename for NVIDIA's Hopper architecture (GPUs such as H100 and H200).
H-series GPUs refer to NVIDIA's Hopper-architecture data center GPUs, mainly including the H100 and H200; the H100 is equipped with 80GB HBM3 memory, and the H200 increases memory to 141GB and uses HBM3e.
On the 2.8-trillion-parameter KimiK3, the per-megawatt throughput of GB300-NVL72 is about 80 times that of H200, while pushing interactivity to about 215 tokens/second/user.
Moonshot Kimi K2.5 has a total parameter count of 1.04T (32B active, 384 experts), and even with QAT-INT4 quantization it still requires 8×H200 to hold the weights.
Data-center-class H100/H200 chips are priced at tens of thousands of dollars and their procurement channels are restricted, while consumer flagship cards have a clear advantage in cost per unit of VRAM.
In GPU Index's price data snapshot, the lowest price for H100 is $1.37/hr (Vast.ai, spot instance), for H200 is $2.62/hr (CoreWeave, spot), for A100 is $1.15/hr (Paperspace, three-year contract), for L40S is $0.79/hr (RunPod, community cloud), and for B200 is $4.26/hr (CoreWeave, spot).
These optimizations brought up to a 10% throughput improvement and up to a 9% improvement in time to first token (TTFT) on H200 GPUs.
29 more timeline events
All Facts (20)
The retail price of a single NVIDIA H100/H200 GPU is as high as $30,000–$40,000.
90%VerifiedNVIDIA's H100/H200 series GPUs hold more than 80% of the AI training and inference market.
90%VerifiedBoth the NVIDIA H100/H200 and the AMD MI300X integrate HBM.
75%UnverifiedOpenAI、Google DeepMind、Anthropic等头部公司同时争夺有限的H100/H200 GPU供应
80%UnverifiedNVIDIA H100/H200 series GPUs have long been in a state of supply shortage.
70%UnverifiedThe GPU generational progression from A100 to H100 and then to H200 has pushed the inference efficiency frontier outward with each new generation.
60%UnverifiedComposer 2.5的高速推理依赖推测性解码（Speculative Decoding）、KV Cache优化以及专用推理硬件（如H100/H200 GPU集群）的协同加速
60%UnverifiedThe power supply and cooling capacity of hyperscale data centers are scarce resources, and the economic return from freeing up 300 racks of space to deploy H100/H200 clusters is far higher than keeping A100s.
50%UnverifiedThe experiment selected four frontier coding agents—Claude Opus, Claude Sonnet, Codex, and Gemini CLI—each allocated eight H200 GPUs and 12 hours of runtime.
50%UnverifiedSM90 is the streaming multiprocessor codename for NVIDIA's Hopper architecture (GPUs such as H100 and H200).
50%UnverifiedH-series GPUs refer to NVIDIA's Hopper-architecture data center GPUs, mainly including the H100 and H200; the H100 is equipped with 80GB HBM3 memory, and the H200 increases memory to 141GB and uses HBM3e.
50%UnverifiedData-center-class H100/H200 chips are priced at tens of thousands of dollars and their procurement channels are restricted, while consumer flagship cards have a clear advantage in cost per unit of VRAM.
50%UnverifiedThese optimizations brought up to a 10% throughput improvement and up to a 9% improvement in time to first token (TTFT) on H200 GPUs.
50%UnverifiedJetBrains self-reported data: Mellum2.1 reaches 7,969 tokens/s on a single H200, while Qwen3.5-9B reaches 4,347 tokens/s.
50%UnverifiedHigh-end AI accelerators such as the NVIDIA H100 integrate HBM3
50%UnverifiedUnder the AI boom, NVIDIA's production capacity and distribution resources have been prioritized toward high-end models such as the H100 and H200.
50%UnverifiedOracle has made massive investments in large-scale procurement of NVIDIA H100/H200 GPU clusters and bare-metal instance supply.
50%UnverifiedNvidia's high-end GPUs include the H100, H200, and the Blackwell series.
50%UnverifiedAI companies are compelled to accept stringent long-term contract terms in order to secure scarce GPU resources, especially NVIDIA H100/H200 clusters.
50%UnverifiedThe core upgrade of the H200 over the previous-generation H100 lies in VRAM capacity and bandwidth.
50%