A100
Core Facts
Since 2022, the United States has imposed advanced semiconductor export controls on China, restricting the export of high-end AI training chips such as the NVIDIA A100/H100 and their successor products.
Training a model at the level of GPT-4 requires tens of thousands of A100/H100 GPUs working in concert for several months.
Each SM on the NVIDIA A100 has 65,536 32-bit registers.
Timeline (last 90 days)
Nvidia's AI chips (such as H100 and A100) have been placed on the export control list by the US Bureau of Industry and Security (BIS) since 2022, under the Export Administration Regulations (EAR).
The A100's CUDA ecosystem is mature and its memory capacity can still meet most inference scenarios, so it still retains relatively high residual value in the second-hand market.
The power supply and cooling capacity of hyperscale data centers are scarce resources, and the economic return from freeing up 300 racks of space to deploy H100/H200 clusters is far higher than keeping A100s.
When large compute rental companies sell off equipment in concentrated fashion, the short-term supply curve shifts right and prices often overshoot; however, second-hand cards face issues such as voided warranties, driver support cycles, and lagging energy efficiency, so the buyer pool is limited and liquidity is far lower than for new cards.
The intelligent computing platform provides professional cards such as A100 and V100 through GPU leasing, billed by the hour.
Different GPU architectures have different SM counts, Tensor Core generations, memory bandwidth, and L2 cache sizes, so the relative merits of the same tactic on A100 and H100 may be completely opposite.
High-Flyer Quant has been stockpiling tens of thousands of A100 GPUs since 2021 and built the Fire-Flyer cluster.
NVIDIA's MIG technology provides near-native isolation by partitioning models such as the A100 into multiple independent instances at the hardware level, but not all GPU models support it and the partition granularity is fixed.
For small- and medium-scale tasks, a single A100 or RTX 4090 may offer better cost-effectiveness than an H100; the H100's advantages truly show only in ultra-large-scale parallel training.
When traffic doubles to 2x, the OpenAI bill is about $980/month, while self-hosting requires two A100s at about $700/month, narrowing the savings.
40 more timeline events
All Facts (20)
Since 2022, the United States has imposed advanced semiconductor export controls on China, restricting the export of high-end AI training chips such as the NVIDIA A100/H100 and their successor products.
90%VerifiedTraining a model at the level of GPT-4 requires tens of thousands of A100/H100 GPUs working in concert for several months.
90%VerifiedEach SM on the NVIDIA A100 has 65,536 32-bit registers.
80%VerifiedThe RTX 4090's GDDR6X memory interface bandwidth reaches up to 1008 GB/s
80%VerifiedNVIDIA's MIG technology enables hardware-level resource isolation on high-end GPUs such as the A100/H100.
80%VerifiedIndustry estimates put the training cost of GPT-4 at over $100 million, involving tens of thousands of A100 GPUs, months of continuous computation, and petabyte-scale training data.
80%VerifiedThe model weights alone of a 175-billion-parameter model (in BF16 precision) require 350 GB of storage, and the Adam optimizer must maintain two sets of states—first-order moments and second-order moments—totaling more than 1 TB.
75%VerifiedEven after quantization, 70B-level models often require more than 40 GB of VRAM or a multi-GPU configuration.
75%VerifiedIn October 2022, the Biden administration introduced its first export controls targeting advanced computing chips, restricting the export of high-end GPUs such as the NVIDIA A100 and H100 to China.
75%VerifiedThe KV Cache of a 70B-parameter model with 128K context support can require hundreds of gigabytes of VRAM at full load, exceeding the 80GB VRAM capacity of a single H100.
70%VerifiedNVLink 4.0 (used in the H100) can achieve a unidirectional bandwidth of 900 GB/s.
70%VerifiedA Mac Studio M4 Ultra equipped with 192 GB of unified memory can theoretically run complete 70B or even larger parameter models.
70%VerifiedMIG leverages the GPC physical isolation capability of the Ampere architecture to partition GPU compute units, L2 cache, and memory bandwidth into independent partitions at the hardware level, providing true hardware-level isolation.
65%VerifiedThe U.S. Department of Commerce has included high-end GPUs such as the A100 and H100 on its export control list since 2022.
65%UnverifiedNVIDIA MIG仅支持A100/H100等高端卡且切分粒度有限，vGPU方案需要额外购买商业许可证
85%UnverifiedNVIDIA MIG仅A100/H100支持
85%Unverified单卡RTX 4090即可运行GPT-OSS 20B，单卡A100即可运行GPT-OSS 120B
80%Partially VerifiedNVIDIA's H100/A100 GPUs hold approximately 80% of the AI market share.
75%Unverified部署DeepSeek-R1 671B MoE模型需要至少数张高端GPU（如H100或A100），显存需求超过320GB
75%Partially VerifiedNVIDIA A100专业训练卡拥有40GB或80GB HBM2e显存，单卡售价超过1万美元
65%