HBM
High Bandwidth Memory (HBM) is a high-performance memory standard based on three-dimensional stacking technology. It achieves extremely high data transfer bandwidth and energy efficiency by vertically stacking multiple layers of DRAM dies interconnected via through-silicon vias (TSVs). HBM is used primarily in scenarios with demanding memory bandwidth requirements, including AI training, high-performance computing, and graphics processing, and is an important component of modern GPUs and AI accelerator chips.
Core Facts
The HBM3 memory used by NVIDIA's H100 has a bandwidth of up to 3.35 TB/s.
HBM is a packaging technology that vertically stacks multiple layers of DRAM chips through Through-Silicon Via (TSV) technology and integrates them with the GPU via a silicon interposer.
The core innovation of FlashAttention is to reduce the memory access complexity of standard attention computation from O(N²) to near-linear by controlling the data transfer pattern between the GPU's SRAM and HBM
Timeline (last 90 days)
UMA's memory bandwidth leads on mobile, but still lags behind the HBM bandwidth of data-center-grade GPUs, which becomes a bottleneck in high-concurrency inference or training scenarios.
The growing demand for HBM from AI servers will directly crowd out the general-purpose capacity of DRAM production lines, thereby increasing the overall supply tightness of memory chips.
GPU memory bandwidth growth has long lagged behind compute growth, causing the actual time of operations like attention to be determined by data movement between HBM and on-chip SRAM rather than pure computation volume.
HBM has shoreline bandwidth limitations, power consumption, and thermal issues; three- to five-layer stacking may be the sweet spot, while 16-layer stacking is nearly impossible in terms of yield.
HBM consumes far more capacity per wafer than ordinary DDR dies and has significantly higher gross margins.
Memory manufacturers are tilting capacity toward higher-margin server and HBM products, causing consumer-grade DRAM supply to tighten continuously.
According to a Bilibili video, Changxin Memory successfully developed a 4F2 memory architecture, reducing the single-cell area by about 30% and raising the capacity ceiling of domestic HBM.
Samsung has sold out its entire 2027 HBM capacity.
The market expects HBM prices to surge 200% next year.
For local inference scenarios, high-bandwidth memory (such as Apple's unified memory architecture, or the HBM of high-end GPUs) is more important than multi-core CPUs.
40 more timeline events
All Facts (20)
The HBM3 memory used by NVIDIA's H100 has a bandwidth of up to 3.35 TB/s.
90%VerifiedHBM is a packaging technology that vertically stacks multiple layers of DRAM chips through Through-Silicon Via (TSV) technology and integrates them with the GPU via a silicon interposer.
90%VerifiedThe core innovation of FlashAttention is to reduce the memory access complexity of standard attention computation from O(N²) to near-linear by controlling the data transfer pattern between the GPU's SRAM and HBM
85%VerifiedThe HBM bandwidth of the A100 GPU is approximately 2 TB/s, while the on-chip SRAM bandwidth can reach 19 TB/s, a difference of nearly 10x.
85%VerifiedBoth the NVIDIA H100/H200 and the AMD MI300X integrate HBM.
75%VerifiedCompared to the previous-generation V4 Flash, V4.1 Flash reduces the demand for high-bandwidth memory (HBM) to one-quarter and the demand for SSD persistent storage to one-eighth.
75%VerifiedHBM achieves high-density interlayer interconnection by vertically stacking multiple layers of DRAM chips using through-silicon via (TSV) technology, delivering memory bandwidth up to 3–5 times higher than conventional GDDR6 memory, while consuming less power and occupying a smaller package footprint.
75%VerifiedThe GB300 integrates HBM3e high-bandwidth memory, with a single-card capacity of up to 288GB and memory bandwidth exceeding 10TB/s
75%VerifiedHBM is a core component of Nvidia's AI training chips including the A100, H100, and H200.
75%VerifiedMainstream NVMe SSDs can achieve sequential read speeds of 3500–7000 MB/s.
75%VerifiedThe complex manufacturing process and difficult yield ramp-up of HBM have led to persistently tight supply.
65%VerifiedWhen running large models locally, GPU VRAM is often more critical than compute power; hardware selection should prioritize VRAM capacity over performance benchmarks.
65%VerifiedThe Prefill stage of LLM inference is compute-intensive with GPU utilization close to 100%, while the Decode stage is memory-bandwidth-intensive with the bottleneck being HBM VRAM bandwidth.
65%VerifiedCompared to conventional GDDR6 memory, HBM can deliver 3–5 times higher memory bandwidth while consuming less power and occupying a smaller package footprint.
65%UnverifiedTraditional GPU memory relies primarily on HBM, which achieves high bandwidth through 3D stacking and ultra-wide buses but has limited capacity, high expense, and significant power consumption
90%UnverifiedStorage-class memory exhibits notable disadvantages in access latency and bandwidth compared to HBM's multi-terabyte-per-second bandwidth
85%Unverified内存行业的晶圆产能需要在DDR、LPDDR和HBM三种主要内存类型之间分配
85%Unverified在AI浪潮到来之前，HBM仅占晶圆分配的2%
85%Unverified每生产1GB的HBM所消耗的晶圆产能是生产1GB DDR或LPDDR的三倍以上
80%UnverifiedFlashAttention reduces the number of reads and writes to HBM by retaining intermediate results in SRAM as much as possible, thereby breaking through the memory bandwidth bottleneck to achieve speedup.
60%