cuDNN
A deep neural network acceleration library released by NVIDIA that provides highly optimized implementations of standard operators. It serves as a core computational library for deep learning training and inference within the CUDA ecosystem.
Core Facts
Since its introduction in 2006, CUDA has become the de facto industry standard in the AI/deep learning field, with mainstream frameworks such as PyTorch and TensorFlow using CUDA as their default backend.
CUDA是NVIDIA于2006年推出的并行计算平台和编程模型
As of 2024, the CUDA developer community has exceeded 4 million members, and software libraries optimized based on CUDA number more than 900.
Timeline (last 90 days)
CUDA's moat includes optimized libraries such as cuBLAS, cuDNN, and NCCL, the Nsight series of profilers and debuggers, and more than a decade of accumulated tutorials, Q&A, and third-party framework adaptations.
Manually setting up an AI development environment requires handling dozens of components such as NVIDIA drivers, the CUDA toolkit, cuDNN, the Python interpreter, Visual Studio Build Tools, Git, Node.js and FFmpeg, and involves version compatibility and PATH ordering issues.
The Ubuntu installer writes environment variables into the profile files shared by all users, so that CUDA_HOME, CUDA_PATH, CUDNN_PATH and the CUDA tool paths take effect for all users; libraries are registered at the system level, so no LD_LIBRARY_PATH is needed; the use-cuda 12.9 command can temporarily switch a single terminal to CUDA 12.9; old configuration lines in the user's ~/.bashrc that switch CUDA versions are backed up and then removed.
The Ubuntu installer installs: Python 3.10 to 3.14 plus the free-threaded 3.13t and 3.14t builds (Ubuntu's own python3 is left unchanged), CUDA 12.9 and 13.2 (13.4 on Ubuntu 26.04), cuDNN 9.20, GCC, Clang, CMake, Ninja, Git and Git LFS, Node.js 24, .NET 10 SDK, FFmpeg 9, uv, and a 100GB swap file; the NVIDIA driver installs the latest version offered by Ubuntu (R595 or newer, R615 on 26.04), uses Ubuntu's signed open-source kernel modules, only upgrades and never downgrades on each run, and is executed as the final step.
The Windows installer places cuDNN and CUDA 13.2 first in PATH, followed by the Windows system directories, Python 3.12 and other versions, the C++ build paths and the FFmpeg shared libraries; the user's existing entries for other programs are kept at the end and not deleted; the pre-install PATH backup can be viewed in the report and restored with one command via --restore-env.
The Windows installer executes 20 steps and installs: the latest compatible NVIDIA WHQL driver (RTX 20 series and above), Python 3.10 to 3.14 (3.12 as default), CUDA 12.9 and 13.2 installed side by side (13.2 as default), cuDNN 9.20 (only this version registered), Visual Studio 2022 Build Tools, the latest VC++ runtime, Git, Node.js 24, .NET 10 SDK, FFmpeg 9, PATH ordering tuning, Windows long path support, and a 100GB page file.
Achieving bit-wise determinism in LLM inference requires fixing all reduction orders, disabling non-deterministic GPU operators (such as some cuBLAS/cuDNN kernels and atomic adds), disabling dynamic batching, and padding inputs to a fixed length; these constraints significantly reduce throughput and memory efficiency.
GPUs have mature software stacks such as CUDA, cuDNN and NCCL, while NPUs mostly rely on vendor-proprietary SDKs (such as Ascend's CANN or the TPU's XLA path), and their cross-platform portability and operator coverage are far inferior to those of GPUs.
cudnn.benchmark auto-tunes on every run and may select different, non-deterministic convolution algorithms, adding a second layer of non-determinism.
Disabling cudnn.benchmark accounts for approximately 3% of the performance overhead on this model.
14 more timeline events
All Facts (20)
Since its introduction in 2006, CUDA has become the de facto industry standard in the AI/deep learning field, with mainstream frameworks such as PyTorch and TensorFlow using CUDA as their default backend.
80%VerifiedCUDA是NVIDIA于2006年推出的并行计算平台和编程模型
80%VerifiedAs of 2024, the CUDA developer community has exceeded 4 million members, and software libraries optimized based on CUDA number more than 900.
75%VerifiedThe CUDA ecosystem encompasses core components such as cuDNN, cuBLAS, and NCCL; mainstream frameworks including PyTorch and TensorFlow all use CUDA as their primary acceleration backend.
65%VerifiedThe barriers of the CUDA ecosystem are deeply rooted in operator libraries (cuBLAS, cuDNN), compilation optimization toolchains (NVCC, TensorRT), and distributed training frameworks (NCCL), forming NVIDIA's deep moat.
65%VerifiedThe core of CUDA includes the CUDA runtime and driver layer, the PTX intermediate instruction set, and companion libraries such as cuDNN, cuBLAS, and TensorRT.
65%VerifiedSince CUDA was introduced by NVIDIA in 2007, it has formed a complete accelerated library ecosystem including cuBLAS, cuDNN, and TensorRT.
65%VerifiedSince its release in 2007, CUDA has accumulated nearly two decades of development, with its moat reflected in low-level optimization libraries such as cuDNN, TensorRT, and NCCL.
65%UnverifiedOllama将CUDA、cuDNN等底层依赖封装在内部，用户无需手动配置GPU环境
90%UnverifiedCompileIQ 可用于优化 cuBLAS、cuDNN、CUTLASS 等底层库的内核级编译
80%Partially VerifiedNVIDIA currently holds a near-monopoly on the AI training and inference chip market, with its CUDA ecosystem forming an extremely high software moat.
65%UnverifiedCUDA's moat includes optimized libraries such as cuBLAS, cuDNN, and NCCL, the Nsight series of profilers and debuggers, and more than a decade of accumulated tutorials, Q&A, and third-party framework adaptations.
50%UnverifiedThe Ubuntu installer installs: Python 3.10 to 3.14 plus the free-threaded 3.13t and 3.14t builds (Ubuntu's own python3 is left unchanged), CUDA 12.9 and 13.2 (13.4 on Ubuntu 26.04), cuDNN 9.20, GCC, Clang, CMake, Ninja, Git and Git LFS, Node.js 24, .NET 10 SDK, FFmpeg 9, uv, and a 100GB swap file; the NVIDIA driver installs the latest version offered by Ubuntu (R595 or newer, R615 on 26.04), uses Ubuntu's signed open-source kernel modules, only upgrades and never downgrades on each run, and is executed as the final step.
50%UnverifiedThe Ubuntu installer writes environment variables into the profile files shared by all users, so that CUDA_HOME, CUDA_PATH, CUDNN_PATH and the CUDA tool paths take effect for all users; libraries are registered at the system level, so no LD_LIBRARY_PATH is needed; the use-cuda 12.9 command can temporarily switch a single terminal to CUDA 12.9; old configuration lines in the user's ~/.bashrc that switch CUDA versions are backed up and then removed.
50%UnverifiedManually setting up an AI development environment requires handling dozens of components such as NVIDIA drivers, the CUDA toolkit, cuDNN, the Python interpreter, Visual Studio Build Tools, Git, Node.js and FFmpeg, and involves version compatibility and PATH ordering issues.
50%UnverifiedThe Windows installer executes 20 steps and installs: the latest compatible NVIDIA WHQL driver (RTX 20 series and above), Python 3.10 to 3.14 (3.12 as default), CUDA 12.9 and 13.2 installed side by side (13.2 as default), cuDNN 9.20 (only this version registered), Visual Studio 2022 Build Tools, the latest VC++ runtime, Git, Node.js 24, .NET 10 SDK, FFmpeg 9, PATH ordering tuning, Windows long path support, and a 100GB page file.
50%UnverifiedThe Windows installer places cuDNN and CUDA 13.2 first in PATH, followed by the Windows system directories, Python 3.12 and other versions, the C++ build paths and the FFmpeg shared libraries; the user's existing entries for other programs are kept at the end and not deleted; the pre-install PATH backup can be viewed in the report and restored with one command via --restore-env.
50%UnverifiedAchieving bit-wise determinism in LLM inference requires fixing all reduction orders, disabling non-deterministic GPU operators (such as some cuBLAS/cuDNN kernels and atomic adds), disabling dynamic batching, and padding inputs to a fixed length; these constraints significantly reduce throughput and memory efficiency.
50%UnverifiedGPUs have mature software stacks such as CUDA, cuDNN and NCCL, while NPUs mostly rely on vendor-proprietary SDKs (such as Ascend's CANN or the TPU's XLA path), and their cross-platform portability and operator coverage are far inferior to those of GPUs.
50%UnverifiedDisabling cudnn.benchmark accounts for approximately 3% of the performance overhead on this model.
50%