PyTorch
PyTorch is an open-source deep learning framework created by Meta AI that supports dynamic computational graphs and GPU-accelerated computation.
Core Facts
vLLM is an open-source large language model inference engine project built on top of PyTorch.
Triton is a GPU programming language developed by OpenAI.
Hugging Face's Transformers library supports three major frameworks: PyTorch, TensorFlow, and JAX.
Timeline (last 90 days)
The vLLM maintainer community is active in deep learning framework communities such as PyTorch.
Efficient operators such as FlashAttention have been deeply integrated into mainstream frameworks such as PyTorch, TensorRT-LLM, and vLLM.
The underlying NCCL version is still determined by the user's environment, and when upgrading one needs to check driver and NCCL version compatibility.
The flight recorder continuously records communication metadata in a ring buffer and exports a snapshot on timeout or crash for analysis by torch.distributed's debugging tools.
PyTorch's torch.distributed wraps NCCL through ProcessGroupNCCL.
PyTorch's distributed data parallel (DDP) and FSDP are both built on NCCL.
PyTorch 2.15 is officially released, and the NCCL backend has been completely rewritten and made the default option.
"Backend rewrite" usually means that on the PyTorch side, the wrapping logic of ProcessGroupNCCL, stream management, error handling and interaction with CUDA streams have been refactored, rather than the NCCL library itself being replaced.
Vanilla Stable Diffusion requires users to install Python themselves, configure a virtual environment, and choose the corresponding PyTorch version for their graphics card model; when versions mismatch, error messages are often obscure and hard to understand, which is the main barrier that makes beginners give up.
Around 2019, PyTorch quickly became popular in academia thanks to its dynamic computation graph design and became the default framework for paper reproduction and open-source models (such as most models on Hugging Face).
40 more timeline events
All Facts (20)
vLLM is an open-source large language model inference engine project built on top of PyTorch.
90%VerifiedTriton is a GPU programming language developed by OpenAI.
90%VerifiedHugging Face's Transformers library supports three major frameworks: PyTorch, TensorFlow, and JAX.
90%VerifiedSince its introduction in 2006, CUDA has become the de facto industry standard in the AI/deep learning field, with mainstream frameworks such as PyTorch and TensorFlow using CUDA as their default backend.
80%VerifiedPyTorch uses dynamic computation graphs (Define-by-Run), rebuilding the graph anew with each forward pass.
80%VerifiedPyTorch's Autograd engine builds a directed acyclic graph (DAG) on the fly during tensor operations, recording the order of computations to enable gradient calculation during backpropagation.
80%VerifiedPython deep learning frameworks include TensorFlow and PyTorch
80%VerifiedPyTorch is currently the most popular deep learning framework in both academia and industry.
80%VerifiedPyTorch was open-sourced in 2016 by Meta (formerly Facebook) AI Research.
80%VerifiedTensorFlow 1.x is a classic example of a static computation graph, requiring the entire computation process to be declared and then compiled before execution.
80%VerifiedPyTorch is widely popular in the research community, and its dynamic computation graph mechanism is one of the important reasons.
80%VerifiedCUDA是NVIDIA于2006年推出的并行计算平台和编程模型
80%VerifiedTriton Inference Server is an open-source model serving framework from NVIDIA that supports dynamic batching and concurrent model execution, and is compatible with inference backends such as TensorRT, PyTorch, TensorFlow, and ONNX Runtime.
80%VerifiedTorch Neuron is the open-source native PyTorch backend AWS built for Trainium, responsible for translating, compiling and scheduling PyTorch computation graphs onto Trainium hardware for execution.
75%VerifiedExecuTorch is an on-device inference runtime framework launched by Meta, built on the Export mechanism of PyTorch 2.0.
75%VerifiedMLX supports lazy computation and function transformations, with an API style that draws from both NumPy and PyTorch.
75%VerifiedTorchTitan is the official large-scale training reference implementation released by PyTorch, focused on efficiently training large models using native PyTorch features.
75%VerifiedTorchDynamo is the entry layer of the PyTorch 2.0 compilation stack, converting dynamic graph code into optimizable intermediate representations through Python bytecode interception.
75%VerifiedThe CUDA Toolkit includes the compiler nvcc, debuggers, performance analysis tools, and math libraries such as cuBLAS and cuFFT.
75%VerifiedPyTorch provides dynamic computation graphs and automatic differentiation mechanisms, allowing users to implement backpropagation without doing so manually.
75%