#PyTorch
36 related articles
PyTorch 2.15 Released: NCCL Backend Rewritten and Made Default
PyTorch 2.15 is released: the rewritten NCCL backend becomes the default and a flight recorder is introduced for debugging communication issues. This article explains the impact on multi-GPU distributed training and upgrade considerations.
Developer Builds Custom PyTorch Optimizer ApexW: Half the Memory, 1.5x Faster
Developer releases PyTorch optimizer ApexW with compact optimizer state and Triton execution path, about 1.5x faster than fused AdamW and roughly half the state memory in optimizer-only benchmarks, but small tensors and automatic mode still have clear weaknesses; the author openly seeks benchmark and implementation feedback.
11-Hour Video Course: Building Reasoning Models from Scratch
An 11+ hour video course teaches building reasoning models from scratch with Python and PyTorch, covering evaluation, inference-time scaling, RL, and distillation.
Building a Transformer from Scratch: A 53K-Parameter Dinosaur Name Generator
A developer hand-builds a 53K-parameter Transformer called DinoGPT in PyTorch, trained on 1,535 dinosaur names and wrapped in a Streamlit demo. We unpack the technical approach, the value of tiny models for learning, and product lessons.
Implementing the Transformer Architecture from Scratch: Training Acceleration and Performance Improvement in Practice
A Reddit developer implemented the Transformer from scratch but training was slow and results poor. This article analyzes bottlenecks and offers PyTorch 2.0 attention optimization, mixed precision, learning rate scheduling, and small-data improvement strategies.
TorchTitan Update: New SOTA Models, Performance at Scale, and TitanRL
PyTorch's official TorchTitan project update adds SOTA model support, large-scale training performance optimizations, and the TitanRL reinforcement learning component.
Google Open-Sources TorchTPU: A Native PyTorch Backend Bridges the TPU Ecosystem
Google open-sources TorchTPU, providing a native PyTorch backend for TPU. This article analyzes how it differs from the XLA route, its potential impact on the CUDA ecosystem, and the costs and performance issues developers should watch during migration.
Google Open-Sources TorchTPU: How a PyTorch Native Backend Reshapes the TPU Ecosystem
Google and Meta open-source TorchTPU, a PyTorch native backend for Cloud TPU with eager execution by default and torch.compile/XLA support, set to replace PyTorch/XLA for new workloads. This article analyzes its architecture, execution trade-offs, and impact on developers and the TPU ecosystem.
Nscale Joins PyTorch Foundation as Platinum Member
The PyTorch Foundation welcomes Nscale as a Platinum Member, its highest corporate tier. We unpack Nscale's background, what Platinum governance means, and the real impact on developers.
TorchTPU Released: Native PyTorch Support for Google TPU
Google and Meta jointly release TorchTPU, providing a native TPU backend for PyTorch so models run out of the box. This article examines the open-source collaboration's technical positioning, impact on developer migration costs, and performance and governance questions still to be verified.
Nebius Joins PyTorch Foundation as Platinum Member
The PyTorch Foundation announces AI cloud company Nebius has joined as a Platinum member. This article analyzes the impact on the open-source AI ecosystem, compute vendors, and framework community relations, plus the competitive logic behind Platinum seats.
PyTorch Foundation Adds Nine New Members, Shifting the Open Source AI Landscape
The PyTorch Foundation adds nine new members, with Nebius, Nscale, and Shopify joining as Platinum members, as governance competition and the center of gravity shift in the open source AI ecosystem.
TorchNeuron: A Native PyTorch Backend for AWS Trainium
PyTorch officially releases TorchNeuron, a native PyTorch backend for AWS Trainium. We unpack the background, what a native backend means, and its potential impact on the AI infrastructure landscape.
PyTorch Runs Natively on AWS Trainium: What Zero Code Changes Really Means
AWS announced native PyTorch support on Trainium at the PyTorch Conference, running eager mode, Torch.compile, and standard distributed APIs with zero code changes. This analysis covers the open-source Torch Neuron backend, six hands-on sessions, and the real impact on training hardware selection.
torch-pointcloud: A timm-style Pretrained Model Library for 3D Point Clouds
torch-pointcloud is a pure PyTorch point cloud library modeled after timm, unifying interfaces and pretrained weights for PointNet++, Point Transformer v3, PointMamba, and more across classification, segmentation, detection, and self-supervised pretraining to solve code reproduction and inconsistent evaluation in point cloud research.
How Spyre Became a PyTorch Native Device
PyTorch's official blog explains how Spyre became a native device via PrivateUse1, connecting device, allocator, stream, and compiler abstractions for AI accelerator integration.
Evolution of the PyTorch Media Processing Ecosystem: Dividing Responsibilities Between TorchCodec and TorchVision
PyTorch's official blog clarifies its media processing ecosystem: use TorchCodec for media decoding or encoding, and TorchVision for image and video transformations. This article explains the practical impact of this division on developers and offers pipeline adjustment suggestions.
Set a Random Seed but Got Different Models? The Truth About PyTorch Reproducibility
Set a random seed but still getting different models? This ResNet18 experiment reveals the truth about PyTorch reproducibility: GPU floating-point nondeterminism causes divergence from step 1, with only 16% real performance cost to fix it.
PyTorch Community Spotlight: On-Device AI and Inference Optimization
A PyTorch Ambassador from Mitsubishi Electric highlights four key technical areas: on-device AI, efficient inference, voice agents, and ML compilers — reflecting the core priorities of AI deployment engineering.
New VS Code Extension: Visualize PyTorch Model Architecture at a Glance
A new VS Code extension brings PyTorch model visualization directly into the editor, helping developers understand complex architectures without switching tools.