AMD ROCm
An open-source software platform developed by AMD for general-purpose GPU computing. It provides a complete runtime, compiler, and math libraries that enable deep learning frameworks to schedule tensor operations on AMD GPUs, serving as AMD's alternative to NVIDIA CUDA.
Core Facts
ROCm is AMD's open-source software platform for high-performance computing and AI workloads, including the HIP programming interface, which allows CUDA code to be ported to AMD GPUs at relatively low cost.
Ollama supports CPU AVX2/AVX512 instruction set acceleration, as well as multiple GPU backends including NVIDIA CUDA, Apple Metal, and AMD ROCm.
Timeline (last 90 days)
Local inference on Linux and Windows platforms usually relies on NVIDIA CUDA, AMD ROCm or pure CPU, with more complex hardware combinations and greater differences in both adaptation effort and performance.
The current version of Ollama is v0.40.3, and the official images support NVIDIA CUDA, AMD ROCm and the built-in Vulkan runtime.
Lucebox supports mixing hardware backends, with NVIDIA RTX, AMD ROCm, Apple Silicon and pure CPU able to appear in the same inference pipeline.
AMD ROCm Docker images with Unsloth Studio and JupyterLab have been added.
NVIDIA GPUs dominate local deployment hardware due to the mature CUDA ecosystem, while AMD ROCm, Apple Silicon, and CPU-only solutions are also continuously improving.
vLLM supports NVIDIA GPU, AMD ROCm, and certain TPU hardware backends.
The new version adds AMD ROCm images for Linux hosts, available alongside CUDA images
vLLM supports the AMD ROCm platform and is continuously improving that support.
Neural Engine is an AI inference framework developed in-house by Blackmagic Design, capable of running on NVIDIA CUDA, AMD ROCm, and the Neural Engine hardware unit of Apple Silicon.
A unified distribution layer can abstract away the differences in programming models across hardware including NVIDIA GPU, AMD ROCm, Intel GPU, Google TPU, and Graphcore IPU.
1 more timeline events
All Facts (13)
ROCm is AMD's open-source software platform for high-performance computing and AI workloads, including the HIP programming interface, which allows CUDA code to be ported to AMD GPUs at relatively low cost.
90%VerifiedOllama supports CPU AVX2/AVX512 instruction set acceleration, as well as multiple GPU backends including NVIDIA CUDA, Apple Metal, and AMD ROCm.
80%UnverifiedvLLM supports the AMD ROCm platform and is continuously improving that support.
70%UnverifiedLocal inference on Linux and Windows platforms usually relies on NVIDIA CUDA, AMD ROCm or pure CPU, with more complex hardware combinations and greater differences in both adaptation effort and performance.
50%UnverifiedThe current version of Ollama is v0.40.3, and the official images support NVIDIA CUDA, AMD ROCm and the built-in Vulkan runtime.
50%UnverifiedLucebox supports mixing hardware backends, with NVIDIA RTX, AMD ROCm, Apple Silicon and pure CPU able to appear in the same inference pipeline.
50%UnverifiedNVIDIA GPUs dominate local deployment hardware due to the mature CUDA ecosystem, while AMD ROCm, Apple Silicon, and CPU-only solutions are also continuously improving.
50%UnverifiedvLLM supports NVIDIA GPU, AMD ROCm, and certain TPU hardware backends.
50%UnverifiedNeural Engine is an AI inference framework developed in-house by Blackmagic Design, capable of running on NVIDIA CUDA, AMD ROCm, and the Neural Engine hardware unit of Apple Silicon.
50%UnverifiedA unified distribution layer can abstract away the differences in programming models across hardware including NVIDIA GPU, AMD ROCm, Intel GPU, Google TPU, and Graphcore IPU.
50%UnverifiedTorchInductor is a code generation backend based on OpenAI Triton, and Triton's backend is being extended to AMD ROCm and Intel GPUs.
50%UnverifiedAMD ROCm Docker images with Unsloth Studio and JupyterLab have been added.
70%UnverifiedThe new version adds AMD ROCm images for Linux hosts, available alongside CUDA images
50%