DenseCore
A lightweight deep learning framework written in Rust, emphasizing bit-level parity with PyTorch's CPU implementation. It is approximately 2 MB in size and currently supports only the Apple Silicon Mac platform.
Timeline (last 90 days)
DenseCore, a deep learning framework written in Rust by a CS student, produces bit-for-bit identical training results to PyTorch when running nanoGPT on an Apple Silicon Mac, with the initial weights, losses, gradients and weight updates completely identical for 50 consecutive steps, even with dropout enabled.
DenseCore can only match PyTorch's bit-level consistency on Apple Silicon Macs.
DenseCore is about 2MB in size, while PyTorch is about 580MB.
The author proposed three possible development directions for DenseCore: continue to replicate PyTorch, become a lightweight drop-in replacement, or transform into a diagnostic tool.
Turning DenseCore into a PyTorch difference diagnostic tool may be the most practical.
DenseCore currently only covers small models like nanoGPT.
DenseCore's GitHub repository address is https://github.com/lncch/DenseCore, and the nanoGPT comparison results can be verified by running `python scripts/demo_bit_exact.py`.
All Facts (7)
DenseCore, a deep learning framework written in Rust by a CS student, produces bit-for-bit identical training results to PyTorch when running nanoGPT on an Apple Silicon Mac, with the initial weights, losses, gradients and weight updates completely identical for 50 consecutive steps, even with dropout enabled.
50%UnverifiedDenseCore can only match PyTorch's bit-level consistency on Apple Silicon Macs.
50%UnverifiedDenseCore is about 2MB in size, while PyTorch is about 580MB.
50%UnverifiedThe author proposed three possible development directions for DenseCore: continue to replicate PyTorch, become a lightweight drop-in replacement, or transform into a diagnostic tool.
50%UnverifiedTurning DenseCore into a PyTorch difference diagnostic tool may be the most practical.
50%UnverifiedDenseCore currently only covers small models like nanoGPT.
50%UnverifiedDenseCore's GitHub repository address is https://github.com/lncch/DenseCore, and the nanoGPT comparison results can be verified by running `python scripts/demo_bit_exact.py`.
50%