BitRun
A zero-compilation, single-file 1-bit LLM inference runner with statically linked built-in NEON DotProd and AVX2 native SIMD kernels. It runs ternary-quantized models on pure CPU without build dependencies.
Timeline (last 90 days)
BitRun was released by the author IshekKhal, and its initial alpha received feedback from thousands of developers.
BitRun can automatically write background running logs to ~/.bitrun/engine.log.
BitRun v0.2 can run PrismML's 8-billion-parameter model bonsai-8b entirely in CPU memory, using about 2.08 GB of memory, without needing a discrete GPU, a cloud API, or compilation.
BitRun rebuilt the inference runtime from the ground up to run the 8B model, including decoupling from legacy submodules, fixing an 8B tensor offset crash, fixing matrix memory boundary and stride safety issues, and architecture-specific SIMD acceleration.
BitRun integrates ARM NEON dot-product kernels and squared ReLU activation on Apple Silicon, while providing an AVX2 fallback for Windows and Linux PCs.
On the x86 platform, the 8B model still goes through the generic portable loop in arch-fallback.h, which is the root cause of the sharp performance drop for large models on x86.
BitRun can start an OpenAI-compatible REST interface with the command bitrun serve falcon3-1b --port 11435.
BitRun is a statically linked standalone runner with built-in native SIMD kernels (NEON DotProd on Mac, AVX2 on Windows/Linux) and requires no build dependencies.
BitRun's GitHub repository is IshekKhal/bitrun, with released binaries under the v0.1.0-alpha tag; the related upstream PRs #642 and #644 are in microsoft/BitNet.
On an Apple Silicon M4 Pro (12 cores, ARM NEON), Falcon3 1B Instruct reaches 137.5 t/s prompt processing and 98.6 t/s generation speed, with 1.9 GB memory usage.
4 more timeline events
All Facts (10)
On the x86 platform, the 8B model still goes through the generic portable loop in arch-fallback.h, which is the root cause of the sharp performance drop for large models on x86.
50%UnverifiedBitRun can automatically write background running logs to ~/.bitrun/engine.log.
50%UnverifiedBitRun v0.2 can run PrismML's 8-billion-parameter model bonsai-8b entirely in CPU memory, using about 2.08 GB of memory, without needing a discrete GPU, a cloud API, or compilation.
50%UnverifiedBitRun rebuilt the inference runtime from the ground up to run the 8B model, including decoupling from legacy submodules, fixing an 8B tensor offset crash, fixing matrix memory boundary and stride safety issues, and architecture-specific SIMD acceleration.
50%UnverifiedBitRun integrates ARM NEON dot-product kernels and squared ReLU activation on Apple Silicon, while providing an AVX2 fallback for Windows and Linux PCs.
50%UnverifiedBitRun was released by the author IshekKhal, and its initial alpha received feedback from thousands of developers.
50%UnverifiedBitRun can be installed with a one-line curl command and supports macOS and Linux; there is also a one-line install command for Windows PowerShell, and after installation you can pull the model with bitrun pull falcon3-1b and run it with bitrun run falcon3-1b.
50%UnverifiedBitRun can start an OpenAI-compatible REST interface with the command bitrun serve falcon3-1b --port 11435.
50%UnverifiedBitRun's GitHub repository is IshekKhal/bitrun, with released binaries under the v0.1.0-alpha tag; the related upstream PRs #642 and #644 are in microsoft/BitNet.
50%UnverifiedBitRun is a statically linked standalone runner with built-in native SIMD kernels (NEON DotProd on Mac, AVX2 on Windows/Linux) and requires no build dependencies.
50%