17MB Speech Transcription Model Whistle: A New Option for On-Device Deployment

A Speech Transcription Model That's Only 17MB

Speech-to-text models usually come with a "big and heavy" reputation. Solutions like Whisper and Moonshine often run tens or hundreds of megabytes, making deployment on phones or embedded devices far from easy. Whistle, released by Cactus Compute, breaks this conventional thinking: the model file is only 16.9MB, yet its transcription quality approaches that of Whisper Base and Moonshine Tiny V2.

What's most noteworthy about this model isn't that it's the "fastest" or the "most accurate" — it's the extreme portability. A single .cact file can be copied onto a phone, Raspberry Pi, microcontroller, robot, smart home device, or in-car system and just run — no internet connection, no API dependency. For hardware projects that need offline speech capabilities, this offers an option that was previously unrealistic.

Cactus Compute was previously known for its Needle series (tool-calling and structured-output models), and Whistle is their new foray into the speech domain. Interestingly, Whistle's precision is specified as 2-4 bit quantization, which is the key to compressing its size to this degree. While quantization brings size advantages, whether it sacrifices recognition quality is the question that most needs verification in actual use.

Quantization is one of the core techniques of model compression. It maps neural network weights and activations from high-precision floating point (such as 32-bit or 16-bit) to low-bit integers (such as 2-bit, 4-bit, 8-bit), thereby greatly reducing storage footprint and memory bandwidth requirements. 2-4 bit quantization means each parameter is represented by only 2 to 4 binary bits, achieving 4-8x compression compared to 16-bit floating point. The cost is a narrower numerical representation range and precision loss, which may lead to higher recognition error rates. Common quantization methods include post-training quantization (PTQ) and quantization-aware training (QAT); the latter simulates quantization errors during training and typically preserves accuracy better. Whistle's ability to maintain usable performance at 2-4 bit suggests its training pipeline was likely optimized for low-bit inference.

Hands-On: Transcription Performance on PC and Mobile

Quick Desktop Verification

In the video, the creator demonstrates the installation process on a PC. The core steps aren't complex: create a project with uv init --no-package, then install the package via uv add cactus-needle. The official example uses a WAV file as input, but you can also call the microphone directly:

uv run needle-whistle-playground

The first run will prompt that NumPy is missing, requiring reinstalling with microphone dependencies: uv add cactus-needle[mic].

During testing, the creator transcribed statements at normal speaking speed and fast, mumbled sentences (such as "Python is an interpreted language"), and the model correctly recognized essentially all of them. This shows that under 2-4 bit quantization, Whistle's robustness remains intact.

Packaging into an APK to Run on a Phone

A more complete verification was done on mobile. The creator used a Flutter project to package Whistle into an APK; the entire app is about 34MB, with the .cact model file placed in the assets directory and declared via pubspec.yaml. At the code level, libneedle.so needs to be loaded — this dynamic library is responsible for parsing the weights, while the .cact file itself contains only the weight data. After initialization, the transcribe function handles recognition, and the UI is only responsible for displaying results.

After installing on the phone, tapping the microphone button in the lower-right corner of the interface invokes the built-in model for transcription. The creator retested with the same mumbled, fast sentences, and the results "looked very good." The entire app works independently without a network connection — exactly the core value scenario of Whistle.

libneedle.so is the dynamic link library (.so is the Linux/Android shared object format) of the Cactus Needle runtime on Android. It is responsible for loading the .cact model file, parsing weights, executing inference computations, and exposing interfaces such as transcribe to upper layers. Separating model weights (.cact) from the inference engine (libneedle.so) offers two benefits: first, weight files can be independently updated or replaced without recompiling the entire application; second, the same runtime can support multiple models simply by switching weight files. In a Flutter project, you need to declare .cact as an asset via pubspec.yaml and call libneedle.so to complete initialization and transcription, either at the native layer or via FFI.

A Few Judgments on On-Device Speech Models

The deployment freedom bought by small size is a real gain. A 34MB Flutter app means speech transcription can be bundled into any app like an ordinary feature, without requiring users to download additional models or paying for cloud APIs. It's especially appealing for privacy-sensitive scenarios (such as medical or in-car).

Quantization is a double-edged sword and needs to be validated per scenario. The 2-4 bit precision performs well in the demo, but the demo samples are mostly short sentences with clear pronunciation. Actual performance on long audio, strong accents, and noisy environments still needs more testing. The creator himself admits that the reason for choosing Whistle is "not that it's the best, nor that it's the fastest, but that it's the most portable."

Comparisons with Whisper Base and Moonshine must be based on fair benchmarks. In terms of size, Whisper Base is far larger than Whistle, while Moonshine Tiny V2's quantized version is 34.4MB and its 16-bit floating-point version is 109MB. Whistle's 16.9MB has a clear size advantage, but its performance is "in the same order of magnitude" rather than superior — this positioning needs to be kept clear.

Possibilities for hardware applications have been opened up. When a speech transcription model is small enough to fit in a microcontroller, the design space for interaction in robots, wearables, and smart homes will expand accordingly. This is a concrete, actionable advance in the on-device AI trend.

Who It's For

If you're building hardware or mobile applications that need offline speech-to-text, Whistle is worth a try: the installation barrier is low, and there are ready paths for both Dart/Flutter and Python, while a single model file also simplifies distribution. If what you're after is the highest recognition accuracy or stability on the longest audio, it may not be sufficient at this stage, and you'll need to evaluate it with more comprehensive benchmarks.

The real significance of the project lies in demonstrating a direction: compressing a good-enough model to be small enough that AI capabilities can be embedded into any device like an ordinary code library. Once this path is proven, the imaginative space for on-device AI will be much larger than it is now.

One-line Summary

Cactus Compute launches Whistle, a speech transcription model of just 16.9MB, extremely portable and deployable offline on phones and microcontrollers.

Paragraph Summary

Cactus Compute's Whistle speech transcription model file is only 16.9MB, using 2-4 bit quantization, and can run fully offline on devices such as phones, Raspberry Pi, and microcontrollers. In hands-on testing, after installing via uv on desktop, it can transcribe microphone input in real time with stable recognition of short sentences; the Flutter-packaged APK is about 34MB, and the built-in model can independently complete mobile transcription. Whistle's performance is in the same order of magnitude as Whisper Base and Moonshine Tiny V2, with a clear size advantage, but its positioning is portability rather than leading in accuracy or speed. The article argues that the deployment freedom brought by small size is a real gain, quantization effects need to be validated per scenario, and small on-device models open up the possibility of offline voice interaction for hardware scenarios such as robots, wearables, and smart homes.