29M-Parameter Earlybird: Predictive Streaming ASR That Emits Words Before the Speaker Finishes

Title

29M-Parameter Earlybird: Predictive Streaming ASR That Emits Words Before the Speaker Finishes

Meta Description

Earlybird is a 29M-parameter predictive streaming ASR model that can output predictions before the speaker finishes a word. At 4-bit quantization it is only 16.8 MB, leads on word emission timing across multiple test sets, and has calibrated confidence.

Article Content

Streaming speech recognition typically waits for the speaker to finish a word before producing output, often lagging by tens to hundreds of milliseconds. For voice assistants, this delay means search and response planning must be queued behind it. Earlybird changes this rhythm with a model of only 29M parameters (16.8 MB after 4-bit quantization): it learns from randomly truncated audio, including clips where a word is cut off mid-utterance, and requires the model to fully predict all words that should have appeared up to the truncation point, plus guess the next two words.

Core Idea: Turning "Listen Then Write" into "Guess While Speaking"

The output rhythm of traditional streaming recognizers is constrained by the accumulation of acoustic evidence—if the second half of a word has not been uttered yet, the model dares not make a judgment. Earlybird's training approach does the opposite: audio is cut at random positions, and the model must output all complete words that "should have appeared by then" based on the available segment, plus guesses for the next two words.

This training objective has a direct consequence: the model has raw token probabilities for every word (including the word currently being spoken), which can be used as confidence scores. In other words, Earlybird is not just transcribing—it is doing online prediction with confidence.

In streaming speech recognition, "word emission latency" usually refers to the time difference between when acoustic evidence is sufficient to determine a word and when the model actually outputs that word. Traditional models use "frame-synchronous" decoding, where each audio frame only outputs the token corresponding to the current frame, and word boundaries must wait for posterior probabilities to stabilize, thus inherently lagging. Earlybird's "predictive" training breaks this constraint: it does not require the model to output only the token of the current frame at any moment, but allows the model to output words that will only fully appear in the future while audio is still mid-stream. This training approach resembles a "fill-in-the-blank" task, but the blank positions are dynamic truncation points, and the model must learn to make probabilistic judgments when information is incomplete.

Timing Performance: Median Words Emitted Before the Speaker Finishes

On clean read speech (LibriSpeech test-clean), Earlybird can give correct predictions for the median word before the speaker finishes, whereas published streaming models generally output only after the word ends. There is a clear time difference between the two, in completely opposite directions.

On six test sets timed by the team—covering audiobooks, TED talks, public speeches, earnings calls, and synthetic speech—Earlybird is the earliest to emit words. On the same audiobook data, most words are correctly predicted before completion, while the proportion for comparison models ranges from single digits to the low twenties percent.

This lead is not achieved at the expense of accuracy. On eight English Open ASR Leaderboard test sets, Earlybird outperforms Moonshine Tiny on every set and is roughly on par with the same-sized Whistle.

Confidence Calibration: Daring to Guess, and Knowing How Accurate the Guess Is

The prerequisite for early prediction is that the model can judge how certain it is. Earlybird's confidence for "the word currently being spoken" is calibrated, with expected calibration error (ECE) between 3.7 and 5.5. This means downstream voice agents can make decisions based on confidence: high-confidence words go directly into semantic parsing, while low-confidence words wait for more audio.

For voice interaction systems, this has more engineering value than raw word error rate—it turns early output into a controllable risk rather than a blind gamble.

Expected Calibration Error (ECE) is a metric measuring how well a model's confidence matches its actual accuracy. Specifically, predictions are bucketed by confidence, the absolute difference between the average confidence and the true accuracy in each bucket is computed, and then a weighted average is taken by sample size. The lower the ECE, the more the model's certainty matches reality. For an early-prediction model like Earlybird, an ECE between 3.7 and 5.5 means that when the model gives 80% confidence, the actual accuracy is roughly between 74.5% and 76.3%, with controllable deviation. This allows downstream systems to set thresholds: predictions above the threshold are accepted directly, while those below wait for more audio, enabling a dynamic trade-off between latency and accuracy.

Practical Significance of a Small Model

With 29M parameters and a size of 16.8 MB after 4-bit quantization, Earlybird can be deployed on-device or in resource-constrained environments. Combined with predictive output, it suits voice agent scenarios sensitive to response latency: before the user finishes speaking, the system is already preparing the next action.

Currently, public information focuses on recognition quality and word emission timing. Multilingual support, performance in noisy environments, and actual end-to-end latency still require more independent evaluation.

4-bit quantization compresses model weights from 16-bit floating point to 4-bit integers, reducing size to about one quarter while maintaining accuracy through quantization-aware training or post-training quantization. A 16.8 MB model can easily be embedded in mobile applications or edge devices, enabling streaming recognition without an internet connection. For voice agents, on-device deployment means audio does not need to be uploaded to the cloud, reducing latency and improving privacy. Combined with predictive output, the system can start local semantic parsing and response planning while the user is still speaking, compressing end-to-end response time from "hundreds of milliseconds after speaking" to "respond as soon as speaking ends" or even "ready before speaking ends."

One-line Summary

Earlybird achieves predictive streaming ASR with 29M parameters, emitting words before the speaker finishes with calibratable confidence.

Paragraph Summary

Earlybird proposes a predictive streaming ASR method: during training, audio is randomly truncated, and the model must output all words that should appear before the truncation point and guess the next two words. With only 29M parameters and 16.8 MB after 4-bit quantization, the model can correctly predict the median word on LibriSpeech test-clean before the speaker finishes, whereas comparison models generally output only after the word ends. It emits words earliest on six test sets, outperforms Moonshine Tiny on all eight Open ASR Leaderboard sets, and is on par with Whistle. The model provides calibrated confidence (ECE 3.7–5.5) for the word currently being spoken, enabling downstream agents to make confidence-based decisions. Its small size suits on-device deployment, but multilingual support, noisy environments, and end-to-end latency still require independent evaluation.