The First Open-Source Speech Language Model for Basque: Breaking the Low-Resource Bottleneck with Synthetic Data

The Speech Interaction Dilemma for Low-Resource Languages and How to Break It

Multimodal interaction capabilities of large language models are spreading rapidly, but minority languages are often left behind. According to a report on the tech blog rss, a new work has released the first open-source Speech Language Model for Basque, designed specifically for low-latency, end-to-end speech interaction. This means Basque users may soon be able to converse directly with AI systems via voice, without first switching to English or Spanish.

What is most noteworthy about this work is not the model size, but that it validates a viable path for low-resource languages: use a modular architecture to carry over existing language capabilities, then fill in speech training material with synthetic data. For the thousands of languages worldwide that lack annotated speech data, this approach is replicable.

Technical Architecture: Standing on the Shoulders of LLaMA-Omni and Latxa

Using LLaMA-Omni as the Skeleton

The research team chose LLaMA-Omni as the base architecture. This is an end-to-end speech interaction design that eliminates the multi-stage cascade of "speech recognition → text processing → speech synthesis" found in traditional cascaded approaches, thereby reducing latency. For voice assistants seeking a natural conversational experience, an end-to-end architecture can preserve paralinguistic information such as intonation and pauses, and makes response speed closer to that of real human conversation.

End-to-end speech language models differ fundamentally in architecture from traditional cascaded voice assistants. A cascaded solution typically consists of three independent modules in series: automatic speech recognition (ASR) converts user speech to text, a large language model processes the text and generates reply text, and text-to-speech (TTS) synthesizes the reply into audio. Each stage introduces latency and error accumulation, and the intermediate text representation loses paralinguistic information such as intonation, speaking rate, and emotion. End-to-end models, by contrast, map speech input directly to speech output without forcing an intermediate discrete text bottleneck, thus achieving lower latency and more natural conversational rhythm. LLaMA-Omni is one of the representative works in this paradigm; it extends a pretrained language model into a speech interaction model through a speech encoder, speech adapter, and speech decoder, providing a reusable engineering skeleton for subsequent low-resource language adaptation.

Injecting Basque Capabilities with Latxa

The model's backbone uses Latxa—an existing Basque large language model. The real ingenuity lies in capability transfer: the team fine-tunes several modules to transfer Latxa's accumulated Basque understanding and generation capabilities from text to the speech modality. In other words, the model does not need to learn Basque from scratch; it "inherits" existing language knowledge and only needs to learn how to listen and speak.

The advantage of this modular design is a clear division of labor and low iteration cost. Language capability and speech capability are decoupled, so if the language backbone is upgraded in the future, the speech adaptation modules can be adjusted relatively independently.

Latxa is a series of Basque large language models released by the University of the Basque Country (UPV/EHU) and other institutions, built by continuing pretraining and instruction tuning on Basque corpora based on open-weight models such as Llama 2. Its significance lies in the fact that Basque is a language isolate with no known kinship to the Indo-European family, so mainstream large models naturally have weak coverage of it, while Latxa provides systematically trained Basque text understanding and generation capabilities. This work uses Latxa as the language backbone, essentially reusing existing achievements at the "language intelligence" level and training only additional speech encoding and decoding adaptation modules, thereby concentrating limited compute on modality alignment. This decoupled design of "text backbone + speech adaptation" is also a key engineering prerequisite for low-resource language speech models to get off the ground quickly.

Data Strategy: A Combined Approach of Translation and Speech Synthesis

The biggest obstacle for low-resource languages is data scarcity, and Basque is no exception. The team's approach is direct and effective: translate the English VoiceAssistant-400K corpus into Basque, then synthesize corresponding audio using a Basque TTS system.

This pipeline bypasses the hard constraint of "having to record large amounts of real speech." Collecting real speech is costly, time-consuming, and involves privacy and licensing issues; translation plus synthesis can build a sizable speech instruction dataset in a relatively short cycle. Of course, there is a domain gap between synthetic speech and human speech, which is also an aspect that requires close attention in subsequent evaluation.

VoiceAssistant-400K is a large-scale instruction corpus for voice assistant scenarios, containing approximately 400,000 text pairs of user requests and assistant replies, covering everyday interaction types such as Q&A, scheduling, and chit-chat. Its value lies in providing instruction diversity close to real usage distributions, but the original corpus is predominantly English. The team in this work machine-translated it into Basque and then synthesized audio using Basque TTS, which amounts to obtaining "text instruction diversity" and "speech signals" in a decoupled way: the text side borrows the corpus scale of a high-resource language, and the speech side borrows the zero marginal cost of TTS. It should be noted that machine translation may introduce translationese and semantic drift, while TTS-synthesized speech has a domain gap from human recordings in prosody, accent, and background noise; these two factors jointly affect the model's performance in real scenarios.

Evaluation Results: Usability Verified, but Room for Improvement Remains

The evaluation data released by the team gives a relatively clear picture:

• Content quality: 3.86/5, indicating that the model's generated replies are basically reliable at the semantic level;
• Speech intelligibility: word error rate (WER) of 9.94%, meaning that accuracy at the recognition and transcription level has entered the usable range;
• Audio quality: UTMOS score of 3.07, reflecting that the naturalness of the synthesized speech is acceptable but still some distance from human level.

Taken together, these three metrics point to one conclusion: the system as a whole is feasible, and the end-to-end speech interaction pipeline has been proven to work, but audio naturalness is a relative weak point. Considering that the training audio mainly comes from TTS synthesis, this result is not surprising—the domain gap brought by synthetic data is likely to directly affect the audio quality score.

The three metrics in the article correspond to different levels of quality. Word error rate (WER) measures the word-level edit distance between speech recognition or transcription results and the reference text; lower is better, and 9.94% means roughly one word wrong in every ten, which is close to the practical threshold. UTMOS is an automatic metric that scores the naturalness of synthesized speech based on a MOS prediction model, typically on a scale of 1 to 5; 3.07 is roughly in the range of "understandable but noticeably mechanical," a considerable gap from human recordings, which usually score above 4. Content quality 3.86/5 is mostly scored by human or LLM judges for the semantic relevance and usefulness of replies. Taken together, it can be seen that the semantic and recognition pipeline basically passes, and the bottleneck is concentrated in acoustic naturalness, a shortcoming highly correlated with the fact that all training speech comes from TTS synthesis.

Implications for Low-Resource Language AI

The value of this work goes beyond Basque itself. It proves that under low-resource conditions, advanced speech interaction systems are not out of reach: the modular architecture frees the team from reinventing the wheel, and synthetic data provides a workaround for the data bottleneck.

According to the summary from the rss source, the study "demonstrates that even in languages with scarce high-resource data, advanced speech interaction systems can be developed through modular architectures and synthetic data." This judgment has practical significance for linguistic diversity preservation—while mainstream languages iterate voice assistants on a monthly basis, minority languages no longer have to wait years.

What needs to be viewed calmly is that the robustness of synthetic-data-driven models in real scenarios, their coverage of accents and dialects, and audio quality degradation over long-term use all require further verification. The significance of the first open-source model lies more in opening up the field than in being the endpoint.

One-line Summary

The first open-source Basque speech language model uses a modular architecture plus synthetic data to prove end-to-end speech interaction

Paragraph Summary

The article introduces a work that released the first open-source speech language model for Basque. The model uses the end-to-end speech interaction architecture of LLaMA-Omni as its skeleton and the Basque large model Latxa as its language backbone, transferring text language capabilities to the speech modality through fine-tuning. On the data side, the team translated the English VoiceAssistant-400K corpus into Basque and synthesized audio with TTS, bypassing the bottleneck of scarce real speech data. Evaluation shows content quality of 3.86/5, word error rate of 9.94%, and UTMOS of 3.07, indicating that the system as a whole is feasible, but audio quality and naturalness brought by synthetic speech remain weak points. The article argues that its true value lies in validating a replicable path of modular architecture plus synthetic data, which has implications for speech interaction development for low-resource languages worldwide.