ElevenLabs Realtime Translation: Simultaneous Interpretation in 90+ Languages via a Single WebSocket for Speech-to-Speech

ElevenLabs has launched a new product called Realtime Translation on Product Hunt, positioned as a tool that "turns live speech into translated audio"—not subtitles, but direct output of emotional speech in the target language. It targets telecom operators, call centers, and live events, supporting over 90 languages, so each person speaks their native tongue and hears the other's language.

What problem does it solve?

Traditional remote translation solutions typically involve three steps: speech recognition (STT) to text, large model translation, and then speech synthesis (TTS) output. The problem with this serial pipeline is compounded latency, loss of intonation, and inconsistent speaker timbre across the stages.

ElevenLabs' approach compresses the entire chain into a single WebSocket connection, generating expressive translated audio directly from real-time speech. The product page emphasizes that it differs from tools that offer "subtitles only" and from stitched pipelines—the translated speech maintains a single, consistent timbre, whether from the system voice library or your own cloned voice.

This builds on ElevenLabs' expertise in voice cloning and streaming TTS. Making timbre consistency a selling point means users still "sound like themselves" in cross-language conversations, which has practical value for brand customer service and live streaming scenarios.

The STT, LLM, TTS three-stage pipeline refers to the typical architecture of traditional speech translation: first, automatic speech recognition (Speech-to-Text) converts source language audio to text, then a large language model (or machine translation model) translates the text into the target language text, and finally text-to-speech (Text-to-Speech) synthesizes the target language audio. Each stage is an independent model inference and network round trip, and when chained, latency adds up noticeably; meanwhile, the intermediate text step loses prosodic, stress, and emotional paralinguistic information, and the timbres maintained by the three models are hard to align. ElevenLabs' single WebSocket solution essentially replaces these three stages with one end-to-end streaming model, thereby eliminating serial overhead and timbre drift at the architectural level.

Key features breakdown

Single connection, streaming output

The entire translation process runs over one WebSocket, which directly benefits latency control. Compared to three serial calls, a single connection reduces handshake and queuing overhead, making it more suitable for latency-sensitive scenarios like conversations.

WebSocket is an application-layer protocol that provides full-duplex communication over a single TCP connection, allowing clients and servers to push messages to each other at any time without re-establishing connections for each request like HTTP. For real-time speech scenarios, this means audio chunks can continuously upload, and translated audio and subtitles can continuously download, forming a true streaming pipeline. Compared to polling or multiple HTTP calls, WebSocket eliminates repeated handshakes and header overhead, and allows the server to start inference as soon as partial input arrives, rather than waiting for the entire speech segment to end. This is the concrete source of "reducing compounded latency" at the protocol level, and one of its engineering differences from stitched pipelines.

Timbre consistency

The product supports using the official voice library and user-owned cloned voices. The translated audio continues with the same timbre, avoiding the disjointed feeling of "changing language means changing person."

Voice Cloning refers to training or conditioning a TTS model with a small number of speech samples from a target speaker, so that the generated speech approximates the speaker's timbre and quality. ElevenLabs initially built its reputation on few-shot cloning: users upload a few minutes of audio and get a reusable "voice ID." In real-time translation scenarios, the cloned voice is used as the timbre condition for the translation output, so the translated target language speech retains the speaker's original timbre characteristics. It's important to distinguish that what's cloned is timbre, not all features like accent or speaking rate; across languages, the model still needs to reconstruct prosody within the target language's phonetic system, so "sounding like yourself" is more about continuity at the timbre level rather than a complete replication of the original voice.

Synchronized subtitles and transcription

In addition to audio, the product also streams subtitles and transcription text. For scenarios requiring archiving, compliance review, or accessibility, this text layer is a necessary supplement.

Adjustable speed and accuracy

Different use cases have different trade-offs between latency and accuracy. Live events may prioritize low latency, while business negotiations may care more about translation quality. The product allows adjusting the balance between speed and accuracy per use case, which is not common among similar tools.

Target scenarios and practical considerations

The three customer types officially named—telecom operators, call centers, and live events—all have hard requirements for real-time performance. Call centers need agents and customers to communicate across languages, and live events need speakers' words to be conveyed instantly to multilingual audiences.

However, the challenges of real-time speech translation go beyond the model. Accents, background noise, multiple people talking over each other, and technical terminology all affect the final experience. The product page does not disclose specific latency data or accuracy performance per language—these are key metrics for assessing practical usability, and for now we can only wait for more real-world feedback.

Another point worth noting is "expressiveness." Translated audio retaining emotion and intonation means it's not just swapping words but also conveying tone. In customer service scenarios, this may affect customer perception, but it also brings risks—misjudging tone could amplify misunderstandings.

Competitive landscape

The real-time speech translation space is not empty. There are dedicated hardware and software solutions focused on meeting scenarios, as well as translation capabilities layered on general models by big players. ElevenLabs' differentiation lies in packaging voice cloning, streaming TTS, and translation into an enterprise-oriented voice layer, rather than a point tool.

For developers, the single WebSocket interface reduces integration complexity; for enterprises, timbre consistency and adjustable speed-accuracy trade-offs are more practical selling points. Whether it can maintain performance in real noisy environments will determine whether it can move from demo to production.

Signals from launch data

The product received 82 upvotes on Product Hunt, ranking 13th, with 1 comment. The vote count shows it has gained attention, but the sparse comments mean community discussion has yet to unfold, and actual user feedback is limited. For an enterprise-oriented infrastructure product, this is a normal starting state.

Summary

ElevenLabs Realtime Translation advances real-time translation from "reading subtitles" to "hearing speech," entering enterprise communication scenarios with a single-connection architecture and timbre consistency. Its engineering approach is clear, but key data such as latency, noise robustness, and multilingual accuracy have not been disclosed, and actual performance remains to be verified. For call center and live event operators, this is an option worth adding to the evaluation list.

One-line Summary

ElevenLabs launches real-time speech translation, delivering emotional translated speech directly over a single connection.

Paragraph Summary

ElevenLabs released Realtime Translation on Product Hunt, positioned to directly convert real-time speech into emotional target-language audio rather than just providing subtitles, supporting over 90 languages and targeting telecom operators, call centers, and live events. It uses a single WebSocket to carry the entire chain, replacing the traditional STT+LLM+TTS three-stage stitching to reduce compounded latency and maintain timbre consistency, using either the official voice library or user-cloned voices. The product also streams subtitles and transcription and allows adjusting speed and accuracy per use case. The article notes that its engineering approach is clear, but key data such as latency, noise robustness, and multilingual accuracy have not been disclosed, and actual performance remains to be verified.