MOS
A widely used subjective evaluation metric for speech quality, in which human listeners rate speech samples on a scale of 1 to 5. MOS is broadly applied to assess the naturalness of text-to-speech and speech enhancement systems.
Core Facts
Timeline (last 90 days)
Traditional TTS is trained with mel-spectrogram reconstruction loss or adversarial loss, and the optimization objective deviates from human subjective perception; MOS-type subjective metrics are not differentiable, so a learnable judge is suitable as a proxy reward for direct optimization via policy gradients.
It is generally believed that an MOS above 4.0 is already close to human naturalness, and beyond 4.5 the differences between systems often require more refined testing to distinguish.
WER (word error rate) measures the difference between ASR transcription and reference text, and MOS (mean opinion score) measures the naturalness of synthesized speech; these were the two core metrics for speech evaluation in the past.
In voice agent scenarios, neither WER nor MOS is sufficient: WER only covers comprehension and does not reflect intent understanding; MOS only measures the naturalness of single-sentence synthesis and does not measure multi-turn dialogue context consistency or emotional expression.
In speech synthesis, the evaluation standard for quantization quality is generally comparison with the output of the original FP16 model, commonly measured using similarity metrics and subjective audio naturalness ratings (such as MOS).
Emotion itself is highly subjective, and cross-cultural and cross-individual differences are difficult for a single MOS to fully capture.
Audio dimension scoring involves Time to First Frame (TTFF), end-to-end latency, speech synthesis naturalness (MOS score), and ASR word error rate (WER) under background noise.
The mainstream metric for evaluating speech naturalness is Mean Opinion Score (MOS), where human listeners rate samples on a scale of 1 to 5.
All Facts (8)
The mainstream metric for evaluating speech naturalness is Mean Opinion Score (MOS), where human listeners rate samples on a scale of 1 to 5.
75%UnverifiedTraditional TTS is trained with mel-spectrogram reconstruction loss or adversarial loss, and the optimization objective deviates from human subjective perception; MOS-type subjective metrics are not differentiable, so a learnable judge is suitable as a proxy reward for direct optimization via policy gradients.
50%UnverifiedIt is generally believed that an MOS above 4.0 is already close to human naturalness, and beyond 4.5 the differences between systems often require more refined testing to distinguish.
50%UnverifiedWER (word error rate) measures the difference between ASR transcription and reference text, and MOS (mean opinion score) measures the naturalness of synthesized speech; these were the two core metrics for speech evaluation in the past.
50%UnverifiedIn voice agent scenarios, neither WER nor MOS is sufficient: WER only covers comprehension and does not reflect intent understanding; MOS only measures the naturalness of single-sentence synthesis and does not measure multi-turn dialogue context consistency or emotional expression.
50%UnverifiedIn speech synthesis, the evaluation standard for quantization quality is generally comparison with the output of the original FP16 model, commonly measured using similarity metrics and subjective audio naturalness ratings (such as MOS).
50%UnverifiedEmotion itself is highly subjective, and cross-cultural and cross-individual differences are difficult for a single MOS to fully capture.
50%UnverifiedAudio dimension scoring involves Time to First Frame (TTFF), end-to-end latency, speech synthesis naturalness (MOS score), and ASR word error rate (WER) under background noise.
50%