[KongchangAI]
BenchmarkWER / Word Error Rate

Word Error Rate

A standardized evaluation metric for measuring the accuracy of speech recognition and text transcription systems. It is calculated as the sum of substitutions, insertions, and deletions divided by the total number of words in the reference transcript; lower values indicate higher transcription quality.

Core Facts

Timeline (last 90 days)

Oct 26

When evaluating Arabic STT, one cannot look only at word error rate (WER) but must also look at key entity recognition rates (station names, ticket numbers, dates, amounts, etc.), because entity recognition errors will crash all downstream processes.

Unverified50%
Oct 22

When comparing WER across models, one must ensure the test corpus and text normalization rules are completely identical, otherwise a 1-2 percentage point gap may come entirely from the evaluation pipeline rather than the model itself.

Unverified50%
Oct 22

The absolute value of WER is highly dependent on test-set difficulty: on read-speech, low-noise corpora, the same model's WER may be only 5%; switched to real recordings with accents and background noise, the same model's WER may double or even higher.

Unverified50%
Oct 20

Looking at the WER published by vendors alone is of limited value; it must be judged in combination with the noise conditions and languages of the test set.

Unverified50%
Oct 20

WER is suitable as a baseline metric for regression testing but cannot serve as acceptance criteria on its own.

Unverified50%
Oct 20

Word error rate (WER) is defined as (substitutions + deletions + insertions) / number of reference words, with lower values indicating closer match to the reference text.

Unverified50%
Oct 18

Public word error rate (WER) figures come from benchmark datasets and cannot represent accuracy under a user's own accent, microphone, or industry terminology.

Unverified50%
Oct 13

Word error rate (WER) is the core metric for measuring speech transcription accuracy, calculated as the sum of substitutions, deletions and insertions divided by the total number of words in the reference text.

Verified75%
Oct 13

For scenarios such as meeting minutes, a WER exceeding 15% may severely affect readability.

Unverified50%
Oct 12

WER is typically measured on standard test sets with clear pronunciation and quiet backgrounds, and real-world performance in actual meeting environments is often worse than official figures.

Unverified50%

7 more timeline events

All Facts (18)

Verified

Word Error Rate (WER) is the most universal performance metric in the speech recognition field, calculated as (number of substitutions + number of deletions + number of insertions) / total number of words in the reference text.

90%
Verified

Word Error Rate (WER) is calculated by dividing the sum of substitution, deletion, and insertion errors between the recognition output and the reference text by the total number of words in the reference text.

80%
Verified

Word error rate (WER) is the core metric for measuring speech transcription accuracy, calculated as the sum of substitutions, deletions and insertions divided by the total number of words in the reference text.

75%
Verified

Word Error Rate (WER) is calculated as (substitution errors + deletion errors + insertion errors) divided by the total number of reference words.

65%
Unverified

When evaluating Arabic STT, one cannot look only at word error rate (WER) but must also look at key entity recognition rates (station names, ticket numbers, dates, amounts, etc.), because entity recognition errors will crash all downstream processes.

50%
Unverified

When comparing WER across models, one must ensure the test corpus and text normalization rules are completely identical, otherwise a 1-2 percentage point gap may come entirely from the evaluation pipeline rather than the model itself.

50%
Unverified

The absolute value of WER is highly dependent on test-set difficulty: on read-speech, low-noise corpora, the same model's WER may be only 5%; switched to real recordings with accents and background noise, the same model's WER may double or even higher.

50%
Unverified

Looking at the WER published by vendors alone is of limited value; it must be judged in combination with the noise conditions and languages of the test set.

50%
Unverified

Word error rate (WER) is defined as (substitutions + deletions + insertions) / number of reference words, with lower values indicating closer match to the reference text.

50%
Unverified

WER is suitable as a baseline metric for regression testing but cannot serve as acceptance criteria on its own.

50%
Unverified

Public word error rate (WER) figures come from benchmark datasets and cannot represent accuracy under a user's own accent, microphone, or industry terminology.

50%
Unverified

For scenarios such as meeting minutes, a WER exceeding 15% may severely affect readability.

50%
Unverified

WER is typically measured on standard test sets with clear pronunciation and quiet backgrounds, and real-world performance in actual meeting environments is often worse than official figures.

50%
Unverified

Earnings22 is a benchmark dataset specifically designed for speech recognition evaluation in the financial domain, consisting of real earnings call recordings from listed companies, containing a large number of industry terms, numerical expressions, and non-native accents, with word error rates generally higher than in general-domain scenarios.

50%
Unverified

In scenarios where regularization is needed, mini-KFAC EWC can achieve lower word error rates and a more stable optimization process compared to standard diagonal EWC.

50%
Unverified

Models fine-tuned on MSA corpora experience a sharp spike in word error rate when encountering real spoken language.

50%
Unverified

Word error rate measures the proportion of edit distance between a model's output and the ground-truth transcription, and is the most core accuracy metric.

50%
Unverified

对于追求智能而非逐字照搬的转写系统,可能需要引入语义相似度(如 BERTScore)、可读性评分、信息完整度等更综合的评估维度

50%

Source Articles