Confusion Matrix
A visualization tool for evaluating classification model performance. It presents the correspondence between model predictions and ground-truth labels in matrix form, enabling analysis of misclassification patterns across individual categories.
Timeline (last 90 days)
In IDP scoring, before mapping to a confusion matrix, the extraction result for each field should be faithfully recorded as one of five raw states: correct, misread, missed, invented, correct blank.
For misreads of text fields, there are different conventions such as counting only FN, only FP, or both FN and FP; different conventions directly change the final score, so the convention adopted must be stated when publishing results.
Precision is essentially computing, given that a prediction is positive, the posterior probability that it is truly positive.
Precision, recall, F1 score, and accuracy can all be derived from the four numbers of the confusion matrix.
When both Prevalence and FPR are derived from the same confusion matrix, the Bayesian formula naturally simplifies to TP/(TP+FP).
When evaluating a judge, one should look at the confusion matrix rather than just accuracy; if 90% of the samples are good answers to begin with, a judge that passes everything can still show 90% accuracy.
The confusion matrix lists the four combinations—true positives (TP), false positives (FP), false negatives (FN), and true negatives (TN)—in a 2×2 table.
In practice, precision and recall are commonly used to quantify false positives and false negatives, and confusion matrices are used to analyze model performance.
In a binary classification confusion matrix, the F1 score is the harmonic mean of precision and recall.
All Facts (10)
混淆矩阵的概念最早可追溯到1904年Karl Pearson在统计学中对分类误差的研究
60%UnverifiedFor misreads of text fields, there are different conventions such as counting only FN, only FP, or both FN and FP; different conventions directly change the final score, so the convention adopted must be stated when publishing results.
50%UnverifiedIn IDP scoring, before mapping to a confusion matrix, the extraction result for each field should be faithfully recorded as one of five raw states: correct, misread, missed, invented, correct blank.
50%UnverifiedPrecision, recall, F1 score, and accuracy can all be derived from the four numbers of the confusion matrix.
50%UnverifiedPrecision is essentially computing, given that a prediction is positive, the posterior probability that it is truly positive.
50%UnverifiedWhen both Prevalence and FPR are derived from the same confusion matrix, the Bayesian formula naturally simplifies to TP/(TP+FP).
50%UnverifiedWhen evaluating a judge, one should look at the confusion matrix rather than just accuracy; if 90% of the samples are good answers to begin with, a judge that passes everything can still show 90% accuracy.
50%UnverifiedThe confusion matrix lists the four combinations—true positives (TP), false positives (FP), false negatives (FN), and true negatives (TN)—in a 2×2 table.
50%UnverifiedIn practice, precision and recall are commonly used to quantify false positives and false negatives, and confusion matrices are used to analyze model performance.
50%UnverifiedIn a binary classification confusion matrix, the F1 score is the harmonic mean of precision and recall.
50%