ROC-AUC
A standard evaluation metric that measures the overall discriminative capability of a binary classification model across different decision thresholds. Values range from 0 to 1, where 0.5 indicates random guessing and 1.0 represents perfect classification. It is widely used for assessing machine learning model performance.
Core Facts
The core advantage of AUC-ROC is its insensitivity to class imbalance; it reflects the model's overall discriminative ability across different classification thresholds, with AUC values ranging from 0.5 to 1.0.
An AUC=1 on the ROC curve indicates a perfect classifier, while AUC=0.5 is equivalent to random guessing.
Timeline (last 90 days)
A linear probe predicting pose failure on VGGT's frozen features achieved an ROC-AUC of 93.5% on unseen categories.
Data Autopsy's target leakage check looks for several suspicious signals, including features identical to the target, very strong numeric relationships, unusually strong categorical class purity, and, where applicable, high ROC-AUC scores, but these signals alone cannot prove leakage.
On synthetic data with 10,000 samples, a 1% positive rate, and 495 negatives scoring 0.90 while the positives score 0.80 and 0.60 respectively, computing with a 0.5 threshold gives: ROC-AUC=0.9500, Average Precision=0.1449, precision=0.1681, recall=1.0000, accuracy=0.9505, TP=100, FP=495, TN=9405, FN=0.
Calibration and discrimination are two independent metrics: discrimination measures the model's ability to rank positives ahead of negatives (such as ROC-AUC), while calibration measures whether the model's output probabilities match true frequencies; commonly used calibration measures include Expected Calibration Error (ECE) and reliability diagrams, and correction methods include Platt scaling, isotonic regression and temperature scaling.
ReachNT's urgency model achieves a ROC-AUC of 0.84 on unseen phrasing, and as part of an overall system that includes human review it catches 99.3% of dangerous work orders.
A learner, when detecting forged receipts on this dataset, found that regardless of how the model was adjusted, the ROC-AUC consistently hovered between 0.46 and 0.54, close to random guessing.
The approach using frozen visual features yielded a ROC-AUC of only 0.482, meaning the model essentially learned nothing.
All Facts (9)
The core advantage of AUC-ROC is its insensitivity to class imbalance; it reflects the model's overall discriminative ability across different classification thresholds, with AUC values ranging from 0.5 to 1.0.
75%VerifiedAn AUC=1 on the ROC curve indicates a perfect classifier, while AUC=0.5 is equivalent to random guessing.
70%UnverifiedA linear probe predicting pose failure on VGGT's frozen features achieved an ROC-AUC of 93.5% on unseen categories.
50%UnverifiedData Autopsy's target leakage check looks for several suspicious signals, including features identical to the target, very strong numeric relationships, unusually strong categorical class purity, and, where applicable, high ROC-AUC scores, but these signals alone cannot prove leakage.
50%UnverifiedOn synthetic data with 10,000 samples, a 1% positive rate, and 495 negatives scoring 0.90 while the positives score 0.80 and 0.60 respectively, computing with a 0.5 threshold gives: ROC-AUC=0.9500, Average Precision=0.1449, precision=0.1681, recall=1.0000, accuracy=0.9505, TP=100, FP=495, TN=9405, FN=0.
50%UnverifiedReachNT's urgency model achieves a ROC-AUC of 0.84 on unseen phrasing, and as part of an overall system that includes human review it catches 99.3% of dangerous work orders.
50%UnverifiedCalibration and discrimination are two independent metrics: discrimination measures the model's ability to rank positives ahead of negatives (such as ROC-AUC), while calibration measures whether the model's output probabilities match true frequencies; commonly used calibration measures include Expected Calibration Error (ECE) and reliability diagrams, and correction methods include Platt scaling, isotonic regression and temperature scaling.
50%UnverifiedThe approach using frozen visual features yielded a ROC-AUC of only 0.482, meaning the model essentially learned nothing.
50%UnverifiedA learner, when detecting forged receipts on this dataset, found that regardless of how the model was adjusted, the ROC-AUC consistently hovered between 0.46 and 0.54, close to random guessing.
50%