Correctness AUROC
A threshold-independent metric for evaluating the quality of model uncertainty. It measures the model's ability to distinguish correct predictions from incorrect ones, with values ranging from 0.5 to 1.0.
Timeline (last 90 days)
The GPT-4o-mini few-shot configuration had only one confidence value before and after calibration, and its correctness AUROC was only 0.500, indicating that the model has almost no ability to distinguish correct from incorrect predictions.
Correctness AUROC measures the ability of a model to distinguish "correct prediction" from "incorrect prediction", with a value of 0.5 indicating no discriminative power and 1.0 indicating perfect discrimination; it is a threshold-independent metric.
All Facts (2)
The GPT-4o-mini few-shot configuration had only one confidence value before and after calibration, and its correctness AUROC was only 0.500, indicating that the model has almost no ability to distinguish correct from incorrect predictions.
50%UnverifiedCorrectness AUROC measures the ability of a model to distinguish "correct prediction" from "incorrect prediction", with a value of 0.5 indicating no discriminative power and 1.0 indicating perfect discrimination; it is a threshold-independent metric.
50%