CLAIM C17 of entrylib:2026.000008 Evaluating forecasts and classifiers: which measures, scoring rules and tests to use licence CC-BY-4.0; contributions C17 extensions review 0.1 DEFINITIONS D22 [variable] AUC: The area under the ROC curve: the probability that a randomly chosen case of the positive class receives a higher score than a randomly chosen case of the negative class. [source B14] CLAIMS C17 [methodological; principal; synthesis; support: argument W17; uses C42; limitations L1, L5; strength moderate (author); verifiability open; planning exploratory; outcome supported] [review: kind recommendation] [[D22:AUC]] should not be the measure of forecast class probabilities. scope: data_type: forecast class probabilities, ranked by a classifier's scores C42 [methodological; supporting; synthesis; support: cited B14; limitations L1, L5; verifiability open; planning exploratory; outcome supported] [review: kind summary] The [[D22:AUC]] is equivalent to averaging the misclassification loss over a distribution of misclassification costs that depends on the classifier. scope: data_type: classifiers that score cases WARRANTS W17 [deductive; premises C42] The AUC averages the misclassification loss over a cost distribution that depends on the classifier, so different classifiers are compared with different weightings of the costs of errors; and it depends only on how the scores rank cases, not on whether the probabilities are right. LIMITATIONS L1 [data_quality; affects C42, C17; could_reverse; mitigation none] These claims rest only on works checked by their abstracts and bibliographic records (Brier 1950, Hand 2009: lab notes NT7 and NT8); a full text could qualify what the abstract states. L5 [researcher_position; affects C1, C2, C3, C4, C5, C6, C7, C8, C9, C10, C11, C12, C13, C14, C15, C16, C17, C18, C19, C20, C21, C22, C23, C24, C25, C26, C27, C28, C29, C30, C31, C32, C33, C34, C35, C36, C37, C38, C39, C40, C41, C42, C43, C44, C45, C46, C47, C48, C49, C50, C51, C52, C53; could_bias_magnitude; mitigation none] The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. REFERENCES B14 10.1007/s10994-009-5119-5 (doi) HOW TO CITE This claim: Svetunkov, I. (guarantor) (2026). Claim C17, in: Evaluating forecasts and classifiers: which measures, scoring rules and tests to use. EntryLib. entrylib:2026.000008@v2#C17. https://entrylib.org/p/2026.000008/C17 The claimset it is from: Svetunkov, I. (guarantor) (2026). Evaluating forecasts and classifiers: which measures, scoring rules and tests to use. EntryLib. entrylib:2026.000008@v2. https://entrylib.org/p/2026.000008@v2