CLAIM C15 of entrylib:2026.000008 Evaluating forecasts and classifiers: which measures, scoring rules and tests to use licence CC-BY-4.0; contributions C15 extensions review 0.1 DEFINITIONS D2 [concept] proper scoring rule: A scoring rule for probabilistic forecasts is proper when its expected value is best when the forecast distribution equals the distribution of the outcome, and strictly proper when it is best only then. [source B3] D16 [variable] log score: The logarithmic score: the negative logarithm of the forecast density (or probability) at the outcome. [source B3] D17 [variable] Brier score: The mean squared difference between forecast probabilities of a class and the outcome coded 1 if the class occurred and 0 otherwise. [source B13] CLAIMS C15 [methodological; principal; synthesis; support: argument W15; uses C38, C37; limitations L5; strength high (author); verifiability open; planning exploratory; outcome supported] [review: kind recommendation] Forecast probabilities of classes should be evaluated with a [[D2:proper scoring rule]], such as the [[D16:log score]] or the [[D17:Brier score]]. scope: data_type: forecast probabilities of classes C37 [methodological; supporting; synthesis; support: cited B3, at Section 4.1; limitations L3, L5; verifiability open; planning exploratory; outcome supported] [review: extracts [{"reference": "B3", "locator": "Section 4.1", "lab_event": "XT11", "exact": "The logarithmic score is strictly proper", "prefix": "rithmic score (Ta- 2002). ", "suffix": " relative to the\nble 1), the asy", "start": 53590, "end": 53630, "artefact": "A2"}]; kind summary] The [[D16:log score]] is a strictly [[D2:proper scoring rule]]. scope: data_type: predictive distributions C38 [methodological; supporting; synthesis; support: cited B3, at Section 3, Example 1 (Quadratic or Brier score), cited B13; limitations L2, L3, L5; verifiability open; planning exploratory; outcome supported] [review: extracts [{"reference": "B3", "locator": "Section 3, Example 1 (Quadratic or Brier score)", "lab_event": "XT12", "exact": "Example 1 (Quadratic or Brier score)", "prefix": " \u0005\n ", "suffix": ". If G(p) = m j=1 pj −\n ", "start": 33270, "end": 33306, "artefact": "A2"}]; kind summary] The [[D17:Brier score]] is a [[D2:proper scoring rule]] for forecast probabilities of classes. scope: data_type: forecast probabilities of classes WARRANTS W15 [deductive; premises C38, C37] The Brier score is proper and the log score strictly proper, so both are best in expectation for the true probabilities of the classes. LIMITATIONS L2 [data_quality; affects C38; reduces_precision; mitigation none] Part of the support of these claims comes from a work checked only by its abstract (Brier 1950, lab note NT7); the rest comes from works read in full. L3 [measurement; affects C29, C30, C31, C32, C33, C34, C35, C36, C37, C38, C39, C44, C46; reduces_precision; mitigation none] Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions. L5 [researcher_position; affects C1, C2, C3, C4, C5, C6, C7, C8, C9, C10, C11, C12, C13, C14, C15, C16, C17, C18, C19, C20, C21, C22, C23, C24, C25, C26, C27, C28, C29, C30, C31, C32, C33, C34, C35, C36, C37, C38, C39, C40, C41, C42, C43, C44, C45, C46, C47, C48, C49, C50, C51, C52, C53; could_bias_magnitude; mitigation none] The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. ARTEFACTS A2 [data] Strictly proper scoring rules, prediction, and estimation (Gneiting and Raftery 2007, JASA 102(477), 359-378, doi:10.1198/016214506000001437), author's copy (application/pdf), saved by hand on 2026-11-13; recorded in the lab record as SR2 on 2026-11-13. venue: Journal of the American Statistical Association. Written by others. REFERENCES B3 10.1198/016214506000001437 (doi) B13 10.1175/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2 (doi) HOW TO CITE This claim: Svetunkov, I. (guarantor) (2026). Claim C15, in: Evaluating forecasts and classifiers: which measures, scoring rules and tests to use. EntryLib. entrylib:2026.000008@v2#C15. https://entrylib.org/p/2026.000008/C15 The claimset it is from: Svetunkov, I. (guarantor) (2026). Evaluating forecasts and classifiers: which measures, scoring rules and tests to use. EntryLib. entrylib:2026.000008@v2. https://entrylib.org/p/2026.000008@v2