CLAIM C13 of entrylib:2026.000008 Evaluating forecasts and classifiers: which measures, scoring rules and tests to use licence CC-BY-4.0; contributions C13 extensions review 0.1 DEFINITIONS D2 [concept] proper scoring rule: A scoring rule for probabilistic forecasts is proper when its expected value is best when the forecast distribution equals the distribution of the outcome, and strictly proper when it is best only then. [source B3] D13 [variable] interval score: The score of a central (1 - alpha) prediction interval: its width plus 2/alpha times the distance by which the outcome falls outside it. [source B3] D19 [concept] calibration: Statistical consistency between forecast distributions and outcomes: events forecast with a given probability occur with that frequency. [source B4] D20 [concept] sharpness: The concentration of the forecast distributions, a property of the forecasts alone; among calibrated forecasts, sharper ones are better. [source B4] CLAIMS C13 [methodological; principal; synthesis; support: argument W13; uses C39, C35; limitations L5; strength high (author); verifiability open; planning exploratory; outcome supported] [review: kind recommendation] Predictive distributions and prediction intervals should first be checked for [[D19:calibration]], and their [[D20:sharpness]] compared only among forecasts that are calibrated. scope: data_type: predictive distributions and prediction intervals C35 [methodological; supporting; synthesis; support: cited B3, at Section 1 (Introduction), cited B3, at Abstract; limitations L3, L5; verifiability open; planning exploratory; outcome supported] [review: extracts [{"reference": "B3", "locator": "Section 1 (Introduction)", "lab_event": "XT8", "exact": "a scoring rule for prediction intervals that is proper", "prefix": " the general idea dates back at\n", "suffix": " and has in- least to Brier ", "start": 15726, "end": 15780, "artefact": "A2"}, {"reference": "B3", "locator": "Abstract", "lab_event": "XT9", "exact": "propose the intuitively appealing interval score as a utility function in interval estimation that addresses width as well as\n coverage", "prefix": "tile\n estimation, and ", "suffix": ".\n KEY WORDS: Bayes fa", "start": 2499, "end": 2644, "artefact": "A2"}]; kind summary] The [[D13:interval score]] is a [[D2:proper scoring rule]] for central prediction intervals that addresses both their width and their coverage. scope: data_type: central prediction intervals C39 [methodological; supporting; synthesis; support: cited B4, at Abstract; limitations L3, L5; verifiability open; planning exploratory; outcome supported] [review: extracts [{"reference": "B4", "locator": "Abstract", "lab_event": "XT13", "exact": "is based on the paradigm of maximizing the sharpness of\n the predictive distributions subject to calibration", "prefix": " of predictive performance that ", "suffix": ". Calibration refers to the stat", "start": 635, "end": 748, "artefact": "A3"}]; kind summary] Predictive distributions are assessed by maximising their [[D20:sharpness]] subject to their [[D19:calibration]]. scope: data_type: predictive distributions WARRANTS W13 [deductive; premises C39, C35] Predictive distributions are assessed by maximising their sharpness subject to their calibration: a sharper forecast that is not calibrated is overconfident, so sharpness tells which of two forecasts is better only when both are calibrated. A prediction interval summarises a predictive distribution, and the interval score weighs its width against its misses in the same way. LIMITATIONS L3 [measurement; affects C29, C30, C31, C32, C33, C34, C35, C36, C37, C38, C39, C44, C46; reduces_precision; mitigation none] Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions. L5 [researcher_position; affects C1, C2, C3, C4, C5, C6, C7, C8, C9, C10, C11, C12, C13, C14, C15, C16, C17, C18, C19, C20, C21, C22, C23, C24, C25, C26, C27, C28, C29, C30, C31, C32, C33, C34, C35, C36, C37, C38, C39, C40, C41, C42, C43, C44, C45, C46, C47, C48, C49, C50, C51, C52, C53; could_bias_magnitude; mitigation none] The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. ARTEFACTS A2 [data] Strictly proper scoring rules, prediction, and estimation (Gneiting and Raftery 2007, JASA 102(477), 359-378, doi:10.1198/016214506000001437), author's copy (application/pdf), saved by hand on 2026-11-13; recorded in the lab record as SR2 on 2026-11-13. venue: Journal of the American Statistical Association. Written by others. A3 [data] Probabilistic forecasts, calibration and sharpness (Gneiting, Balabdaoui and Raftery 2007, JRSS B 69(2), 243-268, doi:10.1111/j.1467-9868.2007.00587.x), author's copy (application/pdf), saved by hand on 2026-11-13; recorded in the lab record as SR3 on 2026-11-13. venue: Journal of the Royal Statistical Society B. Written by others. REFERENCES B3 10.1198/016214506000001437 (doi) B4 10.1111/j.1467-9868.2007.00587.x (doi) HOW TO CITE This claim: Svetunkov, I. (guarantor) (2026). Claim C13, in: Evaluating forecasts and classifiers: which measures, scoring rules and tests to use. EntryLib. entrylib:2026.000008@v2#C13. https://entrylib.org/p/2026.000008/C13 The claimset it is from: Svetunkov, I. (guarantor) (2026). Evaluating forecasts and classifiers: which measures, scoring rules and tests to use. EntryLib. entrylib:2026.000008@v2. https://entrylib.org/p/2026.000008@v2