Claim C14 · methodological · entrylib:2026.000008@v2 · 2026-11-14

Corrected · 14 November 2026 at 12:59 UTC

Version 1 labelled the review's findings and recommendations extension, with an extends relation to the very works cited as their evidence, which said they extend those works; they state what those works establish, so version 2 labels them synthesis (claimset v0.43) and drops those relations, keeping extension only for C16, the one rule that goes beyond its sources, which extends Gneiting and Raftery (2007); and, at the guarantor's request, it rewords C3 (RMSSE as an example of the scaling), C4 (MAE and MASE as the main measure only when the mean and median forecasts coincide), C8 (absolute-error measures avoided on intermittent series), C13 (sharpness compared only among calibrated distributions and intervals) and C24 (choose a measure for what the methods predict, even if precedent used another).

By Ivan Svetunkov (ORCID 0000-0001-7826-0281)

Evaluating forecasts and classifiers: which measures, scoring rules and tests to use · C14

Principal claim C14 of this paper (Svetunkov, I. (guarantor), 2026), with what it rests on. Read the whole paper.

Text file for AI

Text file: the claim with what it rests on as plain text, to attach when your assistant cannot open the address.

The message

Read https://entrylib.org/p/2026.000008/C14, one claim of a research paper with the evidence it rests on. Tell me briefly what it says and how well it is supported. If you cannot open the link, tell me instead of guessing. Then wait for my questions.

The claim

SupportS 74C14 · methodological · novelty: summarises earlier work, by the authors

Predictive distributions of counts, such as forecasts of intermittent sales, should be evaluated with proper scoring rules and a randomised probability integral transform.

Claims it rests on

SupportS 93C36 · methodological · novelty: summarises earlier work, by the authors

The CRPS is a proper scoring rule for predictive distributions of continuous outcomes.

SupportS 93C37 · methodological · novelty: summarises earlier work, by the authors

The log score is a strictly proper scoring rule.

SupportS 78C40 · methodological · novelty: summarises earlier work, by the authors

The calibration of predictive distributions of counts can be assessed with a randomised probability integral transform.

Evidence

W14 · argument · deductive · premises C40, C36, C37

The probability integral transform of a discrete distribution takes few values and is not uniform even when the forecast is calibrated, so the calibration of count forecasts is checked with the randomised transform; proper scoring rules apply to count distributions as to any other.

Limitations

L3 · measurement · reduces precision · affects C29, C30, C31, C32, C33, C34, C35, C36, C37, C38, C39, C44, C46 · mitigation: none

Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions.

L5 · researcher position · could bias magnitude · affects C1, C2, C3, C4, C5, C6, C7, C8, C9, C10, C11, C12, C13, C14, C15, C16, C17, C18, C19, C20, C21, C22, C23, C24, C25, C26, C27, C28, C29, C30, C31, C32, C33, C34, C35, C36, C37, C38, C39, C40, C41, C42, C43, C44, C45, C46, C47, C48, C49, C50, C51, C52, C53 · mitigation: none

The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it.

L6 · data quality · reduces precision · affects C30, C33, C40, C41, C45, C47, C48, C49, C52, C53 · mitigation: none

These claims rest in part on works read in full by the guarantor, not by the agent that wrote the claimset (lab note NT14); no passage of those works is quoted, so a reader checks these claims against the works themselves.

Definitions

Variables and measures 2

D15 CRPS: Continuous ranked probability score: the integral over thresholds of the squared difference between the forecast's cumulative distribution function and the step function at the outcome. Source: Gneiting and Raftery (2007).

D16 log score: The logarithmic score: the negative logarithm of the forecast density (or probability) at the outcome. Source: Gneiting and Raftery (2007).

Concepts 2

D2 proper scoring rule: A scoring rule for probabilistic forecasts is proper when its expected value is best when the forecast distribution equals the distribution of the outcome, and strictly proper when it is best only then. Source: Gneiting and Raftery (2007).

D18 probability integral transform: The value of the forecast's cumulative distribution function at the outcome; for a calibrated forecast its values are uniformly distributed. Source: Gneiting et al. (2007).

References

  1. Gneiting, T., Raftery, A. E., 2007. Strictly Proper Scoring Rules, Prediction, and Estimation. Journal of the American Statistical Association 102, 359–378. https://doi.org/10.1198/016214506000001437 Used by: C36, C37, D2, D15, D16, A2
  2. Gneiting, T., Balabdaoui, F., Raftery, A. E., 2007. Probabilistic Forecasts, Calibration and Sharpness. Journal of the Royal Statistical Society Series B: Statistical Methodology 69, 243–268. https://doi.org/10.1111/j.1467-9868.2007.00587.x Used by: D18, A3
  3. Kolassa, S., 2016. Evaluating predictive count data distributions in retail sales forecasting. International Journal of Forecasting 32, 788–803. https://doi.org/10.1016/j.ijforecast.2015.12.004 Used by: C40

Artefacts

  • A2 · data · Gneiting and Raftery (2007) Strictly proper scoring rules, prediction, and estimation (Gneiting and Raftery 2007, JASA 102(477), 359-378, doi:10.1198/016214506000001437), author's copy (application/pdf), saved by hand on 2026-11-13; recorded in the lab record as SR2 on 2026-11-13. venue: Journal of the American Statistical Association. Written by others. Gneiting2007jasa.pdf
  • A3 · data · Gneiting et al. (2007) Probabilistic forecasts, calibration and sharpness (Gneiting, Balabdaoui and Raftery 2007, JRSS B 69(2), 243-268, doi:10.1111/j.1467-9868.2007.00587.x), author's copy (application/pdf), saved by hand on 2026-11-13; recorded in the lab record as SR3 on 2026-11-13. venue: Journal of the Royal Statistical Society B. Written by others. Gneiting2007jrssb.pdf

Score

Scores are computed by rules from the claimset file (EntryLib rubric 0.7): they check the record, not the science, and may change as the rubric and reviews develop. The rubric.

C14: 74, Partly supported

Novelty: summarises earlier work, by the authors

  1. −15 Evidence Rests on cited work only; cited work read in full, without the passage it rests on; an argument from other claims. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
  2. −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
  3. −3 Plan The review's search for the literature: none is recorded; the claims were confirmed against works the authors knew. To remove it: Record the protocol, then the searches and the screening of what they found (claimset lab protocol, search, screen), and export them.
  4. −1 Open problems Limitation L3 (reduces precision, not mitigated): Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
  5. −1 Open problems Limitation L6 (reduces precision, not mitigated): These claims rest in part on works read in full by the guarantor, not by the agent that wrote the claimset (lab note NT14); no passage of those works is quoted, so a reader checks these claims against the works themselves. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.

C36: 93, Well supported

Novelty: summarises earlier work, by the authors

  1. −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
  2. −1 Open problems Limitation L3 (reduces precision, not mitigated): Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.

C37: 93, Well supported

Novelty: summarises earlier work, by the authors

  1. −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
  2. −1 Open problems Limitation L3 (reduces precision, not mitigated): Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.

C40: 78, Partly supported

Novelty: summarises earlier work, by the authors

  1. −15 Evidence Rests on cited work only; cited work read in full, without the passage it rests on; an argument from other claims. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
  2. −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
  3. −1 Open problems Limitation L6 (reduces precision, not mitigated): These claims rest in part on works read in full by the guarantor, not by the agent that wrote the claimset (lab note NT14); no passage of those works is quoted, so a reader checks these claims against the works themselves. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.