The claim
Claims it rests on
The interval score is a proper scoring rule for central prediction intervals that addresses both their width and their coverage.
Scope: data: central prediction intervals · Cited: Gneiting and Raftery (2007), Section 1 (Introduction) · Cited: Gneiting and Raftery (2007), Abstract · Limitations: L3, L5 · Used by: C28
Scale-dependent error measures, such as RMSE and MAE, cannot be compared across series on different scales.
Scope: data: error measures over series on different scales · Cited: Hyndman and Koehler (2006), Section 2.1 (Scale-dependent measures) · Cited: Hyndman and Koehler (2006), Section 2.4 (Relative measures) · Limitations: L3, L5 · Used by: C28
The M4 competition scored prediction intervals with the mean scaled interval score: the interval score of each series scaled by an in-sample error of that series.
Scope: data: prediction intervals in the M4 competition · Cited: Makridakis et al. (2020) · Limitations: L5, L6 · Used by: C28
Evidence
Interval scores of series on different scales cannot be compared unscaled; dividing each series' interval score by a positive quantity of that series that does not depend on the forecasts keeps it proper, as the mean scaled interval score of the M4 competition does.
Limitations
Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions.
The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it.
These claims rest in part on works read in full by the guarantor, not by the agent that wrote the claimset (lab note NT14); no passage of those works is quoted, so a reader checks these claims against the works themselves.
Definitions
Variables and measures 3
D5 RMSE: Root mean squared error: the square root of the mean of the squared differences between outcomes and point forecasts. Source: Hyndman and Koehler (2006).
D7 MAE: Mean absolute error: the mean of the absolute differences between outcomes and point forecasts. Source: Hyndman and Koehler (2006).
D13 interval score: The score of a central (1 - alpha) prediction interval: its width plus 2/alpha times the distance by which the outcome falls outside it. Source: Gneiting and Raftery (2007).
Concepts 1
D2 proper scoring rule: A scoring rule for probabilistic forecasts is proper when its expected value is best when the forecast distribution equals the distribution of the outcome, and strictly proper when it is best only then. Source: Gneiting and Raftery (2007).
References
- Gneiting, T., Raftery, A. E., 2007. Strictly Proper Scoring Rules, Prediction, and Estimation. Journal of the American Statistical Association 102, 359–378. https://doi.org/10.1198/016214506000001437 Used by: C35, D2, D13, A2
- Hyndman, R. J., Koehler, A. B., 2006. Another look at measures of forecast accuracy. International Journal of Forecasting 22, 679–688. https://doi.org/10.1016/j.ijforecast.2006.03.001 Used by: C44, D5, D7, A7
- Makridakis, S., Spiliotis, E., Assimakopoulos, V., 2020. The M4 Competition: 100,000 time series and 61 forecasting methods. International Journal of Forecasting 36, 54–74. https://doi.org/10.1016/j.ijforecast.2019.04.014 Used by: C49
Artefacts
- A2 · data · Gneiting and Raftery (2007) Strictly proper scoring rules, prediction, and estimation (Gneiting and Raftery 2007, JASA 102(477), 359-378, doi:10.1198/016214506000001437), author's copy (application/pdf), saved by hand on 2026-11-13; recorded in the lab record as SR2 on 2026-11-13. venue: Journal of the American Statistical Association. Written by others. Gneiting2007jasa.pdf
- A7 · data · Hyndman and Koehler (2006) Another look at measures of forecast accuracy (preprint of Hyndman and Koehler 2006, IJF 22(4), 679-688, doi:10.1016/j.ijforecast.2006.03.001) (application/pdf), saved by hand on 2026-11-13; recorded in the lab record as SR7 on 2026-11-13. venue: International Journal of Forecasting. Written by others. mase.pdf
Score
Scores are computed by rules from the claimset file (EntryLib rubric 0.7): they check the record, not the science, and may change as the rubric and reviews develop. The rubric.
C28: 74, Partly supported
Novelty: summarises earlier work, by the authors
- −15 Evidence Rests on cited work only; cited work read in full, without the passage it rests on; an argument from other claims. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −3 Plan The review's search for the literature: none is recorded; the claims were confirmed against works the authors knew. To remove it: Record the protocol, then the searches and the screening of what they found (claimset lab protocol, search, screen), and export them.
- −1 Open problems Limitation L3 (reduces precision, not mitigated): Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L6 (reduces precision, not mitigated): These claims rest in part on works read in full by the guarantor, not by the agent that wrote the claimset (lab note NT14); no passage of those works is quoted, so a reader checks these claims against the works themselves. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C35: 93, Well supported
Novelty: summarises earlier work, by the authors
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L3 (reduces precision, not mitigated): Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C44: 93, Well supported
Novelty: summarises earlier work, by the authors
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L3 (reduces precision, not mitigated): Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C49: 78, Partly supported
Novelty: summarises earlier work, by the authors
- −15 Evidence Rests on cited work only; cited work read in full, without the passage it rests on; an argument from other claims. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L6 (reduces precision, not mitigated): These claims rest in part on works read in full by the guarantor, not by the agent that wrote the claimset (lab note NT14); no passage of those works is quoted, so a reader checks these claims against the works themselves. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.