Summary
How should a study measure and compare predictions? This review states what the methodological literature establishes and the rules it draws from it. A measure has to match what a prediction targets: squared error (RMSE) for forecasts of the mean, absolute error (MAE) for the median, the pinball loss for a quantile, the interval score with coverage for intervals, CRPS or the log score for distributions, and log loss or the Brier score for class probabilities. MAPE rewards forecasts that are too low, sMAPE prefers forecasts that are too high, and AUC and accuracy do not judge probabilities. Errors of series on different scales should be scaled per series before they are averaged, never summed as they are. The mean, median and quartiles of a scaled measure summarise accuracy over many series; rank-based tests (Friedman with Nemenyi, MCB) and the Diebold-Mariano test tell whether similar methods differ statistically.
Highlights
- SupportS 79Point forecasts should be evaluated with a consistent scoring function for the functional they target, such as the mean, the median or a quantile. C1
- SupportS 79To aggregate the accuracy of point forecasts over series on different scales, each series' errors should first be scaled by a quantity of that series that does not depend on the compared forecasts, such as the in-sample error of its one-step naive forecast in MASE or RMSSE, before the scaled errors are averaged. C26
- SupportS 80MAPE should not be the measure for point forecasts of the mean or the median. C5
- SupportS 80Predictive distributions should be evaluated with a proper scoring rule, such as the CRPS or the log score. C12
- SupportS 79Forecast probabilities of classes should be evaluated with a proper scoring rule, such as the log score or the Brier score. C15
Context
- Extends Gneiting and Raftery (2007) · from C16
- Gneiting and Raftery (2007) show that the Brier and logarithmic scores are proper for forecast probabilities of classes.
Definitions
Methods 4
D25 Friedman test: A non-parametric test of whether several methods differ, applied to their ranks on each of several data sets. Source: https://jmlr.org/papers/v7/demsar06a.html.
D26 Nemenyi test: A post-hoc test after the Friedman test that compares every pair of methods by their mean ranks against a critical difference. Source: https://jmlr.org/papers/v7/demsar06a.html.
D27 MCB: Multiple comparisons with the best: a test, on mean ranks over series, of which methods differ significantly from the best-ranked one. Source: Koning et al. (2005).
D28 Diebold-Mariano test: A test of equal expected loss of two competing forecasts of the same series, based on the mean loss differential and its long-run variance. Source: Diebold and Mariano (1995).
Data 1
D24 intermittent series: A time series with many zero values, such as the sales of a slow-moving item. Source: Kolassa (2016).
Variables and measures 14
D5 RMSE: Root mean squared error: the square root of the mean of the squared differences between outcomes and point forecasts. Source: Hyndman and Koehler (2006).
D6 RMSSE: Root mean squared scaled error: the RMSE of a series' forecasts divided by the in-sample RMSE of the one-step naive forecast of that series, so that series on different scales can be averaged. Source: Makridakis et al. (2022a).
D7 MAE: Mean absolute error: the mean of the absolute differences between outcomes and point forecasts. Source: Hyndman and Koehler (2006).
D8 MASE: Mean absolute scaled error: the MAE of a series' forecasts divided by the in-sample MAE of the one-step naive forecast of that series. Source: Hyndman and Koehler (2006).
D9 MAPE: Mean absolute percentage error: the mean of the absolute differences between outcomes and point forecasts, each divided by the outcome. Source: Hyndman and Koehler (2006).
D10 sMAPE: Symmetric mean absolute percentage error: the mean of the absolute differences between outcomes and point forecasts, each divided by the mean of the absolute outcome and the absolute forecast. Source: Goodwin and Lawton (1999).
D12 pinball loss: The quantile (asymmetric piecewise linear) loss at level tau: tau times the error when the outcome is above the forecast, and (1 - tau) times the absolute error when it is below. Source: Gneiting (2011).
D13 interval score: The score of a central (1 - alpha) prediction interval: its width plus 2/alpha times the distance by which the outcome falls outside it. Source: Gneiting and Raftery (2007).
D14 coverage: The share of outcomes that fall inside the prediction intervals, compared with their nominal level. Source: Gneiting and Raftery (2007).
D15 CRPS: Continuous ranked probability score: the integral over thresholds of the squared difference between the forecast's cumulative distribution function and the step function at the outcome. Source: Gneiting and Raftery (2007).
D16 log score: The logarithmic score: the negative logarithm of the forecast density (or probability) at the outcome. Source: Gneiting and Raftery (2007).
D17 Brier score: The mean squared difference between forecast probabilities of a class and the outcome coded 1 if the class occurred and 0 otherwise. Source: BRIER (1950).
D21 accuracy: The share of cases whose predicted class, after thresholding the predicted probabilities, equals the observed class.
D22 AUC: The area under the ROC curve: the probability that a randomly chosen case of the positive class receives a higher score than a randomly chosen case of the negative class. Source: Hand (2009).
Concepts 9
D1 consistent scoring function: A scoring function (an error measure for point forecasts) is consistent for a functional of the predictive distribution, such as the mean, the median or a quantile, when its expected value is lowest when the point forecast equals that functional. Source: Gneiting (2011).
D2 proper scoring rule: A scoring rule for probabilistic forecasts is proper when its expected value is best when the forecast distribution equals the distribution of the outcome, and strictly proper when it is best only then. Source: Gneiting and Raftery (2007).
D3 point forecast of the mean: A single-valued forecast meant to equal the mean (expected value) of the outcome's distribution given the information available. Source: Gneiting (2011).
D4 point forecast of the median: A single-valued forecast meant to equal the median of the outcome's distribution given the information available. Source: Gneiting (2011).
D11 median of order -1: The beta-median with beta = -1: the point forecast that minimises the expected absolute percentage error; for positive outcomes it lies below the median. Source: Gneiting (2011).
D18 probability integral transform: The value of the forecast's cumulative distribution function at the outcome; for a calibrated forecast its values are uniformly distributed. Source: Gneiting et al. (2007).
D19 calibration: Statistical consistency between forecast distributions and outcomes: events forecast with a given probability occur with that frequency. Source: Gneiting et al. (2007).
D20 sharpness: The concentration of the forecast distributions, a property of the forecasts alone; among calibrated forecasts, sharper ones are better. Source: Gneiting et al. (2007).
D23 precision-recall curve: The curve of precision (the share of predicted positives that are positive) against recall (the share of positives predicted as positive) over the classification thresholds. Source: Saito and Rehmsmeier (2015).
Principal claims
Point forecasts should be evaluated with a consistent scoring function for the functional they target, such as the mean, the median or a quantile.
Scope: data: point forecasts · Argument: W1 · Uses: C30 · Limitations: L5
Point forecasts of the mean should be evaluated with squared error, as in RMSE.
Scope: data: point forecasts of the mean · Argument: W2 · Uses: C29, C30 · Limitations: L5
When point forecasts of the mean are compared over series on different scales, each series' RMSE should be scaled by the in-sample RMSE of its one-step naive forecast, for example, as in RMSSE.
Scope: data: point forecasts of the mean, over series on different scales · Argument: W3 · Uses: C47, C29, C46 · Limitations: L5
MAE and MASE can be used as the main measure of point forecasts only when the mean and median forecasts coincide.
Scope: data: point forecasts of the mean · Argument: W4 · Uses: C31, C29 · Limitations: L5
MAPE should not be the measure for point forecasts of the mean or the median.
Scope: data: point forecasts of positive outcomes · Argument: W5 · Uses: C32 · Limitations: L5
sMAPE should not be the measure for point forecasts.
Scope: data: point forecasts · Argument: W6 · Uses: C33 · Limitations: L5
Point forecasts of the median should be evaluated with absolute error, as in MAE or, across series on different scales, MASE.
Scope: data: point forecasts of the median · Argument: W7 · Uses: C31, C46 · Limitations: L5
On intermittent series, absolute-error measures should be avoided.
Scope: data: point forecasts of intermittent series · Argument: W8 · Uses: C41, C31 · Limitations: L5
Quantile forecasts should be evaluated with the pinball loss at the quantile's level, averaged over the levels when there are several.
Scope: data: quantile forecasts · Argument: W9 · Uses: C34 · Limitations: L5
Prediction intervals should be evaluated with the interval score, reported together with their coverage against the nominal level.
Scope: data: central prediction intervals · Argument: W10 · Uses: C35 · Limitations: L5
Coverage alone should not be the measure of prediction intervals.
Scope: data: prediction intervals · Argument: W11 · Uses: C35 · Limitations: L5
Predictive distributions should be evaluated with a proper scoring rule, such as the CRPS or the log score.
Scope: data: predictive distributions · Argument: W12 · Uses: C36, C37 · Limitations: L5
Predictive distributions and prediction intervals should first be checked for calibration, and their sharpness compared only among forecasts that are calibrated.
Scope: data: predictive distributions and prediction intervals · Argument: W13 · Uses: C39, C35 · Limitations: L5
Predictive distributions of counts, such as forecasts of intermittent sales, should be evaluated with proper scoring rules and a randomised probability integral transform.
Scope: data: predictive distributions of counts · Argument: W14 · Uses: C40, C36, C37 · Limitations: L5
Forecast probabilities of classes should be evaluated with a proper scoring rule, such as the log score or the Brier score.
Scope: data: forecast probabilities of classes · Argument: W15 · Uses: C38, C37 · Limitations: L5
Accuracy should not be the measure of forecast class probabilities.
Scope: data: forecast class probabilities, thresholded to labels · Argument: W16 · Uses: C38, C37 · Limitations: L4, L5 · Extends: Gneiting and Raftery (2007)
AUC should not be the measure of forecast class probabilities.
Scope: data: forecast class probabilities, ranked by a classifier's scores · Argument: W17 · Uses: C42 · Limitations: L1, L5
With rare positive cases, the precision-recall curve should be reported rather than the ROC curve alone.
Scope: data: binary classifiers; conditions: imbalanced classes · Argument: W18 · Uses: C43 · Limitations: L5
When the compared methods target different functionals, one the mean and another the median, each should be evaluated with a measure consistent for its own target, or the methods made to target the same functional.
Scope: data: point forecasts of methods that target different functionals · Argument: W19 · Uses: C30 · Limitations: L5
When methods are compared over many series, their accuracy should be summarised by the mean, the median and the quartiles of a scaled measure across the series, such as RMSSE or MASE.
Scope: data: comparisons of methods over many series · Argument: W20 · Uses: C46, C50 · Limitations: L5
To tell whether several methods whose aggregate accuracy is similar differ statistically significantly over many series or data sets, their ranks on each series should be tested with the Friedman test followed by Nemenyi test post-hoc comparisons.
Scope: data: comparisons of several methods over many series or data sets · Argument: W21 · Uses: C51 · Limitations: L5
To tell which methods differ statistically significantly from the best one over many series, multiple comparisons with the best (MCB) on ranks should be used.
Scope: data: comparisons of methods over many series · Argument: W22 · Uses: C52 · Limitations: L5
The accuracy of two forecasts of the same series should be compared with the Diebold-Mariano test.
Scope: data: two forecasts of one series · Argument: W23 · Uses: C53 · Limitations: L5
An error measure should be chosen to match what the compared methods predict, even if competitions or well-known studies used a different one.
Scope: data: evaluations of point forecasts · Argument: W24 · Uses: C30 · Limitations: L5
Scale-dependent error measures, such as RMSE and MAE, should not be summed or averaged across series on different scales.
Scope: data: point forecasts over series on different scales · Argument: W25 · Uses: C44, C45 · Limitations: L5
To aggregate the accuracy of point forecasts over series on different scales, each series' errors should first be scaled by a quantity of that series that does not depend on the compared forecasts, such as the in-sample error of its one-step naive forecast in MASE or RMSSE, before the scaled errors are averaged.
Scope: data: point forecasts over series on different scales · Argument: W26 · Uses: C44, C46, C47 · Limitations: L5
Supporting claims
Squared error is a consistent scoring function for the mean.
Scope: data: point forecasts · Cited: Gneiting (2011), Section 5 (Discussion); Theorem 3.1 (arXiv:0912.0902 numbering) · Limitations: L3, L5 · Used by: C2, C3, C4
Which point forecast has the lowest expected score depends on the scoring function: a consistent scoring function for a functional, such as the mean or a quantile, is lowest in expectation for the forecast that equals that functional of the outcome's distribution.
Scope: data: point forecasts · Cited: Gneiting (2011), Abstract (arXiv:0912.0902 numbering) · Cited: Kolassa (2020) · Limitations: L3, L5, L6 · Used by: C1, C2, C19, C24
Absolute error is a consistent scoring function for the median.
Scope: data: point forecasts · Cited: Gneiting (2011), Section 5 (Discussion) (arXiv:0912.0902 numbering) · Limitations: L3, L5 · Used by: C4, C7, C8
The absolute percentage error averaged by MAPE is consistent for the median of order -1, a functional that supports severe underforecasts compared with the mean or the median.
Scope: data: point forecasts of positive outcomes · Cited: Gneiting (2011), Section 5 (Discussion); the beta-median in Section 1.3 (arXiv:0912.0902 numbering) · Cited: Gneiting (2011), Section 5 (Discussion) (arXiv:0912.0902 numbering) · Limitations: L3, L5 · Used by: C5
For the same outcome, sMAPE penalises a forecast below the outcome more heavily than a forecast above it.
Scope: data: point forecasts of positive outcomes · Cited: Hyndman and Koehler (2006), Section 2.2 (Measures based on percentage errors) · Cited: Goodwin and Lawton (1999) · Limitations: L3, L5, L6 · Used by: C6
The pinball loss at a level is a consistent scoring function for the quantile at that level.
Scope: data: quantile forecasts · Cited: Gneiting (2011), Abstract; Theorem 3.3 (arXiv:0912.0902 numbering) · Limitations: L3, L5 · Used by: C9, C27
The interval score is a proper scoring rule for central prediction intervals that addresses both their width and their coverage.
Scope: data: central prediction intervals · Cited: Gneiting and Raftery (2007), Section 1 (Introduction) · Cited: Gneiting and Raftery (2007), Abstract · Limitations: L3, L5 · Used by: C10, C11, C13, C28
The CRPS is a proper scoring rule for predictive distributions of continuous outcomes.
Scope: data: predictive distributions of continuous outcomes · Cited: Gneiting and Raftery (2007), Section 4.2 (Continuous ranked probability score) · Limitations: L3, L5 · Used by: C12, C14
The log score is a strictly proper scoring rule.
Scope: data: predictive distributions · Cited: Gneiting and Raftery (2007), Section 4.1 · Limitations: L3, L5 · Used by: C12, C14, C15, C16
The Brier score is a proper scoring rule for forecast probabilities of classes.
Scope: data: forecast probabilities of classes · Cited: Gneiting and Raftery (2007), Section 3, Example 1 (Quadratic or Brier score) · Cited: BRIER (1950) · Limitations: L2, L3, L5 · Used by: C15, C16
Predictive distributions are assessed by maximising their sharpness subject to their calibration.
Scope: data: predictive distributions · Cited: Gneiting et al. (2007), Abstract · Limitations: L3, L5 · Used by: C13
The calibration of predictive distributions of counts can be assessed with a randomised probability integral transform.
Scope: data: predictive distributions of counts · Cited: Kolassa (2016) · Limitations: L5, L6 · Used by: C14
On intermittent series, absolute-error measures give their best scores to forecasts close to zero.
Scope: data: point forecasts of intermittent series · Cited: https://github.com/Mcompetitions/M5-methods/blob/master/M5-Competitors-Guide.pdf, Point forecasts · Cited: Kolassa (2016) · Limitations: L5, L6 · Used by: C8
The AUC is equivalent to averaging the misclassification loss over a distribution of misclassification costs that depends on the classifier.
Scope: data: classifiers that score cases · Cited: Hand (2009) · Limitations: L1, L5 · Used by: C17
On imbalanced data, the precision-recall curve is more informative than the ROC curve for evaluating binary classifiers.
Scope: data: binary classifiers on imbalanced data · Cited: Saito and Rehmsmeier (2015), Title · Cited: Saito and Rehmsmeier (2015), Abstract · Limitations: L5 · Used by: C18
Scale-dependent error measures, such as RMSE and MAE, cannot be compared across series on different scales.
Scope: data: error measures over series on different scales · Cited: Hyndman and Koehler (2006), Section 2.1 (Scale-dependent measures) · Cited: Hyndman and Koehler (2006), Section 2.4 (Relative measures) · Limitations: L3, L5 · Used by: C25, C26, C27, C28
When the errors of series on different scales are summed or averaged without scaling, the series with the largest values dominate the result.
Scope: data: error measures over series on different scales · Cited: Athanasopoulos and Kourentzes (2023) · Limitations: L5, L6 · Used by: C25
Scaled errors, as in MASE, are independent of the scale of the data and can be averaged over series on different scales.
Scope: data: error measures over series on different scales · Cited: Hyndman and Koehler (2006), Abstract; Section 3 (Scaled errors) · Limitations: L3, L5 · Used by: C3, C7, C20, C26
The M5 accuracy competition scored point forecasts with the RMSSE: each series' squared errors divided by the in-sample mean squared error of its one-step naive forecast.
Scope: data: point forecasts in the M5 accuracy competition · Cited: https://github.com/Mcompetitions/M5-methods/blob/master/M5-Competitors-Guide.pdf, Point forecasts · Cited: Makridakis et al. (2022a) · Limitations: L5, L6 · Used by: C3, C26
The M5 uncertainty competition scored quantile forecasts with the scaled pinball loss: each series' pinball loss divided by the in-sample mean absolute error of its one-step naive forecast.
Scope: data: quantile forecasts in the M5 uncertainty competition · Cited: https://github.com/Mcompetitions/M5-methods/blob/master/M5-Competitors-Guide.pdf, Probabilistic forecasts · Cited: Makridakis et al. (2022b) · Limitations: L5, L6 · Used by: C27
The M4 competition scored prediction intervals with the mean scaled interval score: the interval score of each series scaled by an in-sample error of that series.
Scope: data: prediction intervals in the M4 competition · Cited: Makridakis et al. (2020) · Limitations: L5, L6 · Used by: C28
Averages of a performance measure over many data sets are susceptible to outliers.
Scope: data: comparisons of methods over many data sets · Cited: https://jmlr.org/papers/v7/demsar06a.html, Section 3.1 (Comparisons of two classifiers) · Limitations: L5 · Used by: C20
The Friedman test, followed by the Nemenyi test, compares several methods over many data sets by their ranks on each data set, assuming only limited commensurability of the measure across data sets.
Scope: data: comparisons of several methods over many data sets · Cited: https://jmlr.org/papers/v7/demsar06a.html, Section 3.2.2 (Friedman test) · Cited: https://jmlr.org/papers/v7/demsar06a.html, Section 5 (Conclusion) · Limitations: L5 · Used by: C21
MCB on ranks was used to test the differences between the methods of the M3 competition.
Scope: data: comparisons of the methods of the M3 competition · Cited: Koning et al. (2005) · Limitations: L5, L6 · Used by: C22
The Diebold-Mariano test of equal accuracy of two forecasts allows forecast errors that are serially correlated and not normally distributed.
Scope: data: two forecasts of one series · Cited: Diebold and Mariano (1995) · Limitations: L5, L6 · Used by: C23
Evidence
A forecast is built to target one functional, and only a scoring function consistent for that functional is lowest in expectation for it; a scoring function consistent for another functional ranks forecasts by how close they come to that other functional.
Squared error is consistent for the mean, so its expected value is lowest for the forecast that equals the mean; the RMSE, the square root of the mean squared error, ranks forecasts in the same order.
Dividing a series' squared errors by a fixed positive number, such as the in-sample mean squared error of its one-step naive forecast, does not change which forecast minimises their expected value, so the scaled RMSE (the RMSSE) keeps the target of the RMSE, the mean, and the scaled values of different series can be averaged, as in the M5 accuracy competition.
Absolute error, which MAE averages and MASE scales, is consistent for the median, and squared error for the mean. When the mean and median forecasts coincide (a symmetric predictive distribution), both reward the same forecast; otherwise MAE and MASE reward the median forecast over a forecast that is correct for the mean.
MAPE is consistent for the median of order -1, which supports severe underforecasts compared with the mean or the median; it therefore rewards forecasts that are too low for either target.
sMAPE penalises forecasts below the outcome more heavily than forecasts above it, so it rewards forecasts that are too high; and it divides by the mean of the absolute outcome and the absolute forecast, which is zero when both are zero, so it is then undefined.
Absolute error is consistent for the median, so its expected value is lowest for the median forecast; MASE divides each series' absolute errors by a fixed in-sample quantity, which keeps that target and lets series on different scales be averaged.
On intermittent series the median of the outcome can be zero, and absolute error, consistent for the median, then gives its best score to a forecast of zero in every period, which says nothing of the demand to be met; a measure of the mean or of the whole distribution does not reward it.
The pinball loss at a level is consistent for the quantile at that level, so its expected value is lowest for the correct quantile; averaging over the levels gives each quantile the same weight.
The interval score is proper for central intervals and addresses width as well as coverage; coverage against the nominal level shows how far the intervals' calibration departs from what they claim.
Coverage counts only whether each outcome falls inside its interval; widening an interval never lowers its coverage, so coverage alone cannot penalise needless width, which the interval score does.
The CRPS and the log score are proper: their expected values are best for the true distribution, so they never reward a forecaster for reporting a distribution other than the one believed.
Predictive distributions are assessed by maximising their sharpness subject to their calibration: a sharper forecast that is not calibrated is overconfident, so sharpness tells which of two forecasts is better only when both are calibrated. A prediction interval summarises a predictive distribution, and the interval score weighs its width against its misses in the same way.
The probability integral transform of a discrete distribution takes few values and is not uniform even when the forecast is calibrated, so the calibration of count forecasts is checked with the randomised transform; proper scoring rules apply to count distributions as to any other.
The Brier score is proper and the log score strictly proper, so both are best in expectation for the true probabilities of the classes.
Accuracy uses a forecast probability only through which side of a threshold it falls, and counts every misclassification as the same loss; two forecasts with different probabilities but the same thresholded labels score the same, and unequal costs of errors are not reflected. The Brier and log scores, which are proper, reward the probabilities themselves.
The AUC averages the misclassification loss over a cost distribution that depends on the classifier, so different classifiers are compared with different weightings of the costs of errors; and it depends only on how the scores rank cases, not on whether the probabilities are right.
On imbalanced data the ROC curve can be deceptive about how reliable a classifier is, and the precision-recall curve is the more informative of the two; reporting it shows what the ROC curve can hide.
Each scoring function is lowest in expectation for the functional it is consistent for, so one measure favours the methods that target that functional; comparing methods that target different functionals with one measure compares their targets, not their skill.
A scaled measure can be averaged over series, but averages over many data sets are susceptible to outliers; the median and the quartiles show how accuracy is spread over the series, which the mean alone hides.
The Friedman test, followed by the Nemenyi test, compares methods by their ranks on each data set and assumes only limited commensurability of the measure across data sets; it tells whether differences between methods whose aggregate accuracy is close are larger than chance.
Multiple comparisons with the best on ranks test which methods differ from the best one, and were used for this on the methods of the M3 competition.
The Diebold-Mariano test allows forecast errors that are serially correlated and not normally distributed, as the errors of multi-step forecasts of one series usually are; tests that assume independent, normal errors do not hold for them.
A measure fits a study when it is consistent for what the compared methods predict; that a competition or a well-known study used another measure says nothing about whether that one is consistent for the forecasts of this study.
Scale-dependent measures cannot be compared across series on different scales, and summed or averaged without scaling they are dominated by the series with the largest values; the result then says little about accuracy on the other series.
Scaled errors are independent of the scale of the data and can be averaged over series; dividing by a quantity of the series that does not depend on the compared forecasts keeps the target that the unscaled measure is consistent for.
Pinball losses of series on different scales cannot be compared unscaled; dividing each series' pinball loss by a quantity of that series that does not depend on the forecasts keeps the quantile it is consistent for, as the scaled pinball loss of the M5 uncertainty competition does.
Interval scores of series on different scales cannot be compared unscaled; dividing each series' interval score by a positive quantity of that series that does not depend on the forecasts keeps it proper, as the mean scaled interval score of the M4 competition does.
Limitations
These claims rest only on works checked by their abstracts and bibliographic records (Brier 1950, Hand 2009: lab notes NT7 and NT8); a full text could qualify what the abstract states.
Part of the support of these claims comes from a work checked only by its abstract (Brier 1950, lab note NT7); the rest comes from works read in full.
Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions.
No cited work states this recommendation or its reason; it rests on an argument from the definition of accuracy (its warrant), with the properness of the Brier and log scores as premises.
The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it.
These claims rest in part on works read in full by the guarantor, not by the agent that wrote the claimset (lab note NT14); no passage of those works is quoted, so a reader checks these claims against the works themselves.
References
- Athanasopoulos, G., Kourentzes, N., 2023. On the evaluation of hierarchical forecasts. International Journal of Forecasting 39, 1502–1511. https://doi.org/10.1016/j.ijforecast.2022.08.003 Used by: C45
- BRIER, G. W., 1950. VERIFICATION OF FORECASTS EXPRESSED IN TERMS OF PROBABILITY. Monthly Weather Review 78, 1–3. https://doi.org/10.1175/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2 Used by: C38, D17
- Diebold, F. X., Mariano, R. S., 1995. Comparing Predictive Accuracy. Journal of Business & Economic Statistics 13, 253–263. https://doi.org/10.1080/07350015.1995.10524599 Used by: C53, D28
- Gneiting, T., 2011. Making and Evaluating Point Forecasts. Journal of the American Statistical Association 106, 746–762. https://doi.org/10.1198/jasa.2011.r10138 Used by: C29, C30, C31, C32, C34, D1, D3, D4, D11, D12, A1
- Gneiting, T., Raftery, A. E., 2007. Strictly Proper Scoring Rules, Prediction, and Estimation. Journal of the American Statistical Association 102, 359–378. https://doi.org/10.1198/016214506000001437 Used by: Context, C35, C36, C37, C38, D2, D13, D14, D15, D16, A2
- Gneiting, T., Balabdaoui, F., Raftery, A. E., 2007. Probabilistic Forecasts, Calibration and Sharpness. Journal of the Royal Statistical Society Series B: Statistical Methodology 69, 243–268. https://doi.org/10.1111/j.1467-9868.2007.00587.x Used by: C39, D18, D19, D20, A3
- Goodwin, P., Lawton, R., 1999. On the asymmetry of the symmetric MAPE. International Journal of Forecasting 15, 405–408. https://doi.org/10.1016/S0169-2070(99)00007-2 Used by: C33, D10
- Hand, D. J., 2009. Measuring classifier performance: a coherent alternative to the area under the ROC curve. Machine Learning 77, 103–123. https://doi.org/10.1007/s10994-009-5119-5 Used by: C42, D22
- Hyndman, R. J., Koehler, A. B., 2006. Another look at measures of forecast accuracy. International Journal of Forecasting 22, 679–688. https://doi.org/10.1016/j.ijforecast.2006.03.001 Used by: C33, C44, C46, D5, D7, D8, D9, A7
- Kolassa, S., 2016. Evaluating predictive count data distributions in retail sales forecasting. International Journal of Forecasting 32, 788–803. https://doi.org/10.1016/j.ijforecast.2015.12.004 Used by: C40, C41, D24
- Kolassa, S., 2020. Why the “best” point forecast depends on the error or accuracy measure. International Journal of Forecasting 36, 208–211. https://doi.org/10.1016/j.ijforecast.2019.02.017 Used by: C30
- Koning, A. J., Franses, P. H., Hibon, M., Stekler, H. O., 2005. The M3 competition: Statistical tests of the results. International Journal of Forecasting 21, 397–409. https://doi.org/10.1016/j.ijforecast.2004.10.003 Used by: C52, D27
- Makridakis, S., Spiliotis, E., Assimakopoulos, V., 2020. The M4 Competition: 100,000 time series and 61 forecasting methods. International Journal of Forecasting 36, 54–74. https://doi.org/10.1016/j.ijforecast.2019.04.014 Used by: C49
- Makridakis, S., Spiliotis, E., Assimakopoulos, V., 2022a. M5 accuracy competition: Results, findings, and conclusions. International Journal of Forecasting 38, 1346–1364. https://doi.org/10.1016/j.ijforecast.2021.11.013 Used by: C47, D6
- Makridakis, S., Spiliotis, E., Assimakopoulos, V., Chen, Z., Gaba, A., Tsetlin, I., Winkler, R. L., 2022b. The M5 uncertainty competition: Results, findings and conclusions. International Journal of Forecasting 38, 1365–1385. https://doi.org/10.1016/j.ijforecast.2021.10.009 Used by: C48
- Saito, T., Rehmsmeier, M., 2015. The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Classifiers on Imbalanced Datasets. PLOS ONE 10, e0118432. https://doi.org/10.1371/journal.pone.0118432 Used by: C43, D23, A8
- https://github.com/Mcompetitions/M5-methods/blob/master/M5-Competitors-Guide.pdf Used by: C41, C47, C48, A9
- https://jmlr.org/papers/v7/demsar06a.html Used by: C50, C51, D25, D26, A4
Artefacts
- A1 · data · Gneiting (2011) Making and evaluating point forecasts (arXiv:0912.0902, the preprint of Gneiting 2011, JASA 106(494), 746-762, doi:10.1198/jasa.2011.r10138) (application/pdf), saved by hand on 2026-11-13; recorded in the lab record as SR1 on 2026-11-13. venue: arXiv. Written by others. 0912.0902
- A2 · data · Gneiting and Raftery (2007) Strictly proper scoring rules, prediction, and estimation (Gneiting and Raftery 2007, JASA 102(477), 359-378, doi:10.1198/016214506000001437), author's copy (application/pdf), saved by hand on 2026-11-13; recorded in the lab record as SR2 on 2026-11-13. venue: Journal of the American Statistical Association. Written by others. Gneiting2007jasa.pdf
- A3 · data · Gneiting et al. (2007) Probabilistic forecasts, calibration and sharpness (Gneiting, Balabdaoui and Raftery 2007, JRSS B 69(2), 243-268, doi:10.1111/j.1467-9868.2007.00587.x), author's copy (application/pdf), saved by hand on 2026-11-13; recorded in the lab record as SR3 on 2026-11-13. venue: Journal of the Royal Statistical Society B. Written by others. Gneiting2007jrssb.pdf
- A4 · data · https://jmlr.org/papers/v7/demsar06a.html Statistical comparisons of classifiers over multiple data sets (Demsar 2006, JMLR 7, 1-30) (application/pdf), saved by hand on 2026-11-13; recorded in the lab record as SR4 on 2026-11-13. venue: Journal of Machine Learning Research. Written by others. demsar06a.pdf
- A7 · data · Hyndman and Koehler (2006) Another look at measures of forecast accuracy (preprint of Hyndman and Koehler 2006, IJF 22(4), 679-688, doi:10.1016/j.ijforecast.2006.03.001) (application/pdf), saved by hand on 2026-11-13; recorded in the lab record as SR7 on 2026-11-13. venue: International Journal of Forecasting. Written by others. mase.pdf
- A8 · data · Saito and Rehmsmeier (2015) The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets (Saito and Rehmsmeier 2015, PLoS ONE 10(3), e0118432, doi:10.1371/journal.pone.0118432) (application/pdf), saved by hand on 2026-11-13; recorded in the lab record as SR8 on 2026-11-13. venue: PLoS ONE; published: 2015-03-04. Written by others. saito-rehmsmeier2015.pdf
- A9 · data · https://github.com/Mcompetitions/M5-methods/blob/master/M5-Competitors-Guide.pdf The M5 Competition: Competitors' Guide (the organisers' definition of the RMSSE, the scaled pinball loss and the reasons for them) (application/pdf), saved by hand on 2026-11-13; recorded in the lab record as SR9 on 2026-11-13. venue: M Open Forecasting Center, GitHub repository Mcompetitions/M5-methods. Written by others. M5-Competitors-Guide.pdf
- A10 · lab_record Lab record of the study: questions, hypotheses, plans, runs, results and decisions, as an append-only, hash-chained event log (Claimset lab record specification v0.22, begun 2026-11-13; exported with claimset 0.43.0). lab.jsonl
Score
Support of the paper: 78, Partly supported
Scores are computed by rules from the claimset file (EntryLib rubric 0.7): they check the record, not the science, and may change as the rubric and reviews develop. The rubric. Every issue as text, for an agent preparing the next version.
The mean of its 28 principal claims; the lowest is C17 at 64. Open problems in the principal claims and what they rest on, each counted once, with the most it costs a claim:
- −20 Evidence Rests on cited work only; cited work read in full, without the passage it rests on; an argument from other claims. In C1, C2, C3, C4, C5, C6, C7, C8, C9, C10, C11, C12, C13, C14, C15, C16, C17, C18, C19, C20, C21, C22, C23, C24, C25, C26, C27, C28, C40, C42, C45, C49, C52, C53. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −10 Open problems Limitation L4 (could reverse, not mitigated): No cited work states this recommendation or its reason; it rests on an argument from the definition of accuracy (its warrant), with the properness of the Brier and log scores as premises. In C16. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −10 Open problems Limitation L1 (could reverse, not mitigated): These claims rest only on works checked by their abstracts and bibliographic records (Brier 1950, Hand 2009: lab notes NT7 and NT8); a full text could qualify what the abstract states. In C17, C42. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −3 Plan The review's search for the literature: none is recorded; the claims were confirmed against works the authors knew. In C1, C2, C3, C4, C5, C6, C7, C8, C9, C10, C11, C12, C13, C14, C15, C16, C17, C18, C19, C20, C21, C22, C23, C24, C25, C26, C27, C28. To remove it: Record the protocol, then the searches and the screening of what they found (claimset lab protocol, search, screen), and export them.
- −1 Open problems Limitation L2 (reduces precision, not mitigated): Part of the support of these claims comes from a work checked only by its abstract (Brier 1950, lab note NT7); the rest comes from works read in full. In C15, C16, C38. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L3 (reduces precision, not mitigated): Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions. In C1, C2, C3, C4, C5, C6, C7, C8, C9, C10, C11, C12, C13, C14, C15, C16, C19, C20, C24, C25, C26, C27, C28, C29, C30, C31, C32, C33, C34, C35, C36, C37, C38, C39, C44, C46. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. In C1, C2, C3, C4, C5, C6, C7, C8, C9, C10, C11, C12, C13, C14, C15, C16, C17, C18, C19, C20, C21, C22, C23, C24, C25, C26, C27, C28, C29, C30, C31, C32, C33, C34, C35, C36, C37, C38, C39, C40, C41, C42, C43, C44, C45, C46, C47, C48, C49, C50, C51, C52, C53. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L6 (reduces precision, not mitigated): These claims rest in part on works read in full by the guarantor, not by the agent that wrote the claimset (lab note NT14); no passage of those works is quoted, so a reader checks these claims against the works themselves. In C1, C2, C3, C6, C8, C14, C19, C22, C23, C24, C25, C26, C27, C28, C30, C33, C40, C41, C45, C47, C48, C49, C52, C53. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
Claims
C1: 79, Partly supported
Novelty: summarises earlier work, by the authors
- −10 Evidence Rests on cited work only; cited work read in full, without the passage it rests on; an argument from other claims. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −3 Plan The review's search for the literature: none is recorded; the claims were confirmed against works the authors knew. To remove it: Record the protocol, then the searches and the screening of what they found (claimset lab protocol, search, screen), and export them.
- −1 Open problems Limitation L3 (reduces precision, not mitigated): Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L6 (reduces precision, not mitigated): These claims rest in part on works read in full by the guarantor, not by the agent that wrote the claimset (lab note NT14); no passage of those works is quoted, so a reader checks these claims against the works themselves. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C2: 79, Partly supported
Novelty: summarises earlier work, by the authors
- −10 Evidence Rests on cited work only; cited work read in full, without the passage it rests on; an argument from other claims. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −3 Plan The review's search for the literature: none is recorded; the claims were confirmed against works the authors knew. To remove it: Record the protocol, then the searches and the screening of what they found (claimset lab protocol, search, screen), and export them.
- −1 Open problems Limitation L3 (reduces precision, not mitigated): Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L6 (reduces precision, not mitigated): These claims rest in part on works read in full by the guarantor, not by the agent that wrote the claimset (lab note NT14); no passage of those works is quoted, so a reader checks these claims against the works themselves. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C3: 79, Partly supported
Novelty: summarises earlier work, by the authors
- −10 Evidence Rests on cited work only; cited work read in full, without the passage it rests on; an argument from other claims. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −3 Plan The review's search for the literature: none is recorded; the claims were confirmed against works the authors knew. To remove it: Record the protocol, then the searches and the screening of what they found (claimset lab protocol, search, screen), and export them.
- −1 Open problems Limitation L3 (reduces precision, not mitigated): Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L6 (reduces precision, not mitigated): These claims rest in part on works read in full by the guarantor, not by the agent that wrote the claimset (lab note NT14); no passage of those works is quoted, so a reader checks these claims against the works themselves. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C4: 80, Well supported
Novelty: summarises earlier work, by the authors
- −10 Evidence Rests on cited work only; cited work read in full, without the passage it rests on; an argument from other claims. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −3 Plan The review's search for the literature: none is recorded; the claims were confirmed against works the authors knew. To remove it: Record the protocol, then the searches and the screening of what they found (claimset lab protocol, search, screen), and export them.
- −1 Open problems Limitation L3 (reduces precision, not mitigated): Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C5: 80, Well supported
Novelty: summarises earlier work, by the authors
- −10 Evidence Rests on cited work only; cited work read in full, without the passage it rests on; an argument from other claims. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −3 Plan The review's search for the literature: none is recorded; the claims were confirmed against works the authors knew. To remove it: Record the protocol, then the searches and the screening of what they found (claimset lab protocol, search, screen), and export them.
- −1 Open problems Limitation L3 (reduces precision, not mitigated): Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C6: 79, Partly supported
Novelty: summarises earlier work, by the authors
- −10 Evidence Rests on cited work only; cited work read in full, without the passage it rests on; an argument from other claims. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −3 Plan The review's search for the literature: none is recorded; the claims were confirmed against works the authors knew. To remove it: Record the protocol, then the searches and the screening of what they found (claimset lab protocol, search, screen), and export them.
- −1 Open problems Limitation L3 (reduces precision, not mitigated): Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L6 (reduces precision, not mitigated): These claims rest in part on works read in full by the guarantor, not by the agent that wrote the claimset (lab note NT14); no passage of those works is quoted, so a reader checks these claims against the works themselves. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C7: 80, Well supported
Novelty: summarises earlier work, by the authors
- −10 Evidence Rests on cited work only; cited work read in full, without the passage it rests on; an argument from other claims. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −3 Plan The review's search for the literature: none is recorded; the claims were confirmed against works the authors knew. To remove it: Record the protocol, then the searches and the screening of what they found (claimset lab protocol, search, screen), and export them.
- −1 Open problems Limitation L3 (reduces precision, not mitigated): Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C8: 79, Partly supported
Novelty: summarises earlier work, by the authors
- −10 Evidence Rests on cited work only; cited work read in full, without the passage it rests on; an argument from other claims. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −3 Plan The review's search for the literature: none is recorded; the claims were confirmed against works the authors knew. To remove it: Record the protocol, then the searches and the screening of what they found (claimset lab protocol, search, screen), and export them.
- −1 Open problems Limitation L3 (reduces precision, not mitigated): Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L6 (reduces precision, not mitigated): These claims rest in part on works read in full by the guarantor, not by the agent that wrote the claimset (lab note NT14); no passage of those works is quoted, so a reader checks these claims against the works themselves. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C9: 80, Well supported
Novelty: summarises earlier work, by the authors
- −10 Evidence Rests on cited work only; cited work read in full, without the passage it rests on; an argument from other claims. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −3 Plan The review's search for the literature: none is recorded; the claims were confirmed against works the authors knew. To remove it: Record the protocol, then the searches and the screening of what they found (claimset lab protocol, search, screen), and export them.
- −1 Open problems Limitation L3 (reduces precision, not mitigated): Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C10: 80, Well supported
Novelty: summarises earlier work, by the authors
- −10 Evidence Rests on cited work only; cited work read in full, without the passage it rests on; an argument from other claims. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −3 Plan The review's search for the literature: none is recorded; the claims were confirmed against works the authors knew. To remove it: Record the protocol, then the searches and the screening of what they found (claimset lab protocol, search, screen), and export them.
- −1 Open problems Limitation L3 (reduces precision, not mitigated): Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C11: 80, Well supported
Novelty: summarises earlier work, by the authors · in question: close match on EntryLib not acknowledged
- −10 Evidence Rests on cited work only; cited work read in full, without the passage it rests on; an argument from other claims. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −3 Plan The review's search for the literature: none is recorded; the claims were confirmed against works the authors knew. To remove it: Record the protocol, then the searches and the screening of what they found (claimset lab protocol, search, screen), and export them.
- −1 Open problems Limitation L3 (reduces precision, not mitigated): Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C12: 80, Well supported
Novelty: summarises earlier work, by the authors
- −10 Evidence Rests on cited work only; cited work read in full, without the passage it rests on; an argument from other claims. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −3 Plan The review's search for the literature: none is recorded; the claims were confirmed against works the authors knew. To remove it: Record the protocol, then the searches and the screening of what they found (claimset lab protocol, search, screen), and export them.
- −1 Open problems Limitation L3 (reduces precision, not mitigated): Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C13: 80, Well supported
Novelty: summarises earlier work, by the authors
- −10 Evidence Rests on cited work only; cited work read in full, without the passage it rests on; an argument from other claims. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −3 Plan The review's search for the literature: none is recorded; the claims were confirmed against works the authors knew. To remove it: Record the protocol, then the searches and the screening of what they found (claimset lab protocol, search, screen), and export them.
- −1 Open problems Limitation L3 (reduces precision, not mitigated): Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C14: 74, Partly supported
Novelty: summarises earlier work, by the authors
- −15 Evidence Rests on cited work only; cited work read in full, without the passage it rests on; an argument from other claims. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −3 Plan The review's search for the literature: none is recorded; the claims were confirmed against works the authors knew. To remove it: Record the protocol, then the searches and the screening of what they found (claimset lab protocol, search, screen), and export them.
- −1 Open problems Limitation L3 (reduces precision, not mitigated): Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L6 (reduces precision, not mitigated): These claims rest in part on works read in full by the guarantor, not by the agent that wrote the claimset (lab note NT14); no passage of those works is quoted, so a reader checks these claims against the works themselves. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C15: 79, Partly supported
Novelty: summarises earlier work, by the authors
- −10 Evidence Rests on cited work only; cited work read in full, without the passage it rests on; an argument from other claims. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −3 Plan The review's search for the literature: none is recorded; the claims were confirmed against works the authors knew. To remove it: Record the protocol, then the searches and the screening of what they found (claimset lab protocol, search, screen), and export them.
- −1 Open problems Limitation L2 (reduces precision, not mitigated): Part of the support of these claims comes from a work checked only by its abstract (Brier 1950, lab note NT7); the rest comes from works read in full. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L3 (reduces precision, not mitigated): Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C16: 72, Partly supported
Novelty: extends earlier work, by the authors
- −10 Evidence Rests on cited work only; cited work read in full, without the passage it rests on; an argument from other claims. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −10 Open problems Limitation L4 (could reverse, not mitigated): No cited work states this recommendation or its reason; it rests on an argument from the definition of accuracy (its warrant), with the properness of the Brier and log scores as premises. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −3 Plan The review's search for the literature: none is recorded; the claims were confirmed against works the authors knew. To remove it: Record the protocol, then the searches and the screening of what they found (claimset lab protocol, search, screen), and export them.
- −3 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L2 (reduces precision, not mitigated): Part of the support of these claims comes from a work checked only by its abstract (Brier 1950, lab note NT7); the rest comes from works read in full. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L3 (reduces precision, not mitigated): Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C17: 64, Partly supported
Novelty: summarises earlier work, by the authors
- −20 Evidence Rests on cited work only; cited work read in full, without the passage it rests on; an argument from other claims. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −10 Open problems Limitation L1 (could reverse, not mitigated): These claims rest only on works checked by their abstracts and bibliographic records (Brier 1950, Hand 2009: lab notes NT7 and NT8); a full text could qualify what the abstract states. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −3 Plan The review's search for the literature: none is recorded; the claims were confirmed against works the authors knew. To remove it: Record the protocol, then the searches and the screening of what they found (claimset lab protocol, search, screen), and export them.
- −3 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C18: 81, Well supported
Novelty: summarises earlier work, by the authors
- −10 Evidence Rests on cited work only; cited work read in full, without the passage it rests on; an argument from other claims. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −3 Plan The review's search for the literature: none is recorded; the claims were confirmed against works the authors knew. To remove it: Record the protocol, then the searches and the screening of what they found (claimset lab protocol, search, screen), and export them.
C19: 79, Partly supported
Novelty: summarises earlier work, by the authors
- −10 Evidence Rests on cited work only; cited work read in full, without the passage it rests on; an argument from other claims. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −3 Plan The review's search for the literature: none is recorded; the claims were confirmed against works the authors knew. To remove it: Record the protocol, then the searches and the screening of what they found (claimset lab protocol, search, screen), and export them.
- −1 Open problems Limitation L3 (reduces precision, not mitigated): Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L6 (reduces precision, not mitigated): These claims rest in part on works read in full by the guarantor, not by the agent that wrote the claimset (lab note NT14); no passage of those works is quoted, so a reader checks these claims against the works themselves. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C20: 80, Well supported
Novelty: summarises earlier work, by the authors
- −10 Evidence Rests on cited work only; cited work read in full, without the passage it rests on; an argument from other claims. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −3 Plan The review's search for the literature: none is recorded; the claims were confirmed against works the authors knew. To remove it: Record the protocol, then the searches and the screening of what they found (claimset lab protocol, search, screen), and export them.
- −1 Open problems Limitation L3 (reduces precision, not mitigated): Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C21: 81, Well supported
Novelty: summarises earlier work, by the authors
- −10 Evidence Rests on cited work only; cited work read in full, without the passage it rests on; an argument from other claims. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −3 Plan The review's search for the literature: none is recorded; the claims were confirmed against works the authors knew. To remove it: Record the protocol, then the searches and the screening of what they found (claimset lab protocol, search, screen), and export them.
C22: 75, Partly supported
Novelty: summarises earlier work, by the authors
- −15 Evidence Rests on cited work only; cited work read in full, without the passage it rests on; an argument from other claims. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −3 Plan The review's search for the literature: none is recorded; the claims were confirmed against works the authors knew. To remove it: Record the protocol, then the searches and the screening of what they found (claimset lab protocol, search, screen), and export them.
- −1 Open problems Limitation L6 (reduces precision, not mitigated): These claims rest in part on works read in full by the guarantor, not by the agent that wrote the claimset (lab note NT14); no passage of those works is quoted, so a reader checks these claims against the works themselves. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C23: 75, Partly supported
Novelty: summarises earlier work, by the authors
- −15 Evidence Rests on cited work only; cited work read in full, without the passage it rests on; an argument from other claims. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −3 Plan The review's search for the literature: none is recorded; the claims were confirmed against works the authors knew. To remove it: Record the protocol, then the searches and the screening of what they found (claimset lab protocol, search, screen), and export them.
- −1 Open problems Limitation L6 (reduces precision, not mitigated): These claims rest in part on works read in full by the guarantor, not by the agent that wrote the claimset (lab note NT14); no passage of those works is quoted, so a reader checks these claims against the works themselves. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C24: 79, Partly supported
Novelty: summarises earlier work, by the authors
- −10 Evidence Rests on cited work only; cited work read in full, without the passage it rests on; an argument from other claims. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −3 Plan The review's search for the literature: none is recorded; the claims were confirmed against works the authors knew. To remove it: Record the protocol, then the searches and the screening of what they found (claimset lab protocol, search, screen), and export them.
- −1 Open problems Limitation L3 (reduces precision, not mitigated): Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L6 (reduces precision, not mitigated): These claims rest in part on works read in full by the guarantor, not by the agent that wrote the claimset (lab note NT14); no passage of those works is quoted, so a reader checks these claims against the works themselves. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C25: 74, Partly supported
Novelty: summarises earlier work, by the authors
- −15 Evidence Rests on cited work only; cited work read in full, without the passage it rests on; an argument from other claims. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −3 Plan The review's search for the literature: none is recorded; the claims were confirmed against works the authors knew. To remove it: Record the protocol, then the searches and the screening of what they found (claimset lab protocol, search, screen), and export them.
- −1 Open problems Limitation L3 (reduces precision, not mitigated): Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L6 (reduces precision, not mitigated): These claims rest in part on works read in full by the guarantor, not by the agent that wrote the claimset (lab note NT14); no passage of those works is quoted, so a reader checks these claims against the works themselves. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C26: 79, Partly supported
Novelty: summarises earlier work, by the authors
- −10 Evidence Rests on cited work only; cited work read in full, without the passage it rests on; an argument from other claims. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −3 Plan The review's search for the literature: none is recorded; the claims were confirmed against works the authors knew. To remove it: Record the protocol, then the searches and the screening of what they found (claimset lab protocol, search, screen), and export them.
- −1 Open problems Limitation L3 (reduces precision, not mitigated): Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L6 (reduces precision, not mitigated): These claims rest in part on works read in full by the guarantor, not by the agent that wrote the claimset (lab note NT14); no passage of those works is quoted, so a reader checks these claims against the works themselves. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C27: 79, Partly supported
Novelty: summarises earlier work, by the authors
- −10 Evidence Rests on cited work only; cited work read in full, without the passage it rests on; an argument from other claims. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −3 Plan The review's search for the literature: none is recorded; the claims were confirmed against works the authors knew. To remove it: Record the protocol, then the searches and the screening of what they found (claimset lab protocol, search, screen), and export them.
- −1 Open problems Limitation L3 (reduces precision, not mitigated): Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L6 (reduces precision, not mitigated): These claims rest in part on works read in full by the guarantor, not by the agent that wrote the claimset (lab note NT14); no passage of those works is quoted, so a reader checks these claims against the works themselves. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C28: 74, Partly supported
Novelty: summarises earlier work, by the authors
- −15 Evidence Rests on cited work only; cited work read in full, without the passage it rests on; an argument from other claims. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −3 Plan The review's search for the literature: none is recorded; the claims were confirmed against works the authors knew. To remove it: Record the protocol, then the searches and the screening of what they found (claimset lab protocol, search, screen), and export them.
- −1 Open problems Limitation L3 (reduces precision, not mitigated): Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L6 (reduces precision, not mitigated): These claims rest in part on works read in full by the guarantor, not by the agent that wrote the claimset (lab note NT14); no passage of those works is quoted, so a reader checks these claims against the works themselves. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C29: 93, Well supported
Novelty: summarises earlier work, by the authors
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L3 (reduces precision, not mitigated): Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C30: 92, Well supported
Novelty: summarises earlier work, by the authors
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L3 (reduces precision, not mitigated): Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L6 (reduces precision, not mitigated): These claims rest in part on works read in full by the guarantor, not by the agent that wrote the claimset (lab note NT14); no passage of those works is quoted, so a reader checks these claims against the works themselves. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C31: 93, Well supported
Novelty: summarises earlier work, by the authors
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L3 (reduces precision, not mitigated): Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C32: 93, Well supported
Novelty: summarises earlier work, by the authors
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L3 (reduces precision, not mitigated): Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C33: 92, Well supported
Novelty: summarises earlier work, by the authors
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L3 (reduces precision, not mitigated): Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L6 (reduces precision, not mitigated): These claims rest in part on works read in full by the guarantor, not by the agent that wrote the claimset (lab note NT14); no passage of those works is quoted, so a reader checks these claims against the works themselves. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C34: 93, Well supported
Novelty: summarises earlier work, by the authors
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L3 (reduces precision, not mitigated): Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C35: 93, Well supported
Novelty: summarises earlier work, by the authors
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L3 (reduces precision, not mitigated): Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C36: 93, Well supported
Novelty: summarises earlier work, by the authors
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L3 (reduces precision, not mitigated): Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C37: 93, Well supported
Novelty: summarises earlier work, by the authors
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L3 (reduces precision, not mitigated): Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C38: 92, Well supported
Novelty: summarises earlier work, by the authors
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L2 (reduces precision, not mitigated): Part of the support of these claims comes from a work checked only by its abstract (Brier 1950, lab note NT7); the rest comes from works read in full. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L3 (reduces precision, not mitigated): Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C39: 93, Well supported
Novelty: summarises earlier work, by the authors
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L3 (reduces precision, not mitigated): Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C40: 78, Partly supported
Novelty: summarises earlier work, by the authors
- −15 Evidence Rests on cited work only; cited work read in full, without the passage it rests on; an argument from other claims. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L6 (reduces precision, not mitigated): These claims rest in part on works read in full by the guarantor, not by the agent that wrote the claimset (lab note NT14); no passage of those works is quoted, so a reader checks these claims against the works themselves. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C41: 93, Well supported
Novelty: summarises earlier work, by the authors
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L6 (reduces precision, not mitigated): These claims rest in part on works read in full by the guarantor, not by the agent that wrote the claimset (lab note NT14); no passage of those works is quoted, so a reader checks these claims against the works themselves. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C42: 67, Partly supported
Novelty: summarises earlier work, by the authors
- −20 Evidence Rests on cited work only; cited work read in full, without the passage it rests on; an argument from other claims. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −10 Open problems Limitation L1 (could reverse, not mitigated): These claims rest only on works checked by their abstracts and bibliographic records (Brier 1950, Hand 2009: lab notes NT7 and NT8); a full text could qualify what the abstract states. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −3 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C43: 94, Well supported
Novelty: summarises earlier work, by the authors
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C44: 93, Well supported
Novelty: summarises earlier work, by the authors
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L3 (reduces precision, not mitigated): Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C45: 78, Partly supported
Novelty: summarises earlier work, by the authors
- −15 Evidence Rests on cited work only; cited work read in full, without the passage it rests on; an argument from other claims. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L6 (reduces precision, not mitigated): These claims rest in part on works read in full by the guarantor, not by the agent that wrote the claimset (lab note NT14); no passage of those works is quoted, so a reader checks these claims against the works themselves. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C46: 93, Well supported
Novelty: summarises earlier work, by the authors
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L3 (reduces precision, not mitigated): Section and theorem numbers for Gneiting 2011, Gneiting and Raftery 2007, Gneiting, Balabdaoui and Raftery 2007 and Hyndman and Koehler 2006 were read in their preprints or authors' copies (arXiv:0912.0902 and the authors' web pages), whose numbering may differ from the published versions. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C47: 93, Well supported
Novelty: summarises earlier work, by the authors
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L6 (reduces precision, not mitigated): These claims rest in part on works read in full by the guarantor, not by the agent that wrote the claimset (lab note NT14); no passage of those works is quoted, so a reader checks these claims against the works themselves. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C48: 93, Well supported
Novelty: summarises earlier work, by the authors
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L6 (reduces precision, not mitigated): These claims rest in part on works read in full by the guarantor, not by the agent that wrote the claimset (lab note NT14); no passage of those works is quoted, so a reader checks these claims against the works themselves. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C49: 78, Partly supported
Novelty: summarises earlier work, by the authors
- −15 Evidence Rests on cited work only; cited work read in full, without the passage it rests on; an argument from other claims. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L6 (reduces precision, not mitigated): These claims rest in part on works read in full by the guarantor, not by the agent that wrote the claimset (lab note NT14); no passage of those works is quoted, so a reader checks these claims against the works themselves. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C50: 94, Well supported
Novelty: summarises earlier work, by the authors
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C51: 94, Well supported
Novelty: summarises earlier work, by the authors
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C52: 78, Partly supported
Novelty: summarises earlier work, by the authors
- −15 Evidence Rests on cited work only; cited work read in full, without the passage it rests on; an argument from other claims. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L6 (reduces precision, not mitigated): These claims rest in part on works read in full by the guarantor, not by the agent that wrote the claimset (lab note NT14); no passage of those works is quoted, so a reader checks these claims against the works themselves. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C53: 78, Partly supported
Novelty: summarises earlier work, by the authors
- −15 Evidence Rests on cited work only; cited work read in full, without the passage it rests on; an argument from other claims. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −6 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L6 (reduces precision, not mitigated): These claims rest in part on works read in full by the guarantor, not by the agent that wrote the claimset (lab note NT14); no passage of those works is quoted, so a reader checks these claims against the works themselves. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
Versions
- Version 2 (this page) · 14 November 2026 at 12:59 UTC · updated
- Version 1 · 14 November 2026 at 09:28 UTC · active
Every version is kept in the public registry (github.com/entrylib/registry).