TRAIL OF H3 (claim C3) Statement: When [[TM3:point forecasts of the mean]] are compared over series on different scales, each series' [[TM5:RMSE]] should be scaled by the in-sample RMSE of its one-step naive forecast, for example, as in [[TM6:RMSSE]]. Status: supported In log order: the question, each version of the hypothesis, the plans that name it with their approvals, deviations and runs, its statuses, and the checks, decisions and notes that refer to them. Results are given by number and file. 2026-11-13 13:18 Q1 question by AC1: Which measures and tests should a study use to evaluate and compare predictions (point forecasts of a mean, a median or a quantile, prediction intervals, predi… 2026-11-13 13:18 TM3 term by AC1: point forecast of the mean [concept] 2026-11-13 13:18 TM5 term by AC1: RMSE [variable] 2026-11-13 13:18 TM6 term by AC1: RMSSE [variable] 2026-11-13 13:20 H3 hypothesis by AC1: When [[TM3:point forecasts of the mean]] are compared over series on different scales, the [[TM5:RMSE]] of each series should be scaled by the in-sample RMSE o… [a_priori] 2026-11-13 13:30 ST3 status by AC1: H3 -> supported, evidence NT3, SR1: The M5 accuracy competition (NT3) ranked point forecasts by the RMSSE: the RMSE of each series scaled by the in-sample… 2026-11-13 14:08 ST34 status by AC1: H3 -> supported, evidence SR9, NT3, SR1: The M5 Competitors' Guide (SR9, Point forecasts): the RMSSE divides each series' squared errors by the in-sample mean s… 2026-11-13 16:16 H36 hypothesis by AC1: Squared error is a [[TM1:consistent scoring function]] for the mean. [a_priori] 2026-11-13 16:18 H53 hypothesis by AC1: Scaled errors, as in [[TM8:MASE]], are independent of the scale of the data and can be averaged over series on different scales. [a_priori] 2026-11-13 16:18 H54 hypothesis by AC1: The M5 accuracy competition scored point forecasts with the [[TM6:RMSSE]]: each series' squared errors divided by the in-sample mean squared error of its one-s… [a_priori] 2026-11-13 16:20 H64 hypothesis by AC1: When [[TM3:point forecasts of the mean]] are compared over series on different scales, each series' [[TM5:RMSE]] should be scaled by the in-sample RMSE of its… [a_priori] 2026-11-13 16:20 ST71 status by AC1: H64 -> supported, evidence H54, H36, H53: Dividing a series' squared errors by a fixed positive number, the in-sample mean squared error of its one-step naive fo… 2026-11-14 12:49 H93 hypothesis by AC1: When [[TM3:point forecasts of the mean]] are compared over series on different scales, each series' [[TM5:RMSE]] should be scaled by the in-sample RMSE of its… [a_priori] 2026-11-14 12:49 ST100 status by AC1: H93 -> supported, evidence H54, H36, H53: Dividing a series' squared errors by a fixed positive number, such as the in-sample mean squared error of its one-step…