TRAIL OF H23 (claim C21) Statement: To tell whether several methods whose aggregate accuracy is similar differ statistically significantly over many series or data sets, their ranks on each series should be tested with the [[TM25:Friedman test]] followed by [[TM26:Nemenyi test]] post-hoc comparisons. Status: supported In log order: the question, each version of the hypothesis, the plans that name it with their approvals, deviations and runs, its statuses, and the checks, decisions and notes that refer to them. Results are given by number and file. 2026-11-13 13:18 Q1 question by AC1: Which measures and tests should a study use to evaluate and compare predictions (point forecasts of a mean, a median or a quantile, prediction intervals, predi… 2026-11-13 13:19 TM25 term by AC1: Friedman test [method] 2026-11-13 13:19 TM26 term by AC1: Nemenyi test [method] 2026-11-13 13:20 H23 hypothesis by AC1: When several methods are compared over many data sets, their differences should be tested with the [[TM25:Friedman test]] on ranks followed by [[TM26:Nemenyi t… [a_priori] 2026-11-13 13:31 ST23 status by AC1: H23 -> supported, evidence SR4: Demsar (2006, SR4, abstract): 'we recommend a set of simple, yet safe and robust non-parametric tests', 'the Friedman t… 2026-11-13 14:07 H32 hypothesis by AC1: To tell whether several methods whose aggregate accuracy is similar differ significantly over many series or data sets, their ranks on each series should be te… [a_priori] 2026-11-13 14:08 ST40 status by AC1: H32 -> supported, evidence SR4: Demsar (2006, SR4, abstract; Sections 3.1 and 3.2.2): it recommends 'the Friedman test with the corresponding post-hoc… 2026-11-13 16:19 H58 hypothesis by AC1: The [[TM25:Friedman test]], followed by the [[TM26:Nemenyi test]], compares several methods over many data sets by their ranks on each data set, assuming only… [a_priori] 2026-11-13 16:22 H85 hypothesis by AC1: To tell whether several methods whose aggregate accuracy is similar differ significantly over many series or data sets, their ranks on each series should be te… [a_priori] 2026-11-13 16:22 ST92 status by AC1: H85 -> supported, evidence H58: The Friedman test, followed by the Nemenyi test, compares methods by their ranks on each data set and assumes only limi… 2026-11-13 16:25 H91 hypothesis by AC1: To tell whether several methods whose aggregate accuracy is similar differ statistically significantly over many series or data sets, their ranks on each serie… [a_priori] 2026-11-13 16:25 ST98 status by AC1: H91 -> supported, evidence H58: The Friedman test, followed by the Nemenyi test, compares methods by their ranks on each data set and assumes only limi…