{
 "bundle_of": "C6",
 "schema_version": "0.44",
 "paper": {
  "id": "entrylib:2026.000010",
  "version": 1,
  "title": "Online Detection of Forecast Model Inadequacies Using Forecast Errors",
  "licence": "CC-BY-4.0",
  "status": "active",
  "date": "2026-11-16"
 },
 "claims": [
  {
   "id": "C6",
   "statement": "On [[D19:the Royal Mail forecast errors]], the [[D2:mean detector]] and the [[D3:squared-error detector]] of the [[D1:forecast error monitoring framework]] both signalled a changepoint on 2 December 2020.",
   "type": "descriptive",
   "role": "principal",
   "about": {
    "subject": "D1",
    "relation": "signals a changepoint in",
    "object": "D19",
    "comparator": null,
    "conditions": [
     "monitoring from 1 November 2020"
    ]
   },
   "novelty": "no_prior_found",
   "planning": "exploratory",
   "scope": {
    "setting": "one Royal Mail delivery office",
    "geography": "United Kingdom",
    "period": "1 July 2020 to 31 December 2020"
   },
   "support": [
    {
     "kind": "empirical_quant",
     "evidence": "E9",
     "artefacts": [
      "A5"
     ],
     "results": [
      "E9.R1",
      "E9.R2"
     ]
    }
   ],
   "limitations": [
    "L4"
   ],
   "assessments": [
    {
     "by": "author",
     "strength": "low",
     "rationale": "Exploratory: no plan was recorded beforehand. One office and one season, with no independent record of when the Christmas peak began (L5); the data are not available."
    }
   ],
   "source_spans": [
    "p9.10",
    "p9.11",
    "p9.12",
    "p10.5"
   ],
   "provenance": "author_stated",
   "relations": [
    {
     "type": "addresses",
     "reference": "B1",
     "self": false,
     "note": "The closest result found: Aue et al. (2015) apply residual-based monitoring of ARMA models to EEG and IBM data; here only the forecast errors are known, not the model or the data."
    }
   ],
   "depends_on": [
    "C3",
    "C23"
   ],
   "verifiability": "unverifiable",
   "verifiability_flags": []
  },
  {
   "id": "C1",
   "statement": "Under [[D8:the mean change model]], the one-step-ahead forecast errors of a forecasting model that satisfies the two forecasting-model assumptions have a change in mean of size $\\Delta_{\\mu,m} \\ne 0$ at the changepoint: $e_t = b_\\mu + \\epsilon_{\\mu,t}$ for $t \\le m + k^*$ and $e_t = b_\\mu + \\epsilon_{\\mu,t} + \\Delta_{\\mu,m}$ afterwards.",
   "type": "formal",
   "role": "principal",
   "about": {
    "subject": "D8",
    "relation": "produces a mean change in the one-step-ahead forecast errors of",
    "object": "a forecasting model satisfying the two forecasting-model assumptions",
    "comparator": null,
    "conditions": []
   },
   "novelty": "extension",
   "scope": {
    "data_type": "univariate time series",
    "conditions": [
     "one-step-ahead forecasts",
     "a single changepoint after the training sample"
    ]
   },
   "support": [
    {
     "kind": "formal_proof",
     "evidence": "E1"
    }
   ],
   "assumptions": [
    "C18",
    "C19"
   ],
   "limitations": [
    "L1"
   ],
   "relations": [
    {
     "type": "extends",
     "reference": "B1",
     "self": false,
     "note": "Aue et al. (2015) monitor the residuals of ARMA models only; the claim covers the forecast errors of any forecasting model that satisfies the two assumptions."
    }
   ],
   "verification": "unchecked",
   "source_spans": [
    "p4.2",
    "p4.3",
    "p4.9",
    "p11.8"
   ],
   "provenance": "author_stated",
   "verifiability": "open",
   "verifiability_flags": []
  },
  {
   "id": "C2",
   "statement": "Under [[D9:the mean and variance change model]] and the constant-bias assumption, the change in the mean of the centred squared forecast errors is $$\\Delta_{\\xi,m} = \\mathrm{Var}_{G'}[Y_t|\\tilde{y}_t] - \\mathrm{Var}_F[Y_t|\\tilde{y}_t]$$ when $\\delta_{\\mu,m} = 0$, and $$\\Delta_{\\xi,m} = \\mathrm{Var}_{G'}[Y_t|\\tilde{y}_t] - \\mathrm{Var}_F[Y_t|\\tilde{y}_t] + \\Delta_{\\mu,m}^2$$ when $\\delta_{\\mu,m} \\ne 0$ and the incomplete-adjustment assumption holds.",
   "type": "formal",
   "role": "principal",
   "about": {
    "subject": "D9",
    "relation": "produces a mean change in the centred squared forecast errors equal to the variance change plus the squared mean change of the forecast errors",
    "object": null,
    "comparator": null,
    "conditions": []
   },
   "novelty": "extension",
   "scope": {
    "data_type": "univariate time series",
    "conditions": [
     "one-step-ahead forecasts"
    ]
   },
   "support": [
    {
     "kind": "formal_proof",
     "evidence": "E2"
    }
   ],
   "assumptions": [
    "C18",
    "C19"
   ],
   "relations": [
    {
     "type": "extends",
     "reference": "B1",
     "self": false,
     "note": "Aue et al. (2015) monitor the residuals of ARMA models only; the claim covers the forecast errors of any forecasting model that satisfies the two assumptions."
    }
   ],
   "verification": "unchecked",
   "source_spans": [
    "p4.17",
    "p4.19",
    "p4.20",
    "p4.21"
   ],
   "provenance": "author_stated",
   "verifiability": "open",
   "verifiability_flags": []
  },
  {
   "id": "C3",
   "statement": "For the [[D3:squared-error detector]] under [[D9:the mean and variance change model]], the limit distribution of Theorem 1 of the paper's Supporting Information holds under the null hypothesis of no change, and the limit distribution of its Theorem 2 holds for the stopping time $\\tau_{\\xi,m}$ under the alternative when the change size $\\Delta_{\\xi,m}$ is large enough in the sense of that document's Assumptions C to E.",
   "type": "formal",
   "role": "principal",
   "about": {
    "subject": "D3",
    "relation": "has the limit distributions of Page's CUSUM under the null and the alternative",
    "object": null,
    "comparator": null,
    "conditions": [
     "D9"
    ]
   },
   "novelty": "extension",
   "scope": {
    "conditions": [
     "change sizes satisfying Assumptions C to E of the paper's Supporting Information"
    ]
   },
   "support": [
    {
     "kind": "formal_proof",
     "evidence": "E3"
    }
   ],
   "depends_on": [
    "C2",
    "C24",
    "C25"
   ],
   "assumptions": [
    "C18",
    "C19",
    "C20",
    "C21"
   ],
   "relations": [
    {
     "type": "extends",
     "reference": "B1",
     "self": false,
     "note": "Aue et al. (2015) monitor the residuals of ARMA models only; the claim covers the forecast errors of any forecasting model that satisfies the two assumptions."
    }
   ],
   "verification": "unchecked",
   "source_spans": [
    "p5.6",
    "p5.7",
    "p5.9",
    "p5.10",
    "p5.11",
    "s3.3-s3.5",
    "s4.5-s4.7"
   ],
   "provenance": "author_stated",
   "verifiability": "open",
   "verifiability_flags": []
  },
  {
   "id": "C23",
   "type": "formal",
   "role": "supporting",
   "novelty": "extension",
   "statement": "For the [[D2:mean detector]] under [[D8:the mean change model]], the limit distribution of Theorem 1 of the paper's Supporting Information holds under the null hypothesis of no change with $D(m,k) = D_\\mu(m,k)$ and $\\hat{\\sigma}_m = \\hat{\\sigma}_{\\mu,m}$, and the limit distribution of its Theorem 2 holds under the alternative with $\\tau_m = \\tau_{\\mu,m}$ and $c_\\alpha = c_{\\mu,\\alpha}$ when the change size $\\Delta_{\\mu,m}$ satisfies its Assumptions C to E.",
   "about": {
    "subject": "D2",
    "relation": "has the limit distributions of Page's CUSUM under the null and the alternative",
    "object": null,
    "comparator": null,
    "conditions": [
     "D8"
    ]
   },
   "scope": {
    "data_type": "univariate time series",
    "conditions": [
     "one-step-ahead forecasts",
     "change sizes satisfying Assumptions C to E of the paper's Supporting Information"
    ]
   },
   "support": [
    {
     "kind": "formal_proof",
     "evidence": "E11"
    }
   ],
   "depends_on": [
    "C1",
    "C24",
    "C25"
   ],
   "assumptions": [
    "C18",
    "C19",
    "C20",
    "C21"
   ],
   "relations": [
    {
     "type": "extends",
     "reference": "B1",
     "self": false,
     "note": "Aue et al. (2015) give the limit distributions for the residuals of ARMA models; the claim gives them for the forecast errors of any forecasting model that satisfies the two assumptions."
    }
   ],
   "verification": "unchecked",
   "source_spans": [
    "p4.9",
    "s4.10",
    "s4.11-s4.12",
    "s4.13",
    "s4.14-s4.15"
   ],
   "provenance": "author_stated",
   "verifiability": "open",
   "verifiability_flags": []
  },
  {
   "id": "C18",
   "statement": "The forecasting model has a constant bias before the changepoint: $\\mathbb{E}_F[Y_t|\\tilde{y}_t] - \\mathbb{E}_{\\mathcal{F}}[Y_t|\\tilde{y}_t] = b_\\mu$ for $t = 1, \\ldots, m + k^*$, where $b_\\mu$ is unknown and may be zero (the paper's Assumption 1, the constant-bias assumption).",
   "type": "formal",
   "role": "supporting",
   "support": [
    {
     "kind": "assumption"
    }
   ],
   "assumption_category": "theorem_condition",
   "check": "unchecked",
   "source_spans": [
    "p3.14",
    "p3.15",
    "p3.16",
    "p3.20"
   ],
   "provenance": "author_stated",
   "verifiability": "not_applicable",
   "verifiability_flags": []
  },
  {
   "id": "C19",
   "statement": "After the changepoint the forecasting model does not fully adjust to the change: $$\\mathbb{E}_G[Y_t|\\tilde{y}_t] - \\mathbb{E}_{\\mathcal{F}}[Y_t|\\tilde{y}_t] = b_\\mu + \\Delta_{\\mu,m}$$ for $t = m + k^* + 1, m + k^* + 2, \\ldots$, where $\\Delta_{\\mu,m} \\ne 0$ is a function of the change size in the raw data (the paper's Assumption 2, the incomplete-adjustment assumption).",
   "type": "formal",
   "role": "supporting",
   "support": [
    {
     "kind": "assumption"
    }
   ],
   "assumption_category": "theorem_condition",
   "check": "unchecked",
   "source_spans": [
    "p3.17",
    "p3.18",
    "p3.19",
    "p3.20",
    "p4.1"
   ],
   "provenance": "author_stated",
   "verifiability": "not_applicable",
   "verifiability_flags": []
  },
  {
   "id": "C20",
   "statement": "The zero-mean variables $\\epsilon_t$ of the monitored series, here $\\epsilon_{\\mu,t}$ and $\\epsilon_{\\xi,t}$ left in the forecast errors after the bias is removed, satisfy the weak invariance principle of Assumptions A and B of the paper's Supporting Information: $\\left|\\sum_{t=1}^{m} \\epsilon_t\\right| = O_P(\\sqrt{m})$, and there are Wiener processes $\\{W_m(i) : i \\ge 0, m \\ge 1\\}$ and a constant $\\sigma > 0$ with $$\\sup_{1/m \\le i < \\infty} \\frac{1}{(mi)^{1/\\nu}} \\left| \\sum_{t=m+1}^{m+mi} \\epsilon_t - \\sigma W_m(mi) \\right| = O_P(1) \\quad \\text{for some } \\nu > 2$$.",
   "type": "formal",
   "role": "supporting",
   "support": [
    {
     "kind": "assumption"
    }
   ],
   "assumption_category": "statistical",
   "check": "unchecked",
   "source_spans": [
    "p4.4",
    "p4.9",
    "p4.18",
    "s1.8",
    "s2.1",
    "s2.2"
   ],
   "provenance": "author_stated",
   "verifiability": "not_applicable",
   "verifiability_flags": []
  },
  {
   "id": "C21",
   "statement": "The change is large enough and bounded: the changepoint is $k^* = \\lfloor \\theta m^\\beta \\rfloor$ for some $\\theta > 0$ and $0 \\le \\beta < 1$, $\\sqrt{m}\\,|\\Delta_m| \\to \\infty$ as $m \\to \\infty$, and $\\Delta_m = O(1)$, where $\\Delta_m$ is the change size in the monitored series, here $\\Delta_{\\mu,m}$ or $\\Delta_{\\xi,m}$, as Assumptions C to E of the paper's Supporting Information require.",
   "type": "formal",
   "role": "supporting",
   "support": [
    {
     "kind": "assumption"
    }
   ],
   "assumption_category": "theorem_condition",
   "check": "unchecked",
   "source_spans": [
    "p5.9",
    "p6.8",
    "p6.11",
    "p7.8",
    "s3.7",
    "s3.8",
    "s3.9",
    "s3.10"
   ],
   "provenance": "author_stated",
   "verifiability": "not_applicable",
   "verifiability_flags": []
  },
  {
   "id": "C24",
   "type": "formal",
   "role": "supporting",
   "novelty": "synthesis",
   "statement": "For [[D24:Page's CUSUM detector]] under the null hypothesis of no change, $$\\lim_{m\\to\\infty} P\\left(\\frac{1}{\\hat{\\sigma}_m} \\sup_{1 \\le k < \\infty} \\frac{D(m,k)}{g(m,k,\\gamma)} \\le c\\right) = P\\left(\\sup_{0<i<1} \\sup_{0 \\le j \\le i} \\frac{1}{i^\\gamma} \\left| W(i) - \\frac{1-i}{1-j} W(j) \\right| \\le c\\right)$$ for every real $c$, where $\\{W(i) : i \\in [0,1]\\}$ is a standard Brownian motion (Theorem 1 of the paper's Supporting Information, from Fremdt 2015).",
   "about": {
    "subject": "D24",
    "relation": "has a known limit distribution under the null hypothesis",
    "object": null,
    "comparator": null,
    "conditions": [
     "no change"
    ]
   },
   "scope": {
    "conditions": [
     "no change in the monitored series",
     "limit for a training sample size going to infinity"
    ]
   },
   "support": [
    {
     "kind": "cited",
     "reference": "B8",
     "self": false
    }
   ],
   "assumptions": [
    "C20"
   ],
   "source_spans": [
    "s3.2",
    "s3.3-s3.5"
   ],
   "provenance": "author_stated",
   "verifiability": "open",
   "verifiability_flags": []
  },
  {
   "id": "C25",
   "type": "formal",
   "role": "supporting",
   "novelty": "synthesis",
   "statement": "For [[D24:Page's CUSUM detector]] under the alternative of a mean change, $$\\lim_{m\\to\\infty} P\\left(\\frac{\\tau_m - a_m(c_\\alpha)}{b_m(c_\\alpha)} \\le x\\right) = \\Psi(x)$$ for every real $x$, where $a_m(c)$ is the unique solution of $$a_m(c) = \\left(\\frac{\\sigma c\\, m^{1/2-\\gamma}}{|\\Delta_m|} + \\frac{k^*}{(a_m(c))^\\gamma}\\right)^{1/(1-\\gamma)},$$ $$b_m(c) = \\frac{\\sigma \\sqrt{a_m(c)}}{|\\Delta_m|} \\left(1 - \\gamma\\left(1 - \\frac{k^*}{a_m(c)}\\right)\\right)^{-1},$$ and $\\Psi(x) = 1 - \\tilde{\\Psi}(-x)$ with $\\tilde{\\Psi}(x)$ equal to $\\Phi(x)$ when $\\tilde{\\Delta}_m = |\\Delta_m| m^{\\beta(1-\\gamma)-1/2+\\gamma} \\to 0$, to $P(\\sup_{d<i<1} W(i) \\le x)$ when $\\tilde{\\Delta}_m \\to \\tilde{C} \\in (0,\\infty)$, and to $P(\\sup_{0<i<1} W(i) \\le x)$ when $\\tilde{\\Delta}_m \\to \\infty$, where $\\Phi$ is the standard normal distribution function, $W$ a standard Brownian motion and $d = d(c)$ the unique solution of $d = 1 - \\frac{c}{C} d^{1-\\gamma}$ with $C = \\theta^{1-\\gamma}\\tilde{C}$ (Theorem 2 of the paper's Supporting Information, from Fremdt 2014).",
   "about": {
    "subject": "D24",
    "relation": "has a stopping time with a known limit distribution under the alternative",
    "object": null,
    "comparator": null,
    "conditions": [
     "a change in mean"
    ]
   },
   "scope": {
    "conditions": [
     "a change in mean satisfying Assumptions C to E of the paper's Supporting Information",
     "limit for a training sample size going to infinity"
    ]
   },
   "support": [
    {
     "kind": "cited",
     "reference": "B9",
     "self": false
    }
   ],
   "assumptions": [
    "C20",
    "C21"
   ],
   "source_spans": [
    "s3.6",
    "s3.11",
    "s3.12",
    "s3.13-s3.14",
    "s4.1-s4.4",
    "s4.5-s4.7",
    "s4.8"
   ],
   "provenance": "author_stated",
   "verifiability": "open",
   "verifiability_flags": []
  }
 ],
 "definitions": [
  {
   "id": "D1",
   "term": "forecast error monitoring framework",
   "kind": "method",
   "definition": "A framework that monitors a forecasting model online by applying sequential changepoint detection to its one-step-ahead forecast errors. For a univariate series $Y_t$ and a forecasting model $\\mathcal{F}$ with one-step-ahead point forecast $\\hat{y}_t(1) = \\mathbb{E}_{\\mathcal{F}}[Y_t | y_{t-1}, \\ldots, y_1]$, the forecast errors are $$e_t = Y_t - \\hat{y}_t(1).$$ The first $m$ forecast errors are the Phase I (training) sample; monitoring starts at $t = m + 1$ and stops when a detector, [[D2:the mean detector]] or [[D3:the squared-error detector]], crosses its threshold. Any forecasting approach can supply the errors: model-based, machine learning or expert judgement.",
   "source_spans": [
    "p2.4",
    "p3.3",
    "p3.4",
    "p3.5"
   ]
  },
  {
   "id": "D2",
   "term": "mean detector $D_\\mu(m,k)$",
   "kind": "method",
   "definition": "[[D24:Page's CUSUM detector]] applied to the forecast errors of the [[D1:forecast error monitoring framework]]: $$\\begin{aligned} Q_\\mu(m,k) &= \\sum_{t=m+1}^{m+k} e_t - \\frac{k}{m} \\sum_{t=1}^{m} e_t \\\\ D_\\mu(m,k) &= \\max_{0 \\le i \\le k} \\left| Q_\\mu(m,k) - Q_\\mu(m,i) \\right| \\end{aligned}$$ with stopping time $$\\tau_{\\mu,m} = \\min\\{k \\ge 1 : D_\\mu(m,k) \\ge \\hat{\\sigma}_{\\mu,m} c_{\\mu,\\alpha} g(m,k,\\gamma)\\},$$ where $\\hat{\\sigma}_{\\mu,m}$ is the standard deviation of the forecast errors in the Phase I sample. The weight function $g(m,k,\\gamma)$ is that of [[D24:Page's CUSUM detector]], and the critical constant $c_{\\mu,\\alpha}$ comes from the limit distribution of the detector under the null hypothesis.",
   "part_of": [
    "D1"
   ],
   "source_spans": [
    "p4.5",
    "p4.7",
    "p4.9",
    "s2.6"
   ],
   "source": "B11"
  },
  {
   "id": "D3",
   "term": "squared-error detector $D_\\xi(m,k)$",
   "kind": "method",
   "definition": "[[D24:Page's CUSUM detector]] applied to the centred squared forecast errors of the [[D1:forecast error monitoring framework]]: $$\\begin{aligned} Q_\\xi(m,k) &= \\sum_{t=m+1}^{m+k} (e_t - \\hat{b}_{\\mu,m})^2 - \\frac{k}{m} \\sum_{t=1}^{m} (e_t - \\hat{b}_{\\mu,m})^2 \\\\ D_\\xi(m,k) &= \\max_{0 \\le i \\le k} \\left| Q_\\xi(m,k) - Q_\\xi(m,i) \\right| \\end{aligned}$$ with stopping time $$\\tau_{\\xi,m} = \\min\\{k \\ge 1 : D_\\xi(m,k) \\ge \\hat{\\sigma}_{\\xi,m} c_{\\xi,\\alpha} g(m,k,\\gamma)\\},$$ where $\\hat{b}_{\\mu,m}$ is the mean of the forecast errors in the $m$ Phase I samples and $\\hat{\\sigma}_{\\xi,m}$ the standard deviation of the centred squared forecast errors in that sample. $g(m,k,\\gamma)$ is that of [[D24:Page's CUSUM detector]] and $c_{\\xi,\\alpha}$ comes from the limit distribution of the detector under the null hypothesis. The paper prints the second sum of $Q_\\xi(m,k)$ without the square, a typographical error that the guarantor, an author of the paper, has confirmed.",
   "part_of": [
    "D1"
   ],
   "source_spans": [
    "p5.3",
    "p5.5",
    "p5.6"
   ]
  },
  {
   "id": "D8",
   "term": "mean change model",
   "kind": "data",
   "definition": "A data-generating process with a changepoint at an unknown location $k^*$: $$\\begin{aligned} Y_t | y_{t-1}, \\ldots, y_1 &\\sim F, \\quad t = 1, \\ldots, m + k^* \\\\ Y_t | y_{t-1}, \\ldots, y_1 &\\sim G, \\quad t = m + k^* + 1, m + k^* + 2, \\ldots \\end{aligned}$$ where $F$ and $G$ differ in expectation by $\\delta_{\\mu,m} \\ne 0$. The raw data may be temporally dependent, trended, seasonal or depend on exogenous variables.",
   "source_spans": [
    "p3.8",
    "p3.9",
    "p3.10",
    "p3.11",
    "p3.12"
   ]
  },
  {
   "id": "D9",
   "term": "mean and variance change model",
   "kind": "data",
   "definition": "The [[D8:mean change model]] extended so that the pre-change distribution $F$ and the post-change distribution $G'$ differ in expectation by $\\delta_{\\mu,m}$ and in variance by $\\delta_{\\xi,m}$ (the paper's Definition 1).",
   "broader": [
    "D8"
   ],
   "source_spans": [
    "p4.13",
    "p4.14",
    "p4.15",
    "p4.16"
   ]
  },
  {
   "id": "D19",
   "term": "Royal Mail forecast errors",
   "kind": "data",
   "definition": "Forecast errors of the daily number of parcels to be delivered from one Royal Mail delivery office, from 1 July 2020 to 31 December 2020. The underlying data and forecasts are not known to the authors. July to October is the Phase I sample; monitoring starts on 1 November 2020.",
   "source_spans": [
    "p8.14",
    "p9.10"
   ]
  },
  {
   "id": "D23",
   "term": "date of detection",
   "kind": "variable",
   "definition": "The date of the observation at which a detector first crosses its threshold.",
   "source_spans": [
    "p9.12"
   ]
  },
  {
   "id": "D24",
   "term": "Page's CUSUM detector $D(m,k)$",
   "kind": "method",
   "definition": "The sequential detector of a mean change in a series $Y_t$ that follows the change in mean model $$Y_t = \\begin{cases} \\lambda + \\epsilon_t, & t = 1, \\ldots, m + k^* \\\\ \\lambda + \\epsilon_t + \\Delta_m, & t = m + k^* + 1, m + k^* + 2, \\ldots \\end{cases}$$ with zero-mean variables $\\epsilon_t$, a training period of length $m$ without changepoints and a change of size $\\Delta_m$ at time $k^*$ after it. With $$\\begin{aligned} Q(m,k) &= \\sum_{t=m+1}^{m+k} Y_t - \\frac{k}{m} \\sum_{t=1}^{m} Y_t \\\\ D(m,k) &= \\max_{0 \\le i \\le k} \\left| Q(m,k) - Q(m,i) \\right| \\end{aligned}$$ the null hypothesis $H_0: \\Delta_m = 0$ is rejected in favour of $H_A: \\Delta_m \\ne 0$ at the stopping time $$\\tau_m = \\min\\{k \\ge 1 : D(m,k) \\ge \\hat{\\sigma}_m c_\\alpha g(m,k,\\gamma)\\},$$ where $\\hat{\\sigma}_m$ is a weakly consistent estimator, from the $m$ training samples, of the constant $\\sigma$ of the weak invariance principle, the weight function is $$g(m,k,\\gamma) = \\sqrt{m}\\left(1 + \\frac{k}{m}\\right)\\left(\\frac{k}{m+k}\\right)^{\\gamma}, \\quad \\gamma \\in [0, 1/2).$$ and the critical value $c_\\alpha$ is taken from the limit distribution under $H_0$, so that $\\lim_{m\\to\\infty} P(\\tau_m < \\infty) = \\alpha$ under $H_0$ for the chosen false alarm rate $\\alpha$ and $\\lim_{m\\to\\infty} P(\\tau_m < \\infty) = 1$ under $H_A$. All limits are for $m \\to \\infty$; the model is trained once and not re-trained during monitoring. The paper takes $\\gamma = 0$.",
   "source": "B10",
   "source_spans": [
    "s1.5",
    "s1.6-s1.7",
    "s2.3-s2.4",
    "s2.5",
    "s2.6",
    "s2.7-s2.10",
    "s3.1"
   ]
  }
 ],
 "evidence": [
  {
   "id": "E1",
   "profile": "formal",
   "description": "Derivation of the mean change in the forecast errors from the two forecasting-model assumptions (Section 2.1 of the paper).",
   "foundation": "Kolmogorov probability axioms",
   "assumptions": [
    "C18",
    "C19"
   ],
   "steps": [
    {
     "id": "E1.1",
     "statement": "Before the changepoint, $$\\mathbb{E}_F[e_t] = \\mathbb{E}_F[Y_t|\\tilde{y}_t] - \\mathbb{E}_{\\mathcal{F}}[Y_t|\\tilde{y}_t] = b_\\mu,$$ so $e_t = b_\\mu + \\epsilon_{\\mu,t}$ with $\\epsilon_{\\mu,t}$ of zero mean.",
     "by": [
      "D1",
      "C18"
     ],
     "rule": "definition of the forecast error"
    },
    {
     "id": "E1.2",
     "statement": "After the changepoint, $\\mathbb{E}_G[e_t] = b_\\mu + \\Delta_{\\mu,m}$, so $e_t = b_\\mu + \\epsilon_{\\mu,t} + \\Delta_{\\mu,m}$ with $\\Delta_{\\mu,m} \\ne 0$.",
     "by": [
      "E1.1",
      "C19"
     ],
     "rule": "substitution"
    }
   ],
   "source_spans": [
    "p3.13",
    "p4.2",
    "p4.3"
   ],
   "availability": {
    "level": "public"
   }
  },
  {
   "id": "E2",
   "profile": "formal",
   "description": "Proof of Proposition 1 of the paper: the change size of the centred squared forecast errors.",
   "foundation": "Kolmogorov probability axioms",
   "assumptions": [
    "C18",
    "C19"
   ],
   "steps": [
    {
     "id": "E2.1",
     "statement": "$$\\Delta_{\\xi,m} = \\mathbb{E}_{G'}[(e_t - b_\\mu)^2] - \\mathbb{E}_F[(e_t - b_\\mu)^2],$$ the difference between the post-change and pre-change expectations of the centred squared forecast errors.",
     "by": [
      "D9"
     ],
     "rule": "definition"
    },
    {
     "id": "E2.2",
     "statement": "With $\\delta_{\\mu,m} = 0$, substituting $e_t = Y_t - \\mathbb{E}_{\\mathcal{F}}[Y_t]$ and $\\mathbb{E}_{\\mathcal{F}}[Y_t] = \\mathbb{E}_F[Y_t] - b_\\mu$ gives $\\Delta_{\\xi,m} = \\mathrm{Var}_{G'}[Y_t] - \\mathrm{Var}_F[Y_t]$.",
     "by": [
      "E2.1",
      "C18"
     ],
     "rule": "substitution"
    },
    {
     "id": "E2.3",
     "statement": "With $\\delta_{\\mu,m} \\ne 0$, substituting $\\mathbb{E}_{\\mathcal{F}}[Y_t] = \\mathbb{E}_G[Y_t] - b_\\mu - \\Delta_{\\mu,m}$ after the change gives $\\Delta_{\\xi,m} = \\mathrm{Var}_G[Y_t] - \\mathrm{Var}_F[Y_t] + \\Delta_{\\mu,m}^2$.",
     "by": [
      "E2.1",
      "C18",
      "C19"
     ],
     "rule": "substitution"
    }
   ],
   "source_spans": [
    "p4.22",
    "p5.1",
    "p5.2"
   ],
   "availability": {
    "level": "public"
   }
  },
  {
   "id": "E3",
   "profile": "formal",
   "description": "Corollary 1 and Proposition 2 of the paper: the limit distributions of the squared-error detector follow from Proposition 1 and from Theorems 1 and 2 of the paper's Supporting Information, the limit theory of Page's CUSUM.",
   "foundation": "Kolmogorov probability axioms",
   "assumptions": [
    "C18",
    "C19",
    "C20",
    "C21"
   ],
   "steps": [
    {
     "id": "E3.1",
     "statement": "The centred squared forecast errors follow a mean change model with change size $\\Delta_{\\xi,m}$ and zero-mean errors $\\epsilon_{\\xi,t}$.",
     "by": [
      "C2",
      "C20"
     ],
     "rule": "Proposition 1"
    },
    {
     "id": "E3.2",
     "statement": "Under the null hypothesis the limit distribution of Theorem 1 of the Supporting Information holds with $D(m,k) = D_\\xi(m,k)$ and $\\hat{\\sigma}_m = \\hat{\\sigma}_{\\xi,m}$.",
     "by": [
      "E3.1",
      "C24"
     ],
     "rule": "Theorem 1 of the Supporting Information"
    },
    {
     "id": "E3.3",
     "statement": "Under the alternative, with $\\Delta_{\\xi,m}$ satisfying Assumptions C to E, the limit distribution of Theorem 2 of the Supporting Information holds with $\\tau_m = \\tau_{\\xi,m}$ and $c_\\alpha = c_{\\xi,\\alpha}$.",
     "by": [
      "E3.1",
      "C21",
      "C25"
     ],
     "rule": "Theorem 2 of the Supporting Information"
    }
   ],
   "source_spans": [
    "p5.7",
    "p5.9",
    "p5.10"
   ],
   "availability": {
    "level": "public"
   }
  },
  {
   "id": "E9",
   "profile": "quant",
   "description": "Both detectors applied sequentially to the forecast errors of daily parcel volumes of one Royal Mail delivery office, with July to October 2020 as the Phase I sample and monitoring from 1 November 2020, using the theoretical thresholds (Section 5.1 and Figure 5 of the paper).",
   "setup": {
    "studied": [
     "D2",
     "D3"
    ],
    "cases": [
     "D19"
    ],
    "procedure": [
     "D1"
    ],
    "lens": [
     "D23"
    ]
   },
   "design": "other",
   "data_origin": "secondary",
   "preregistered": false,
   "artefacts": [
    "A5"
   ],
   "results": [
    {
     "id": "R1",
     "variable": "D23",
     "value": "2020-12-02",
     "conditions": {
      "detector": "D2"
     }
    },
    {
     "id": "R2",
     "variable": "D23",
     "value": "2020-12-02",
     "conditions": {
      "detector": "D3"
     }
    }
   ],
   "source_spans": [
    "p9.10",
    "p9.11",
    "p9.12",
    "p10.5"
   ],
   "availability": {
    "level": "unavailable",
    "reason": "commercial"
   }
  },
  {
   "id": "E11",
   "profile": "formal",
   "foundation": "Kolmogorov probability axioms",
   "description": "Corollaries 1 and 2 of the paper's Supporting Information: the limit distributions of the mean detector follow from the mean change in the forecast errors and from Theorems 1 and 2.",
   "assumptions": [
    "C18",
    "C19",
    "C20",
    "C21"
   ],
   "steps": [
    {
     "id": "E11.1",
     "statement": "The forecast errors follow the change in mean model of [[D24:Page's CUSUM detector]] with $\\lambda = b_\\mu$, $\\Delta_m = \\Delta_{\\mu,m}$ and zero-mean errors $\\epsilon_{\\mu,t}$.",
     "by": [
      "C1",
      "C20"
     ],
     "rule": "substitution"
    },
    {
     "id": "E11.2",
     "statement": "Under the null hypothesis the limit distribution of Theorem 1 holds with $D(m,k) = D_\\mu(m,k)$ and $\\hat{\\sigma}_m = \\hat{\\sigma}_{\\mu,m}$.",
     "by": [
      "E11.1",
      "C24"
     ],
     "rule": "Theorem 1 of the Supporting Information"
    },
    {
     "id": "E11.3",
     "statement": "Under the alternative, with $\\Delta_{\\mu,m}$ satisfying Assumptions C to E, the limit distribution of Theorem 2 holds with $\\tau_m = \\tau_{\\mu,m}$ and $c_\\alpha = c_{\\mu,\\alpha}$.",
     "by": [
      "E11.1",
      "C21",
      "C25"
     ],
     "rule": "Theorem 2 of the Supporting Information"
    }
   ],
   "source_spans": [
    "s4.11-s4.12",
    "s4.14-s4.15"
   ],
   "availability": {
    "level": "public",
    "comment": "in the paper's Supporting Information"
   }
  }
 ],
 "limitations": [
  {
   "id": "L1",
   "category": "model_specification",
   "statement": "Forecasting models that adapt to a change in mean, such as ETS models with a positive smoothing parameter, do not satisfy the incomplete-adjustment assumption, so a mean change does not stay in their forecast errors.",
   "affects": [
    "C1"
   ],
   "impact": "reduces_transferability",
   "mitigation": "none",
   "source_spans": [
    "p6.12",
    "p7.5",
    "p7.6"
   ]
  },
  {
   "id": "L4",
   "category": "measurement",
   "statement": "The start of the Christmas peak season at the delivery office is inferred from the rise in the mean and variance of the forecast errors; the underlying parcel volumes and forecasts are not known to the authors.",
   "affects": [
    "C6"
   ],
   "impact": "reduces_precision",
   "mitigation": "none",
   "source_spans": [
    "p8.14",
    "p9.12"
   ]
  },
  {
   "id": "L5",
   "category": "sample_size",
   "statement": "The application uses one monthly series with a Phase I sample of 36 observations.",
   "affects": [
    "C13",
    "C14",
    "C7"
   ],
   "impact": "reduces_transferability",
   "mitigation": "none",
   "source_spans": [
    "p11.6"
   ]
  }
 ],
 "artefacts": [
  {
   "id": "A5",
   "kind": "figure",
   "function": "data_plot",
   "reference": "B7",
   "locator": "Figure 5",
   "description": "Forecast errors of the number of parcels to be delivered from one Royal Mail delivery office in 2020, and the two detectors with their thresholds.",
   "availability": {
    "level": "public"
   },
   "source_spans": [
    "p10.5"
   ],
   "shows": [
    "E9.R1",
    "E9.R2"
   ]
  }
 ],
 "references": [
  {
   "id": "B7",
   "target": "10.1111/jtsa.12843",
   "id_type": "doi"
  },
  {
   "id": "B8",
   "target": "10.1080/02331888.2013.870568",
   "id_type": "doi"
  },
  {
   "id": "B9",
   "target": "10.1016/j.jspi.2013.09.001",
   "id_type": "doi"
  },
  {
   "id": "B10",
   "target": "10.2307/2171955",
   "id_type": "doi"
  },
  {
   "id": "B11",
   "target": "10.1093/biomet/41.1-2.100",
   "id_type": "doi"
  }
 ]
}
