You are reading a research paper published on EntryLib as separate claims, each with its evidence. How to read this page

Version 1 · 16 November 2026 · Management Science and Operations Research

Online Detection of Forecast Model Inadequacies Using Forecast Errors

Responsible researcher: Ivan Svetunkov (0000-0001-7826-0281)

Authors of the original paper: PersonThomas Grundy (0000-0003-1723-3217), PersonRebecca Killick (0000-0003-0583-3960), PersonIvan Svetunkov (0000-0001-7826-0281)

CC-BY-4.0 · Keywords: sequential changepoint detection, forecast monitoring, forecast errors, CUSUM, structural break, concept drift

Contributions
  • Claude · claimset preparation (AI agent): Writing – original draft

Converted by its authors from their paper: The published article, Journal of Time Series Analysis 47, 715-726 (open access, CC BY)..

Text file for AI

Text file: the whole paper as plain text, to attach when your assistant cannot open the address.

The message

Read https://entrylib.org/p/[email protected], a research paper written as claims with their evidence. Give me a brief summary of its main findings and how well each is supported. If you cannot open the link, tell me instead of guessing. Then wait for my questions.

Summary

Forecasting models go wrong when the process they forecast changes, and the more complex the model, the harder the change is to spot directly. This paper proposes watching the model's one-step-ahead forecast errors instead, with a sequential changepoint test (Page's CUSUM), whatever the forecasting method. It shows that, when the model keeps a constant bias and cannot fully adapt, a change in the mean or variance of the data appears as a change in the mean of the errors or of their squares, with known limit distributions for the test. Exponential smoothing adapts to mean changes, so with it only variance changes stay visible. In simulations of seasonal data, monitoring forecast errors found changes sooner than monitoring the raw data, with few false alarms. Two examples follow: Royal Mail parcel forecasts and hospital admissions for gallstones.

Highlights

  • SupportS 70Under the mean change model, the one-step-ahead forecast errors of a forecasting model that satisfies the two forecasting-model assumptions have a change in mean of size Δμ,m≠0\Delta_{\mu,m} \ne 0 at the changepoint: et=bμ+ϵμ,te_t = b_\mu + \epsilon_{\mu,t} for t≤m+k*t \le m + k^* and et=bμ+ϵμ,t+Δμ,me_t = b_\mu + \epsilon_{\mu,t} + \Delta_{\mu,m} afterwards. C1
  • SupportS 71Under the mean and variance change model and the constant-bias assumption, the change in the mean of the centred squared forecast errors is Δξ,m=VarG′[Yt|y~t]−VarF[Yt|y~t]\Delta_{\xi,m} = \mathrm{Var}_{G'}[Y_t|\tilde{y}_t] - \mathrm{Var}_F[Y_t|\tilde{y}_t] when δμ,m=0\delta_{\mu,m} = 0, and Δξ,m=VarG′[Yt|y~t]−VarF[Yt|y~t]+Δμ,m2\Delta_{\xi,m} = \mathrm{Var}_{G'}[Y_t|\tilde{y}_t] - \mathrm{Var}_F[Y_t|\tilde{y}_t] + \Delta_{\mu,m}^2 when δμ,m≠0\delta_{\mu,m} \ne 0 and the incomplete-adjustment assumption holds. C2
  • SupportS 71On the seasonal mean change scenario, the seasonal trend change scenario and the autoregressive parameter change scenarios, monitoring forecast errors with the forecast error monitoring framework detected the changes with a shorter average detection delay (ADD) than Raw CUSUM, at a low false detection probability (FDP). C5
  • SupportS 75For an ETS(A,N,N) forecasting model under the mean change model with pre-change expectation λ\lambda, the expected forecast error ss steps after the changepoint is 𝔼[ek*+s]=δμ,m\mathbb{E}[e_{k^*+s}] = \delta_{\mu,m} for s=1s = 1 and 𝔼[ek*+s]=(λ+δμ,m)(1−∑i=0s−2α(1−α)i)−λ(α(1−α)s−1−(1−α)s)\mathbb{E}[e_{k^*+s}] = (\lambda + \delta_{\mu,m})\left(1 - \sum_{i=0}^{s-2} \alpha(1-\alpha)^i\right) - \lambda\left(\alpha(1-\alpha)^{s-1} - (1-\alpha)^s\right) for s=2,3,…s = 2, 3, \ldots, an expectation that returns to zero as ss grows when α>0\alpha > 0 and does not satisfy the incomplete-adjustment assumption unless α=0\alpha = 0. C4

Context

Extends Aue et al. (2015) · from C1
Aue et al. (2015) monitor the residuals of ARMA models only; the claim covers the forecast errors of any forecasting model that satisfies the two assumptions.
Extends Aue et al. (2015) · from C2
Aue et al. (2015) monitor the residuals of ARMA models only; the claim covers the forecast errors of any forecasting model that satisfies the two assumptions.
Extends Aue et al. (2015) · from C3
Aue et al. (2015) monitor the residuals of ARMA models only; the claim covers the forecast errors of any forecasting model that satisfies the two assumptions.
Addresses Aue et al. (2015) · from C4
The closest result found: Aue et al. (2015) give the change size in ARMA residuals; the claim gives it for an exponential smoothing forecasting model, where it vanishes.
Addresses Robbins et al. (2011) · from C5
The closest result found: Robbins et al. (2011) showed, for offline changepoint detection, that using residuals within the CUSUM procedure improves on the raw data under temporal dependence; the claim concerns sequential detection with forecast errors of any forecasting model.
Addresses Aue et al. (2015) · from C6
The closest result found: Aue et al. (2015) apply residual-based monitoring of ARMA models to EEG and IBM data; here only the forecast errors are known, not the model or the data.
Addresses Taib et al. (2021) · own earlier work · from C7
Taib et al. (2021) found the changepoint retrospectively, with all the data from 2010 to 2020; the claim is about detecting it sequentially.
Addresses Robbins et al. (2011) · from C8
Robbins et al. (2011) compared residuals with raw data for offline mean shift testing; this is the sequential case.
Addresses Horváth et al. (2004) · from C9
Horváth et al. (2004) monitor changes in linear models, including trend, within a specified model; here the trend change is detected through forecast errors.
Addresses Aue et al. (2015) · from C10
Aue et al. (2015) detect changes in ARMA parameters from ARMA residuals; here the comparison is with monitoring the raw data, and includes an ETS forecasting model.
Extends Aue et al. (2015) · from C23
Aue et al. (2015) give the limit distributions for the residuals of ARMA models; the claim gives them for the forecast errors of any forecasting model that satisfies the two assumptions.
Addresses Inclán and Tiao (1994) · from C17
The closest result found: Inclán and Tiao (1994) detect variance changes from the centred squared data; the claim applies the idea to the squared forecast errors of an ETS model.

Definitions

Methods 11

D1 forecast error monitoring framework: A framework that monitors a forecasting model online by applying sequential changepoint detection to its one-step-ahead forecast errors. For a univariate series YtY_t and a forecasting model ℱ\mathcal{F} with one-step-ahead point forecast y^t(1)=𝔼ℱ[Yt|yt−1,…,y1]\hat{y}_t(1) = \mathbb{E}_{\mathcal{F}}[Y_t | y_{t-1}, \ldots, y_1], the forecast errors are et=Yt−y^t(1).e_t = Y_t - \hat{y}_t(1). The first mm forecast errors are the Phase I (training) sample; monitoring starts at t=m+1t = m + 1 and stops when a detector, the mean detector or the squared-error detector, crosses its threshold. Any forecasting approach can supply the errors: model-based, machine learning or expert judgement.

D2 · part of D1 mean detector $D_\mu(m,k)$: Page's CUSUM detector applied to the forecast errors of the forecast error monitoring framework: Qμ(m,k)=∑t=m+1m+ket−km∑t=1metDμ(m,k)=max0≤i≤k⁡|Qμ(m,k)−Qμ(m,i)|\begin{aligned} Q_\mu(m,k) &= \sum_{t=m+1}^{m+k} e_t - \frac{k}{m} \sum_{t=1}^{m} e_t \\ D_\mu(m,k) &= \max_{0 \le i \le k} \left| Q_\mu(m,k) - Q_\mu(m,i) \right| \end{aligned} with stopping time τμ,m=min⁡{k≥1:Dμ(m,k)≥σ^μ,mcμ,αg(m,k,γ)},\tau_{\mu,m} = \min\{k \ge 1 : D_\mu(m,k) \ge \hat{\sigma}_{\mu,m} c_{\mu,\alpha} g(m,k,\gamma)\}, where σ^μ,m\hat{\sigma}_{\mu,m} is the standard deviation of the forecast errors in the Phase I sample. The weight function g(m,k,γ)g(m,k,\gamma) is that of Page's CUSUM detector, and the critical constant cμ,αc_{\mu,\alpha} comes from the limit distribution of the detector under the null hypothesis. Source: PAGE (1954).

D3 · part of D1 squared-error detector $D_\xi(m,k)$: Page's CUSUM detector applied to the centred squared forecast errors of the forecast error monitoring framework: Qξ(m,k)=∑t=m+1m+k(et−b^μ,m)2−km∑t=1m(et−b^μ,m)2Dξ(m,k)=max0≤i≤k⁡|Qξ(m,k)−Qξ(m,i)|\begin{aligned} Q_\xi(m,k) &= \sum_{t=m+1}^{m+k} (e_t - \hat{b}_{\mu,m})^2 - \frac{k}{m} \sum_{t=1}^{m} (e_t - \hat{b}_{\mu,m})^2 \\ D_\xi(m,k) &= \max_{0 \le i \le k} \left| Q_\xi(m,k) - Q_\xi(m,i) \right| \end{aligned} with stopping time τξ,m=min⁡{k≥1:Dξ(m,k)≥σ^ξ,mcξ,αg(m,k,γ)},\tau_{\xi,m} = \min\{k \ge 1 : D_\xi(m,k) \ge \hat{\sigma}_{\xi,m} c_{\xi,\alpha} g(m,k,\gamma)\}, where b^μ,m\hat{b}_{\mu,m} is the mean of the forecast errors in the mm Phase I samples and σ^ξ,m\hat{\sigma}_{\xi,m} the standard deviation of the centred squared forecast errors in that sample. g(m,k,γ)g(m,k,\gamma) is that of Page's CUSUM detector and cξ,αc_{\xi,\alpha} comes from the limit distribution of the detector under the null hypothesis. The paper prints the second sum of Qξ(m,k)Q_\xi(m,k) without the square, a typographical error that the guarantor, an author of the paper, has confirmed.

D5 · a kind of D1 ARMA Forecast Errors: The detector of the forecast error monitoring framework, [[D2:Dμ(m,k)D_\mu(m,k)]] or [[D3:Dξ(m,k)D_\xi(m,k)]] by the type of change, applied to the forecast errors of an ARMA forecasting model fitted with the forecast R package. For seasonal data the model is a seasonal ARMA model chosen by auto.arima using AIC, with no differencing. The standard deviation is the traditional estimator, the errors being taken as i.i.d. Source: https://CRAN.R-project.org/package=forecast.

D6 · a kind of D1 ETS Forecast Errors: The detector of the forecast error monitoring framework, [[D2:Dμ(m,k)D_\mu(m,k)]] or [[D3:Dξ(m,k)D_\xi(m,k)]] by the type of change, applied to the forecast errors of an ETS forecasting model fitted with the es function of the smooth R package. The standard deviation is the traditional estimator, the errors being taken as i.i.d. Source: https://CRAN.R-project.org/package=smooth.

D4 Raw CUSUM: Page's CUSUM detector applied to the raw data instead of forecast errors; for variance changes, to the centred squared raw data. In the paper's simulations its standard deviation σ^m\hat{\sigma}_m is the Bartlett long run variance estimator from the sandwich R package. Source: Inclán and Tiao (1994).

D7 compared detectors: The three detectors compared in the paper's simulations. Members: D4, D5, D6.

D10 ETS(A,N,N) forecasting model: The local level exponential smoothing model with additive error, no trend and no seasonality, whose one-step-ahead forecasts follow y^t(1)=y^t−1(1)+αet−1,\hat{y}_t(1) = \hat{y}_{t-1}(1) + \alpha e_{t-1}, where et−1e_{t-1} is the forecast error at time t−1t-1 and 0<α<10 < \alpha < 1 is the smoothing parameter.

D11 ARMA forecasting model: A forecasting model for the causal and invertible ARMA(pp,qq) process ϕ(B)(Yt−λt)=θ(B)ϵt,\phi(B)(Y_t - \lambda_t) = \theta(B)\epsilon_t, with mean parameters λt\lambda_t, autoregressive polynomial ϕ(B)=1−ϕ1B−⋯−ϕpBp\phi(B) = 1 - \phi_1 B - \cdots - \phi_p B^p, moving average polynomial θ(B)=1+θ1B+⋯+θqBq\theta(B) = 1 + \theta_1 B + \cdots + \theta_q B^q and independent zero-mean errors ϵt\epsilon_t with variance σt2\sigma_t^2.

D21 model-based sequential changepoint detection: Sequential changepoint detection that fits the model to the data before and after a potential changepoint and decides by a likelihood-ratio style statistic.

D24 Page's CUSUM detector $D(m,k)$: The sequential detector of a mean change in a series YtY_t that follows the change in mean model Yt={λ+ϵt,t=1,…,m+k*λ+ϵt+Δm,t=m+k*+1,m+k*+2,…Y_t = \begin{cases} \lambda + \epsilon_t, & t = 1, \ldots, m + k^* \\ \lambda + \epsilon_t + \Delta_m, & t = m + k^* + 1, m + k^* + 2, \ldots \end{cases} with zero-mean variables ϵt\epsilon_t, a training period of length mm without changepoints and a change of size Δm\Delta_m at time k*k^* after it. With Q(m,k)=∑t=m+1m+kYt−km∑t=1mYtD(m,k)=max0≤i≤k⁡|Q(m,k)−Q(m,i)|\begin{aligned} Q(m,k) &= \sum_{t=m+1}^{m+k} Y_t - \frac{k}{m} \sum_{t=1}^{m} Y_t \\ D(m,k) &= \max_{0 \le i \le k} \left| Q(m,k) - Q(m,i) \right| \end{aligned} the null hypothesis H0:Δm=0H_0: \Delta_m = 0 is rejected in favour of HA:Δm≠0H_A: \Delta_m \ne 0 at the stopping time τm=min⁡{k≥1:D(m,k)≥σ^mcαg(m,k,γ)},\tau_m = \min\{k \ge 1 : D(m,k) \ge \hat{\sigma}_m c_\alpha g(m,k,\gamma)\}, where σ^m\hat{\sigma}_m is a weakly consistent estimator, from the mm training samples, of the constant σ\sigma of the weak invariance principle, the weight function is g(m,k,γ)=m(1+km)(km+k)γ,γ∈[0,1/2).g(m,k,\gamma) = \sqrt{m}\left(1 + \frac{k}{m}\right)\left(\frac{k}{m+k}\right)^{\gamma}, \quad \gamma \in [0, 1/2). and the critical value cαc_\alpha is taken from the limit distribution under H0H_0, so that limm→∞⁡P(τm<∞)=α\lim_{m\to\infty} P(\tau_m < \infty) = \alpha under H0H_0 for the chosen false alarm rate α\alpha and limm→∞⁡P(τm<∞)=1\lim_{m\to\infty} P(\tau_m < \infty) = 1 under HAH_A. All limits are for m→∞m \to \infty; the model is trained once and not re-trained during monitoring. The paper takes γ=0\gamma = 0. Source: Chu et al. (1996).

Data 7

D8 mean change model: A data-generating process with a changepoint at an unknown location k*k^*: Yt|yt−1,…,y1∼F,t=1,…,m+k*Yt|yt−1,…,y1∼G,t=m+k*+1,m+k*+2,…\begin{aligned} Y_t | y_{t-1}, \ldots, y_1 &\sim F, \quad t = 1, \ldots, m + k^* \\ Y_t | y_{t-1}, \ldots, y_1 &\sim G, \quad t = m + k^* + 1, m + k^* + 2, \ldots \end{aligned} where FF and GG differ in expectation by δμ,m≠0\delta_{\mu,m} \ne 0. The raw data may be temporally dependent, trended, seasonal or depend on exogenous variables.

D9 · a kind of D8 mean and variance change model: The mean change model extended so that the pre-change distribution FF and the post-change distribution G′G' differ in expectation by δμ,m\delta_{\mu,m} and in variance by δξ,m\delta_{\xi,m} (the paper's Definition 1).

D16 seasonal mean change scenario: Simulated data whose errors follow an ARMA(2,1) model with ϕ=(−0.6,0.3)\phi = (-0.6, 0.3), θ=−0.3\theta = -0.3, λ=0\lambda = 0 and σ2=1\sigma^2 = 1, with a mean change of size δμ\delta_\mu at k*=100k^* = 100 and a seasonal component of frequency 12 whose seasonal means are 12 equally spaced points of y=10sin⁡(x)y = 10\sin(x), x∈[0,π]x \in [0, \pi].

D17 seasonal trend change scenario: Simulated data yt=1[t>m+k*]βt+ϵty_t = 1[t > m + k^*]\beta t + \epsilon_t whose errors follow an ARMA(1,1) model with ϕ=0.2\phi = 0.2, θ=0.2\theta = 0.2, λ=0\lambda = 0 and σ2=1\sigma^2 = 1: no trend before the change and a trend of gradient β\beta after it, with a seasonal component of frequency 7 whose seasonal means are 7 equally spaced points of y=10cos⁡(x)y = 10\cos(x), x∈[0,2π]x \in [0, 2\pi].

D18 autoregressive parameter change scenarios: Simulated data with AR(1) errors, λ=0\lambda = 0, whose autoregressive parameter changes at the changepoint from ϕpre=0.2\phi_{\text{pre}} = 0.2 (scenario 1) or ϕpre=0.8\phi_{\text{pre}} = 0.8 (scenario 2) to varying values ϕpost\phi_{\text{post}}, with quarterly seasonality given by quarter means (−2,5,7,−10)(-2, 5, 7, -10). The ETS forecasting model is ETS(A,N,A) with estimated smoothing parameters.

D19 Royal Mail forecast errors: Forecast errors of the daily number of parcels to be delivered from one Royal Mail delivery office, from 1 July 2020 to 31 December 2020. The underlying data and forecasts are not known to the authors. July to October is the Phase I sample; monitoring starts on 1 November 2020.

D20 GRP admissions series: The monthly proportion of A&E admissions related to gallstone-related pathology (GRP) in participating hospitals across England, 2010 to 2020, based on NHS Hospital Episode Statistics. A SARMA(1,0,0)(1,0,0)12_{12} model fitted to the data up to 2013 gives one-step-ahead forecasts for the rest; 2013 to 2016 is the Phase I sample of size m=36m = 36 and monitoring starts in 2016. Source: Taib et al. (2021).

Variables and measures 5

D12 average detection delay (ADD): The mean of the detection delays over the simulated series in which a change was signalled within the length of the data.

D13 detection probability (DP): The proportion of simulated series in which a change was detected within the length of the monitoring data.

D14 false detection probability (FDP): The proportion of simulated series in which a change was detected before the true change had occurred.

D22 detection delay: The number of observations between a change and the time a detector signals it.

D23 date of detection: The date of the observation at which a detector first crosses its threshold.

Study design 1

D15 simulation design of the paper: Each scenario is simulated 1000 times with normally distributed innovations. Forecasting models are trained on the first ntrain=300n_{\text{train}} = 300 observations and then produce one-step-ahead forecasts sequentially. A change is signalled by the theoretical thresholds of Page's CUSUM detector with γ=0\gamma = 0 and a false alarm rate of 0.05.

Assumptions

C18 · theorem_condition · unchecked

The forecasting model has a constant bias before the changepoint: 𝔼F[Yt|y~t]−𝔼ℱ[Yt|y~t]=bμ\mathbb{E}_F[Y_t|\tilde{y}_t] - \mathbb{E}_{\mathcal{F}}[Y_t|\tilde{y}_t] = b_\mu for t=1,…,m+k*t = 1, \ldots, m + k^*, where bμb_\mu is unknown and may be zero (the paper's Assumption 1, the constant-bias assumption). Rested on by C1, C2, C3, C23, C17, E1, E2, E3, E11.

C19 · theorem_condition · unchecked

After the changepoint the forecasting model does not fully adjust to the change: 𝔼G[Yt|y~t]−𝔼ℱ[Yt|y~t]=bμ+Δμ,m\mathbb{E}_G[Y_t|\tilde{y}_t] - \mathbb{E}_{\mathcal{F}}[Y_t|\tilde{y}_t] = b_\mu + \Delta_{\mu,m} for t=m+k*+1,m+k*+2,…t = m + k^* + 1, m + k^* + 2, \ldots, where Δμ,m≠0\Delta_{\mu,m} \ne 0 is a function of the change size in the raw data (the paper's Assumption 2, the incomplete-adjustment assumption). Rested on by C1, C2, C3, C23, E1, E2, E3, E11.

C20 · statistical · unchecked

The zero-mean variables ϵt\epsilon_t of the monitored series, here ϵμ,t\epsilon_{\mu,t} and ϵξ,t\epsilon_{\xi,t} left in the forecast errors after the bias is removed, satisfy the weak invariance principle of Assumptions A and B of the paper's Supporting Information: |∑t=1mϵt|=OP(m)\left|\sum_{t=1}^{m} \epsilon_t\right| = O_P(\sqrt{m}), and there are Wiener processes {Wm(i):i≥0,m≥1}\{W_m(i) : i \ge 0, m \ge 1\} and a constant σ>0\sigma > 0 with sup1/m≤i<∞⁡1(mi)1/ν|∑t=m+1m+miϵt−σWm(mi)|=OP(1)for some ν>2.\sup_{1/m \le i < \infty} \frac{1}{(mi)^{1/\nu}} \left| \sum_{t=m+1}^{m+mi} \epsilon_t - \sigma W_m(mi) \right| = O_P(1) \quad \text{for some } \nu > 2. Rested on by C3, C23, C24, C25, E3, E11.

C21 · theorem_condition · unchecked

The change is large enough and bounded: the changepoint is k*=⌊θmβ⌋k^* = \lfloor \theta m^\beta \rfloor for some θ>0\theta > 0 and 0≤β<10 \le \beta < 1, m|Δm|→∞\sqrt{m}\,|\Delta_m| \to \infty as m→∞m \to \infty, and Δm=O(1)\Delta_m = O(1), where Δm\Delta_m is the change size in the monitored series, here Δμ,m\Delta_{\mu,m} or Δξ,m\Delta_{\xi,m}, as Assumptions C to E of the paper's Supporting Information require. Rested on by C3, C23, C25, E3, E11.

C22 · theorem_condition · unchecked

The ARMA process is causal and invertible, ϕ(x)≠0\phi(x) \ne 0 and θ(x)≠0\theta(x) \ne 0 for all |x|≤1|x| \le 1, and the ARMA forecasting model is perfectly estimated, so that its residuals and forecast errors are identical. Rested on by C15, C16.

Principal claims

SupportS 70C1 · formal · novelty: extends earlier work, by the authors

Under the mean change model, the one-step-ahead forecast errors of a forecasting model that satisfies the two forecasting-model assumptions have a change in mean of size Δμ,m≠0\Delta_{\mu,m} \ne 0 at the changepoint: et=bμ+ϵμ,te_t = b_\mu + \epsilon_{\mu,t} for t≤m+k*t \le m + k^* and et=bμ+ϵμ,t+Δμ,me_t = b_\mu + \epsilon_{\mu,t} + \Delta_{\mu,m} afterwards.

SupportS 71C2 · formal · novelty: extends earlier work, by the authors

Under the mean and variance change model and the constant-bias assumption, the change in the mean of the centred squared forecast errors is Δξ,m=VarG′[Yt|y~t]−VarF[Yt|y~t]\Delta_{\xi,m} = \mathrm{Var}_{G'}[Y_t|\tilde{y}_t] - \mathrm{Var}_F[Y_t|\tilde{y}_t] when δμ,m=0\delta_{\mu,m} = 0, and Δξ,m=VarG′[Yt|y~t]−VarF[Yt|y~t]+Δμ,m2\Delta_{\xi,m} = \mathrm{Var}_{G'}[Y_t|\tilde{y}_t] - \mathrm{Var}_F[Y_t|\tilde{y}_t] + \Delta_{\mu,m}^2 when δμ,m≠0\delta_{\mu,m} \ne 0 and the incomplete-adjustment assumption holds.

SupportS 69C3 · formal · novelty: extends earlier work, by the authors

For the squared-error detector under the mean and variance change model, the limit distribution of Theorem 1 of the paper's Supporting Information holds under the null hypothesis of no change, and the limit distribution of its Theorem 2 holds for the stopping time τξ,m\tau_{\xi,m} under the alternative when the change size Δξ,m\Delta_{\xi,m} is large enough in the sense of that document's Assumptions C to E.

SupportS 75C4 · formal · novelty: new, by the authors

For an ETS(A,N,N) forecasting model under the mean change model with pre-change expectation λ\lambda, the expected forecast error ss steps after the changepoint is 𝔼[ek*+s]=δμ,m\mathbb{E}[e_{k^*+s}] = \delta_{\mu,m} for s=1s = 1 and 𝔼[ek*+s]=(λ+δμ,m)(1−∑i=0s−2α(1−α)i)−λ(α(1−α)s−1−(1−α)s)\mathbb{E}[e_{k^*+s}] = (\lambda + \delta_{\mu,m})\left(1 - \sum_{i=0}^{s-2} \alpha(1-\alpha)^i\right) - \lambda\left(\alpha(1-\alpha)^{s-1} - (1-\alpha)^s\right) for s=2,3,…s = 2, 3, \ldots, an expectation that returns to zero as ss grows when α>0\alpha > 0 and does not satisfy the incomplete-adjustment assumption unless α=0\alpha = 0.

SupportS 71C5 · descriptive · novelty: new, by the authors

On the seasonal mean change scenario, the seasonal trend change scenario and the autoregressive parameter change scenarios, monitoring forecast errors with the forecast error monitoring framework detected the changes with a shorter average detection delay (ADD) than Raw CUSUM, at a low false detection probability (FDP).

SupportS 15C6 · descriptive · novelty: new, by the authors

On the Royal Mail forecast errors, the mean detector and the squared-error detector of the forecast error monitoring framework both signalled a changepoint on 2 December 2020.

SupportS 39C7 · descriptive · novelty: new, by the authors

On the GRP admissions series, both detectors of the forecast error monitoring framework have a shorter detection delay than model-based sequential changepoint detection, whose minimum possible detection delay on data with yearly seasonality is 24 months.

Supporting claims

SupportS 86C8 · descriptive · novelty: new, by the authors

On the seasonal mean change scenario, ARMA Forecast Errors has a high detection probability (DP) and a lower average detection delay (ADD) than Raw CUSUM across all change sizes.

SupportS 86C9 · descriptive · novelty: new, by the authors

On the seasonal trend change scenario, ARMA Forecast Errors and Raw CUSUM both have an extremely high detection probability (DP), and ARMA Forecast Errors has the lower average detection delay (ADD) across all change sizes.

SupportS 78C10 · descriptive · novelty: new, by the authors

On the autoregressive parameter change scenarios, ARMA Forecast Errors and ETS Forecast Errors each give a better detection probability (DP) and average detection delay (ADD) than Raw CUSUM in all scenarios.

SupportS 89C11 · descriptive

On the autoregressive parameter change scenarios, the false detection probability (FDP) is below 0.075 for each of the compared detectors.

SupportS 87C12 · descriptive

On the seasonal mean change scenario and the seasonal trend change scenario, the false detection probability (FDP) of the compared detectors is low across all change sizes.

SupportS 54C13 · descriptive

On the GRP admissions series, the detection delay of the mean detector was 4 data points, counted from the changepoint that the retrospective analysis of Taib et al. (2021) identified.

SupportS 54C14 · descriptive

On the GRP admissions series, the detection delay of the squared-error detector was 2 time points.

SupportS 68C23 · formal · novelty: extends earlier work, by the authors

For the mean detector under the mean change model, the limit distribution of Theorem 1 of the paper's Supporting Information holds under the null hypothesis of no change with D(m,k)=Dμ(m,k)D(m,k) = D_\mu(m,k) and σ^m=σ^μ,m\hat{\sigma}_m = \hat{\sigma}_{\mu,m}, and the limit distribution of its Theorem 2 holds under the alternative with τm=τμ,m\tau_m = \tau_{\mu,m} and cα=cμ,αc_\alpha = c_{\mu,\alpha} when the change size Δμ,m\Delta_{\mu,m} satisfies its Assumptions C to E.

SupportS 82C15 · formal · novelty: summarises earlier work, by the authors

An ARMA forecasting model satisfies the two forecasting-model assumptions for a change of size δμ,m\delta_{\mu,m} in its mean parameter λ\lambda, the change in its forecast errors being Δμ,m=δμ,m(1−∑j=1pϕj)∑l=0∞ψl(θ)\Delta_{\mu,m} = \delta_{\mu,m}\left(1 - \sum_{j=1}^{p}\phi_j\right)\sum_{l=0}^{\infty}\psi_l(\theta) with 1/θ(z)=∑l=0∞ψl(θ)1/\theta(z) = \sum_{l=0}^{\infty}\psi_l(\theta).

SupportS 70C16 · formal · novelty: summarises earlier work, by the authors

For a change of size δξ,m\delta_{\xi,m} in the variance of the data, the change in the squared forecast errors of an ARMA forecasting model is Δξ,m=δξ,m\Delta_{\xi,m} = \delta_{\xi,m}, in agreement with the variance-change formula of the paper's Proposition 1.

SupportS 64C17 · formal · novelty: new, by the authors

A variance change in the raw data produces a mean change in the squared forecast errors of an ETS forecasting model that the squared-error detector can detect.

SupportS 77C24 · formal · novelty: summarises earlier work, by the authors

For Page's CUSUM detector under the null hypothesis of no change, limm→∞⁡P(1σ^msup1≤k<∞⁡D(m,k)g(m,k,γ)≤c)=P(sup0<i<1⁡sup0≤j≤i⁡1iγ|W(i)−1−i1−jW(j)|≤c)\lim_{m\to\infty} P\left(\frac{1}{\hat{\sigma}_m} \sup_{1 \le k < \infty} \frac{D(m,k)}{g(m,k,\gamma)} \le c\right) = P\left(\sup_{0<i<1} \sup_{0 \le j \le i} \frac{1}{i^\gamma} \left| W(i) - \frac{1-i}{1-j} W(j) \right| \le c\right) for every real cc, where {W(i):i∈[0,1]}\{W(i) : i \in [0,1]\} is a standard Brownian motion (Theorem 1 of the paper's Supporting Information, from Fremdt 2015).

SupportS 76C25 · formal · novelty: summarises earlier work, by the authors

For Page's CUSUM detector under the alternative of a mean change, limm→∞⁡P(τm−am(cα)bm(cα)≤x)=Ψ(x)\lim_{m\to\infty} P\left(\frac{\tau_m - a_m(c_\alpha)}{b_m(c_\alpha)} \le x\right) = \Psi(x) for every real xx, where am(c)a_m(c) is the unique solution of am(c)=(σcm1/2−γ|Δm|+k*(am(c))γ)1/(1−γ),a_m(c) = \left(\frac{\sigma c\, m^{1/2-\gamma}}{|\Delta_m|} + \frac{k^*}{(a_m(c))^\gamma}\right)^{1/(1-\gamma)}, bm(c)=σam(c)|Δm|(1−γ(1−k*am(c)))−1,b_m(c) = \frac{\sigma \sqrt{a_m(c)}}{|\Delta_m|} \left(1 - \gamma\left(1 - \frac{k^*}{a_m(c)}\right)\right)^{-1}, and Ψ(x)=1−Ψ~(−x)\Psi(x) = 1 - \tilde{\Psi}(-x) with Ψ~(x)\tilde{\Psi}(x) equal to Φ(x)\Phi(x) when Δ~m=|Δm|mβ(1−γ)−1/2+γ→0\tilde{\Delta}_m = |\Delta_m| m^{\beta(1-\gamma)-1/2+\gamma} \to 0, to P(supd<i<1⁡W(i)≤x)P(\sup_{d<i<1} W(i) \le x) when Δ~m→C~∈(0,∞)\tilde{\Delta}_m \to \tilde{C} \in (0,\infty), and to P(sup0<i<1⁡W(i)≤x)P(\sup_{0<i<1} W(i) \le x) when Δ~m→∞\tilde{\Delta}_m \to \infty, where Φ\Phi is the standard normal distribution function, WW a standard Brownian motion and d=d(c)d = d(c) the unique solution of d=1−cCd1−γd = 1 - \frac{c}{C} d^{1-\gamma} with C=θ1−γC~C = \theta^{1-\gamma}\tilde{C} (Theorem 2 of the paper's Supporting Information, from Fremdt 2014).

Evidence

E1 · formal · cited by C1

Derivation of the mean change in the forecast errors from the two forecasting-model assumptions (Section 2.1 of the paper).

  1. Before the changepoint, 𝔼F[et]=𝔼F[Yt|y~t]−𝔼ℱ[Yt|y~t]=bμ,\mathbb{E}_F[e_t] = \mathbb{E}_F[Y_t|\tilde{y}_t] - \mathbb{E}_{\mathcal{F}}[Y_t|\tilde{y}_t] = b_\mu, so et=bμ+ϵμ,te_t = b_\mu + \epsilon_{\mu,t} with ϵμ,t\epsilon_{\mu,t} of zero mean. (definition of the forecast error; from D1, C18)
  2. After the changepoint, 𝔼G[et]=bμ+Δμ,m\mathbb{E}_G[e_t] = b_\mu + \Delta_{\mu,m}, so et=bμ+ϵμ,t+Δμ,me_t = b_\mu + \epsilon_{\mu,t} + \Delta_{\mu,m} with Δμ,m≠0\Delta_{\mu,m} \ne 0. (substitution; from E1.1, C19)

E2 · formal · cited by C2

Proof of Proposition 1 of the paper: the change size of the centred squared forecast errors.

  1. Δξ,m=𝔼G′[(et−bμ)2]−𝔼F[(et−bμ)2],\Delta_{\xi,m} = \mathbb{E}_{G'}[(e_t - b_\mu)^2] - \mathbb{E}_F[(e_t - b_\mu)^2], the difference between the post-change and pre-change expectations of the centred squared forecast errors. (definition; from D9)
  2. With δμ,m=0\delta_{\mu,m} = 0, substituting et=Yt−𝔼ℱ[Yt]e_t = Y_t - \mathbb{E}_{\mathcal{F}}[Y_t] and 𝔼ℱ[Yt]=𝔼F[Yt]−bμ\mathbb{E}_{\mathcal{F}}[Y_t] = \mathbb{E}_F[Y_t] - b_\mu gives Δξ,m=VarG′[Yt]−VarF[Yt]\Delta_{\xi,m} = \mathrm{Var}_{G'}[Y_t] - \mathrm{Var}_F[Y_t]. (substitution; from E2.1, C18)
  3. With δμ,m≠0\delta_{\mu,m} \ne 0, substituting 𝔼ℱ[Yt]=𝔼G[Yt]−bμ−Δμ,m\mathbb{E}_{\mathcal{F}}[Y_t] = \mathbb{E}_G[Y_t] - b_\mu - \Delta_{\mu,m} after the change gives Δξ,m=VarG[Yt]−VarF[Yt]+Δμ,m2\Delta_{\xi,m} = \mathrm{Var}_G[Y_t] - \mathrm{Var}_F[Y_t] + \Delta_{\mu,m}^2. (substitution; from E2.1, C18, C19)

E3 · formal · cited by C3

Corollary 1 and Proposition 2 of the paper: the limit distributions of the squared-error detector follow from Proposition 1 and from Theorems 1 and 2 of the paper's Supporting Information, the limit theory of Page's CUSUM.

  1. The centred squared forecast errors follow a mean change model with change size Δξ,m\Delta_{\xi,m} and zero-mean errors ϵξ,t\epsilon_{\xi,t}. (Proposition 1; from C2, C20)
  2. Under the null hypothesis the limit distribution of Theorem 1 of the Supporting Information holds with D(m,k)=Dξ(m,k)D(m,k) = D_\xi(m,k) and σ^m=σ^ξ,m\hat{\sigma}_m = \hat{\sigma}_{\xi,m}. (Theorem 1 of the Supporting Information; from E3.1, C24)
  3. Under the alternative, with Δξ,m\Delta_{\xi,m} satisfying Assumptions C to E, the limit distribution of Theorem 2 of the Supporting Information holds with τm=τξ,m\tau_m = \tau_{\xi,m} and cα=cξ,αc_\alpha = c_{\xi,\alpha}. (Theorem 2 of the Supporting Information; from E3.1, C21, C25)

E4 · formal · cited by C4

Proof of Proposition 3 of the paper: the expected forecast errors of the ETS(A,N,N) forecasting model after a mean change.

  1. The forecast error after the changepoint is written by repeated substitution of the previous forecast errors into y^t(1)=y^t−1(1)+αet−1\hat{y}_t(1) = \hat{y}_{t-1}(1) + \alpha e_{t-1} until the pre-change regime is reached, which gives the stated expectation. (repeated substitution; from D10, D8)
  2. As ss increases the expectation returns to zero when α>0\alpha > 0, and faster for larger α\alpha, so no constant change Δμ,m≠0\Delta_{\mu,m} \ne 0 remains. (limit of a geometric sum; from E4.1)

E5 · simulation · cited by C5, C8, C9, C12

Simulation of a change in mean and a change in trend in seasonal data (Section 4.1 and Figure 3 of the paper): detection probability, false detection probability and average detection delay of the three detectors, 1000 repetitions per change size. The paper reports the values as a figure only.

E6 · simulation · cited by C5, C10, C11

Simulation of a change in the autoregressive parameter of AR(1) errors in quarterly seasonal data (Section 4.2 and Figure 4 of the paper), for pre-change parameters 0.2 and 0.8: detection probability, false detection probability and average detection delay of the three detectors, 1000 repetitions per scenario. The paper reports the values as a figure only. The paper also notes that detection probability and delay are better for all three detectors when the dependence increases after the change.

E7 · simulation · cited by C15

One simulated series from an ARMA(1,1) model with a change in mean at time point 400, its forecast errors from an ARMA forecasting model and the mean detector with its threshold (Figure 1 of the paper).

E8 · simulation · cited by C17

One simulated series with a trend and a variance change at time point 400, forecast with an ETS(A,A,N) model, its squared forecast errors and the squared-error detector with its threshold (Figure 2 of the paper).

E9 · quant · other · secondary · cited by C6

Both detectors applied sequentially to the forecast errors of daily parcel volumes of one Royal Mail delivery office, with July to October 2020 as the Phase I sample and monitoring from 1 November 2020, using the theoretical thresholds (Section 5.1 and Figure 5 of the paper).

E10 · quant · other · secondary · n 36 · cited by C13, C14

Both detectors applied sequentially to the one-step-ahead forecast errors of a SARMA(1,0,0)(1,0,0)12_{12} model of the monthly proportion of GRP A&E admissions, fitted on the data up to 2013, with 2013 to 2016 as the Phase I sample of size 36 and monitoring from 2016 (Section 6 and Figure 6 of the paper).

E11 · formal · cited by C23

Corollaries 1 and 2 of the paper's Supporting Information: the limit distributions of the mean detector follow from the mean change in the forecast errors and from Theorems 1 and 2.

  1. The forecast errors follow the change in mean model of Page's CUSUM detector with λ=bμ\lambda = b_\mu, Δm=Δμ,m\Delta_m = \Delta_{\mu,m} and zero-mean errors ϵμ,t\epsilon_{\mu,t}. (substitution; from C1, C20)
  2. Under the null hypothesis the limit distribution of Theorem 1 holds with D(m,k)=Dμ(m,k)D(m,k) = D_\mu(m,k) and σ^m=σ^μ,m\hat{\sigma}_m = \hat{\sigma}_{\mu,m}. (Theorem 1 of the Supporting Information; from E11.1, C24)
  3. Under the alternative, with Δμ,m\Delta_{\mu,m} satisfying Assumptions C to E, the limit distribution of Theorem 2 holds with τm=τμ,m\tau_m = \tau_{\mu,m} and cα=cμ,αc_\alpha = c_{\mu,\alpha}. (Theorem 2 of the Supporting Information; from E11.1, C21, C25)

W1 · argument · deductive · premises C2, C3, C4

The variance-change formula of Proposition 1 needs only the constant-bias assumption, which an ETS forecasting model can satisfy, so a variance change in the raw data gives a mean change in its squared forecast errors; the limit distributions of the squared-error detector then apply. The mean of the forecast errors themselves returns to zero after a mean change, so only the variance change remains detectable.

W2 · argument · deductive · premises C13, C14

A model-based sequential detector must fit the post-change model, here a SARMA(1,0,0)(1,0,0) model with a 12-month season, which the paper states needs at least two years of data after the change; the detectors on the forecast errors signalled four and two observations after it.

Limitations

L1 · model specification · reduces transferability · affects C1 · mitigation: none

Forecasting models that adapt to a change in mean, such as ETS models with a positive smoothing parameter, do not satisfy the incomplete-adjustment assumption, so a mean change does not stay in their forecast errors.

L2 · model specification · could bias magnitude · affects C5, C8, C9, C10, C11, C12 · mitigation: none

The two forecasting-model assumptions that make the thresholds theoretically valid are not necessarily satisfied in all the simulated scenarios.

L3 · sampling · reduces transferability · affects C5, C8, C9, C10 · mitigation: none

All simulated data have an ARMA error structure with normally distributed innovations and a training sample of 300 observations.

L4 · measurement · reduces precision · affects C6 · mitigation: none

The start of the Christmas peak season at the delivery office is inferred from the rise in the mean and variance of the forecast errors; the underlying parcel volumes and forecasts are not known to the authors.

L5 · sample size · reduces transferability · affects C13, C14, C7 · mitigation: none

The application uses one monthly series with a Phase I sample of 36 observations.

Open questions

O1 · from C1, C2

Whether a multivariate sequential changepoint algorithm on the vector of forecast errors of several variables or products detects changes, and how it should treat the sparsity of a change, is left as future work.

O2 · from C1

How to combine the information in forecast errors from several forecast horizons, beyond the one-step-ahead errors studied, is not established.

References

  1. Aue, A., Dienes, C., Fremdt, S., Steinebach, J., 2015. Reaction times of monitoring schemes for ARMA time series. Bernoulli 21, 1238–1259. https://doi.org/10.3150/14-BEJ604 Used by: Context, C15, C16
  2. Chu, C.-S. J., Stinchcombe, M., White, H., 1996. Monitoring Structural Change. Econometrica 64, 1045. https://doi.org/10.2307/2171955 Used by: D24
  3. Fremdt, S., 2014. Asymptotic distribution of the delay time in page's sequential procedure. Journal of Statistical Planning and Inference 145, 74–91. https://doi.org/10.1016/j.jspi.2013.09.001 Used by: C25
  4. Fremdt, S., 2015. Page's sequential procedure for change-point detection in time series regression. Statistics 49, 128–155. https://doi.org/10.1080/02331888.2013.870568 Used by: C24
  5. Grundy, T., Killick, R., Svetunkov, I., 2026. Online Detection of Forecast Model Inadequacies Using Forecast Errors. Journal of Time Series Analysis 47, 715–726. https://doi.org/10.1111/jtsa.12843 Used by: A0, A1, A2, A3, A4, A5, A6, A7
  6. Horváth, L., Hušková, M., Kokoszka, P., Steinebach, J., 2004. Monitoring changes in linear models. Journal of Statistical Planning and Inference 126, 225–251. https://doi.org/10.1016/j.jspi.2003.07.014 Used by: Context
  7. Inclán, C., Tiao, G. C., 1994. Use of Cumulative Sums of Squares for Retrospective Detection of Changes of Variance. Journal of the American Statistical Association 89, 913–923. https://doi.org/10.1080/01621459.1994.10476824 Used by: Context, D4
  8. PAGE, E. S., 1954. CONTINUOUS INSPECTION SCHEMES. Biometrika 41, 100–115. https://doi.org/10.1093/biomet/41.1-2.100 Used by: D2
  9. Robbins, M., Gallagher, C., Lund, R., Aue, A., 2011. Mean shift testing in correlated data. Journal of Time Series Analysis 32, 498–511. https://doi.org/10.1111/j.1467-9892.2010.00707.x Used by: Context
  10. Taib, A., Killick, R., Hussain, K., Patel, H., Obeidallah, M. R., 2021. Is there seasonal variation in gallstone related admissions in England?. HPB 23, 1732–1743. https://doi.org/10.1016/j.hpb.2021.04.009 Used by: Context, D20
  11. https://CRAN.R-project.org/package=forecast Used by: D5, A8
  12. https://CRAN.R-project.org/package=sandwich Used by: A10
  13. https://CRAN.R-project.org/package=smooth Used by: D6, A9

Artefacts

  • A0 · manuscript · Grundy et al. (2026) The published article, Journal of Time Series Analysis 47, 715-726 (open access, CC BY). jtsa.12843
  • A1 · figure · data_plot · Grundy et al. (2026) Data from an ARMA(1,1) model with a change in mean at time point 400, the forecast errors of an ARMA forecasting model, and the mean detector with its threshold.
  • A2 · figure · data_plot · Grundy et al. (2026) Data with a variance change at time point 400 with the prediction intervals of an ETS forecasting model, the squared forecast errors, and the squared-error detector with its threshold.
  • A3 · figure · data_plot · Grundy et al. (2026) Detection probability, false detection probability and average detection delay of the three detectors for a change in mean (top) and a change in trend (bottom) in seasonal data.
  • A4 · figure · data_plot · Grundy et al. (2026) Detection probability, false detection probability and average detection delay of the three detectors for a change in the autoregressive parameter, pre-change parameter 0.2 (top) and 0.8 (bottom).
  • A5 · figure · data_plot · Grundy et al. (2026) Forecast errors of the number of parcels to be delivered from one Royal Mail delivery office in 2020, and the two detectors with their thresholds.
  • A6 · figure · data_plot · Grundy et al. (2026) The proportion of GRP A&E admissions, the forecast errors of a SARIMA(1,0,0)(1,0,0)12 model, and the two detectors with their thresholds.
  • A7 · manuscript · Grundy et al. (2026) The article's Supporting Information: the sequential changepoint set-up, Assumptions A to E, Theorems 1 and 2, Corollaries 1 and 2 for the mean detector, and additional simulations, which this claimset does not restate. jtsa.12843
  • A8 · software · https://CRAN.R-project.org/package=forecast The forecast R package (Arima and auto.arima), used for the ARMA forecasting models. The paper's reference gives no version; 8.21.1 is the last version released in 2023, when the paper was written. package=forecast
  • A9 · software · https://CRAN.R-project.org/package=smooth The smooth R package (es function), used for the ETS forecasting models, in the version the paper's reference states. package=smooth
  • A10 · software · https://CRAN.R-project.org/package=sandwich The sandwich R package, used for the Bartlett long run variance estimator of Raw CUSUM. The paper's reference gives no version; 3.1-0 is the last version released in 2023, when the paper was written. package=sandwich

Score

Support of the paper: 59, Weakly supported

Scores are computed by rules from the claimset file (EntryLib rubric 0.7): they check the record, not the science, and may change as the rubric and reviews develop. The rubric. Every issue as text, for an agent preparing the next version.

The mean of its 7 principal claims; the lowest is C6 at 15. Open problems in the principal claims and what they rest on, each counted once, with the most it costs a claim:

  1. −45 Checkable Verifiability: unverifiable. In C6, C7, C13, C14. To remove it: Make the data, code and models the claim rests on available, or state why they cannot be.
  2. −25 Evidence Rests on an unchecked proof; evidence named without the observations the claim reports; an argument from other claims; cited work only. In C1, C2, C3, C4, C5, C6, C7, C10, C15, C16, C17, C23, C24, C25. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
  3. −7 Checkable Flags: availability not stated, figure without data. In C5, C8, C9, C10, C11, C12, C15, C17. To remove it: Publish what each flag says is missing.
  4. −6 Open problems Limitation L2 (could bias magnitude, not mitigated): The two forecasting-model assumptions that make the thresholds theoretically valid are not necessarily satisfied in all the simulated scenarios. In C5, C8, C9, C10, C11, C12. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
  5. −5 Plan A principal claim reported without a plan recorded beforehand. In C5, C6, C7. To remove it: Plan the measurement in the lab record or a registration before running it, or publish the lab record that shows every run.
  6. −3 Open problems Assumption C18 is not checked: The forecasting model has a constant bias before the changepoint: 𝔼F[Yt|y~t]−𝔼ℱ[Yt|y~t]=bμ\mathbb{E}_F[Y_t|\tilde{y}_t] - \mathbb{E}_{\mathcal{F}}[Y_t|\tilde{y}_t] = b_\mu for t=1,…,m+k*t = 1, \ldots, m + k^*, where bμb_\mu is unknown and may be zero (the paper's Assumption 1, the constant-bias assumption). In C1, C2, C3, C6, C16, C17, C23. To remove it: Test the assumption, or analyse how sensitive the claim is to it.
  7. −1 Open problems Limitation L1 (reduces transferability, not mitigated): Forecasting models that adapt to a change in mean, such as ETS models with a positive smoothing parameter, do not satisfy the incomplete-adjustment assumption, so a mean change does not stay in their forecast errors. In C1, C6, C23. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
  8. −1 Open problems Limitation L3 (reduces transferability, not mitigated): All simulated data have an ARMA error structure with normally distributed innovations and a training sample of 300 observations. In C5, C8, C9, C10. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
  9. −3 Open problems Limitation L4 (reduces precision, not mitigated): The start of the Christmas peak season at the delivery office is inferred from the rise in the mean and variance of the forecast errors; the underlying parcel volumes and forecasts are not known to the authors. In C6. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
  10. −1 Open problems Limitation L5 (reduces transferability, not mitigated): The application uses one monthly series with a Phase I sample of 36 observations. In C7, C13, C14. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
  11. −1 Open problems Assumption C19 is not checked: After the changepoint the forecasting model does not fully adjust to the change: 𝔼G[Yt|y~t]−𝔼ℱ[Yt|y~t]=bμ+Δμ,m\mathbb{E}_G[Y_t|\tilde{y}_t] - \mathbb{E}_{\mathcal{F}}[Y_t|\tilde{y}_t] = b_\mu + \Delta_{\mu,m} for t=m+k*+1,m+k*+2,…t = m + k^* + 1, m + k^* + 2, \ldots, where Δμ,m≠0\Delta_{\mu,m} \ne 0 is a function of the change size in the raw data (the paper's Assumption 2, the incomplete-adjustment assumption). In C1, C2, C3, C6, C16, C17, C23. To remove it: Test the assumption, or analyse how sensitive the claim is to it.
  12. −3 Open problems Assumption C20 is not checked: The zero-mean variables ϵt\epsilon_t of the monitored series, here ϵμ,t\epsilon_{\mu,t} and ϵξ,t\epsilon_{\xi,t} left in the forecast errors after the bias is removed, satisfy the weak invariance principle of Assumptions A and B of the paper's Supporting Information: |∑t=1mϵt|=OP(m)\left|\sum_{t=1}^{m} \epsilon_t\right| = O_P(\sqrt{m}), and there are Wiener processes {Wm(i):i≥0,m≥1}\{W_m(i) : i \ge 0, m \ge 1\} and a constant σ>0\sigma > 0 with sup1/m≤i<∞⁡1(mi)1/ν|∑t=m+1m+miϵt−σWm(mi)|=OP(1)for some ν>2\sup_{1/m \le i < \infty} \frac{1}{(mi)^{1/\nu}} \left| \sum_{t=m+1}^{m+mi} \epsilon_t - \sigma W_m(mi) \right| = O_P(1) \quad \text{for some } \nu > 2. In C3, C6, C17, C23, C24, C25. To remove it: Test the assumption, or analyse how sensitive the claim is to it.
  13. −1 Open problems Assumption C21 is not checked: The change is large enough and bounded: the changepoint is k*=⌊θmβ⌋k^* = \lfloor \theta m^\beta \rfloor for some θ>0\theta > 0 and 0≤β<10 \le \beta < 1, m|Δm|→∞\sqrt{m}\,|\Delta_m| \to \infty as m→∞m \to \infty, and Δm=O(1)\Delta_m = O(1), where Δm\Delta_m is the change size in the monitored series, here Δμ,m\Delta_{\mu,m} or Δξ,m\Delta_{\xi,m}, as Assumptions C to E of the paper's Supporting Information require. In C3, C6, C17, C23, C25. To remove it: Test the assumption, or analyse how sensitive the claim is to it.

Claims

C1: 70, Partly supported

Novelty: extends earlier work, by the authors

  1. −25 Evidence Rests on an unchecked proof; evidence named without the observations the claim reports; an argument from other claims; cited work only. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
  2. −3 Open problems Assumption C18 is not checked: The forecasting model has a constant bias before the changepoint: 𝔼F[Yt|y~t]−𝔼ℱ[Yt|y~t]=bμ\mathbb{E}_F[Y_t|\tilde{y}_t] - \mathbb{E}_{\mathcal{F}}[Y_t|\tilde{y}_t] = b_\mu for t=1,…,m+k*t = 1, \ldots, m + k^*, where bμb_\mu is unknown and may be zero (the paper's Assumption 1, the constant-bias assumption). To remove it: Test the assumption, or analyse how sensitive the claim is to it.
  3. −1 Open problems Assumption C19 is not checked: After the changepoint the forecasting model does not fully adjust to the change: 𝔼G[Yt|y~t]−𝔼ℱ[Yt|y~t]=bμ+Δμ,m\mathbb{E}_G[Y_t|\tilde{y}_t] - \mathbb{E}_{\mathcal{F}}[Y_t|\tilde{y}_t] = b_\mu + \Delta_{\mu,m} for t=m+k*+1,m+k*+2,…t = m + k^* + 1, m + k^* + 2, \ldots, where Δμ,m≠0\Delta_{\mu,m} \ne 0 is a function of the change size in the raw data (the paper's Assumption 2, the incomplete-adjustment assumption). To remove it: Test the assumption, or analyse how sensitive the claim is to it.
  4. −1 Open problems Limitation L1 (reduces transferability, not mitigated): Forecasting models that adapt to a change in mean, such as ETS models with a positive smoothing parameter, do not satisfy the incomplete-adjustment assumption, so a mean change does not stay in their forecast errors. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.

C2: 71, Partly supported

Novelty: extends earlier work, by the authors

  1. −25 Evidence Rests on an unchecked proof; evidence named without the observations the claim reports; an argument from other claims; cited work only. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
  2. −3 Open problems Assumption C18 is not checked: The forecasting model has a constant bias before the changepoint: 𝔼F[Yt|y~t]−𝔼ℱ[Yt|y~t]=bμ\mathbb{E}_F[Y_t|\tilde{y}_t] - \mathbb{E}_{\mathcal{F}}[Y_t|\tilde{y}_t] = b_\mu for t=1,…,m+k*t = 1, \ldots, m + k^*, where bμb_\mu is unknown and may be zero (the paper's Assumption 1, the constant-bias assumption). To remove it: Test the assumption, or analyse how sensitive the claim is to it.
  3. −1 Open problems Assumption C19 is not checked: After the changepoint the forecasting model does not fully adjust to the change: 𝔼G[Yt|y~t]−𝔼ℱ[Yt|y~t]=bμ+Δμ,m\mathbb{E}_G[Y_t|\tilde{y}_t] - \mathbb{E}_{\mathcal{F}}[Y_t|\tilde{y}_t] = b_\mu + \Delta_{\mu,m} for t=m+k*+1,m+k*+2,…t = m + k^* + 1, m + k^* + 2, \ldots, where Δμ,m≠0\Delta_{\mu,m} \ne 0 is a function of the change size in the raw data (the paper's Assumption 2, the incomplete-adjustment assumption). To remove it: Test the assumption, or analyse how sensitive the claim is to it.

C3: 69, Partly supported

Novelty: extends earlier work, by the authors

  1. −25 Evidence Rests on an unchecked proof; evidence named without the observations the claim reports; an argument from other claims; cited work only. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
  2. −3 Open problems Assumption C18 is not checked: The forecasting model has a constant bias before the changepoint: 𝔼F[Yt|y~t]−𝔼ℱ[Yt|y~t]=bμ\mathbb{E}_F[Y_t|\tilde{y}_t] - \mathbb{E}_{\mathcal{F}}[Y_t|\tilde{y}_t] = b_\mu for t=1,…,m+k*t = 1, \ldots, m + k^*, where bμb_\mu is unknown and may be zero (the paper's Assumption 1, the constant-bias assumption). To remove it: Test the assumption, or analyse how sensitive the claim is to it.
  3. −1 Open problems Assumption C19 is not checked: After the changepoint the forecasting model does not fully adjust to the change: 𝔼G[Yt|y~t]−𝔼ℱ[Yt|y~t]=bμ+Δμ,m\mathbb{E}_G[Y_t|\tilde{y}_t] - \mathbb{E}_{\mathcal{F}}[Y_t|\tilde{y}_t] = b_\mu + \Delta_{\mu,m} for t=m+k*+1,m+k*+2,…t = m + k^* + 1, m + k^* + 2, \ldots, where Δμ,m≠0\Delta_{\mu,m} \ne 0 is a function of the change size in the raw data (the paper's Assumption 2, the incomplete-adjustment assumption). To remove it: Test the assumption, or analyse how sensitive the claim is to it.
  4. −1 Open problems Assumption C20 is not checked: The zero-mean variables ϵt\epsilon_t of the monitored series, here ϵμ,t\epsilon_{\mu,t} and ϵξ,t\epsilon_{\xi,t} left in the forecast errors after the bias is removed, satisfy the weak invariance principle of Assumptions A and B of the paper's Supporting Information: |∑t=1mϵt|=OP(m)\left|\sum_{t=1}^{m} \epsilon_t\right| = O_P(\sqrt{m}), and there are Wiener processes {Wm(i):i≥0,m≥1}\{W_m(i) : i \ge 0, m \ge 1\} and a constant σ>0\sigma > 0 with sup1/m≤i<∞⁡1(mi)1/ν|∑t=m+1m+miϵt−σWm(mi)|=OP(1)for some ν>2\sup_{1/m \le i < \infty} \frac{1}{(mi)^{1/\nu}} \left| \sum_{t=m+1}^{m+mi} \epsilon_t - \sigma W_m(mi) \right| = O_P(1) \quad \text{for some } \nu > 2. To remove it: Test the assumption, or analyse how sensitive the claim is to it.
  5. −1 Open problems Assumption C21 is not checked: The change is large enough and bounded: the changepoint is k*=⌊θmβ⌋k^* = \lfloor \theta m^\beta \rfloor for some θ>0\theta > 0 and 0≤β<10 \le \beta < 1, m|Δm|→∞\sqrt{m}\,|\Delta_m| \to \infty as m→∞m \to \infty, and Δm=O(1)\Delta_m = O(1), where Δm\Delta_m is the change size in the monitored series, here Δμ,m\Delta_{\mu,m} or Δξ,m\Delta_{\xi,m}, as Assumptions C to E of the paper's Supporting Information require. To remove it: Test the assumption, or analyse how sensitive the claim is to it.

C4: 75, Partly supported

Novelty: new, by the authors

  1. −25 Evidence Rests on an unchecked proof; evidence named without the observations the claim reports; an argument from other claims; cited work only. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).

C5: 71, Partly supported

Novelty: new, by the authors

  1. −10 Evidence Rests on an unchecked proof; evidence named without the observations the claim reports; an argument from other claims; cited work only. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
  2. −7 Checkable Flags: availability not stated, figure without data. To remove it: Publish what each flag says is missing.
  3. −6 Open problems Limitation L2 (could bias magnitude, not mitigated): The two forecasting-model assumptions that make the thresholds theoretically valid are not necessarily satisfied in all the simulated scenarios. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
  4. −5 Plan A principal claim reported without a plan recorded beforehand. To remove it: Plan the measurement in the lab record or a registration before running it, or publish the lab record that shows every run.
  5. −1 Open problems Limitation L3 (reduces transferability, not mitigated): All simulated data have an ARMA error structure with normally distributed innovations and a training sample of 300 observations. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.

C6: 15, Weakly supported

Novelty: new, by the authors

  1. −45 Checkable Verifiability: unverifiable. To remove it: Make the data, code and models the claim rests on available, or state why they cannot be.
  2. −25 Evidence Rests on an unchecked proof; evidence named without the observations the claim reports; an argument from other claims; cited work only. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
  3. −5 Plan A principal claim reported without a plan recorded beforehand. To remove it: Plan the measurement in the lab record or a registration before running it, or publish the lab record that shows every run.
  4. −3 Open problems Assumption C18 is not checked: The forecasting model has a constant bias before the changepoint: 𝔼F[Yt|y~t]−𝔼ℱ[Yt|y~t]=bμ\mathbb{E}_F[Y_t|\tilde{y}_t] - \mathbb{E}_{\mathcal{F}}[Y_t|\tilde{y}_t] = b_\mu for t=1,…,m+k*t = 1, \ldots, m + k^*, where bμb_\mu is unknown and may be zero (the paper's Assumption 1, the constant-bias assumption). To remove it: Test the assumption, or analyse how sensitive the claim is to it.
  5. −3 Open problems Limitation L4 (reduces precision, not mitigated): The start of the Christmas peak season at the delivery office is inferred from the rise in the mean and variance of the forecast errors; the underlying parcel volumes and forecasts are not known to the authors. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
  6. −1 Open problems Assumption C19 is not checked: After the changepoint the forecasting model does not fully adjust to the change: 𝔼G[Yt|y~t]−𝔼ℱ[Yt|y~t]=bμ+Δμ,m\mathbb{E}_G[Y_t|\tilde{y}_t] - \mathbb{E}_{\mathcal{F}}[Y_t|\tilde{y}_t] = b_\mu + \Delta_{\mu,m} for t=m+k*+1,m+k*+2,…t = m + k^* + 1, m + k^* + 2, \ldots, where Δμ,m≠0\Delta_{\mu,m} \ne 0 is a function of the change size in the raw data (the paper's Assumption 2, the incomplete-adjustment assumption). To remove it: Test the assumption, or analyse how sensitive the claim is to it.
  7. −1 Open problems Assumption C20 is not checked: The zero-mean variables ϵt\epsilon_t of the monitored series, here ϵμ,t\epsilon_{\mu,t} and ϵξ,t\epsilon_{\xi,t} left in the forecast errors after the bias is removed, satisfy the weak invariance principle of Assumptions A and B of the paper's Supporting Information: |∑t=1mϵt|=OP(m)\left|\sum_{t=1}^{m} \epsilon_t\right| = O_P(\sqrt{m}), and there are Wiener processes {Wm(i):i≥0,m≥1}\{W_m(i) : i \ge 0, m \ge 1\} and a constant σ>0\sigma > 0 with sup1/m≤i<∞⁡1(mi)1/ν|∑t=m+1m+miϵt−σWm(mi)|=OP(1)for some ν>2\sup_{1/m \le i < \infty} \frac{1}{(mi)^{1/\nu}} \left| \sum_{t=m+1}^{m+mi} \epsilon_t - \sigma W_m(mi) \right| = O_P(1) \quad \text{for some } \nu > 2. To remove it: Test the assumption, or analyse how sensitive the claim is to it.
  8. −1 Open problems Assumption C21 is not checked: The change is large enough and bounded: the changepoint is k*=⌊θmβ⌋k^* = \lfloor \theta m^\beta \rfloor for some θ>0\theta > 0 and 0≤β<10 \le \beta < 1, m|Δm|→∞\sqrt{m}\,|\Delta_m| \to \infty as m→∞m \to \infty, and Δm=O(1)\Delta_m = O(1), where Δm\Delta_m is the change size in the monitored series, here Δμ,m\Delta_{\mu,m} or Δξ,m\Delta_{\xi,m}, as Assumptions C to E of the paper's Supporting Information require. To remove it: Test the assumption, or analyse how sensitive the claim is to it.
  9. −1 Open problems Limitation L1 (reduces transferability, not mitigated): Forecasting models that adapt to a change in mean, such as ETS models with a positive smoothing parameter, do not satisfy the incomplete-adjustment assumption, so a mean change does not stay in their forecast errors. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.

C7: 39, Weakly supported

Novelty: new, by the authors

  1. −45 Checkable Verifiability: unverifiable. To remove it: Make the data, code and models the claim rests on available, or state why they cannot be.
  2. −10 Evidence Rests on an unchecked proof; evidence named without the observations the claim reports; an argument from other claims; cited work only. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
  3. −5 Plan A principal claim reported without a plan recorded beforehand. To remove it: Plan the measurement in the lab record or a registration before running it, or publish the lab record that shows every run.
  4. −1 Open problems Limitation L5 (reduces transferability, not mitigated): The application uses one monthly series with a Phase I sample of 36 observations. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.

C8: 86, Well supported

Novelty: new, by the authors

  1. −7 Checkable Flags: availability not stated, figure without data. To remove it: Publish what each flag says is missing.
  2. −6 Open problems Limitation L2 (could bias magnitude, not mitigated): The two forecasting-model assumptions that make the thresholds theoretically valid are not necessarily satisfied in all the simulated scenarios. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
  3. −1 Open problems Limitation L3 (reduces transferability, not mitigated): All simulated data have an ARMA error structure with normally distributed innovations and a training sample of 300 observations. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.

C9: 86, Well supported

Novelty: new, by the authors

  1. −7 Checkable Flags: availability not stated, figure without data. To remove it: Publish what each flag says is missing.
  2. −6 Open problems Limitation L2 (could bias magnitude, not mitigated): The two forecasting-model assumptions that make the thresholds theoretically valid are not necessarily satisfied in all the simulated scenarios. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
  3. −1 Open problems Limitation L3 (reduces transferability, not mitigated): All simulated data have an ARMA error structure with normally distributed innovations and a training sample of 300 observations. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.

C10: 78, Partly supported

Novelty: new, by the authors

  1. −10 Evidence Rests on an unchecked proof; evidence named without the observations the claim reports; an argument from other claims; cited work only. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
  2. −6 Open problems Limitation L2 (could bias magnitude, not mitigated): The two forecasting-model assumptions that make the thresholds theoretically valid are not necessarily satisfied in all the simulated scenarios. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
  3. −5 Checkable Flags: availability not stated, figure without data. To remove it: Publish what each flag says is missing.
  4. −1 Open problems Limitation L3 (reduces transferability, not mitigated): All simulated data have an ARMA error structure with normally distributed innovations and a training sample of 300 observations. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.

C11: 89, Well supported

  1. −6 Open problems Limitation L2 (could bias magnitude, not mitigated): The two forecasting-model assumptions that make the thresholds theoretically valid are not necessarily satisfied in all the simulated scenarios. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
  2. −5 Checkable Flags: availability not stated, figure without data. To remove it: Publish what each flag says is missing.

C12: 87, Well supported

  1. −7 Checkable Flags: availability not stated, figure without data. To remove it: Publish what each flag says is missing.
  2. −6 Open problems Limitation L2 (could bias magnitude, not mitigated): The two forecasting-model assumptions that make the thresholds theoretically valid are not necessarily satisfied in all the simulated scenarios. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.

C13: 54, Weakly supported

  1. −45 Checkable Verifiability: unverifiable. To remove it: Make the data, code and models the claim rests on available, or state why they cannot be.
  2. −1 Open problems Limitation L5 (reduces transferability, not mitigated): The application uses one monthly series with a Phase I sample of 36 observations. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.

C14: 54, Weakly supported

  1. −45 Checkable Verifiability: unverifiable. To remove it: Make the data, code and models the claim rests on available, or state why they cannot be.
  2. −1 Open problems Limitation L5 (reduces transferability, not mitigated): The application uses one monthly series with a Phase I sample of 36 observations. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.

C23: 68, Partly supported

Novelty: extends earlier work, by the authors

  1. −25 Evidence Rests on an unchecked proof; evidence named without the observations the claim reports; an argument from other claims; cited work only. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
  2. −3 Open problems Assumption C18 is not checked: The forecasting model has a constant bias before the changepoint: 𝔼F[Yt|y~t]−𝔼ℱ[Yt|y~t]=bμ\mathbb{E}_F[Y_t|\tilde{y}_t] - \mathbb{E}_{\mathcal{F}}[Y_t|\tilde{y}_t] = b_\mu for t=1,…,m+k*t = 1, \ldots, m + k^*, where bμb_\mu is unknown and may be zero (the paper's Assumption 1, the constant-bias assumption). To remove it: Test the assumption, or analyse how sensitive the claim is to it.
  3. −1 Open problems Assumption C19 is not checked: After the changepoint the forecasting model does not fully adjust to the change: 𝔼G[Yt|y~t]−𝔼ℱ[Yt|y~t]=bμ+Δμ,m\mathbb{E}_G[Y_t|\tilde{y}_t] - \mathbb{E}_{\mathcal{F}}[Y_t|\tilde{y}_t] = b_\mu + \Delta_{\mu,m} for t=m+k*+1,m+k*+2,…t = m + k^* + 1, m + k^* + 2, \ldots, where Δμ,m≠0\Delta_{\mu,m} \ne 0 is a function of the change size in the raw data (the paper's Assumption 2, the incomplete-adjustment assumption). To remove it: Test the assumption, or analyse how sensitive the claim is to it.
  4. −1 Open problems Assumption C20 is not checked: The zero-mean variables ϵt\epsilon_t of the monitored series, here ϵμ,t\epsilon_{\mu,t} and ϵξ,t\epsilon_{\xi,t} left in the forecast errors after the bias is removed, satisfy the weak invariance principle of Assumptions A and B of the paper's Supporting Information: |∑t=1mϵt|=OP(m)\left|\sum_{t=1}^{m} \epsilon_t\right| = O_P(\sqrt{m}), and there are Wiener processes {Wm(i):i≥0,m≥1}\{W_m(i) : i \ge 0, m \ge 1\} and a constant σ>0\sigma > 0 with sup1/m≤i<∞⁡1(mi)1/ν|∑t=m+1m+miϵt−σWm(mi)|=OP(1)for some ν>2\sup_{1/m \le i < \infty} \frac{1}{(mi)^{1/\nu}} \left| \sum_{t=m+1}^{m+mi} \epsilon_t - \sigma W_m(mi) \right| = O_P(1) \quad \text{for some } \nu > 2. To remove it: Test the assumption, or analyse how sensitive the claim is to it.
  5. −1 Open problems Assumption C21 is not checked: The change is large enough and bounded: the changepoint is k*=⌊θmβ⌋k^* = \lfloor \theta m^\beta \rfloor for some θ>0\theta > 0 and 0≤β<10 \le \beta < 1, m|Δm|→∞\sqrt{m}\,|\Delta_m| \to \infty as m→∞m \to \infty, and Δm=O(1)\Delta_m = O(1), where Δm\Delta_m is the change size in the monitored series, here Δμ,m\Delta_{\mu,m} or Δξ,m\Delta_{\xi,m}, as Assumptions C to E of the paper's Supporting Information require. To remove it: Test the assumption, or analyse how sensitive the claim is to it.
  6. −1 Open problems Limitation L1 (reduces transferability, not mitigated): Forecasting models that adapt to a change in mean, such as ETS models with a positive smoothing parameter, do not satisfy the incomplete-adjustment assumption, so a mean change does not stay in their forecast errors. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.

C15: 82, Well supported

Novelty: summarises earlier work, by the authors

  1. −10 Evidence Rests on an unchecked proof; evidence named without the observations the claim reports; an argument from other claims; cited work only. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
  2. −5 Checkable Flags: availability not stated, figure without data. To remove it: Publish what each flag says is missing.
  3. −3 Open problems Assumption C22 is not checked: The ARMA process is causal and invertible, ϕ(x)≠0\phi(x) \ne 0 and θ(x)≠0\theta(x) \ne 0 for all |x|≤1|x| \le 1, and the ARMA forecasting model is perfectly estimated, so that its residuals and forecast errors are identical. To remove it: Test the assumption, or analyse how sensitive the claim is to it.

C16: 70, Partly supported

Novelty: summarises earlier work, by the authors

  1. −25 Evidence Rests on an unchecked proof; evidence named without the observations the claim reports; an argument from other claims; cited work only. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
  2. −3 Open problems Assumption C22 is not checked: The ARMA process is causal and invertible, ϕ(x)≠0\phi(x) \ne 0 and θ(x)≠0\theta(x) \ne 0 for all |x|≤1|x| \le 1, and the ARMA forecasting model is perfectly estimated, so that its residuals and forecast errors are identical. To remove it: Test the assumption, or analyse how sensitive the claim is to it.
  3. −1 Open problems Assumption C19 is not checked: After the changepoint the forecasting model does not fully adjust to the change: 𝔼G[Yt|y~t]−𝔼ℱ[Yt|y~t]=bμ+Δμ,m\mathbb{E}_G[Y_t|\tilde{y}_t] - \mathbb{E}_{\mathcal{F}}[Y_t|\tilde{y}_t] = b_\mu + \Delta_{\mu,m} for t=m+k*+1,m+k*+2,…t = m + k^* + 1, m + k^* + 2, \ldots, where Δμ,m≠0\Delta_{\mu,m} \ne 0 is a function of the change size in the raw data (the paper's Assumption 2, the incomplete-adjustment assumption). To remove it: Test the assumption, or analyse how sensitive the claim is to it.
  4. −1 Open problems Assumption C18 is not checked: The forecasting model has a constant bias before the changepoint: 𝔼F[Yt|y~t]−𝔼ℱ[Yt|y~t]=bμ\mathbb{E}_F[Y_t|\tilde{y}_t] - \mathbb{E}_{\mathcal{F}}[Y_t|\tilde{y}_t] = b_\mu for t=1,…,m+k*t = 1, \ldots, m + k^*, where bμb_\mu is unknown and may be zero (the paper's Assumption 1, the constant-bias assumption). To remove it: Test the assumption, or analyse how sensitive the claim is to it.

C17: 64, Partly supported

Novelty: new, by the authors

  1. −25 Evidence Rests on an unchecked proof; evidence named without the observations the claim reports; an argument from other claims; cited work only. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
  2. −5 Checkable Flags: availability not stated, figure without data. To remove it: Publish what each flag says is missing.
  3. −3 Open problems Assumption C18 is not checked: The forecasting model has a constant bias before the changepoint: 𝔼F[Yt|y~t]−𝔼ℱ[Yt|y~t]=bμ\mathbb{E}_F[Y_t|\tilde{y}_t] - \mathbb{E}_{\mathcal{F}}[Y_t|\tilde{y}_t] = b_\mu for t=1,…,m+k*t = 1, \ldots, m + k^*, where bμb_\mu is unknown and may be zero (the paper's Assumption 1, the constant-bias assumption). To remove it: Test the assumption, or analyse how sensitive the claim is to it.
  4. −1 Open problems Assumption C19 is not checked: After the changepoint the forecasting model does not fully adjust to the change: 𝔼G[Yt|y~t]−𝔼ℱ[Yt|y~t]=bμ+Δμ,m\mathbb{E}_G[Y_t|\tilde{y}_t] - \mathbb{E}_{\mathcal{F}}[Y_t|\tilde{y}_t] = b_\mu + \Delta_{\mu,m} for t=m+k*+1,m+k*+2,…t = m + k^* + 1, m + k^* + 2, \ldots, where Δμ,m≠0\Delta_{\mu,m} \ne 0 is a function of the change size in the raw data (the paper's Assumption 2, the incomplete-adjustment assumption). To remove it: Test the assumption, or analyse how sensitive the claim is to it.
  5. −1 Open problems Assumption C20 is not checked: The zero-mean variables ϵt\epsilon_t of the monitored series, here ϵμ,t\epsilon_{\mu,t} and ϵξ,t\epsilon_{\xi,t} left in the forecast errors after the bias is removed, satisfy the weak invariance principle of Assumptions A and B of the paper's Supporting Information: |∑t=1mϵt|=OP(m)\left|\sum_{t=1}^{m} \epsilon_t\right| = O_P(\sqrt{m}), and there are Wiener processes {Wm(i):i≥0,m≥1}\{W_m(i) : i \ge 0, m \ge 1\} and a constant σ>0\sigma > 0 with sup1/m≤i<∞⁡1(mi)1/ν|∑t=m+1m+miϵt−σWm(mi)|=OP(1)for some ν>2\sup_{1/m \le i < \infty} \frac{1}{(mi)^{1/\nu}} \left| \sum_{t=m+1}^{m+mi} \epsilon_t - \sigma W_m(mi) \right| = O_P(1) \quad \text{for some } \nu > 2. To remove it: Test the assumption, or analyse how sensitive the claim is to it.
  6. −1 Open problems Assumption C21 is not checked: The change is large enough and bounded: the changepoint is k*=⌊θmβ⌋k^* = \lfloor \theta m^\beta \rfloor for some θ>0\theta > 0 and 0≤β<10 \le \beta < 1, m|Δm|→∞\sqrt{m}\,|\Delta_m| \to \infty as m→∞m \to \infty, and Δm=O(1)\Delta_m = O(1), where Δm\Delta_m is the change size in the monitored series, here Δμ,m\Delta_{\mu,m} or Δξ,m\Delta_{\xi,m}, as Assumptions C to E of the paper's Supporting Information require. To remove it: Test the assumption, or analyse how sensitive the claim is to it.

C24: 77, Partly supported

Novelty: summarises earlier work, by the authors

  1. −20 Evidence Rests on an unchecked proof; evidence named without the observations the claim reports; an argument from other claims; cited work only. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
  2. −3 Open problems Assumption C20 is not checked: The zero-mean variables ϵt\epsilon_t of the monitored series, here ϵμ,t\epsilon_{\mu,t} and ϵξ,t\epsilon_{\xi,t} left in the forecast errors after the bias is removed, satisfy the weak invariance principle of Assumptions A and B of the paper's Supporting Information: |∑t=1mϵt|=OP(m)\left|\sum_{t=1}^{m} \epsilon_t\right| = O_P(\sqrt{m}), and there are Wiener processes {Wm(i):i≥0,m≥1}\{W_m(i) : i \ge 0, m \ge 1\} and a constant σ>0\sigma > 0 with sup1/m≤i<∞⁡1(mi)1/ν|∑t=m+1m+miϵt−σWm(mi)|=OP(1)for some ν>2\sup_{1/m \le i < \infty} \frac{1}{(mi)^{1/\nu}} \left| \sum_{t=m+1}^{m+mi} \epsilon_t - \sigma W_m(mi) \right| = O_P(1) \quad \text{for some } \nu > 2. To remove it: Test the assumption, or analyse how sensitive the claim is to it.

C25: 76, Partly supported

Novelty: summarises earlier work, by the authors

  1. −20 Evidence Rests on an unchecked proof; evidence named without the observations the claim reports; an argument from other claims; cited work only. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
  2. −3 Open problems Assumption C20 is not checked: The zero-mean variables ϵt\epsilon_t of the monitored series, here ϵμ,t\epsilon_{\mu,t} and ϵξ,t\epsilon_{\xi,t} left in the forecast errors after the bias is removed, satisfy the weak invariance principle of Assumptions A and B of the paper's Supporting Information: |∑t=1mϵt|=OP(m)\left|\sum_{t=1}^{m} \epsilon_t\right| = O_P(\sqrt{m}), and there are Wiener processes {Wm(i):i≥0,m≥1}\{W_m(i) : i \ge 0, m \ge 1\} and a constant σ>0\sigma > 0 with sup1/m≤i<∞⁡1(mi)1/ν|∑t=m+1m+miϵt−σWm(mi)|=OP(1)for some ν>2\sup_{1/m \le i < \infty} \frac{1}{(mi)^{1/\nu}} \left| \sum_{t=m+1}^{m+mi} \epsilon_t - \sigma W_m(mi) \right| = O_P(1) \quad \text{for some } \nu > 2. To remove it: Test the assumption, or analyse how sensitive the claim is to it.
  3. −1 Open problems Assumption C21 is not checked: The change is large enough and bounded: the changepoint is k*=⌊θmβ⌋k^* = \lfloor \theta m^\beta \rfloor for some θ>0\theta > 0 and 0≤β<10 \le \beta < 1, m|Δm|→∞\sqrt{m}\,|\Delta_m| \to \infty as m→∞m \to \infty, and Δm=O(1)\Delta_m = O(1), where Δm\Delta_m is the change size in the monitored series, here Δμ,m\Delta_{\mu,m} or Δξ,m\Delta_{\xi,m}, as Assumptions C to E of the paper's Supporting Information require. To remove it: Test the assumption, or analyse how sensitive the claim is to it.

Versions

  • Version 1 (this page) · 16 November 2026 at 20:14 UTC · active

Every version is kept in the public registry (github.com/entrylib/registry).