Claim C1 · formal · entrylib:2026.000010@v1 · 2026-11-16

Online Detection of Forecast Model Inadequacies Using Forecast Errors · C1

Principal claim C1 of this paper (Svetunkov, I. (guarantor), 2026), with what it rests on. Read the whole paper.

Text file for AI

Text file: the claim with what it rests on as plain text, to attach when your assistant cannot open the address.

The message

Read https://entrylib.org/p/2026.000010/C1, one claim of a research paper with the evidence it rests on. Tell me briefly what it says and how well it is supported. If you cannot open the link, tell me instead of guessing. Then wait for my questions.

The claim

SupportS 70C1 · formal · novelty: extends earlier work, by the authors

Under the mean change model, the one-step-ahead forecast errors of a forecasting model that satisfies the two forecasting-model assumptions have a change in mean of size Δμ,m≠0\Delta_{\mu,m} \ne 0 at the changepoint: et=bμ+ϵμ,te_t = b_\mu + \epsilon_{\mu,t} for t≤m+k*t \le m + k^* and et=bμ+ϵμ,t+Δμ,me_t = b_\mu + \epsilon_{\mu,t} + \Delta_{\mu,m} afterwards.

Evidence

E1 · formal · cited by C1

Derivation of the mean change in the forecast errors from the two forecasting-model assumptions (Section 2.1 of the paper).

  1. Before the changepoint, 𝔼F[et]=𝔼F[Yt|y~t]−𝔼ℱ[Yt|y~t]=bμ,\mathbb{E}_F[e_t] = \mathbb{E}_F[Y_t|\tilde{y}_t] - \mathbb{E}_{\mathcal{F}}[Y_t|\tilde{y}_t] = b_\mu, so et=bμ+ϵμ,te_t = b_\mu + \epsilon_{\mu,t} with ϵμ,t\epsilon_{\mu,t} of zero mean. (definition of the forecast error; from D1, C18)
  2. After the changepoint, 𝔼G[et]=bμ+Δμ,m\mathbb{E}_G[e_t] = b_\mu + \Delta_{\mu,m}, so et=bμ+ϵμ,t+Δμ,me_t = b_\mu + \epsilon_{\mu,t} + \Delta_{\mu,m} with Δμ,m≠0\Delta_{\mu,m} \ne 0. (substitution; from E1.1, C19)

Limitations

L1 · model specification · reduces transferability · affects C1 · mitigation: none

Forecasting models that adapt to a change in mean, such as ETS models with a positive smoothing parameter, do not satisfy the incomplete-adjustment assumption, so a mean change does not stay in their forecast errors.

Assumptions

C18 · theorem_condition · unchecked

The forecasting model has a constant bias before the changepoint: 𝔼F[Yt|y~t]−𝔼ℱ[Yt|y~t]=bμ\mathbb{E}_F[Y_t|\tilde{y}_t] - \mathbb{E}_{\mathcal{F}}[Y_t|\tilde{y}_t] = b_\mu for t=1,…,m+k*t = 1, \ldots, m + k^*, where bμb_\mu is unknown and may be zero (the paper's Assumption 1, the constant-bias assumption). Rested on by C1, E1.

C19 · theorem_condition · unchecked

After the changepoint the forecasting model does not fully adjust to the change: 𝔼G[Yt|y~t]−𝔼ℱ[Yt|y~t]=bμ+Δμ,m\mathbb{E}_G[Y_t|\tilde{y}_t] - \mathbb{E}_{\mathcal{F}}[Y_t|\tilde{y}_t] = b_\mu + \Delta_{\mu,m} for t=m+k*+1,m+k*+2,…t = m + k^* + 1, m + k^* + 2, \ldots, where Δμ,m≠0\Delta_{\mu,m} \ne 0 is a function of the change size in the raw data (the paper's Assumption 2, the incomplete-adjustment assumption). Rested on by C1, E1.

Definitions

Methods 4

D1 forecast error monitoring framework: A framework that monitors a forecasting model online by applying sequential changepoint detection to its one-step-ahead forecast errors. For a univariate series YtY_t and a forecasting model ℱ\mathcal{F} with one-step-ahead point forecast y^t(1)=𝔼ℱ[Yt|yt−1,…,y1]\hat{y}_t(1) = \mathbb{E}_{\mathcal{F}}[Y_t | y_{t-1}, \ldots, y_1], the forecast errors are et=Yt−y^t(1).e_t = Y_t - \hat{y}_t(1). The first mm forecast errors are the Phase I (training) sample; monitoring starts at t=m+1t = m + 1 and stops when a detector, the mean detector or the squared-error detector, crosses its threshold. Any forecasting approach can supply the errors: model-based, machine learning or expert judgement.

D2 · part of D1 mean detector $D_\mu(m,k)$: Page's CUSUM detector applied to the forecast errors of the forecast error monitoring framework: Qμ(m,k)=∑t=m+1m+ket−km∑t=1metDμ(m,k)=max0≤i≤k⁡|Qμ(m,k)−Qμ(m,i)|\begin{aligned} Q_\mu(m,k) &= \sum_{t=m+1}^{m+k} e_t - \frac{k}{m} \sum_{t=1}^{m} e_t \\ D_\mu(m,k) &= \max_{0 \le i \le k} \left| Q_\mu(m,k) - Q_\mu(m,i) \right| \end{aligned} with stopping time τμ,m=min⁡{k≥1:Dμ(m,k)≥σ^μ,mcμ,αg(m,k,γ)},\tau_{\mu,m} = \min\{k \ge 1 : D_\mu(m,k) \ge \hat{\sigma}_{\mu,m} c_{\mu,\alpha} g(m,k,\gamma)\}, where σ^μ,m\hat{\sigma}_{\mu,m} is the standard deviation of the forecast errors in the Phase I sample. The weight function g(m,k,γ)g(m,k,\gamma) is that of Page's CUSUM detector, and the critical constant cμ,αc_{\mu,\alpha} comes from the limit distribution of the detector under the null hypothesis. Source: PAGE (1954).

D3 · part of D1 squared-error detector $D_\xi(m,k)$: Page's CUSUM detector applied to the centred squared forecast errors of the forecast error monitoring framework: Qξ(m,k)=∑t=m+1m+k(et−b^μ,m)2−km∑t=1m(et−b^μ,m)2Dξ(m,k)=max0≤i≤k⁡|Qξ(m,k)−Qξ(m,i)|\begin{aligned} Q_\xi(m,k) &= \sum_{t=m+1}^{m+k} (e_t - \hat{b}_{\mu,m})^2 - \frac{k}{m} \sum_{t=1}^{m} (e_t - \hat{b}_{\mu,m})^2 \\ D_\xi(m,k) &= \max_{0 \le i \le k} \left| Q_\xi(m,k) - Q_\xi(m,i) \right| \end{aligned} with stopping time τξ,m=min⁡{k≥1:Dξ(m,k)≥σ^ξ,mcξ,αg(m,k,γ)},\tau_{\xi,m} = \min\{k \ge 1 : D_\xi(m,k) \ge \hat{\sigma}_{\xi,m} c_{\xi,\alpha} g(m,k,\gamma)\}, where b^μ,m\hat{b}_{\mu,m} is the mean of the forecast errors in the mm Phase I samples and σ^ξ,m\hat{\sigma}_{\xi,m} the standard deviation of the centred squared forecast errors in that sample. g(m,k,γ)g(m,k,\gamma) is that of Page's CUSUM detector and cξ,αc_{\xi,\alpha} comes from the limit distribution of the detector under the null hypothesis. The paper prints the second sum of Qξ(m,k)Q_\xi(m,k) without the square, a typographical error that the guarantor, an author of the paper, has confirmed.

D24 Page's CUSUM detector $D(m,k)$: The sequential detector of a mean change in a series YtY_t that follows the change in mean model Yt={λ+ϵt,t=1,…,m+k*λ+ϵt+Δm,t=m+k*+1,m+k*+2,…Y_t = \begin{cases} \lambda + \epsilon_t, & t = 1, \ldots, m + k^* \\ \lambda + \epsilon_t + \Delta_m, & t = m + k^* + 1, m + k^* + 2, \ldots \end{cases} with zero-mean variables ϵt\epsilon_t, a training period of length mm without changepoints and a change of size Δm\Delta_m at time k*k^* after it. With Q(m,k)=∑t=m+1m+kYt−km∑t=1mYtD(m,k)=max0≤i≤k⁡|Q(m,k)−Q(m,i)|\begin{aligned} Q(m,k) &= \sum_{t=m+1}^{m+k} Y_t - \frac{k}{m} \sum_{t=1}^{m} Y_t \\ D(m,k) &= \max_{0 \le i \le k} \left| Q(m,k) - Q(m,i) \right| \end{aligned} the null hypothesis H0:Δm=0H_0: \Delta_m = 0 is rejected in favour of HA:Δm≠0H_A: \Delta_m \ne 0 at the stopping time τm=min⁡{k≥1:D(m,k)≥σ^mcαg(m,k,γ)},\tau_m = \min\{k \ge 1 : D(m,k) \ge \hat{\sigma}_m c_\alpha g(m,k,\gamma)\}, where σ^m\hat{\sigma}_m is a weakly consistent estimator, from the mm training samples, of the constant σ\sigma of the weak invariance principle, the weight function is g(m,k,γ)=m(1+km)(km+k)γ,γ∈[0,1/2).g(m,k,\gamma) = \sqrt{m}\left(1 + \frac{k}{m}\right)\left(\frac{k}{m+k}\right)^{\gamma}, \quad \gamma \in [0, 1/2). and the critical value cαc_\alpha is taken from the limit distribution under H0H_0, so that limm→∞⁡P(τm<∞)=α\lim_{m\to\infty} P(\tau_m < \infty) = \alpha under H0H_0 for the chosen false alarm rate α\alpha and limm→∞⁡P(τm<∞)=1\lim_{m\to\infty} P(\tau_m < \infty) = 1 under HAH_A. All limits are for m→∞m \to \infty; the model is trained once and not re-trained during monitoring. The paper takes γ=0\gamma = 0. Source: Chu et al. (1996).

Data 1

D8 mean change model: A data-generating process with a changepoint at an unknown location k*k^*: Yt|yt−1,…,y1∼F,t=1,…,m+k*Yt|yt−1,…,y1∼G,t=m+k*+1,m+k*+2,…\begin{aligned} Y_t | y_{t-1}, \ldots, y_1 &\sim F, \quad t = 1, \ldots, m + k^* \\ Y_t | y_{t-1}, \ldots, y_1 &\sim G, \quad t = m + k^* + 1, m + k^* + 2, \ldots \end{aligned} where FF and GG differ in expectation by δμ,m≠0\delta_{\mu,m} \ne 0. The raw data may be temporally dependent, trended, seasonal or depend on exogenous variables.

Context

Extends B1 · from C1
Aue et al. (2015) monitor the residuals of ARMA models only; the claim covers the forecast errors of any forecasting model that satisfies the two assumptions.

References

  1. Chu, C.-S. J., Stinchcombe, M., White, H., 1996. Monitoring Structural Change. Econometrica 64, 1045. https://doi.org/10.2307/2171955 Used by: D24
  2. PAGE, E. S., 1954. CONTINUOUS INSPECTION SCHEMES. Biometrika 41, 100–115. https://doi.org/10.1093/biomet/41.1-2.100 Used by: D2

Score

Scores are computed by rules from the claimset file (EntryLib rubric 0.7): they check the record, not the science, and may change as the rubric and reviews develop. The rubric.

C1: 70, Partly supported

Novelty: extends earlier work, by the authors

  1. −25 Evidence Rests on an unchecked proof; evidence named without the observations the claim reports; an argument from other claims; cited work only. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
  2. −3 Open problems Assumption C18 is not checked: The forecasting model has a constant bias before the changepoint: 𝔼F[Yt|y~t]−𝔼ℱ[Yt|y~t]=bμ\mathbb{E}_F[Y_t|\tilde{y}_t] - \mathbb{E}_{\mathcal{F}}[Y_t|\tilde{y}_t] = b_\mu for t=1,…,m+k*t = 1, \ldots, m + k^*, where bμb_\mu is unknown and may be zero (the paper's Assumption 1, the constant-bias assumption). To remove it: Test the assumption, or analyse how sensitive the claim is to it.
  3. −1 Open problems Assumption C19 is not checked: After the changepoint the forecasting model does not fully adjust to the change: 𝔼G[Yt|y~t]−𝔼ℱ[Yt|y~t]=bμ+Δμ,m\mathbb{E}_G[Y_t|\tilde{y}_t] - \mathbb{E}_{\mathcal{F}}[Y_t|\tilde{y}_t] = b_\mu + \Delta_{\mu,m} for t=m+k*+1,m+k*+2,…t = m + k^* + 1, m + k^* + 2, \ldots, where Δμ,m≠0\Delta_{\mu,m} \ne 0 is a function of the change size in the raw data (the paper's Assumption 2, the incomplete-adjustment assumption). To remove it: Test the assumption, or analyse how sensitive the claim is to it.
  4. −1 Open problems Limitation L1 (reduces transferability, not mitigated): Forecasting models that adapt to a change in mean, such as ETS models with a positive smoothing parameter, do not satisfy the incomplete-adjustment assumption, so a mean change does not stay in their forecast errors. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.