Claim C7 · descriptive · entrylib:2026.000010@v1 · 2026-11-16

Online Detection of Forecast Model Inadequacies Using Forecast Errors · C7

Principal claim C7 of this paper (Svetunkov, I. (guarantor), 2026), with what it rests on. Read the whole paper.

Text file for AI

Text file: the claim with what it rests on as plain text, to attach when your assistant cannot open the address.

The message

Read https://entrylib.org/p/2026.000010/C7, one claim of a research paper with the evidence it rests on. Tell me briefly what it says and how well it is supported. If you cannot open the link, tell me instead of guessing. Then wait for my questions.

The claim

SupportS 39C7 · descriptive · novelty: new, by the authors

On the GRP admissions series, both detectors of the forecast error monitoring framework have a shorter detection delay than model-based sequential changepoint detection, whose minimum possible detection delay on data with yearly seasonality is 24 months.

Claims it rests on

SupportS 54C13 · descriptive

On the GRP admissions series, the detection delay of the mean detector was 4 data points, counted from the changepoint that the retrospective analysis of Taib et al. (2021) identified.

SupportS 54C14 · descriptive

On the GRP admissions series, the detection delay of the squared-error detector was 2 time points.

Evidence

E10 · quant · other · secondary · n 36 · cited by C13, C14

Both detectors applied sequentially to the one-step-ahead forecast errors of a SARMA(1,0,0)(1,0,0)12_{12} model of the monthly proportion of GRP A&E admissions, fitted on the data up to 2013, with 2013 to 2016 as the Phase I sample of size 36 and monitoring from 2016 (Section 6 and Figure 6 of the paper).

W2 · argument · deductive · premises C13, C14

A model-based sequential detector must fit the post-change model, here a SARMA(1,0,0)(1,0,0) model with a 12-month season, which the paper states needs at least two years of data after the change; the detectors on the forecast errors signalled four and two observations after it.

Limitations

L5 · sample size · reduces transferability · affects C13, C14, C7 · mitigation: none

The application uses one monthly series with a Phase I sample of 36 observations.

Definitions

Methods 5

D1 forecast error monitoring framework: A framework that monitors a forecasting model online by applying sequential changepoint detection to its one-step-ahead forecast errors. For a univariate series YtY_t and a forecasting model ℱ\mathcal{F} with one-step-ahead point forecast y^t(1)=𝔼ℱ[Yt|yt−1,…,y1]\hat{y}_t(1) = \mathbb{E}_{\mathcal{F}}[Y_t | y_{t-1}, \ldots, y_1], the forecast errors are et=Yt−y^t(1).e_t = Y_t - \hat{y}_t(1). The first mm forecast errors are the Phase I (training) sample; monitoring starts at t=m+1t = m + 1 and stops when a detector, the mean detector or the squared-error detector, crosses its threshold. Any forecasting approach can supply the errors: model-based, machine learning or expert judgement.

D2 · part of D1 mean detector $D_\mu(m,k)$: Page's CUSUM detector applied to the forecast errors of the forecast error monitoring framework: Qμ(m,k)=∑t=m+1m+ket−km∑t=1metDμ(m,k)=max0≤i≤k⁡|Qμ(m,k)−Qμ(m,i)|\begin{aligned} Q_\mu(m,k) &= \sum_{t=m+1}^{m+k} e_t - \frac{k}{m} \sum_{t=1}^{m} e_t \\ D_\mu(m,k) &= \max_{0 \le i \le k} \left| Q_\mu(m,k) - Q_\mu(m,i) \right| \end{aligned} with stopping time τμ,m=min⁡{k≥1:Dμ(m,k)≥σ^μ,mcμ,αg(m,k,γ)},\tau_{\mu,m} = \min\{k \ge 1 : D_\mu(m,k) \ge \hat{\sigma}_{\mu,m} c_{\mu,\alpha} g(m,k,\gamma)\}, where σ^μ,m\hat{\sigma}_{\mu,m} is the standard deviation of the forecast errors in the Phase I sample. The weight function g(m,k,γ)g(m,k,\gamma) is that of Page's CUSUM detector, and the critical constant cμ,αc_{\mu,\alpha} comes from the limit distribution of the detector under the null hypothesis. Source: PAGE (1954).

D3 · part of D1 squared-error detector $D_\xi(m,k)$: Page's CUSUM detector applied to the centred squared forecast errors of the forecast error monitoring framework: Qξ(m,k)=∑t=m+1m+k(et−b^μ,m)2−km∑t=1m(et−b^μ,m)2Dξ(m,k)=max0≤i≤k⁡|Qξ(m,k)−Qξ(m,i)|\begin{aligned} Q_\xi(m,k) &= \sum_{t=m+1}^{m+k} (e_t - \hat{b}_{\mu,m})^2 - \frac{k}{m} \sum_{t=1}^{m} (e_t - \hat{b}_{\mu,m})^2 \\ D_\xi(m,k) &= \max_{0 \le i \le k} \left| Q_\xi(m,k) - Q_\xi(m,i) \right| \end{aligned} with stopping time τξ,m=min⁡{k≥1:Dξ(m,k)≥σ^ξ,mcξ,αg(m,k,γ)},\tau_{\xi,m} = \min\{k \ge 1 : D_\xi(m,k) \ge \hat{\sigma}_{\xi,m} c_{\xi,\alpha} g(m,k,\gamma)\}, where b^μ,m\hat{b}_{\mu,m} is the mean of the forecast errors in the mm Phase I samples and σ^ξ,m\hat{\sigma}_{\xi,m} the standard deviation of the centred squared forecast errors in that sample. g(m,k,γ)g(m,k,\gamma) is that of Page's CUSUM detector and cξ,αc_{\xi,\alpha} comes from the limit distribution of the detector under the null hypothesis. The paper prints the second sum of Qξ(m,k)Q_\xi(m,k) without the square, a typographical error that the guarantor, an author of the paper, has confirmed.

D21 model-based sequential changepoint detection: Sequential changepoint detection that fits the model to the data before and after a potential changepoint and decides by a likelihood-ratio style statistic.

D24 Page's CUSUM detector $D(m,k)$: The sequential detector of a mean change in a series YtY_t that follows the change in mean model Yt={λ+ϵt,t=1,…,m+k*λ+ϵt+Δm,t=m+k*+1,m+k*+2,…Y_t = \begin{cases} \lambda + \epsilon_t, & t = 1, \ldots, m + k^* \\ \lambda + \epsilon_t + \Delta_m, & t = m + k^* + 1, m + k^* + 2, \ldots \end{cases} with zero-mean variables ϵt\epsilon_t, a training period of length mm without changepoints and a change of size Δm\Delta_m at time k*k^* after it. With Q(m,k)=∑t=m+1m+kYt−km∑t=1mYtD(m,k)=max0≤i≤k⁡|Q(m,k)−Q(m,i)|\begin{aligned} Q(m,k) &= \sum_{t=m+1}^{m+k} Y_t - \frac{k}{m} \sum_{t=1}^{m} Y_t \\ D(m,k) &= \max_{0 \le i \le k} \left| Q(m,k) - Q(m,i) \right| \end{aligned} the null hypothesis H0:Δm=0H_0: \Delta_m = 0 is rejected in favour of HA:Δm≠0H_A: \Delta_m \ne 0 at the stopping time τm=min⁡{k≥1:D(m,k)≥σ^mcαg(m,k,γ)},\tau_m = \min\{k \ge 1 : D(m,k) \ge \hat{\sigma}_m c_\alpha g(m,k,\gamma)\}, where σ^m\hat{\sigma}_m is a weakly consistent estimator, from the mm training samples, of the constant σ\sigma of the weak invariance principle, the weight function is g(m,k,γ)=m(1+km)(km+k)γ,γ∈[0,1/2).g(m,k,\gamma) = \sqrt{m}\left(1 + \frac{k}{m}\right)\left(\frac{k}{m+k}\right)^{\gamma}, \quad \gamma \in [0, 1/2). and the critical value cαc_\alpha is taken from the limit distribution under H0H_0, so that limm→∞⁡P(τm<∞)=α\lim_{m\to\infty} P(\tau_m < \infty) = \alpha under H0H_0 for the chosen false alarm rate α\alpha and limm→∞⁡P(τm<∞)=1\lim_{m\to\infty} P(\tau_m < \infty) = 1 under HAH_A. All limits are for m→∞m \to \infty; the model is trained once and not re-trained during monitoring. The paper takes γ=0\gamma = 0. Source: Chu et al. (1996).

Data 1

D20 GRP admissions series: The monthly proportion of A&E admissions related to gallstone-related pathology (GRP) in participating hospitals across England, 2010 to 2020, based on NHS Hospital Episode Statistics. A SARMA(1,0,0)(1,0,0)12_{12} model fitted to the data up to 2013 gives one-step-ahead forecasts for the rest; 2013 to 2016 is the Phase I sample of size m=36m = 36 and monitoring starts in 2016. Source: Taib et al. (2021).

Variables and measures 1

D22 detection delay: The number of observations between a change and the time a detector signals it.

Context

Addresses Taib et al. (2021) · own earlier work · from C7
Taib et al. (2021) found the changepoint retrospectively, with all the data from 2010 to 2020; the claim is about detecting it sequentially.

References

  1. Chu, C.-S. J., Stinchcombe, M., White, H., 1996. Monitoring Structural Change. Econometrica 64, 1045. https://doi.org/10.2307/2171955 Used by: D24
  2. Grundy, T., Killick, R., Svetunkov, I., 2026. Online Detection of Forecast Model Inadequacies Using Forecast Errors. Journal of Time Series Analysis 47, 715–726. https://doi.org/10.1111/jtsa.12843 Used by: A6
  3. PAGE, E. S., 1954. CONTINUOUS INSPECTION SCHEMES. Biometrika 41, 100–115. https://doi.org/10.1093/biomet/41.1-2.100 Used by: D2
  4. Taib, A., Killick, R., Hussain, K., Patel, H., Obeidallah, M. R., 2021. Is there seasonal variation in gallstone related admissions in England?. HPB 23, 1732–1743. https://doi.org/10.1016/j.hpb.2021.04.009 Used by: Context, D20

Artefacts

  • A6 · figure · data_plot · Grundy et al. (2026) The proportion of GRP A&E admissions, the forecast errors of a SARIMA(1,0,0)(1,0,0)12 model, and the two detectors with their thresholds.

Score

Scores are computed by rules from the claimset file (EntryLib rubric 0.7): they check the record, not the science, and may change as the rubric and reviews develop. The rubric.

C7: 39, Weakly supported

Novelty: new, by the authors

  1. −45 Checkable Verifiability: unverifiable. To remove it: Make the data, code and models the claim rests on available, or state why they cannot be.
  2. −10 Evidence Rests on an unchecked proof; evidence named without the observations the claim reports; an argument from other claims; cited work only. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
  3. −5 Plan A principal claim reported without a plan recorded beforehand. To remove it: Plan the measurement in the lab record or a registration before running it, or publish the lab record that shows every run.
  4. −1 Open problems Limitation L5 (reduces transferability, not mitigated): The application uses one monthly series with a Phase I sample of 36 observations. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.

C13: 54, Weakly supported

  1. −45 Checkable Verifiability: unverifiable. To remove it: Make the data, code and models the claim rests on available, or state why they cannot be.
  2. −1 Open problems Limitation L5 (reduces transferability, not mitigated): The application uses one monthly series with a Phase I sample of 36 observations. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.

C14: 54, Weakly supported

  1. −45 Checkable Verifiability: unverifiable. To remove it: Make the data, code and models the claim rests on available, or state why they cannot be.
  2. −1 Open problems Limitation L5 (reduces transferability, not mitigated): The application uses one monthly series with a Phase I sample of 36 observations. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.