The claim
Under the mean change model, the one-step-ahead forecast errors of a forecasting model that satisfies the two forecasting-model assumptions have a change in mean of size at the changepoint: for and afterwards.
Scope: data: univariate time series; conditions: one-step-ahead forecasts, a single changepoint after the training sample · Proof: E1 · Assumes: C18, C19 · Limitations: L1 · Extends: B1 · Verification: not checked
Evidence
Limitations
Forecasting models that adapt to a change in mean, such as ETS models with a positive smoothing parameter, do not satisfy the incomplete-adjustment assumption, so a mean change does not stay in their forecast errors.
Assumptions
Definitions
Methods 4
D1 forecast error monitoring framework: A framework that monitors a forecasting model online by applying sequential changepoint detection to its one-step-ahead forecast errors. For a univariate series and a forecasting model with one-step-ahead point forecast , the forecast errors are The first forecast errors are the Phase I (training) sample; monitoring starts at and stops when a detector, the mean detector or the squared-error detector, crosses its threshold. Any forecasting approach can supply the errors: model-based, machine learning or expert judgement.
D2 · part of D1 mean detector $D_\mu(m,k)$: Page's CUSUM detector applied to the forecast errors of the forecast error monitoring framework: with stopping time where is the standard deviation of the forecast errors in the Phase I sample. The weight function is that of Page's CUSUM detector, and the critical constant comes from the limit distribution of the detector under the null hypothesis. Source: PAGE (1954).
D3 · part of D1 squared-error detector $D_\xi(m,k)$: Page's CUSUM detector applied to the centred squared forecast errors of the forecast error monitoring framework: with stopping time where is the mean of the forecast errors in the Phase I samples and the standard deviation of the centred squared forecast errors in that sample. is that of Page's CUSUM detector and comes from the limit distribution of the detector under the null hypothesis. The paper prints the second sum of without the square, a typographical error that the guarantor, an author of the paper, has confirmed.
D24 Page's CUSUM detector $D(m,k)$: The sequential detector of a mean change in a series that follows the change in mean model with zero-mean variables , a training period of length without changepoints and a change of size at time after it. With the null hypothesis is rejected in favour of at the stopping time where is a weakly consistent estimator, from the training samples, of the constant of the weak invariance principle, the weight function is and the critical value is taken from the limit distribution under , so that under for the chosen false alarm rate and under . All limits are for ; the model is trained once and not re-trained during monitoring. The paper takes . Source: Chu et al. (1996).
Data 1
D8 mean change model: A data-generating process with a changepoint at an unknown location : where and differ in expectation by . The raw data may be temporally dependent, trended, seasonal or depend on exogenous variables.
Context
References
- Chu, C.-S. J., Stinchcombe, M., White, H., 1996. Monitoring Structural Change. Econometrica 64, 1045. https://doi.org/10.2307/2171955 Used by: D24
- PAGE, E. S., 1954. CONTINUOUS INSPECTION SCHEMES. Biometrika 41, 100–115. https://doi.org/10.1093/biomet/41.1-2.100 Used by: D2
Score
Scores are computed by rules from the claimset file (EntryLib rubric 0.7): they check the record, not the science, and may change as the rubric and reviews develop. The rubric.
C1: 70, Partly supported
Novelty: extends earlier work, by the authors
- −25 Evidence Rests on an unchecked proof; evidence named without the observations the claim reports; an argument from other claims; cited work only. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −3 Open problems Assumption C18 is not checked: The forecasting model has a constant bias before the changepoint: for , where is unknown and may be zero (the paper's Assumption 1, the constant-bias assumption). To remove it: Test the assumption, or analyse how sensitive the claim is to it.
- −1 Open problems Assumption C19 is not checked: After the changepoint the forecasting model does not fully adjust to the change: for , where is a function of the change size in the raw data (the paper's Assumption 2, the incomplete-adjustment assumption). To remove it: Test the assumption, or analyse how sensitive the claim is to it.
- −1 Open problems Limitation L1 (reduces transferability, not mitigated): Forecasting models that adapt to a change in mean, such as ETS models with a positive smoothing parameter, do not satisfy the incomplete-adjustment assumption, so a mean change does not stay in their forecast errors. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.