Claims
A term is defined before the claim that first uses it, and what supports a claim stands under its statement.
Term this claim uses
- mean change model
- A data-generating process with a changepoint at an unknown location : where and differ in expectation by . The raw data may be temporally dependent, trended, seasonal or depend on exogenous variables.
Claim C1
Under the mean change model, the one-step-ahead forecast errors of a forecasting model that satisfies the two forecasting-model assumptions have a change in mean of size at the changepoint: for and afterwards.
Proof 2 steps
Earlier work and other claims
- Extends Aue et al. (2015). Aue et al. (2015) monitor the residuals of ARMA models only; the claim covers the forecast errors of any forecasting model that satisfies the two assumptions.
ProvedRecord
- Claim
- C1 · formal · principal · extends earlier work
- Evidence
- E1 in the record
- Scope
- data: univariate time series; conditions: one-step-ahead forecasts, a single changepoint after the training sample
- Limitations
- L1
- Support score
- SupportS 70 of 100. The score checks the record, not the science.
- Record
- This claim on the record page
- To cite it
- The page of this claim
Term this claim uses
- mean and variance change model
- The mean change model extended so that the pre-change distribution and the post-change distribution differ in expectation by and in variance by (the paper's Definition 1).
Claim C2
Under the mean and variance change model and the constant-bias assumption, the change in the mean of the centred squared forecast errors is when , and when and the incomplete-adjustment assumption holds.
Proof 3 steps
Earlier work and other claims
- Extends Aue et al. (2015). Aue et al. (2015) monitor the residuals of ARMA models only; the claim covers the forecast errors of any forecasting model that satisfies the two assumptions.
ProvedRecord
- Claim
- C2 · formal · principal · extends earlier work
- Evidence
- E2 in the record
- Scope
- data: univariate time series; conditions: one-step-ahead forecasts
- Support score
- SupportS 71 of 100. The score checks the record, not the science.
- Record
- This claim on the record page
- To cite it
- The page of this claim
Term this claim uses
- Page's CUSUM detector $D(m,k)$
- The sequential detector of a mean change in a series that follows the change in mean model with zero-mean variables , a training period of length without changepoints and a change of size at time after it. With the null hypothesis is rejected in favour of at the stopping time where is a weakly consistent estimator, from the training samples, of the constant of the weak invariance principle, the weight function is and the critical value is taken from the limit distribution under , so that under for the chosen false alarm rate and under . All limits are for ; the model is trained once and not re-trained during monitoring. The paper takes .
Claim C24
For Page's CUSUM detector under the null hypothesis of no change, for every real , where is a standard Brownian motion (Theorem 1 of the paper's Supporting Information, from Fremdt 2015).
Rests on Fremdt (2015)
What the record holds of this claimRecord
- Claim
- C24 · formal · supporting · states what earlier work establishes
- Support
- cited
- Scope
- conditions: no change in the monitored series, limit for a training sample size going to infinity
- Support score
- SupportS 77 of 100. The score checks the record, not the science.
- Record
- This claim on the record page
Claim C25
For Page's CUSUM detector under the alternative of a mean change, for every real , where is the unique solution of and with equal to when , to when , and to when , where is the standard normal distribution function, a standard Brownian motion and the unique solution of with (Theorem 2 of the paper's Supporting Information, from Fremdt 2014).
Rests on Fremdt (2014)
What the record holds of this claimRecord
- Claim
- C25 · formal · supporting · states what earlier work establishes
- Support
- cited
- Scope
- conditions: a change in mean satisfying Assumptions C to E of the paper's Supporting Information, limit for a training sample size going to infinity
- Support score
- SupportS 76 of 100. The score checks the record, not the science.
- Record
- This claim on the record page
Terms this claim uses
- mean detector $D_\mu(m,k)$
- Page's CUSUM detector applied to the forecast errors of the forecast error monitoring framework: with stopping time where is the standard deviation of the forecast errors in the Phase I sample. The weight function is that of Page's CUSUM detector, and the critical constant comes from the limit distribution of the detector under the null hypothesis.
- forecast error monitoring framework
- A framework that monitors a forecasting model online by applying sequential changepoint detection to its one-step-ahead forecast errors. For a univariate series and a forecasting model with one-step-ahead point forecast , the forecast errors are The first forecast errors are the Phase I (training) sample; monitoring starts at and stops when a detector, the mean detector or the squared-error detector, crosses its threshold. Any forecasting approach can supply the errors: model-based, machine learning or expert judgement.
- squared-error detector $D_\xi(m,k)$
- Page's CUSUM detector applied to the centred squared forecast errors of the forecast error monitoring framework: with stopping time where is the mean of the forecast errors in the Phase I samples and the standard deviation of the centred squared forecast errors in that sample. is that of Page's CUSUM detector and comes from the limit distribution of the detector under the null hypothesis. The paper prints the second sum of without the square, a typographical error that the guarantor, an author of the paper, has confirmed.
Claim C3
For the squared-error detector under the mean and variance change model, the limit distribution of Theorem 1 of the paper's Supporting Information holds under the null hypothesis of no change, and the limit distribution of its Theorem 2 holds for the stopping time under the alternative when the change size is large enough in the sense of that document's Assumptions C to E.
Proof 3 steps
- The centred squared forecast errors follow a mean change model with change size and zero-mean errors . (by claim C2, claim C20)
- Under the null hypothesis the limit distribution of Theorem 1 of the Supporting Information holds with and . (by claim C24)
- Under the alternative, with satisfying Assumptions C to E, the limit distribution of Theorem 2 of the Supporting Information holds with and . (by claim C21, claim C25) ∎
Earlier work and other claims
- Extends Aue et al. (2015). Aue et al. (2015) monitor the residuals of ARMA models only; the claim covers the forecast errors of any forecasting model that satisfies the two assumptions.
ProvedRecord
- Claim
- C3 · formal · principal · extends earlier work
- Evidence
- E3 in the record
- Scope
- conditions: change sizes satisfying Assumptions C to E of the paper's Supporting Information
- Support score
- SupportS 69 of 100. The score checks the record, not the science.
- Record
- This claim on the record page
- To cite it
- The page of this claim
Term this claim uses
- ETS(A,N,N) forecasting model
- The local level exponential smoothing model with additive error, no trend and no seasonality, whose one-step-ahead forecasts follow where is the forecast error at time and is the smoothing parameter.
Claim C4
For an ETS(A,N,N) forecasting model under the mean change model with pre-change expectation , the expected forecast error steps after the changepoint is for and for , an expectation that returns to zero as grows when and does not satisfy the incomplete-adjustment assumption unless .
Proof 2 steps
- The forecast error after the changepoint is written by repeated substitution of the previous forecast errors into until the pre-change regime is reached, which gives the stated expectation.
- As increases the expectation returns to zero when , and faster for larger , so no constant change remains. ∎
Earlier work and other claims
- Answers Aue et al. (2015). The closest result found: Aue et al. (2015) give the change size in ARMA residuals; the claim gives it for an exponential smoothing forecasting model, where it vanishes.
ProvedRecord
- Claim
- C4 · formal · principal · no earlier result found
- Evidence
- E4 in the record
- Scope
- conditions: ETS(A,N,N), a change in mean
- Support score
- SupportS 75 of 100. The score checks the record, not the science.
- Record
- This claim on the record page
- To cite it
- The page of this claim
Terms this claim uses
- seasonal mean change scenario
- Simulated data whose errors follow an ARMA(2,1) model with , , and , with a mean change of size at and a seasonal component of frequency 12 whose seasonal means are 12 equally spaced points of , .
- seasonal trend change scenario
- Simulated data whose errors follow an ARMA(1,1) model with , , and : no trend before the change and a trend of gradient after it, with a seasonal component of frequency 7 whose seasonal means are 7 equally spaced points of , .
- autoregressive parameter change scenarios
- Simulated data with AR(1) errors, , whose autoregressive parameter changes at the changepoint from (scenario 1) or (scenario 2) to varying values , with quarterly seasonality given by quarter means . The ETS forecasting model is ETS(A,N,A) with estimated smoothing parameters.
- average detection delay (ADD)
- The mean of the detection delays over the simulated series in which a change was signalled within the length of the data.
- Raw CUSUM
- Page's CUSUM detector applied to the raw data instead of forecast errors; for variance changes, to the centred squared raw data. In the paper's simulations its standard deviation is the Bartlett long run variance estimator from the sandwich R package.
- false detection probability (FDP)
- The proportion of simulated series in which a change was detected before the true change had occurred.
Claim C5
On the seasonal mean change scenario, the seasonal trend change scenario and the autoregressive parameter change scenarios, monitoring forecast errors with the forecast error monitoring framework detected the changes with a shorter average detection delay (ADD) than Raw CUSUM, at a low false detection probability (FDP).
Evidence
Everything recorded for this evidence: E5 in the record.
Evidence
Everything recorded for this evidence: E6 in the record.
Earlier work and other claims
- Answers Robbins et al. (2011). The closest result found: Robbins et al. (2011) showed, for offline changepoint detection, that using residuals within the CUSUM procedure improves on the raw data under temporal dependence; the claim concerns sequential detection with forecast errors of any forecasting model.
exploratory · moderate support, by its authorRecord
- Claim
- C5 · descriptive · principal · no earlier result found
- Evidence
- E5, E6 in the record
- Scope
- data: simulated seasonal time series with ARMA errors and normally distributed innovations; conditions: training sample of 300 observations, false alarm rate 0.05
- Limitations
- L2, L3
- Support score
- SupportS 71 of 100. The score checks the record, not the science.
- Record
- This claim on the record page
- To cite it
- The page of this claim
Term this claim uses
- Royal Mail forecast errors
- Forecast errors of the daily number of parcels to be delivered from one Royal Mail delivery office, from 1 July 2020 to 31 December 2020. The underlying data and forecasts are not known to the authors. July to October is the Phase I sample; monitoring starts on 1 November 2020.
Claim C6
On the Royal Mail forecast errors, the mean detector and the squared-error detector of the forecast error monitoring framework both signalled a changepoint on 2 December 2020.
Evidence
- R1 date of detection = 2020-12-02 (detector: mean detector $D_\mu(m,k)$)
- R2 date of detection = 2020-12-02 (detector: squared-error detector $D_\xi(m,k)$)
Everything recorded for this evidence: E9 in the record.
Earlier work and other claims
- Answers Aue et al. (2015). The closest result found: Aue et al. (2015) apply residual-based monitoring of ARMA models to EEG and IBM data; here only the forecast errors are known, not the model or the data.
exploratory · low support, by its authorRecord
- Claim
- C6 · descriptive · principal · no earlier result found
- Evidence
- E9 in the record
- Scope
- setting: one Royal Mail delivery office; where: United Kingdom; period: 1 July 2020 to 31 December 2020
- Limitations
- L4
- Support score
- SupportS 15 of 100. The score checks the record, not the science.
- Record
- This claim on the record page
- To cite it
- The page of this claim
Terms this claim uses
- GRP admissions series
- The monthly proportion of A&E admissions related to gallstone-related pathology (GRP) in participating hospitals across England, 2010 to 2020, based on NHS Hospital Episode Statistics. A SARMA(1,0,0)(1,0,0) model fitted to the data up to 2013 gives one-step-ahead forecasts for the rest; 2013 to 2016 is the Phase I sample of size and monitoring starts in 2016.
- detection delay
- The number of observations between a change and the time a detector signals it.
- model-based sequential changepoint detection
- Sequential changepoint detection that fits the model to the data before and after a potential changepoint and decides by a likelihood-ratio style statistic.
Claim C7
On the GRP admissions series, both detectors of the forecast error monitoring framework have a shorter detection delay than model-based sequential changepoint detection, whose minimum possible detection delay on data with yearly seasonality is 24 months.
Argument
A model-based sequential detector must fit the post-change model, here a SARMA(1,0,0)(1,0,0) model with a 12-month season, which the paper states needs at least two years of data after the change; the detectors on the forecast errors signalled four and two observations after it.
Earlier work and other claims
- Answers Taib et al. (2021). Taib et al. (2021) found the changepoint retrospectively, with all the data from 2010 to 2020; the claim is about detecting it sequentially.
exploratory · low support, by its authorRecord
- Claim
- C7 · descriptive · principal · no earlier result found
- Support
- argument
- Scope
- setting: A&E admissions in participating hospitals; where: England; data: monthly proportions with yearly seasonality
- Limitations
- L5
- Support score
- SupportS 39 of 100. The score checks the record, not the science.
- Record
- This claim on the record page
- To cite it
- The page of this claim
Terms this claim uses
- ARMA Forecast Errors
- The detector of the forecast error monitoring framework, [[D2:]] or [[D3:]] by the type of change, applied to the forecast errors of an ARMA forecasting model fitted with the forecast R package. For seasonal data the model is a seasonal ARMA model chosen by auto.arima using AIC, with no differencing. The standard deviation is the traditional estimator, the errors being taken as i.i.d.
- detection probability (DP)
- The proportion of simulated series in which a change was detected within the length of the monitoring data.
Claim C8
On the seasonal mean change scenario, ARMA Forecast Errors has a high detection probability (DP) and a lower average detection delay (ADD) than Raw CUSUM across all change sizes.
Evidence
Everything recorded for this evidence: E5 in the record.
Earlier work and other claims
- Answers Robbins et al. (2011). Robbins et al. (2011) compared residuals with raw data for offline mean shift testing; this is the sequential case.
exploratory · moderate support, by its authorRecord
- Claim
- C8 · descriptive · supporting · no earlier result found
- Evidence
- E5 in the record
- Scope
- data: simulated monthly-type seasonal data with ARMA(2,1) errors; conditions: mean change sizes 0.5 to 2.5
- Limitations
- L2, L3
- Support score
- SupportS 86 of 100. The score checks the record, not the science.
- Record
- This claim on the record page
Claim C9
On the seasonal trend change scenario, ARMA Forecast Errors and Raw CUSUM both have an extremely high detection probability (DP), and ARMA Forecast Errors has the lower average detection delay (ADD) across all change sizes.
Evidence
Everything recorded for this evidence: E5 in the record.
Earlier work and other claims
- Answers Horváth et al. (2004). Horváth et al. (2004) monitor changes in linear models, including trend, within a specified model; here the trend change is detected through forecast errors.
exploratory · moderate support, by its authorRecord
- Claim
- C9 · descriptive · supporting · no earlier result found
- Evidence
- E5 in the record
- Scope
- data: simulated daily-type seasonal data with ARMA(1,1) errors; conditions: trend gradients 0.008 to 0.042
- Limitations
- L2, L3
- Support score
- SupportS 86 of 100. The score checks the record, not the science.
- Record
- This claim on the record page
Term this claim uses
- ETS Forecast Errors
- The detector of the forecast error monitoring framework, [[D2:]] or [[D3:]] by the type of change, applied to the forecast errors of an ETS forecasting model fitted with the es function of the smooth R package. The standard deviation is the traditional estimator, the errors being taken as i.i.d.
Claim C10
On the autoregressive parameter change scenarios, ARMA Forecast Errors and ETS Forecast Errors each give a better detection probability (DP) and average detection delay (ADD) than Raw CUSUM in all scenarios.
Evidence
Everything recorded for this evidence: E6 in the record.
Earlier work and other claims
- Answers Aue et al. (2015). Aue et al. (2015) detect changes in ARMA parameters from ARMA residuals; here the comparison is with monitoring the raw data, and includes an ETS forecasting model.
exploratory · moderate support, by its authorRecord
- Claim
- C10 · descriptive · supporting · no earlier result found
- Evidence
- E6 in the record
- Scope
- data: simulated quarterly-type seasonal data with AR(1) errors
- Limitations
- L2, L3
- Support score
- SupportS 78 of 100. The score checks the record, not the science.
- Record
- This claim on the record page
Term this claim uses
- compared detectors
- The three detectors compared in the paper's simulations.
Claim C11
On the autoregressive parameter change scenarios, the false detection probability (FDP) is below 0.075 for each of the compared detectors.
Evidence
- R1 false detection probability (FDP) = < 0.075 (upper limit) 1 (compared detectors: each detector, scenario: autoregressive parameter change scenarios)
Everything recorded for this evidence: E6 in the record.
exploratory · moderate support, by its authorRecord
- Claim
- C11 · descriptive · supporting · not applicable
- Evidence
- E6 in the record
- Scope
- data: simulated quarterly-type seasonal data with AR(1) errors; conditions: false alarm rate set at 0.05
- Limitations
- L2
- Support score
- SupportS 89 of 100. The score checks the record, not the science.
- Record
- This claim on the record page
Claim C12
On the seasonal mean change scenario and the seasonal trend change scenario, the false detection probability (FDP) of the compared detectors is low across all change sizes.
Evidence
Everything recorded for this evidence: E5 in the record.
exploratory · moderate support, by its authorRecord
- Claim
- C12 · descriptive · supporting · not applicable
- Evidence
- E5 in the record
- Scope
- data: simulated seasonal data with ARMA errors
- Limitations
- L2
- Support score
- SupportS 87 of 100. The score checks the record, not the science.
- Record
- This claim on the record page
Claim C13
On the GRP admissions series, the detection delay of the mean detector was 4 data points, counted from the changepoint that the retrospective analysis of Taib et al. (2021) identified.
Evidence
- R1 detection delay = 4 observations (detector: mean detector $D_\mu(m,k)$, reference: the changepoint identified by Taib et al. (2021))
Everything recorded for this evidence: E10 in the record.
exploratory · low support, by its authorRecord
- Claim
- C13 · descriptive · supporting · not applicable
- Evidence
- E10 in the record
- Scope
- setting: A&E admissions in participating hospitals; where: England; period: 2010 to 2020; data: monthly proportions
- Limitations
- L5
- Support score
- SupportS 54 of 100. The score checks the record, not the science.
- Record
- This claim on the record page
Claim C14
On the GRP admissions series, the detection delay of the squared-error detector was 2 time points.
Evidence
- R2 detection delay = 2 observations (detector: squared-error detector $D_\xi(m,k)$)
Everything recorded for this evidence: E10 in the record.
exploratory · low support, by its authorRecord
- Claim
- C14 · descriptive · supporting · not applicable
- Evidence
- E10 in the record
- Scope
- setting: A&E admissions in participating hospitals; where: England; period: 2010 to 2020; data: monthly proportions
- Limitations
- L5
- Support score
- SupportS 54 of 100. The score checks the record, not the science.
- Record
- This claim on the record page
Term this claim uses
- ARMA forecasting model
- A forecasting model for the causal and invertible ARMA(,) process with mean parameters , autoregressive polynomial , moving average polynomial and independent zero-mean errors with variance .
Claim C15
An ARMA forecasting model satisfies the two forecasting-model assumptions for a change of size in its mean parameter , the change in its forecast errors being with .
Rests on Aue et al. (2015)
Evidence
Everything recorded for this evidence: E7 in the record.
What the record holds of this claimRecord
- Claim
- C15 · formal · secondary · states what earlier work establishes
- Evidence
- E7 in the record
- Scope
- conditions: causal and invertible ARMA process, perfectly estimated ARMA model
- Support score
- SupportS 82 of 100. The score checks the record, not the science.
- Record
- This claim on the record page
Claim C16
For a change of size in the variance of the data, the change in the squared forecast errors of an ARMA forecasting model is , in agreement with the variance-change formula of the paper's Proposition 1.
Rests on Aue et al. (2015)
What the record holds of this claimRecord
- Claim
- C16 · formal · secondary · states what earlier work establishes
- Support
- cited
- Scope
- conditions: causal and invertible ARMA process, perfectly estimated ARMA model
- Support score
- SupportS 70 of 100. The score checks the record, not the science.
- Record
- This claim on the record page
Claim C17
A variance change in the raw data produces a mean change in the squared forecast errors of an ETS forecasting model that the squared-error detector can detect.
Argument
The variance-change formula of Proposition 1 needs only the constant-bias assumption, which an ETS forecasting model can satisfy, so a variance change in the raw data gives a mean change in its squared forecast errors; the limit distributions of the squared-error detector then apply. The mean of the forecast errors themselves returns to zero after a mean change, so only the variance change remains detectable.
Evidence
Everything recorded for this evidence: E8 in the record.
Earlier work and other claims
- Answers Inclán and Tiao (1994). The closest result found: Inclán and Tiao (1994) detect variance changes from the centred squared data; the claim applies the idea to the squared forecast errors of an ETS model.
What the record holds of this claimRecord
- Claim
- C17 · formal · secondary · no earlier result found
- Evidence
- E8 in the record
- Scope
- conditions: variance change sizes satisfying Assumptions C to E of the paper's Supporting Information
- Support score
- SupportS 64 of 100. The score checks the record, not the science.
- Record
- This claim on the record page
Claim C23
For the mean detector under the mean change model, the limit distribution of Theorem 1 of the paper's Supporting Information holds under the null hypothesis of no change with and , and the limit distribution of its Theorem 2 holds under the alternative with and when the change size satisfies its Assumptions C to E.
Proof 3 steps
- The forecast errors follow the change in mean model of Page's CUSUM detector with , and zero-mean errors . (by claim C1, claim C20)
- Under the null hypothesis the limit distribution of Theorem 1 holds with and . (by claim C24)
- Under the alternative, with satisfying Assumptions C to E, the limit distribution of Theorem 2 holds with and . (by claim C21, claim C25) ∎
Earlier work and other claims
- Extends Aue et al. (2015). Aue et al. (2015) give the limit distributions for the residuals of ARMA models; the claim gives them for the forecast errors of any forecasting model that satisfies the two assumptions.
ProvedRecord
- Claim
- C23 · formal · supporting · extends earlier work
- Evidence
- E11 in the record
- Scope
- data: univariate time series; conditions: one-step-ahead forecasts, change sizes satisfying Assumptions C to E of the paper's Supporting Information
- Support score
- SupportS 68 of 100. The score checks the record, not the science.
- Record
- This claim on the record page
2 more definitions that no claim names (designs, data, cases) are on the record page.
Assumptions
The forecasting model has a constant bias before the changepoint: for , where is unknown and may be zero (the paper's Assumption 1, the constant-bias assumption). Rested on by C1, C2, C3, C23, C17, E1, E2, E3, E11.
After the changepoint the forecasting model does not fully adjust to the change: for , where is a function of the change size in the raw data (the paper's Assumption 2, the incomplete-adjustment assumption). Rested on by C1, C2, C3, C23, E1, E2, E3, E11.
The zero-mean variables of the monitored series, here and left in the forecast errors after the bias is removed, satisfy the weak invariance principle of Assumptions A and B of the paper's Supporting Information: , and there are Wiener processes and a constant with Rested on by C3, C23, C24, C25, E3, E11.
Limitations
Forecasting models that adapt to a change in mean, such as ETS models with a positive smoothing parameter, do not satisfy the incomplete-adjustment assumption, so a mean change does not stay in their forecast errors.
The two forecasting-model assumptions that make the thresholds theoretically valid are not necessarily satisfied in all the simulated scenarios.
All simulated data have an ARMA error structure with normally distributed innovations and a training sample of 300 observations.
The start of the Christmas peak season at the delivery office is inferred from the rise in the mean and variance of the forecast errors; the underlying parcel volumes and forecasts are not known to the authors.
The application uses one monthly series with a Phase I sample of 36 observations.
Open questions
Whether a multivariate sequential changepoint algorithm on the vector of forecast errors of several variables or products detects changes, and how it should treat the sparsity of a change, is left as future work.
How to combine the information in forecast errors from several forecast horizons, beyond the one-step-ahead errors studied, is not established.
References
- Aue, A., Dienes, C., Fremdt, S., Steinebach, J., 2015. Reaction times of monitoring schemes for ARMA time series. Bernoulli 21, 1238–1259. https://doi.org/10.3150/14-BEJ604 Used by: Context, C15, C16
- Chu, C.-S. J., Stinchcombe, M., White, H., 1996. Monitoring Structural Change. Econometrica 64, 1045. https://doi.org/10.2307/2171955 Used by: D24
- Fremdt, S., 2014. Asymptotic distribution of the delay time in page's sequential procedure. Journal of Statistical Planning and Inference 145, 74–91. https://doi.org/10.1016/j.jspi.2013.09.001 Used by: C25
- Fremdt, S., 2015. Page's sequential procedure for change-point detection in time series regression. Statistics 49, 128–155. https://doi.org/10.1080/02331888.2013.870568 Used by: C24
- Grundy, T., Killick, R., Svetunkov, I., 2026. Online Detection of Forecast Model Inadequacies Using Forecast Errors. Journal of Time Series Analysis 47, 715–726. https://doi.org/10.1111/jtsa.12843 Used by: A0, A1, A2, A3, A4, A5, A6, A7
- Horváth, L., Hušková, M., Kokoszka, P., Steinebach, J., 2004. Monitoring changes in linear models. Journal of Statistical Planning and Inference 126, 225–251. https://doi.org/10.1016/j.jspi.2003.07.014 Used by: Context
- Inclán, C., Tiao, G. C., 1994. Use of Cumulative Sums of Squares for Retrospective Detection of Changes of Variance. Journal of the American Statistical Association 89, 913–923. https://doi.org/10.1080/01621459.1994.10476824 Used by: Context, D4
- PAGE, E. S., 1954. CONTINUOUS INSPECTION SCHEMES. Biometrika 41, 100–115. https://doi.org/10.1093/biomet/41.1-2.100 Used by: D2
- Robbins, M., Gallagher, C., Lund, R., Aue, A., 2011. Mean shift testing in correlated data. Journal of Time Series Analysis 32, 498–511. https://doi.org/10.1111/j.1467-9892.2010.00707.x Used by: Context
- Taib, A., Killick, R., Hussain, K., Patel, H., Obeidallah, M. R., 2021. Is there seasonal variation in gallstone related admissions in England?. HPB 23, 1732–1743. https://doi.org/10.1016/j.hpb.2021.04.009 Used by: Context, D20
- https://CRAN.R-project.org/package=forecast Used by: D5, A8
- https://CRAN.R-project.org/package=sandwich Used by: A10
- https://CRAN.R-project.org/package=smooth Used by: D6, A9
Score
Support of the paper: 59, Weakly supported
Scores are computed by rules from the claimset file (EntryLib rubric 0.7): they check the record, not the science, and may change as the rubric and reviews develop. The rubric. Every issue as text, for an agent preparing the next version.
The mean of its 7 principal claims; the lowest is C6 at 15. Open problems in the principal claims and what they rest on, each counted once, with the most it costs a claim:
- −45 Checkable Verifiability: unverifiable. In C6, C7, C13, C14. To remove it: Make the data, code and models the claim rests on available, or state why they cannot be.
- −25 Evidence Rests on an unchecked proof; evidence named without the observations the claim reports; an argument from other claims; cited work only. In C1, C2, C3, C4, C5, C6, C7, C10, C15, C16, C17, C23, C24, C25. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −7 Checkable Flags: availability not stated, figure without data. In C5, C8, C9, C10, C11, C12, C15, C17. To remove it: Publish what each flag says is missing.
- −6 Open problems Limitation L2 (could bias magnitude, not mitigated): The two forecasting-model assumptions that make the thresholds theoretically valid are not necessarily satisfied in all the simulated scenarios. In C5, C8, C9, C10, C11, C12. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −5 Plan A principal claim reported without a plan recorded beforehand. In C5, C6, C7. To remove it: Plan the measurement in the lab record or a registration before running it, or publish the lab record that shows every run.
- −3 Open problems Assumption C18 is not checked: The forecasting model has a constant bias before the changepoint: for , where is unknown and may be zero (the paper's Assumption 1, the constant-bias assumption). In C1, C2, C3, C6, C16, C17, C23. To remove it: Test the assumption, or analyse how sensitive the claim is to it.
- −1 Open problems Limitation L1 (reduces transferability, not mitigated): Forecasting models that adapt to a change in mean, such as ETS models with a positive smoothing parameter, do not satisfy the incomplete-adjustment assumption, so a mean change does not stay in their forecast errors. In C1, C6, C23. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L3 (reduces transferability, not mitigated): All simulated data have an ARMA error structure with normally distributed innovations and a training sample of 300 observations. In C5, C8, C9, C10. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −3 Open problems Limitation L4 (reduces precision, not mitigated): The start of the Christmas peak season at the delivery office is inferred from the rise in the mean and variance of the forecast errors; the underlying parcel volumes and forecasts are not known to the authors. In C6. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L5 (reduces transferability, not mitigated): The application uses one monthly series with a Phase I sample of 36 observations. In C7, C13, C14. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Assumption C19 is not checked: After the changepoint the forecasting model does not fully adjust to the change: for , where is a function of the change size in the raw data (the paper's Assumption 2, the incomplete-adjustment assumption). In C1, C2, C3, C6, C16, C17, C23. To remove it: Test the assumption, or analyse how sensitive the claim is to it.
- −3 Open problems Assumption C20 is not checked: The zero-mean variables of the monitored series, here and left in the forecast errors after the bias is removed, satisfy the weak invariance principle of Assumptions A and B of the paper's Supporting Information: , and there are Wiener processes and a constant with . In C3, C6, C17, C23, C24, C25. To remove it: Test the assumption, or analyse how sensitive the claim is to it.
- −1 Open problems Assumption C21 is not checked: The change is large enough and bounded: the changepoint is for some and , as , and , where is the change size in the monitored series, here or , as Assumptions C to E of the paper's Supporting Information require. In C3, C6, C17, C23, C25. To remove it: Test the assumption, or analyse how sensitive the claim is to it.
Claims
C1: 70, Partly supported
Novelty: extends earlier work, by the authors
- −25 Evidence Rests on an unchecked proof; evidence named without the observations the claim reports; an argument from other claims; cited work only. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −3 Open problems Assumption C18 is not checked: The forecasting model has a constant bias before the changepoint: for , where is unknown and may be zero (the paper's Assumption 1, the constant-bias assumption). To remove it: Test the assumption, or analyse how sensitive the claim is to it.
- −1 Open problems Assumption C19 is not checked: After the changepoint the forecasting model does not fully adjust to the change: for , where is a function of the change size in the raw data (the paper's Assumption 2, the incomplete-adjustment assumption). To remove it: Test the assumption, or analyse how sensitive the claim is to it.
- −1 Open problems Limitation L1 (reduces transferability, not mitigated): Forecasting models that adapt to a change in mean, such as ETS models with a positive smoothing parameter, do not satisfy the incomplete-adjustment assumption, so a mean change does not stay in their forecast errors. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C2: 71, Partly supported
Novelty: extends earlier work, by the authors
- −25 Evidence Rests on an unchecked proof; evidence named without the observations the claim reports; an argument from other claims; cited work only. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −3 Open problems Assumption C18 is not checked: The forecasting model has a constant bias before the changepoint: for , where is unknown and may be zero (the paper's Assumption 1, the constant-bias assumption). To remove it: Test the assumption, or analyse how sensitive the claim is to it.
- −1 Open problems Assumption C19 is not checked: After the changepoint the forecasting model does not fully adjust to the change: for , where is a function of the change size in the raw data (the paper's Assumption 2, the incomplete-adjustment assumption). To remove it: Test the assumption, or analyse how sensitive the claim is to it.
C3: 69, Partly supported
Novelty: extends earlier work, by the authors
- −25 Evidence Rests on an unchecked proof; evidence named without the observations the claim reports; an argument from other claims; cited work only. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −3 Open problems Assumption C18 is not checked: The forecasting model has a constant bias before the changepoint: for , where is unknown and may be zero (the paper's Assumption 1, the constant-bias assumption). To remove it: Test the assumption, or analyse how sensitive the claim is to it.
- −1 Open problems Assumption C19 is not checked: After the changepoint the forecasting model does not fully adjust to the change: for , where is a function of the change size in the raw data (the paper's Assumption 2, the incomplete-adjustment assumption). To remove it: Test the assumption, or analyse how sensitive the claim is to it.
- −1 Open problems Assumption C20 is not checked: The zero-mean variables of the monitored series, here and left in the forecast errors after the bias is removed, satisfy the weak invariance principle of Assumptions A and B of the paper's Supporting Information: , and there are Wiener processes and a constant with . To remove it: Test the assumption, or analyse how sensitive the claim is to it.
- −1 Open problems Assumption C21 is not checked: The change is large enough and bounded: the changepoint is for some and , as , and , where is the change size in the monitored series, here or , as Assumptions C to E of the paper's Supporting Information require. To remove it: Test the assumption, or analyse how sensitive the claim is to it.
C4: 75, Partly supported
Novelty: new, by the authors
- −25 Evidence Rests on an unchecked proof; evidence named without the observations the claim reports; an argument from other claims; cited work only. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
C5: 71, Partly supported
Novelty: new, by the authors
- −10 Evidence Rests on an unchecked proof; evidence named without the observations the claim reports; an argument from other claims; cited work only. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −7 Checkable Flags: availability not stated, figure without data. To remove it: Publish what each flag says is missing.
- −6 Open problems Limitation L2 (could bias magnitude, not mitigated): The two forecasting-model assumptions that make the thresholds theoretically valid are not necessarily satisfied in all the simulated scenarios. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −5 Plan A principal claim reported without a plan recorded beforehand. To remove it: Plan the measurement in the lab record or a registration before running it, or publish the lab record that shows every run.
- −1 Open problems Limitation L3 (reduces transferability, not mitigated): All simulated data have an ARMA error structure with normally distributed innovations and a training sample of 300 observations. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C6: 15, Weakly supported
Novelty: new, by the authors
- −45 Checkable Verifiability: unverifiable. To remove it: Make the data, code and models the claim rests on available, or state why they cannot be.
- −25 Evidence Rests on an unchecked proof; evidence named without the observations the claim reports; an argument from other claims; cited work only. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −5 Plan A principal claim reported without a plan recorded beforehand. To remove it: Plan the measurement in the lab record or a registration before running it, or publish the lab record that shows every run.
- −3 Open problems Assumption C18 is not checked: The forecasting model has a constant bias before the changepoint: for , where is unknown and may be zero (the paper's Assumption 1, the constant-bias assumption). To remove it: Test the assumption, or analyse how sensitive the claim is to it.
- −3 Open problems Limitation L4 (reduces precision, not mitigated): The start of the Christmas peak season at the delivery office is inferred from the rise in the mean and variance of the forecast errors; the underlying parcel volumes and forecasts are not known to the authors. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Assumption C19 is not checked: After the changepoint the forecasting model does not fully adjust to the change: for , where is a function of the change size in the raw data (the paper's Assumption 2, the incomplete-adjustment assumption). To remove it: Test the assumption, or analyse how sensitive the claim is to it.
- −1 Open problems Assumption C20 is not checked: The zero-mean variables of the monitored series, here and left in the forecast errors after the bias is removed, satisfy the weak invariance principle of Assumptions A and B of the paper's Supporting Information: , and there are Wiener processes and a constant with . To remove it: Test the assumption, or analyse how sensitive the claim is to it.
- −1 Open problems Assumption C21 is not checked: The change is large enough and bounded: the changepoint is for some and , as , and , where is the change size in the monitored series, here or , as Assumptions C to E of the paper's Supporting Information require. To remove it: Test the assumption, or analyse how sensitive the claim is to it.
- −1 Open problems Limitation L1 (reduces transferability, not mitigated): Forecasting models that adapt to a change in mean, such as ETS models with a positive smoothing parameter, do not satisfy the incomplete-adjustment assumption, so a mean change does not stay in their forecast errors. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C7: 39, Weakly supported
Novelty: new, by the authors
- −45 Checkable Verifiability: unverifiable. To remove it: Make the data, code and models the claim rests on available, or state why they cannot be.
- −10 Evidence Rests on an unchecked proof; evidence named without the observations the claim reports; an argument from other claims; cited work only. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −5 Plan A principal claim reported without a plan recorded beforehand. To remove it: Plan the measurement in the lab record or a registration before running it, or publish the lab record that shows every run.
- −1 Open problems Limitation L5 (reduces transferability, not mitigated): The application uses one monthly series with a Phase I sample of 36 observations. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C8: 86, Well supported
Novelty: new, by the authors
- −7 Checkable Flags: availability not stated, figure without data. To remove it: Publish what each flag says is missing.
- −6 Open problems Limitation L2 (could bias magnitude, not mitigated): The two forecasting-model assumptions that make the thresholds theoretically valid are not necessarily satisfied in all the simulated scenarios. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L3 (reduces transferability, not mitigated): All simulated data have an ARMA error structure with normally distributed innovations and a training sample of 300 observations. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C9: 86, Well supported
Novelty: new, by the authors
- −7 Checkable Flags: availability not stated, figure without data. To remove it: Publish what each flag says is missing.
- −6 Open problems Limitation L2 (could bias magnitude, not mitigated): The two forecasting-model assumptions that make the thresholds theoretically valid are not necessarily satisfied in all the simulated scenarios. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −1 Open problems Limitation L3 (reduces transferability, not mitigated): All simulated data have an ARMA error structure with normally distributed innovations and a training sample of 300 observations. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C10: 78, Partly supported
Novelty: new, by the authors
- −10 Evidence Rests on an unchecked proof; evidence named without the observations the claim reports; an argument from other claims; cited work only. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −6 Open problems Limitation L2 (could bias magnitude, not mitigated): The two forecasting-model assumptions that make the thresholds theoretically valid are not necessarily satisfied in all the simulated scenarios. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −5 Checkable Flags: availability not stated, figure without data. To remove it: Publish what each flag says is missing.
- −1 Open problems Limitation L3 (reduces transferability, not mitigated): All simulated data have an ARMA error structure with normally distributed innovations and a training sample of 300 observations. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C11: 89, Well supported
- −6 Open problems Limitation L2 (could bias magnitude, not mitigated): The two forecasting-model assumptions that make the thresholds theoretically valid are not necessarily satisfied in all the simulated scenarios. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −5 Checkable Flags: availability not stated, figure without data. To remove it: Publish what each flag says is missing.
C12: 87, Well supported
- −7 Checkable Flags: availability not stated, figure without data. To remove it: Publish what each flag says is missing.
- −6 Open problems Limitation L2 (could bias magnitude, not mitigated): The two forecasting-model assumptions that make the thresholds theoretically valid are not necessarily satisfied in all the simulated scenarios. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C13: 54, Weakly supported
- −45 Checkable Verifiability: unverifiable. To remove it: Make the data, code and models the claim rests on available, or state why they cannot be.
- −1 Open problems Limitation L5 (reduces transferability, not mitigated): The application uses one monthly series with a Phase I sample of 36 observations. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C14: 54, Weakly supported
- −45 Checkable Verifiability: unverifiable. To remove it: Make the data, code and models the claim rests on available, or state why they cannot be.
- −1 Open problems Limitation L5 (reduces transferability, not mitigated): The application uses one monthly series with a Phase I sample of 36 observations. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C23: 68, Partly supported
Novelty: extends earlier work, by the authors
- −25 Evidence Rests on an unchecked proof; evidence named without the observations the claim reports; an argument from other claims; cited work only. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −3 Open problems Assumption C18 is not checked: The forecasting model has a constant bias before the changepoint: for , where is unknown and may be zero (the paper's Assumption 1, the constant-bias assumption). To remove it: Test the assumption, or analyse how sensitive the claim is to it.
- −1 Open problems Assumption C19 is not checked: After the changepoint the forecasting model does not fully adjust to the change: for , where is a function of the change size in the raw data (the paper's Assumption 2, the incomplete-adjustment assumption). To remove it: Test the assumption, or analyse how sensitive the claim is to it.
- −1 Open problems Assumption C20 is not checked: The zero-mean variables of the monitored series, here and left in the forecast errors after the bias is removed, satisfy the weak invariance principle of Assumptions A and B of the paper's Supporting Information: , and there are Wiener processes and a constant with . To remove it: Test the assumption, or analyse how sensitive the claim is to it.
- −1 Open problems Assumption C21 is not checked: The change is large enough and bounded: the changepoint is for some and , as , and , where is the change size in the monitored series, here or , as Assumptions C to E of the paper's Supporting Information require. To remove it: Test the assumption, or analyse how sensitive the claim is to it.
- −1 Open problems Limitation L1 (reduces transferability, not mitigated): Forecasting models that adapt to a change in mean, such as ETS models with a positive smoothing parameter, do not satisfy the incomplete-adjustment assumption, so a mean change does not stay in their forecast errors. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C15: 82, Well supported
Novelty: summarises earlier work, by the authors
- −10 Evidence Rests on an unchecked proof; evidence named without the observations the claim reports; an argument from other claims; cited work only. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −5 Checkable Flags: availability not stated, figure without data. To remove it: Publish what each flag says is missing.
- −3 Open problems Assumption C22 is not checked: The ARMA process is causal and invertible, and for all , and the ARMA forecasting model is perfectly estimated, so that its residuals and forecast errors are identical. To remove it: Test the assumption, or analyse how sensitive the claim is to it.
C16: 70, Partly supported
Novelty: summarises earlier work, by the authors
- −25 Evidence Rests on an unchecked proof; evidence named without the observations the claim reports; an argument from other claims; cited work only. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −3 Open problems Assumption C22 is not checked: The ARMA process is causal and invertible, and for all , and the ARMA forecasting model is perfectly estimated, so that its residuals and forecast errors are identical. To remove it: Test the assumption, or analyse how sensitive the claim is to it.
- −1 Open problems Assumption C19 is not checked: After the changepoint the forecasting model does not fully adjust to the change: for , where is a function of the change size in the raw data (the paper's Assumption 2, the incomplete-adjustment assumption). To remove it: Test the assumption, or analyse how sensitive the claim is to it.
- −1 Open problems Assumption C18 is not checked: The forecasting model has a constant bias before the changepoint: for , where is unknown and may be zero (the paper's Assumption 1, the constant-bias assumption). To remove it: Test the assumption, or analyse how sensitive the claim is to it.
C17: 64, Partly supported
Novelty: new, by the authors
- −25 Evidence Rests on an unchecked proof; evidence named without the observations the claim reports; an argument from other claims; cited work only. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −5 Checkable Flags: availability not stated, figure without data. To remove it: Publish what each flag says is missing.
- −3 Open problems Assumption C18 is not checked: The forecasting model has a constant bias before the changepoint: for , where is unknown and may be zero (the paper's Assumption 1, the constant-bias assumption). To remove it: Test the assumption, or analyse how sensitive the claim is to it.
- −1 Open problems Assumption C19 is not checked: After the changepoint the forecasting model does not fully adjust to the change: for , where is a function of the change size in the raw data (the paper's Assumption 2, the incomplete-adjustment assumption). To remove it: Test the assumption, or analyse how sensitive the claim is to it.
- −1 Open problems Assumption C20 is not checked: The zero-mean variables of the monitored series, here and left in the forecast errors after the bias is removed, satisfy the weak invariance principle of Assumptions A and B of the paper's Supporting Information: , and there are Wiener processes and a constant with . To remove it: Test the assumption, or analyse how sensitive the claim is to it.
- −1 Open problems Assumption C21 is not checked: The change is large enough and bounded: the changepoint is for some and , as , and , where is the change size in the monitored series, here or , as Assumptions C to E of the paper's Supporting Information require. To remove it: Test the assumption, or analyse how sensitive the claim is to it.
C24: 77, Partly supported
Novelty: summarises earlier work, by the authors
- −20 Evidence Rests on an unchecked proof; evidence named without the observations the claim reports; an argument from other claims; cited work only. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −3 Open problems Assumption C20 is not checked: The zero-mean variables of the monitored series, here and left in the forecast errors after the bias is removed, satisfy the weak invariance principle of Assumptions A and B of the paper's Supporting Information: , and there are Wiener processes and a constant with . To remove it: Test the assumption, or analyse how sensitive the claim is to it.
C25: 76, Partly supported
Novelty: summarises earlier work, by the authors
- −20 Evidence Rests on an unchecked proof; evidence named without the observations the claim reports; an argument from other claims; cited work only. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −3 Open problems Assumption C20 is not checked: The zero-mean variables of the monitored series, here and left in the forecast errors after the bias is removed, satisfy the weak invariance principle of Assumptions A and B of the paper's Supporting Information: , and there are Wiener processes and a constant with . To remove it: Test the assumption, or analyse how sensitive the claim is to it.
- −1 Open problems Assumption C21 is not checked: The change is large enough and bounded: the changepoint is for some and , as , and , where is the change size in the monitored series, here or , as Assumptions C to E of the paper's Supporting Information require. To remove it: Test the assumption, or analyse how sensitive the claim is to it.
Versions
- Version 1 (this page) · 16 November 2026 at 20:14 UTC · active
Every version is kept in the public registry (github.com/entrylib/registry).