CLAIM C6 of entrylib:2026.000001 Calibration of the prediction intervals of AutoMSARIMA in smooth on M1, M3, Tourism and M4
licence CC-BY-4.0; contributions C6

DEFINITIONS
D1 [method] MSARIMA analytic interval: The quantiles of `predict(interval="prediction")` of a pure ARIMA in smooth 1.3.0, which dispatches to the approximate interval. The ARIMA is the state space model
$$\begin{aligned} y_t &= w^\top v_{t-l} + \epsilon_t, \\ v_t &= F v_{t-l} + g \epsilon_t, \\ \epsilon_t &\sim N(0, \sigma^2). \end{aligned}$$
The quantile at level $\tau$ and horizon $h$ is
$$q_{\tau,h} = \hat y_{T+h|T} + \Phi^{-1}(\tau) \sqrt{V_h}, \qquad V_h = \hat\sigma^2 \frac{T}{T-k} c_h,$$
with
$$\hat\sigma^2 = \frac{1}{T} \sum_{t=1}^{T} e_t^2, \qquad c_h = 1 + \sum_{j=1}^{h-1} \left(w^\top F^{j-1} g\right)^2,$$
where $k$ is the number of estimated parameters and $c_h$ the $h$-th diagonal element of the analytic covariance of the state space model, written here for a model with lag 1. [source B6]
D4 [variable] coverage: For the quantile $q_{\tau,j}$ at level $\tau$ and horizon $j$, the share of holdout observations at or below it,
$$c_\tau = \frac{1}{h} \sum_{j=1}^{h} 1\left(y_{T+j} \le q_{\tau,j}\right),$$
averaged first within a series and then over series. cov80 is $c_{0.9} - c_{0.1}$, the coverage of the central 80% interval; q50 is $c_{0.5}$.
D5 [data] benchmark: The 5,315 series of M1 (1,001), M3 (3,003) and Tourism (1,311) from fcompdata 0.1.4, with their own holdouts and seasonal periods; lags $(1, m)$ for seasonal series. [members D18, D19, D20]
D7 [design] level terciles: Series split into terciles of the [[D17:level ratio]] across the series of a data set; cov80 reported for each tercile, and the spread as the largest minus the smallest of the three.
D10 [method] AutoMSARIMA: `AutoMSARIMA(lags, initial)` in Python and `auto.msarima()` in R of smooth 1.3.0 / 4.6: ARIMA orders up to AR (3,3), I (2,1), MA (3,3) and a constant selected by AICc, normal errors, the [[D1:MSARIMA analytic interval]] for the quantiles. Backcasting initial states unless stated. [source B6]
D15 [method] ETS forced and selected: ETS of smooth with backcasting: AutoADAM-style selection among ETS models (`ES(model="ZXZ")`, or ZXN for non-seasonal series), or ETS(ANN) or ETS(MNN) fitted to every series. [source B6]
D16 [method] other ARIMA implementations: statsforecast AutoARIMA, pmdarima auto_arima and skforecast Arima, as run in the notebook of the smooth benchmark of 2026-11-05, whose saved forecasts were read.
D17 [variable] level ratio: Mean of the last third of the in-sample data of a series divided by the mean of all its in-sample data.
D18 [data] M1: The 1,001 series of the M1 competition with their holdouts. [source B1]
D19 [data] M3: The 3,003 series of the M3 competition with their holdouts. [source B2]
D20 [data] Tourism: The 1,311 series of the tourism forecasting competition with their holdouts. [source B3]

CLAIMS
C6 [interpretive; principal; support: argument W1; uses C10, C11; strength moderate (author); verifiability open; planning exploratory]
  The wider intervals of ETS than of [[D10:AutoMSARIMA]] on the [[D5:benchmark]] come from the multiplicative-error models in the ETS pool of [[D15:ETS forced and selected]], not from how either package computes the interval.
  scope: data_type: M1, M3 and Tourism series | conditions: backcasting
C10 [descriptive; supporting; support: empirical_quant E13 (E13.T2.r1, E13.T2.r2, E13.T2.r3); limitations L7; strength moderate (author); verifiability open; planning exploratory]
  On the 114 [[D5:benchmark]] series where the selected ETS(ANN) of [[D15:ETS forced and selected]] and the ARIMA(0,1,1) selected by [[D10:AutoMSARIMA]] give the same forecast and width, the two have the same 80% [[D4:coverage]], 0.734.
  scope: data_type: M1, M3 and Tourism series | conditions: backcasting
C11 [descriptive; supporting; support: empirical_quant E14 (E14.T1.r3, E14.T1.r2, E14.T1.r1); strength moderate (author); verifiability open; planning exploratory]
  Forced on all 5,315 [[D5:benchmark]] series, ETS(MNN) of [[D15:ETS forced and selected]] has an 80% [[D4:coverage]] of 0.764, against 0.668 for ETS(ANN) and 0.695 for ARIMA(0,1,1).
  scope: data_type: M1, M3 and Tourism series | conditions: backcasting

EVIDENCE
E13 [quant; design benchmark] Coverage by [[D7:level terciles]] on the 5,315 [[D5:benchmark]] series for [[D10:AutoMSARIMA]] with backcasting (refitted) and for the [[D16:other ARIMA implementations]] (saved forecasts of the smooth benchmark of 2026-11-05); and coverage of ETS and ARIMA on the series where the selected ETS(ANN) of [[D15:ETS forced and selected]] and the selected ARIMA(0,1,1) give the same forecast and width (exploratory run RN15).
  setup: studied D10, D16, D15; cases D5; procedure D7; lens D4, D17
  E13.T1 80% coverage by level tercile of AutoMSARIMA and the other ARIMA implementations on the benchmark {data: D5}
  | row | method | D4 {nominal level: 0.8, D7: low} | D4 {nominal level: 0.8, D7: mid} | D4 {nominal level: 0.8, D7: high} |
  | r1 | D10 | 0.7862 | 0.6677 | 0.5897 |
  | r2 | statsforecast AutoARIMA | 0.7891 | 0.6672 | 0.5893 |
  | r3 | pmdarima auto_arima | 0.7702 | 0.6354 | 0.5825 |
  | r4 | skforecast Arima | 0.7809 | 0.6578 | 0.5854 |
  E13.T2 Series of the benchmark where the selected ETS(ANN) and the selected ARIMA(0,1,1) give the same forecast and width {data: D5}
  | row | method | series | D4 {nominal level: 0.8} |
  | r1 | both | 114 |  |
  | r2 | ETS |  | 0.7336 |
  | r3 | ARIMA |  | 0.7336 |
E14 [quant; design benchmark] [[D15:ETS forced and selected]] and ARIMA(0,1,1) forced on all 5,315 [[D5:benchmark]] series and the selected ETS and [[D10:AutoMSARIMA]] models, backcasting; cov80 per method (exploratory run RN14 on the forecasts of RN13).
  setup: studied D15, D10; cases D5; lens D4
  E14.T1 80% coverage of forced and selected models on all 5,315 benchmark series, backcasting {data: D5}
  | row | method | D4 {nominal level: 0.8} |
  | r1 | ARIMA(0,1,1) | 0.6951 |
  | r2 | ETS(ANN) | 0.6681 |
  | r3 | ETS(MNN) | 0.7635 |
  | r4 | ETS, selected | 0.7492 |
  | r5 | ARIMA, selected | 0.6812 |

WARRANTS
W1 [abductive; premises C10, C11] Where ETS and ARIMA reduce to the same model, their intervals coincide and cover the same share (C17), so the interval computations agree; the additive ETS(ANN) forced on every series covers no more than ARIMA(0,1,1), while the multiplicative ETS(MNN) covers 0.07 to 0.10 more (C22). The difference between the pools is therefore the multiplicative-error models, which AutoMSARIMA does not have.

LIMITATIONS
L7 [sample_size; affects C10; reduces_precision; mitigation none] The series where the selected ETS(ANN) and ARIMA(0,1,1) give the same forecast and width are 114 of 5,315.

ARTEFACTS
A7 [lab_record] Lab record of the study (branch claimset/msarima-interval-calibration): questions, hypotheses, plans, approvals, runs, results and notes as an append-only, hash-chained event log (lab record 0.20, claimset 0.39.0).
A10 [data] Every result recorded in the lab record (A7), one row per result: the lab result ID, the run that produced it, the quantity as the analysis script names it, its conditions as JSON and its full-precision value. The tables of the evidence objects hold these values rounded to four significant digits and at least four decimals.

REFERENCES
B1 10.1002/for.3980010202 (doi)
B2 10.1016/S0169-2070(00)00057-1 (doi)
B3 10.1016/j.ijforecast.2010.04.009 (doi)
B6 10.1201/9781003452652 (doi)

HOW TO CITE
This claim: Svetunkov, I. (guarantor) (2026). Claim C6, in: Calibration of the prediction intervals of AutoMSARIMA in smooth on M1, M3, Tourism and M4. EntryLib. entrylib:2026.000001@v3#C6. https://entrylib.org/p/2026.000001/C6
The claimset it is from: Svetunkov, I. (guarantor) (2026). Calibration of the prediction intervals of AutoMSARIMA in smooth on M1, M3, Tourism and M4. EntryLib. entrylib:2026.000001@v3. https://entrylib.org/p/2026.000001@v3