Calibration of the prediction intervals of AutoMSARIMA in smooth on M1, M3, Tourism and M4 · C6

D1 MSARIMA analytic interval: The quantiles of predict(interval="prediction") of a pure ARIMA in smooth 1.3.0, which dispatches to the approximate interval. The ARIMA is the state space model $$\begin{aligned} y_t &= w^\top v_{t-l} + \epsilon_t, \\ v_t &= F v_{t-l} + g \epsilon_t, \\ \epsilon_t &\sim N(0, \sigma^2). \end{aligned}$$ The quantile at level $\tau$ and horizon $h$ is $$q_{\tau,h} = \hat y_{T+h|T} + \Phi^{-1}(\tau) \sqrt{V_h}, \qquad V_h = \hat\sigma^2 \frac{T}{T-k} c_h,$$ with $$\hat\sigma^2 = \frac{1}{T} \sum_{t=1}^{T} e_t^2, \qquad c_h = 1 + \sum_{j=1}^{h-1} \left(w^\top F^{j-1} g\right)^2,$$ where $k$ is the number of estimated parameters and $c_h$ the $h$-th diagonal element of the analytic covariance of the state space model, written here for a model with lag 1. Source: Svetunkov (2023).

D10 AutoMSARIMA: AutoMSARIMA(lags, initial) in Python and auto.msarima() in R of smooth 1.3.0 / 4.6: ARIMA orders up to AR (3,3), I (2,1), MA (3,3) and a constant selected by AICc, normal errors, the MSARIMA analytic interval for the quantiles. Backcasting initial states unless stated. Source: Svetunkov (2023).

D15 ETS forced and selected: ETS of smooth with backcasting: AutoADAM-style selection among ETS models (ES(model="ZXZ"), or ZXN for non-seasonal series), or ETS(ANN) or ETS(MNN) fitted to every series. Source: Svetunkov (2023).

D16 other ARIMA implementations: statsforecast AutoARIMA, pmdarima auto_arima and skforecast Arima, as run in the notebook of the smooth benchmark of 2026-11-05, whose saved forecasts were read.

D5 benchmark: The 5,315 series of M1 (1,001), M3 (3,003) and Tourism (1,311) from fcompdata 0.1.4, with their own holdouts and seasonal periods; lags $(1, m)$ for seasonal series. Members: D18, D19, D20.

D18 M1: The 1,001 series of the M1 competition with their holdouts. Source: Makridakis et al. (1982).

D19 M3: The 3,003 series of the M3 competition with their holdouts. Source: Makridakis and Hibon (2000).

D20 Tourism: The 1,311 series of the tourism forecasting competition with their holdouts. Source: Athanasopoulos et al. (2011).

D4 coverage: For the quantile $q_{\tau,j}$ at level $\tau$ and horizon $j$, the share of holdout observations at or below it, $$c_\tau = \frac{1}{h} \sum_{j=1}^{h} 1\left(y_{T+j} \le q_{\tau,j}\right),$$ averaged first within a series and then over series. cov80 is $c_{0.9} - c_{0.1}$, the coverage of the central 80% interval; q50 is $c_{0.5}$.

D17 level ratio: Mean of the last third of the in-sample data of a series divided by the mean of all its in-sample data.

D7 level terciles: Series split into terciles of the level ratio across the series of a data set; cov80 reported for each tercile, and the spread as the largest minus the smallest of the three.