Staying Positive: Challenges and Solutions in Using Pure Multiplicative ETS Models: claim C10

The claim

Support 36 C10 · descriptive · novelty: new, by the authors · in question: close match on EntryLib not acknowledged

Across all five data-generating processes in Simulation experiment 2 synthetic series, Akram's log-ETS model achieves lower mean scaled pinball score (SPS) over the 48-step horizon than pure multiplicative ETS(M,Md,M) models with Normal error distribution, Inverse Gaussian error distribution, Log-Normal error distribution, Gamma error distribution, or AICc-selected distributions, driven primarily by lower pinball loss in the central quantiles ( $\tau \in [0.2, 0.8]$ ) while performing similarly in the distribution tails ( $\tau < 0.2$ and $\tau > 0.8$ ).

Scope: conditions: $10,000$ simulated series of length $500$ per DGP, $99$ quantile levels from $0.01$ to $0.99$ , horizon $48$ · Simulation: E5 (T1, T2), artefacts A3 · Assumes: C12, C13 · Limitations: L4 · Extends: doi:10.1111/j.1467-842X.2009.00555.x

Evidence

E1 · formal

Analytical derivation of recursive conditional expectations and point forecasts for pure multiplicative ETS models in logarithmic state space form (Section 2, Eqs. 2.1-2.11).

Foundation: classical_probability · Assumes: C12, C13

Reformulate pure multiplicative ETS in logarithmic state space form: $\log y_t = w' \log(v_{t-l}) + \log(1 + \epsilon_t)$ and $\log v_t = F \log v_{t-l} + \log(1_k + g \epsilon_t)$ . (logarithmic_transformation; from S, e, c, t, i, o, n, , 2, ,, , E, q, ., , (, 2, ., 1, ))
Expand $h$ -steps-ahead recursion into non-seasonal and seasonal components and take conditional expectation $E(\log y_{t+h}|t)$ . (recursive_substitution; from S, e, c, t, i, o, n, , 2, ,, , E, q, s, ., , (, 2, ., 5, ), -, (, 2, ., 6, ))
Apply Jensen's inequality and non-negativity of $g$ to show point forecast $\hat{y}_{t+h} \le \tilde{y}_{t+h} \le \mu_{y,t+h}$ , with equality only at $h=1$ , $h \le m$ without trend, or for $ETS(M,N,N)$ . (jensens_inequality; from S, e, c, t, i, o, n, , 2, ,, , E, q, s, ., , (, 2, ., 8, ), -, (, 2, ., 1, 1, ))

E5 · simulation · cited by C10

Monte Carlo simulation of $10,000$ series of length $500$ for each of five DGPs ( $ETS(M,Md,M)$ with Normal, Inverse Gaussian, Gamma, Log-Normal, and Akram's exponentiated $ETS(A,Ad,A)$ ), evaluating 48-step quantile forecasts across 99 probability levels via scaled pinball score, quantile bias, and quantile precision (Section 5, Tables 4-5).

Setup: studied Normal error distribution, Inverse Gaussian error distribution, Gamma error distribution, Log-Normal error distribution, Akram's log-ETS model · cases Simulation experiment 2 synthetic series · procedure 48-step quantile forecast simulation procedure · lens scaled pinball score (SPS), quantile bias, quantile precision

E5.T1 · Mean scaled pinball scores ( $SPS$ ) across $10,000$ simulated series for applied models (rows) against data-generating processes (columns) over forecast horizon $48$ .
Row	applied_model	scaled pinball score (SPS) (DGP: Normal) (1)	scaled pinball score (SPS) (DGP: Inverse Gaussian) (1)	scaled pinball score (SPS) (DGP: Gamma) (1)	scaled pinball score (SPS) (DGP: Log-Normal) (1)	scaled pinball score (SPS) (DGP: Akram) (1)
r1	Normal	5.255	5.204	5.193	9.417	5.192
r2	Inverse Gaussian	5.3	5.221	5.224	9.464	5.229
r3	Gamma	5.287	5.192	5.231	9.388	5.214
r4	Log-Normal	5.215	5.295	5.224	9.423	5.22
r5	Akram	5.11	5.052	5.074	9.18	5.089
r6	Auto (AICc)	5.258	5.175	5.208	9.332	5.178

E5.T2 · Quantile bias and quantile precision across 99 quantile levels ( $0.01$ to $0.99$ ) for applied models against data-generating processes.
Row	applied_model	quantile bias (DGP: Normal) (1)	quantile bias (DGP: Inverse Gaussian) (1)	quantile bias (DGP: Gamma) (1)	quantile bias (DGP: Log-Normal) (1)	quantile bias (DGP: Akram) (1)	quantile precision (DGP: Normal) (1)	quantile precision (DGP: Inverse Gaussian) (1)	quantile precision (DGP: Gamma) (1)	quantile precision (DGP: Log-Normal) (1)	quantile precision (DGP: Akram) (1)
r1	Normal	-0.002	0.003	0.011	0.007	-0.01	0.007	0.011	0.005	0.009	0.01
r2	Inverse Gaussian	-0.004	0.009	0.001	0.001	-0.013	0.009	0.009	0.006	0.009	0.015
r3	Gamma	-0.004	0.008	0.001	0.003	-0.012	0.011	0.009	0.005	0.008	0.014
r4	Log-Normal	-0.005	0.008	0	0.001	-0.013	0.008	0.004	0.008	0.004	0.014
r5	Akram	0	0.004	0.01	0.006	-0.007	0.004	0.01	0.005	0.008	0.007
r6	Auto (AICc)	-0.003	0.008	0	0.002	-0.012	0.008	0.004	0.009	0.007	0.013

Limitations

L4 · model specification · reduces transferability · affects C9, C10 · mitigation: none

Simulation experiment 2 uses a single fixed parameter vector with error standard deviation $\sigma = 0.1$ , where multiplicative errors remain small enough that Normal and logarithmic approximations behave regularly.

Assumptions

C12 · statistical · unchecked

The multiplicative error term $(1+\epsilon_t)$ in pure multiplicative ETS models is independent and identically distributed with unit expectation $E(1+\epsilon_t) = 1$ . Rested on by C10, E1.

C13 · methodological · tested

Smoothing parameters $\alpha, \beta, \gamma$ in pure multiplicative ETS models lie within $[0, 1]$ so that state update terms $(1 + g\epsilon_t)$ remain strictly positive whenever $1 + \epsilon_t > 0$ . Rested on by C10, E1.

Definitions

Methods 3

D1 ETS(M,*,*): Pure multiplicative exponential smoothing state space model family written in logarithmic state space form as $\begin{aligned} \log y_t &= w' \log(v_{t-l}) + \log(1 + \epsilon_t) \\ \log v_t &= F \log v_{t-l} + \log(1_k + g \epsilon_t) \end{aligned}$ with conventional multi-step point forecast $\hat{y}_{t+h} = \exp\left((w_1' F_1^{h-1} + w_m' F_m^{\lceil h/m \rceil - 1}) \log v_t\right)$ . Source: Svetunkov (2023).

D3 · a kind of D1 ETS(M,Md,M): Pure multiplicative exponential smoothing model with damped multiplicative trend and multiplicative seasonality, defined by $\begin{aligned} y_t &= l_{t-1} b_{t-1}^{\phi} s_{t-m} (1 + \epsilon_t) \\ l_t &= l_{t-1} b_{t-1}^{\phi} (1 + \alpha \epsilon_t) \\ b_t &= b_{t-1}^{\phi} (1 + \beta \epsilon_t) \\ s_t &= s_{t-m} (1 + \gamma \epsilon_t) \end{aligned}$ where $\phi$ is the dampening parameter, $\alpha, \beta, \gamma$ are smoothing parameters, and $m$ is the seasonal lag. Source: 978-3-540-71918-2.

D10 Akram's log-ETS model: Additive error ETS model $ETS(A,Ad,A)$ with Normal errors applied to the natural logarithms of strictly positive time series and exponentiated back to the original scale. Source: Akram et al. (2009).

Data 1

D15 Simulation experiment 2 synthetic series: 10,000 simulated time series of length 500 generated for each of five data-generating processes ( $ETS(M,Md,M)$ with Normal, Inverse Gaussian, Gamma, Log-Normal errors, and Akram's exponentiated $ETS(A,Ad,A)$ ) with $\sigma=0.1$ , $\alpha=0.1$ , $\beta=0.05$ , $\gamma=0.2$ , $\phi=0.95$ , $l_0=100$ , $b_0=1$ , and holdout horizon $48$ .

Variables and measures 4

D9 AICc: Corrected Akaike Information Criterion used to select among ETS component structures and error distributions. Source: doi:10.1080/03610927808837599.

D11 scaled pinball score (SPS): Quantile forecast accuracy measure scaled by the in-sample mean $\bar{y}$ , defined for probability level $\tau$ over horizon $h$ as $SPS = \frac{1}{\bar{y}} \left( (1-\tau) \sum_{y_{t+j} < \hat{q}_{\tau,t+j}} (\hat{q}_{\tau,t+j} - y_{t+j}) + \tau \sum_{y_{t+j} \ge \hat{q}_{\tau,t+j}} (y_{t+j} - \hat{q}_{\tau,t+j}) \right).$

D17 quantile bias: Mean across quantile levels $\tau \in \{0.01, 0.02, \dots, 0.99\}$ of empirical quantile coverage minus nominal target level $\tau$ .

D18 quantile precision: Mean across quantile levels $\tau \in \{0.01, 0.02, \dots, 0.99\}$ of the absolute difference between empirical quantile coverage and nominal target level $\tau$ .

Study design 1

D20 48-step quantile forecast simulation procedure: Estimation of $ETS(M,Md,M)$ with each candidate distribution, AICc-based automatic selection (smooth::auto.adam), and Akram's log-ETS model on 500 observations, evaluating 99 quantile levels ( $\tau = 0.01, \dots, 0.99$ ) over a 48-step holdout.

Concepts 4

D4 Log-Normal error distribution: Probability distribution on $(0, \infty)$ applied to $1+\epsilon_t$ with location $\mu = -\sigma^2/2$ and scale $\sigma^2$ so that $E(1+\epsilon_t) = 1$ and $V(1+\epsilon_t) = e^{\sigma^2} - 1$ .

D5 Gamma error distribution: Probability distribution on $(0, \infty)$ applied to $1+\epsilon_t$ with shape $\xi = \sigma^{-2}$ and scale $\sigma^2$ so that $E(1+\epsilon_t) = 1$ and $V(1+\epsilon_t) = \sigma^2$ .

D6 Inverse Gaussian error distribution: Probability distribution on $(0, \infty)$ applied to $1+\epsilon_t$ with location $\mu = 1$ and scale $\sigma^2$ so that $E(1+\epsilon_t) = 1$ and $V(1+\epsilon_t) = \sigma^2$ .

D7 Normal error distribution: Classical Gaussian distribution on $(-\infty, \infty)$ applied to $1+\epsilon_t$ with mean $\mu = 1$ and variance $\sigma^2$ . Source: 978-3-540-71918-2.

Context

Extends doi:10.1111/j.1467-842X.2009.00555.x · from C10: Benchmarks Akram's logarithmic ETS model against pure multiplicative ETS with positive error distributions across full predictive distributions.

References

978-3-540-71918-2 978-3-540-71918-2 Used by: D3, D7
Akram et al. (2009) Akram, M., Hyndman, R. J., & Ord, J. K. (2009). EXPONENTIAL SMOOTHING AND NON‐NEGATIVE DATA. Australian & New Zealand Journal of Statistics, 51(4), 415–432. doi:10.1111/j.1467-842X.2009.00555.x Used by: D10
doi:10.1080/03610927808837599 doi:10.1080/03610927808837599 Used by: D9
Svetunkov (2023) Svetunkov, I. (2023). Forecasting and Analytics with the Augmented Dynamic Adaptive Model (ADAM). Chapman and Hall/CRC. doi:10.1201/9781003452652 Used by: D1

Artefacts

A3 · table Full simulation results tables (Table 1 from Section 3 and Tables 4-5 from Section 5) exported as CSV. simulation-results.csv

Score

Scores are computed by rules from the claimset file (EntryLib rubric 0.4): they check the record, not the science, and may change as the rubric and reviews develop. The rubric.

C10: 36, Weakly supported

Novelty: new, by the authors · in question: close match on EntryLib not acknowledged

−45 Checkable Verifiability: unverifiable. To remove it: Make the data, code and models the claim rests on available, or state why they cannot be.
−10 Evidence Rests on an unchecked proof. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked.
−5 Plan A principal claim reported without a plan recorded beforehand. To remove it: Plan the measurement in the lab record or a registration before running it, or publish the lab record that shows every run.
−3 Open problems Assumption C12 is not checked: The multiplicative error term $(1+\epsilon_t)$ in pure multiplicative ETS models is independent and identically distributed with unit expectation $E(1+\epsilon_t) = 1$ . To remove it: Test the assumption, or analyse how sensitive the claim is to it.
−1 Open problems Limitation L4 (reduces transferability, not mitigated): Simulation experiment 2 uses a single fixed parameter vector with error standard deviation $\sigma = 0.1$ , where multiplicative errors remain small enough that Normal and logarithmic approximations behave regularly. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.

Staying Positive: Challenges and Solutions in Using Pure Multiplicative ETS Models · C10