CLAIM C19 of entrylib:2026.000013 What does sMAPE reward? An analytical and simulation study licence CC-BY-4.0; contributions C19 DEFINITIONS D2 [variable] sMAPE: Symmetric mean absolute percentage error of a point forecast $f$ over a test set of actuals $y_1, \dots, y_n$: $\frac{200}{n}\sum_t |y_t - f|/(|y_t|+|f|)$, where a term with $y_t = f = 0$ counts as 0. Introduced as the symmetric MAPE by Makridakis (1993), whose denominator had no absolute values; for positive actuals and forecasts the two agree. [source B8] D1 [variable] sMAPE term ($s(y,f)$): The contribution of one actual $y$ and one point forecast $f$ to [[D2:sMAPE]]: $s(y,f) = |y-f|/(|y|+|f|)$, between 0 and 1, with $s = 0$ when $y = f$ (including $y = f = 0$, where the fraction is $0/0$). D3 [variable] expected sMAPE: The expected sMAPE of a fixed forecast $f$ under the distribution of the actual $y$: $200\,\mathrm{E}[s(y, f)]$, where $\mathrm E$ is the expectation over the random actual $y$. Computed from the distribution (by integration or exact sums), never from a sample. D6 [concept] sMAPE target: The value or values of $f$ that minimise expected sMAPE for a given distribution of the actual $y$, $\arg\min_f \mathrm E[s(y,f)]$: the functional sMAPE rewards. It may be a single value or several. D4 [variable] log error ($z$): For a positive actual $y$ and a positive forecast $f$, $z = \ln y - \ln f$. D5 [variable] weight ($w(y,f)$): For a positive actual $y$ and a positive forecast $f$, $w(y,f) = y/(y+f)^2$: how much the actual pulls the forecast that minimises expected sMAPE, since the derivative of $\mathrm E[s(y,f)]$ in $f$ is $2\,\mathrm E[w(y,f)(\mathbf 1\{yf\})]$. Largest at $y = f$, the same for $y/f = r$ and $1/r$. D10 [variable] RMSE: Root mean squared error of a point forecast $f$ over a test set of actuals $y_1, \dots, y_n$: $\sqrt{\frac1n\sum_t (y_t-f)^2}$. D11 [variable] MAE: Mean absolute error of a point forecast $f$ over a test set of actuals $y_1, \dots, y_n$: $\frac1n\sum_t |y_t-f|$. D19 [variable] log-MAE: 100 times the mean absolute error of $\ln f$ as a forecast of $\ln y$: $\frac{100}{n}\sum_t |\ln y_t - \ln f|$. Unbounded; sMAPE's small-error limit (C3); minimised by the median. D28 [variable] Suilin term ($\tilde s(y,f)$): The contribution of one actual $y$ and one forecast $f$ to the Suilin loss: $$\tilde s(y,f) = \frac{|y-f|}{\max(|y|+|f|+\delta,\ 0.5+\delta)}, \qquad \delta = 0.1;$$ the Suilin loss is 200 times its mean over a test set. It replaces the denominator $|y|+|f|$ of the [[D1:sMAPE term]]. Proposed by Suilin (2017) in code for a Kaggle competition, as a smoothed variant of sMAPE; defined in print by Hewamalage, Bergmeir and Bandara (2021, Eq. 15), who write the constant $\delta$ as $\epsilon$ and the factor 2 into the measure differently (a constant factor changes no property here). Hewamalage, Ackermann and Bergmeir (2023) call it msMAPE; Smyl (2025) gives that name to a different measure, so this study follows Kolassa (2026) and calls it the Suilin loss. Version 2 of this paper called it msMAPE. [source B14] D33 [variable] msMAPE term ($s_\nu(y,f)$): The contribution of one actual $y \ge 0$ and one forecast $f \ge 0$ to the sparse-proof sMAPE (msMAPE) of Smyl (2025), as Kolassa (2026, Definition 4) states it, for a small constant $\nu > 0$ (Kolassa writes it $m$, which is the median here): $$s_\nu(y,f) = \frac{|f-y|}{f+\max\{y, \nu\}},$$ which is 0 when $y = f = 0$; msMAPE is 200 times its mean over a test set (Kolassa writes the factor 2 into the term). It replaces the actual in the denominator of the [[D1:sMAPE term]] by $\nu$ when the actual is below $\nu$. [source B13] D8 [variable] sMAPE minimiser of a test set ($\hat f_n$): The forecast that minimises the sMAPE of one test set $y_1, \dots, y_n$, $\hat f_n = \arg\min_f \frac1n\sum_t s(y_t, f)$, found numerically (the sample counterpart of the [[D6:sMAPE target]]). D9 [variable] relative efficiency ($\mathrm{RE}(n)$): For a test-set size $n$ and an outcome distribution where sMAPE and MAE both target the median: $\mathrm{RE}(n) = \operatorname{Var}(\ln \hat m_n) / \operatorname{Var}(\ln \hat f_n)$ over test sets, where $\hat m_n$ is the sample median (the MAE minimiser) and $\hat f_n$ the sMAPE minimiser of the same test set. Above 1, sMAPE's minimiser varies less; below 1, MAE's does. D32 [variable] signal-to-noise ratio of a comparison: For a measure with term $L(y,f)$, two forecasts $f_1$ and $f_2$ and a distribution of the actual $y$: $|\mathrm E[D]|/\mathrm{sd}(D)$, where $D = L(y,f_2) - L(y,f_1)$ is the difference of the two forecasts' scores on one actual. Over a test set of $n$ independent actuals the mean of $D$ is $\sqrt n$ times this many of its standard errors from zero: the larger the ratio, the smaller the test set that tells the two forecasts apart. D31 [method] formula checks: Numerical checks of the formulas of the formal claims (claimset-code/smape-target/check_formulas.py, seed 6; check_jump.py for C11): identities and inequalities on 200000 random pairs, derivatives against finite differences, minimisers, balances and thresholds against quadrature and bisection on every setting of the selection design, and the large-spread expansion against its bound. For the claims new or reworded in version 3 (C3, C4, C12, C17 to C20): check_v3.py (seed 8), check_h39.py and check_corrected.py (seed 8), in the same way. CLAIMS C19 "The sMAPE minimiser in large samples" [formal; principal; support: formal_proof E36, empirical_quant E33, cited B15, at Theorems 5.7 and 5.23; uses C3, C5, C12; verification agent_checked; verifiability open; planning confirmatory; outcome supported] Let $\ln y$ have a density $g$ that is symmetric about $\ln m$, unimodal ($g(\ln m + v_1) \ge g(\ln m + v_2)$ whenever $|v_1| \le |v_2|$, strictly on a set of positive probability) and continuous at $\ln m$. Write $x = \ln y - \ln m$, $A = \mathrm E[\operatorname{sech}^2(x/2)\tanh(|x|/2)]$ and $B = \mathrm E[\operatorname{sech}^4(x/2)]$. (a) $A < 2g(\ln m)$. (b) Over test sets of $n$ independent actuals, the [[D8:sMAPE minimiser of a test set]] $\hat f_n$ (a forecast with the lowest sMAPE of the test set) tends to $m$ in probability, and $\sqrt n(\ln\hat f_n - \ln m)$ is asymptotically normal with mean 0 and variance $B/(2g(\ln m) - A)^2$; for the sample median the variance is $1/(4g(\ln m)^2)$. The ratio of these two limiting variances, the asymptotic relative efficiency of the sMAPE minimiser to the sample median, is $$\mathrm{ARE} = \frac{(2g(\ln m) - A)^2}{4\,g(\ln m)^2\,B};$$ that the [[D9:relative efficiency]] of finite test sets tends to it is not shown here, since the variances of finite test sets need not tend to those of the limits. (c) For lognormal actuals with standard deviation $\sigma$ of $\ln y$: $\mathrm{ARE} \to 1$ as $\sigma \to 0$ and $\mathrm{ARE}\,\sigma^3 \to 6(\ln 2)^2\sqrt{2\pi} \approx 7.23$ as $\sigma \to \infty$. For log-Laplace actuals with scale $b$ of $\ln y$: $\mathrm{ARE} \to 1$ as $b \to 0$ and $\mathrm{ARE}\,b \to 3$ as $b \to \infty$. So on both families the sMAPE minimiser is asymptotically as precise as the sample median for narrow distributions and asymptotically loses all of its precision relative to it as the actuals spread out. in plain words: Where the log of the actual is symmetric with one peak, the forecast that minimises sMAPE on a test set is asymptotically normal about the median. It is asymptotically as precise as the sample median for narrow distributions and loses its precision as the actuals spread out. explains C15, by E33.T4, E33.T6: C15 found by simulation that the sMAPE minimiser of a test set varies more than the sample median, the more so the wider the spread. C19 gives the limiting variances of both and the rates at which their ratio falls. test: formal; supported if The proof of the hypothesis as worded here (proofs.md, section H13 for the signal-to-noise ratios, H14 for the large-sample efficiency, revised to the corrected wording) is complete; the independent checker AC6 (Claude Opus 5.5) records a check of the corrected wording with outcome confirmed; and the deciding run's checks meet the tolerances of the design.; not supported if A counterexample to the hypothesis as worded here, found by the proof, the checker, the supervisor or the deciding run, that a correction of the proof cannot remove.; verdict judged plan: lab plan PL15; planned 2026-11-18T12:19:07Z; anchor git_remote C1 "sMAPE depends only on the ratio of actual to forecast" [formal; principal; extension; support: formal_proof E18, empirical_quant E1; verification agent_checked; verifiability open; planning confirmatory; outcome supported] For $y > 0$ and $f > 0$, the [[D1:sMAPE term]] $s(y,f) = |y-f|/(|y|+|f|)$ depends on $y$ and $f$ only through their ratio $r = y/f$: $s = |r - 1|/(r + 1)$. in plain words: Only the ratio of the actual to the forecast matters to sMAPE, not their size. extends B12: Tofallis (2015) lists the sMAPE term, |y-f|/[(|y|+|f|)/2], among Tornqvist et al.'s measures of relative change; C1 shows it depends on the actual and the forecast only through their ratio. test: formal; supported if The proof of the hypothesis, as now worded, in the study's final proof document (proofs.md, AR1.R2) is complete, every step naming what it rests on; the independent checker AC4 has a check of the current wording with outcome confirmed; and the deciding run's numerical checks of its formulas meet the tolerances of PL5: identities on random pairs within 1e-9 (on the scale of sMAPE), quantities computed by quadrature within 1e-4 relative (or absolute, for a residual the proof says is zero), counts of violations 0, and the expansion's error times sigma^3 below the proof's bound.; not supported if A counterexample to the statement as now worded, found by the proof, a checker, the supervisor or the deciding run, that a correction of the proof cannot remove.; verdict judged plan: lab plan PL9; planned 2026-11-17T09:21:09Z; anchor git_remote C3 "sMAPE is a hyperbolic tangent of the log error" [formal; principal; extension; support: formal_proof E20, empirical_quant E31; uses C1; verification agent_checked; verifiability open; planning confirmatory; outcome supported] For $y > 0$ and $f > 0$, the [[D1:sMAPE term]] is $s = \tanh(|\ln y - \ln f|/2)$ ([[D4:log error]] $\ln y - \ln f$). (a) It is 0 when $y = f$, strictly increasing and concave in $|\ln y - \ln f|$, and bounded by 1, which it approaches as $|\ln y - \ln f|$ grows. (b) $0 \le \tfrac12|\ln y - \ln f| - s \le \tfrac1{24}|\ln y - \ln f|^3$; so when all log errors are small, sMAPE is about 100 times the mean absolute error of $\ln f$ as a forecast of $\ln y$ ([[D19:log-MAE]]). (c) The log-MAE of a test set is minimised by its median; sMAPE is then minimised near the median, not in general at it. For positive actuals $y_1, \dots, y_n$ with a median $m$, a forecast $f^*$ that minimises their sMAPE satisfies $$\frac1n\sum_t|\ln y_t - \ln f^*| - \frac1n\sum_t|\ln y_t - \ln m| \le \frac1{12}\cdot\frac1n\sum_t|\ln y_t - \ln f^*|^3:$$ its mean absolute log error exceeds the smallest possible by at most a twelfth of its mean cubed absolute log error. The minimiser is the median itself only under further conditions on the distribution of the actuals. in plain words: sMAPE measures the error on the log scale. For small errors it is half the absolute log error; for large errors it levels off. For small errors its best forecast is close to the median, but not in general the median. extends B12: Tofallis (2015) relates sMAPE to the log accuracy ratio among relative measures; C3 shows the sMAPE term is exactly the hyperbolic tangent of half the absolute log ratio. test: formal; supported if The proof of the hypothesis as worded here, in the study's proof documents (proofs.md, proof-suilin.md), is complete, every step naming what it rests on; the independent checker (an agent of another model than the author's: Claude Opus 5.5, registered as an actor before it checks) records a check of the hypothesis as worded with outcome confirmed; and the deciding run's numerical checks of its formulas meet the tolerances of the design.; not supported if A counterexample to the hypothesis as worded, found by the proof, the checker, the supervisor or the deciding run, that a correction of the proof cannot remove.; verdict judged plan: lab plan PL12; planned 2026-11-18T12:02:12Z; anchor git_remote C4 "sMAPE targets a weighted median" [formal; principal; support: formal_proof E22, empirical_quant E31; uses C9; verification agent_checked; verifiability open; planning confirmatory; outcome supported] Let the actual $y$ be positive and take no single value with positive probability. A forecast $f$ has the lowest expected sMAPE only if the actuals below $f$ and those above it carry equal total [[D5:weight]] $w(y,f) = y/(y+f)^2$: $$\mathrm E[w(y,f)\,\mathbf 1\{yf\}].$$ So the [[D6:sMAPE target]] is a median of the actuals weighted by $w$, where the ordinary median gives every actual the same weight. An actual's weight is largest when it equals the forecast, is the same for $y/f = r$ and $1/r$, and falls as $y/f$ moves away from 1 in either direction. in plain words: For any distribution, the forecast sMAPE rewards balances the actuals below it and above it. Each actual is weighted by how close it is to the forecast by ratio. explains B4: C5 found sMAPE's preferred candidate close to the median on other distributions; C4 shows why: the target is a weighted median, on the side of the median that carries more weight (C12). addresses B12: Tofallis (2015) states that minimising sMAPE does not produce an established measure of location; C4 identifies the target as a median weighted by y/(y+f)^2. qualifies B10: Hewamalage et al. (2023) say sMAPE may tend towards selecting a slightly overestimating model; C4 shows the target is a weighted median, which lies on the side of the median that carries more weight, above or below it (C12). qualifies C2: C2 compares two forecasts of one known actual. For an uncertain actual, the forecast sMAPE rewards is the weighted median of C4, not a forecast that is too high. test: formal; supported if The proof of the hypothesis as worded here, in the study's proof documents (proofs.md, proof-suilin.md), is complete, every step naming what it rests on; the independent checker (an agent of another model than the author's: Claude Opus 5.5, registered as an actor before it checks) records a check of the hypothesis as worded with outcome confirmed; and the deciding run's numerical checks of its formulas meet the tolerances of the design.; not supported if A counterexample to the hypothesis as worded, found by the proof, the checker, the supervisor or the deciding run, that a correction of the proof cannot remove.; verdict judged plan: lab plan PL12; planned 2026-11-18T12:02:12Z; anchor git_remote C9 "Expected sMAPE is always finite and has a minimiser" [formal; principal; extension; support: formal_proof E21, empirical_quant E1; uses C3; verification agent_checked; verifiability open; planning confirmatory; outcome supported] For actuals and forecasts of any sign, $0 \le s \le 1$. So the [[D3:expected sMAPE]] of every forecast is finite, between 0 and 200, whatever the distribution of the actuals and without any moment of it being finite (unlike RMSE, MAE and MAPE, C8). For a positive random actual $y$, at least one forecast has the lowest expected sMAPE. in plain words: The expected sMAPE of a forecast is always finite, even when the actuals have a heavy tail that makes the expected RMSE, MAE or MAPE infinite. And there is always a best forecast. extends B10: Hewamalage et al. (2023) note that sMAPE is bounded by 200; C9 adds that its expected value is therefore finite for every distribution and that a best forecast exists. test: formal; supported if The proof of the hypothesis, as now worded, in the study's final proof document (proofs.md, AR1.R2) is complete, every step naming what it rests on; the independent checker AC4 has a check of the current wording with outcome confirmed; and the deciding run's numerical checks of its formulas meet the tolerances of PL5: identities on random pairs within 1e-9 (on the scale of sMAPE), quantities computed by quadrature within 1e-4 relative (or absolute, for a residual the proof says is zero), counts of violations 0, and the expansion's error times sigma^3 below the proof's bound.; not supported if A counterexample to the statement as now worded, found by the proof, a checker, the supervisor or the deciding run, that a correction of the proof cannot remove.; verdict judged plan: lab plan PL9; planned 2026-11-17T09:21:09Z; anchor git_remote C12 "When sMAPE targets the ordinary median" [formal; principal; support: formal_proof E29, empirical_quant E31; uses C3, C4, C5, C9; verification agent_checked; verifiability open; planning confirmatory; outcome supported] Let the actual $y$ be positive and take no single value with positive probability, let $m$ be its ordinary (unweighted) median, and let its expected sMAPE have a single local minimum. With the forecast at $m$, compare the total [[D5:weight]] $w(y,m) = y/(y+m)^2$ of the actuals above $m$ with that of the actuals below it. (a) If the actuals above $m$ carry more weight, the [[D6:sMAPE target]] lies above $m$; if those below carry more, it lies below $m$. (b) If the two weights are equal, $m$ is a point where expected sMAPE is flat. It is then the target if, in addition, $$2g(\ln m) > \mathrm E\Big[\operatorname{sech}^2\Big(\frac{\ln y - \ln m}{2}\Big)\tanh\Big(\frac{|\ln y - \ln m|}{2}\Big)\Big],$$ where $g$ is the density of $\ln y$, continuous at $\ln m$: this makes $m$ a strict local minimum. Without it, $m$ can be a flat point of inflection, with the target elsewhere. (c) When $\ln y$ has a density symmetric about $\ln m$ and unimodal, $m$ is the only target; lognormal and log-Laplace actuals are examples. in plain words: The forecast sMAPE rewards lies above the ordinary median when the actuals above it carry more weight, and below it when those below do. When the weights balance and enough actuals lie near the median, the median is rewarded. explains B4: C5 found by simulation that sMAPE prefers the median on lognormal and log-Laplace outcomes; C12 proves it for every symmetric unimodal ln y. test: formal; supported if The proof of the hypothesis as worded here, in the study's proof documents (proofs.md, proof-suilin.md), is complete, every step naming what it rests on; the independent checker (an agent of another model than the author's: Claude Opus 5.5, registered as an actor before it checks) records a check of the hypothesis as worded with outcome confirmed; and the deciding run's numerical checks of its formulas meet the tolerances of the design.; not supported if A counterexample to the hypothesis as worded, found by the proof, the checker, the supervisor or the deciding run, that a correction of the proof cannot remove.; verdict judged plan: lab plan PL12; planned 2026-11-18T12:02:12Z; anchor git_remote C5 "Symmetric, unimodal logs: the target is the median" [formal; supporting; replication; support: formal_proof E23, empirical_quant E1; uses C3; verification agent_checked; verifiability open; planning confirmatory; outcome supported] Let $\ln y$ have a density $g$ symmetric about $\ln m$, $g(\ln m+v) = g(\ln m-v)$, and unimodal: $g(\ln m+v_1) \ge g(\ln m+v_2)$ whenever $|v_1| \le |v_2|$, strictly on a set of positive probability. Then the median $m$ of $y$ is the only [[D6:sMAPE target]]. in plain words: When the logarithm of the actual is symmetric with a single peak, the weights balance at the median, and the median is the forecast sMAPE rewards. Lognormal and log-Laplace actuals are examples. replicates B2: C5 is an elementary case of Anderson's theorem on symmetric unimodal densities, proved here directly for the expected sMAPE. test: formal; supported if The proof of the hypothesis, as now worded, in the study's final proof document (proofs.md, AR1.R2) is complete, every step naming what it rests on; the independent checker AC4 has a check of the current wording with outcome confirmed; and the deciding run's numerical checks of its formulas meet the tolerances of PL5: identities on random pairs within 1e-9 (on the scale of sMAPE), quantities computed by quadrature within 1e-4 relative (or absolute, for a residual the proof says is zero), counts of violations 0, and the expansion's error times sigma^3 below the proof's bound.; not supported if A counterexample to the statement as now worded, found by the proof, a checker, the supervisor or the deciding run, that a correction of the proof cannot remove.; verdict judged plan: lab plan PL9; planned 2026-11-17T09:21:09Z; anchor git_remote EVIDENCE E1 [quant; design other] Numerical checks of the formulas of the formal claims ([[D31:formula checks]]), lab run RN8 under the formal plan that decided them; every check is within the plan's tolerances. They check the formulas on concrete numbers; the proofs are the evidence for the claims. The rows are labelled by the hypotheses of the record's first plan, whose wording kept the factor 200: H1 checks C1; H2 and H10 check C2; H5 checks C5; H7 checks C6; H8 checks C7; H12 checks C8; H13 checks C9; H14 checks C10. H3 and H9 checked C3, and H4 and H6 checked C4, as version 2 worded them; C3, C4 and C12 as now worded are checked in E31. The two results E1.R1 and E1.R2 compare a simulated variance with the large-sample variance that C19 now proves. setup: studied D1, D3, D4, D5, D6; procedure D31 E1.R1 simulated n Var(ln sMAPE minimiser) / large-sample value, lognormal sigma=1 = 0.9944 99% confidence interval 0.9353981515994695 to 1.0598078456840363 {hypothesis: H11, test-set size: 1000, test sets: 4000} E1.R2 simulated n Var(ln sMAPE minimiser) / large-sample value, lognormal sigma=1 = 1.022 99% confidence interval 0.9396347019298308 to 1.0981626206556903 {hypothesis: H11, test-set size: 10000, test sets: 2000} E1.R3 maximum absolute difference from the ratio form = 8.527e-14 {hypothesis: H1} E1.R4 maximum absolute difference between the ratio and its reciprocal = 8.527e-14 {hypothesis: H2} E1.R5 maximum absolute difference from 200 tanh(|z|/2) = 8.527e-14 {hypothesis: H3} E1.R6 violations of monotonicity, concavity or the bound 200 on 0 <= |z| <= 12 = 0 {hypothesis: H3} E1.R7 violations of 0 <= 100|z| - s <= 25|z|^3/3 = 0 {hypothesis: H9} E1.R8 maximum of |s - 100|z| + 25|z|^3/3| / |z|^5 for 0.05 < |z| < 0.5 = 0.8331 {hypothesis: H9} E1.R9 violations of s(y, y+e) < s(y, y-e) = 0 {hypothesis: H10} E1.R10 maximum absolute difference between s(y, ky) and s(y, y/k) = 1.137e-13 {hypothesis: H10} E1.R11 maximum relative difference between the formula for S' and a numerical derivative = 4.823e-08 {hypothesis: H4} E1.R12 maximum distance in ln f between the minimiser and the median = 8.641e-08 {hypothesis: H5} E1.R13 number of local minima, lognormal sigma=1 and log-Laplace b=1 = 1 {hypothesis: H5} E1.R14 maximum balance residual at the minimiser (pull below minus pull above) = 8.633e-09 {hypothesis: H6} E1.R15 settings with one local minimum where the sign of D misplaces the target = 0 {hypothesis: H6} E1.R16 maximum distance between the one-step and the exact quantile level = 0.002514 {hypothesis: H6} E1.R17 2 g(m) - E[sech^2(Z/2) tanh(|Z|/2)] for the equal-mode bimodal setting (negative: median a local maximum) = -0.3475 {hypothesis: H7} E1.R18 number of local minima of the equal-mode bimodal setting = 2 {hypothesis: H7} E1.R19 difference between the two minimal expected sMAPEs, equal-mode bimodal setting = 1.421e-14 {hypothesis: H7} E1.R20 maximum distance between the formula and the bisected threshold = 5.551e-17 {hypothesis: H8} E1.R21 opposite-sign or one-zero pairs not scoring 200 = 0 {hypothesis: H8} E1.R22 relative distance between the 1/y-weighted median and the minimiser of expected MAPE = 6.745e-05 {hypothesis: H12} E1.R23 difference of expected squared errors minus (f1-f2)(f1+f2-2E[y]) = 0 {hypothesis: H12} E1.R24 sMAPE terms outside [0, 200] = 0 {hypothesis: H13} E1.R25 expected sMAPE at f = 1e-6 and f = 1e10 (both near 200) = 200 {hypothesis: H13} E1.R26 the proof's bound on the error times sigma^3, 600 zeta(3) max|g''|, log-Gumbel = 309.9 {hypothesis: H14} E1.R27 limit of sigma^3 times the relative efficiency for normal log-outcomes, from the proof = 7.226 {hypothesis: H14} E1.T1 zero threshold by setting; hypothesis: H8 (lab runs RN8) {hypothesis: H8} | row | setting | zero threshold | | r1 | lognormal sigma=0.3 | 0.4687 | | r2 | lognormal sigma=0.5 | 0.4469 | | r3 | lognormal sigma=1 | 0.394 | | r4 | lognormal sigma=2 | 0.3075 | E1.T2 maximum |S - (200 - 800 ln2 h)| times sigma^3, log-Gumbel right, sigma^3 times the asymptotic relative efficiency, lognormal by sigma; hypothesis: H14 (lab runs RN8) {hypothesis: H14} | row | sigma | maximum |S - (200 - 800 ln2 h)| times sigma^3, log-Gumbel right | sigma^3 times the asymptotic relative efficiency, lognormal | | r1 | 5.0 | 264.5 | 5.613 | | r2 | 10.0 | 292.3 | 6.743 | | r3 | 20.0 | 300.1 | 7.099 | | r4 | 40.0 | 302.1 | 7.193 | E31 [quant; design other] Numerical checks of the formulas of the claims new or reworded in version 3 ([[D31:formula checks]]), lab run RN13 (check_v3.py, seed 8) under the formal plan that decided C3, C4, C12, C17 and C20; every check is within the plan's tolerances. They check the formulas on concrete numbers; the proofs are the evidence for the claims. The tables are labelled by the hypotheses the run's script names: H3 (T1 and T2) checks C3; H4 (T3) checks C4; H34 (T4 and T5) checks C12; H35 (T6 and T7) checks C17; H42 (T8 and T9) checks C20. The tables of H39 (T10 to T12) and H40 (T13 to T17) check the formulas of C18 and C19, whose wording was decided after later checks (E32, E33). setup: studied D1, D28, D33, D6, D8; procedure D31; lens D32, D9, D5, D4, D19, D11, D10 E31.T1 violations of the bound on the log-MAE of the sMAPE minimiser, largest excess of the minimiser's log-MAE over the median's, minus the bound; hypothesis: H3, test_sets: 2000 (lab runs RN13) {hypothesis: H3, test_sets: 2000} | row | violations of the bound on the log-MAE of the sMAPE minimiser | largest excess of the minimiser's log-MAE over the median's, minus the bound | | r1 | 0 | -1.263e-06 | E31.T2 sum of sMAPE terms, forecast at the median, actuals 1, 1, e, e^4, e^4, sum of sMAPE terms, forecast at 1, actuals 1, 1, e, e^4, e^4; hypothesis: H3 (lab runs RN13) {hypothesis: H3} | row | sum of sMAPE terms, forecast at the median, actuals 1, 1, e, e^4, e^4 | sum of sMAPE terms, forecast at 1, actuals 1, 1, e, e^4, e^4 | | r1 | 2.735 | 2.39 | E31.T3 largest difference of the weights below and above the target, over their sum; hypothesis: H4, distributions: 6 (lab runs RN13) {hypothesis: H4, distributions: 6} | row | largest difference of the weights below and above the target, over their sum | | r1 | 2.089e-08 | E31.T4 distributions with more than one local minimum, distributions where the target is not on the side of the median that carries more weight; hypothesis: H34, distributions: 6 (lab runs RN13) {hypothesis: H34, distributions: 6} | row | distributions with more than one local minimum | distributions where the target is not on the side of the median that carries more weight | | r1 | 0 | 0 | E31.T5 largest distance in logs between the target and the median, lognormal and log-Laplace; hypothesis: H34, distributions: 16 (lab runs RN13) {hypothesis: H34, distributions: 16} | row | largest distance in logs between the target and the median, lognormal and log-Laplace | | r1 | 1.753e-07 | E31.T6 Suilin term of a zero forecast of a zero actual; hypothesis: H35 (lab runs RN13) {hypothesis: H35} | row | Suilin term of a zero forecast of a zero actual | | r1 | 0 | E31.T7 largest error of the identity of (a), largest error of the identity of (b), values of the Suilin term outside [0, 1); hypothesis: H35, pairs: 200000 (lab runs RN13) {hypothesis: H35, pairs: 200000} | row | largest error of the identity of (a) | largest error of the identity of (b) | values of the Suilin term outside [0, 1) | | r1 | 1.11e-16 | 0 | 0 | E31.T8 largest error of the identity of (a), largest error of the identity of (b), values of the msMAPE term outside [0, 1], values of the msMAPE term not below 1 for actuals below nu, pairs with a count of at least 1 where the msMAPE and sMAPE terms differ by nu; hypothesis: H42, pairs: 200000 (lab runs RN13) {hypothesis: H42, pairs: 200000} | row | nu | largest error of the identity of (a) | largest error of the identity of (b) | values of the msMAPE term outside [0, 1] | values of the msMAPE term not below 1 for actuals below nu | pairs with a count of at least 1 where the msMAPE and sMAPE terms differ | | r1 | 0.1 | 0 | 0 | 0 | 0 | 0 | | r2 | 0.5 | 0 | 0 | 0 | 0 | 0 | | r3 | 1.0 | 0 | 0 | 0 | 0 | 0 | E31.T9 msMAPE term of a forecast of 1e-9 for a zero actual by nu; hypothesis: H42 (lab runs RN13) {hypothesis: H42} | row | nu | msMAPE term of a forecast of 1e-9 for a zero actual | | r1 | 0.1 | 1e-08 | | r2 | 0.5 | 2e-09 | | r3 | 1.0 | 1e-09 | E31.T10 signal-to-noise ratio of sMAPE over its large-spread formula, signal-to-noise ratio of the absolute error over its limit by error_distribution, c1, c2; hypothesis: H39, sigma: 200 (lab runs RN13) {hypothesis: H39, sigma: 200} | row | error_distribution | c1 | c2 | signal-to-noise ratio of sMAPE over its large-spread formula | signal-to-noise ratio of the absolute error over its limit | | r1 | normal | 0.0 | 0.5 | 0.9999 | 0.9913 | | r2 | normal | 0.0 | 1.0 | 1 | 0.9956 | | r3 | normal | -0.5 | 1.5 | 1.001 | 0.9966 | | r4 | Gumbel | 0.0 | 0.5 | 0.9998 | 0.9647 | | r5 | Gumbel | 0.0 | 1.0 | 1 | 0.9937 | | r6 | Gumbel | -0.5 | 1.5 | 1 | 0.9967 | E31.T11 relative spread of the absolute log error's signal-to-noise ratio over sigma = 2, 10, 50 by error_distribution, c1, c2; hypothesis: H39 (lab runs RN13) {hypothesis: H39} | row | error_distribution | c1 | c2 | relative spread of the absolute log error's signal-to-noise ratio over sigma = 2, 10, 50 | | r1 | normal | 0.0 | 0.5 | 1.686e-15 | | r2 | normal | 0.0 | 1.0 | 8.388e-16 | | r3 | normal | -0.5 | 1.5 | 4.51e-16 | | r4 | Gumbel | 0.0 | 0.5 | 4.186e-15 | | r5 | Gumbel | 0.0 | 1.0 | 4.885e-15 | | r6 | Gumbel | -0.5 | 1.5 | 1.356e-14 | E31.T12 signal-to-noise ratio of the squared error over its formula, lognormal, exp(sigma^2 / 2) times the squared error's ratio, lognormal, formula by c1, c2, sigma; hypothesis: H39 (lab runs RN13) {hypothesis: H39} | row | c1 | c2 | sigma | signal-to-noise ratio of the squared error over its formula, lognormal | exp(sigma^2 / 2) times the squared error's ratio, lognormal, formula | | r1 | 0.0 | 0.5 | 0.5 | 1 | | | r2 | 0.0 | 1.0 | 0.5 | 1 | | | r3 | -0.5 | 1.5 | 0.5 | 1 | | | r4 | 0.0 | 0.5 | 1.0 | 1 | | | r5 | 0.0 | 1.0 | 1.0 | 1 | | | r6 | -0.5 | 1.5 | 1.0 | 1 | | | r7 | 0.0 | 0.5 | 2.0 | 1 | | | r8 | 0.0 | 1.0 | 2.0 | 1 | | | r9 | -0.5 | 1.5 | 2.0 | 1 | | | r10 | 0.0 | 1.0 | 4.0 | | 0.9907 | | r11 | 0.0 | 1.0 | 6.0 | | 1 | E31.T13 distributions compared with the asymptotic values recorded by RN7; hypothesis: H40 (lab runs RN13) {hypothesis: H40} | row | distributions compared with the asymptotic values recorded by RN7 | | r1 | 16 | E31.T14 distributions where A is not below 2 g at the median, largest difference from the asymptotic values recorded by RN7; hypothesis: H40, distributions: 16 (lab runs RN13) {hypothesis: H40, distributions: 16} | row | distributions where A is not below 2 g at the median | largest difference from the asymptotic values recorded by RN7 | | r1 | 0 | 0.00014 | E31.T15 large-sample relative efficiency, formula by family, spread; hypothesis: H40 (lab runs RN13) {hypothesis: H40} | row | family | spread | large-sample relative efficiency, formula | | r1 | lognormal | 0.25 | 0.9696 | | r2 | lognormal | 0.5 | 0.8881 | | r3 | lognormal | 1.0 | 0.6601 | | r4 | lognormal | 1.5 | 0.454 | | r5 | lognormal | 2.0 | 0.3073 | | r6 | lognormal | 3.0 | 0.1474 | | r7 | lognormal | 4.0 | 0.07778 | | r8 | lognormal | 6.0 | 0.0279 | | r9 | log-Laplace | 0.25 | 0.9969 | | r10 | log-Laplace | 0.5 | 0.9765 | | r11 | log-Laplace | 1.0 | 0.8953 | | r12 | log-Laplace | 1.5 | 0.8036 | | r13 | log-Laplace | 2.0 | 0.7207 | | r14 | log-Laplace | 3.0 | 0.5903 | | r15 | log-Laplace | 4.0 | 0.4969 | | r16 | log-Laplace | 6.0 | 0.3755 | E31.T16 distance of the large-sample relative efficiency from 1 at spread 0.01 by family; hypothesis: H40 (lab runs RN13) {hypothesis: H40} | row | family | distance of the large-sample relative efficiency from 1 at spread 0.01 | | r1 | lognormal | 5e-05 | | r2 | log-Laplace | 1.249e-08 | E31.T17 large-sample relative efficiency times sigma^3 over 6 (ln 2)^2 sqrt(2 pi), lognormal, large-sample relative efficiency times b over 3, log-Laplace; hypothesis: H40, spread: 200 (lab runs RN13) {hypothesis: H40, spread: 200} | row | large-sample relative efficiency times sigma^3 over 6 (ln 2)^2 sqrt(2 pi), lognormal | large-sample relative efficiency times b over 3, log-Laplace | | r1 | 0.9998 | 0.9906 | E32 [quant; design other] Numerical checks of the formulas of C18 ([[D31:formula checks]]), lab run RN18 (check_h39.py) under the formal plan that decided C18; every check is within the plan's tolerances. They check the formulas on concrete numbers; the proof is the evidence for the claim. The run's script labels the checks H39, the record's first wording of C18; the formulas are those of C18. setup: studied D1; procedure D31; lens D32, D19, D11, D10 E32.T1 distance of sMAPE's signal-to-noise ratio over its large-spread formula from 1, distance of the absolute error's signal-to-noise ratio over its limit from 1 by error_distribution, c1, c2, sigma; hypothesis: H39 (lab runs RN18) {hypothesis: H39} | row | error_distribution | c1 | c2 | sigma | distance of sMAPE's signal-to-noise ratio over its large-spread formula from 1 | distance of the absolute error's signal-to-noise ratio over its limit from 1 | | r1 | normal | 0.0 | 0.5 | 200 | 5.266e-05 | 0.008727 | | r2 | normal | 0.0 | 0.5 | 2000 | 2.749e-06 | 0.0008713 | | r3 | normal | 0.0 | 1.0 | 200 | 0.0003989 | 0.004379 | | r4 | normal | 0.0 | 1.0 | 2000 | 4.703e-05 | 0.0004373 | | r5 | normal | -0.5 | 1.5 | 200 | 0.00122 | 0.003402 | | r6 | normal | -0.5 | 1.5 | 2000 | 0.0001273 | 0.0003398 | | r7 | Gumbel | 0.0 | 0.5 | 200 | 0.0001627 | 0.03529 | | r8 | Gumbel | 0.0 | 0.5 | 2000 | 5.872e-07 | 0.00352 | | r9 | Gumbel | 0.0 | 1.0 | 200 | 0.0001512 | 0.006295 | | r10 | Gumbel | 0.0 | 1.0 | 2000 | 2.457e-05 | 0.0006281 | | r11 | Gumbel | -0.5 | 1.5 | 200 | 0.0003779 | 0.00326 | | r12 | Gumbel | -0.5 | 1.5 | 2000 | 4.713e-05 | 0.0003255 | E32.T2 relative spread of the absolute log error's signal-to-noise ratio over sigma = 2, 10, 50 by error_distribution, c1, c2; hypothesis: H39 (lab runs RN18) {hypothesis: H39} | row | error_distribution | c1 | c2 | relative spread of the absolute log error's signal-to-noise ratio over sigma = 2, 10, 50 | | r1 | normal | 0.0 | 0.5 | 3.761e-15 | | r2 | normal | 0.0 | 1.0 | 2.397e-16 | | r3 | normal | -0.5 | 1.5 | 1.579e-15 | | r4 | Gumbel | 0.0 | 0.5 | 4.243e-13 | | r5 | Gumbel | 0.0 | 1.0 | 7.932e-13 | | r6 | Gumbel | -0.5 | 1.5 | 1.43e-12 | E32.T3 distance of the squared error's signal-to-noise ratio over its formula from 1, lognormal, distance of exp(sigma^2 / 2) times the squared error's ratio from 1, lognormal, formula by c1, c2, sigma; hypothesis: H39 (lab runs RN18) {hypothesis: H39} | row | c1 | c2 | sigma | distance of the squared error's signal-to-noise ratio over its formula from 1, lognormal | distance of exp(sigma^2 / 2) times the squared error's ratio from 1, lognormal, formula | | r1 | 0.0 | 0.5 | 0.5 | 3.109e-15 | | | r2 | 0.0 | 1.0 | 0.5 | 6.661e-16 | | | r3 | -0.5 | 1.5 | 0.5 | 1.11e-15 | | | r4 | 0.0 | 0.5 | 1.0 | 1.554e-15 | | | r5 | 0.0 | 1.0 | 1.0 | 8.882e-16 | | | r6 | -0.5 | 1.5 | 1.0 | 5.551e-16 | | | r7 | 0.0 | 0.5 | 2.0 | 1.409e-13 | | | r8 | 0.0 | 1.0 | 2.0 | 7.55e-15 | | | r9 | -0.5 | 1.5 | 2.0 | 1.887e-15 | | | r10 | 0.0 | 1.0 | 4.0 | | 0.009325 | | r11 | 0.0 | 1.0 | 6.0 | | 3.08e-06 | | r12 | 0.0 | 1.0 | 8.0 | | 1.888e-11 | E33 [quant; design other] Numerical checks of the formulas of C19 ([[D31:formula checks]]), lab run RN16 (check_corrected.py, seed 8) under the formal plan that decided C19; every check is within the plan's tolerances. They check the formulas on concrete numbers; the proof is the evidence for the claim. The run's script labels the checks by the record's first wordings: H40 (T4 to T7 and the result R1) checks C19; H39 (T1 to T3) checks the formulas of C18, which the same plan tested in an earlier wording. setup: studied D1, D8; procedure D31; lens D32, D9, D19, D11, D10 E33.R1 distributions compared with the asymptotic values recorded by RN7 = 16 {hypothesis: H40} E33.T1 distance of sMAPE's signal-to-noise ratio over its large-spread formula from 1, distance of the absolute error's signal-to-noise ratio over its limit from 1 by error_distribution, c1, c2, sigma; hypothesis: H39 (lab runs RN16) {hypothesis: H39} | row | error_distribution | c1 | c2 | sigma | distance of sMAPE's signal-to-noise ratio over its large-spread formula from 1 | distance of the absolute error's signal-to-noise ratio over its limit from 1 | | r1 | normal | 0.0 | 0.5 | 200 | 5.266e-05 | 0.008727 | | r2 | normal | 0.0 | 0.5 | 2000 | 2.749e-06 | 0.0008713 | | r3 | normal | 0.0 | 1.0 | 200 | 0.0003989 | 0.004379 | | r4 | normal | 0.0 | 1.0 | 2000 | 4.703e-05 | 0.0004373 | | r5 | normal | -0.5 | 1.5 | 200 | 0.00122 | 0.003402 | | r6 | normal | -0.5 | 1.5 | 2000 | 0.0001273 | 0.0003398 | | r7 | Gumbel | 0.0 | 0.5 | 200 | 0.0001627 | 0.03529 | | r8 | Gumbel | 0.0 | 0.5 | 2000 | 5.872e-07 | 0.00352 | | r9 | Gumbel | 0.0 | 1.0 | 200 | 0.0001512 | 0.006295 | | r10 | Gumbel | 0.0 | 1.0 | 2000 | 2.457e-05 | 0.0006281 | | r11 | Gumbel | -0.5 | 1.5 | 200 | 0.0003779 | 0.00326 | | r12 | Gumbel | -0.5 | 1.5 | 2000 | 4.713e-05 | 0.0003255 | E33.T2 relative spread of the absolute log error's signal-to-noise ratio over sigma = 2, 10, 50 by error_distribution, c1, c2; hypothesis: H39 (lab runs RN16) {hypothesis: H39} | row | error_distribution | c1 | c2 | relative spread of the absolute log error's signal-to-noise ratio over sigma = 2, 10, 50 | | r1 | normal | 0.0 | 0.5 | 3.761e-15 | | r2 | normal | 0.0 | 1.0 | 2.397e-16 | | r3 | normal | -0.5 | 1.5 | 1.579e-15 | | r4 | Gumbel | 0.0 | 0.5 | 4.243e-13 | | r5 | Gumbel | 0.0 | 1.0 | 7.932e-13 | | r6 | Gumbel | -0.5 | 1.5 | 1.43e-12 | E33.T3 distance of the squared error's signal-to-noise ratio over its formula from 1, lognormal, distance of exp(sigma^2 / 2) times the squared error's ratio from 1, lognormal, formula by c1, c2, sigma; hypothesis: H39 (lab runs RN16) {hypothesis: H39} | row | c1 | c2 | sigma | distance of the squared error's signal-to-noise ratio over its formula from 1, lognormal | distance of exp(sigma^2 / 2) times the squared error's ratio from 1, lognormal, formula | | r1 | 0.0 | 0.5 | 0.5 | 3.109e-15 | | | r2 | 0.0 | 1.0 | 0.5 | 6.661e-16 | | | r3 | -0.5 | 1.5 | 0.5 | 1.11e-15 | | | r4 | 0.0 | 0.5 | 1.0 | 1.554e-15 | | | r5 | 0.0 | 1.0 | 1.0 | 8.882e-16 | | | r6 | -0.5 | 1.5 | 1.0 | 5.551e-16 | | | r7 | 0.0 | 0.5 | 2.0 | 1.409e-13 | | | r8 | 0.0 | 1.0 | 2.0 | 7.55e-15 | | | r9 | -0.5 | 1.5 | 2.0 | 1.887e-15 | | | r10 | 0.0 | 1.0 | 4.0 | | 0.009325 | | r11 | 0.0 | 1.0 | 6.0 | | 3.08e-06 | | r12 | 0.0 | 1.0 | 8.0 | | 1.888e-11 | E33.T4 large-sample relative efficiency, formula by family, spread; hypothesis: H40 (lab runs RN16) {hypothesis: H40} | row | family | spread | large-sample relative efficiency, formula | | r1 | lognormal | 0.25 | 0.9696 | | r2 | lognormal | 0.5 | 0.8881 | | r3 | lognormal | 1.0 | 0.6601 | | r4 | lognormal | 1.5 | 0.454 | | r5 | lognormal | 2.0 | 0.3073 | | r6 | lognormal | 3.0 | 0.1474 | | r7 | lognormal | 4.0 | 0.07778 | | r8 | lognormal | 6.0 | 0.0279 | | r9 | log-Laplace | 0.25 | 0.9969 | | r10 | log-Laplace | 0.5 | 0.9765 | | r11 | log-Laplace | 1.0 | 0.8953 | | r12 | log-Laplace | 1.5 | 0.8036 | | r13 | log-Laplace | 2.0 | 0.7207 | | r14 | log-Laplace | 3.0 | 0.5903 | | r15 | log-Laplace | 4.0 | 0.4969 | | r16 | log-Laplace | 6.0 | 0.3755 | E33.T5 distance of the large-sample relative efficiency from 1 at spread 0.01 by family; hypothesis: H40 (lab runs RN16) {hypothesis: H40} | row | family | distance of the large-sample relative efficiency from 1 at spread 0.01 | | r1 | lognormal | 5e-05 | | r2 | log-Laplace | 1.249e-08 | E33.T6 distributions where A is not below 2 g at the median, largest difference from the asymptotic values recorded by RN7; hypothesis: H40, distributions: 16 (lab runs RN16) {hypothesis: H40, distributions: 16} | row | distributions where A is not below 2 g at the median | largest difference from the asymptotic values recorded by RN7 | | r1 | 0 | 0.00014 | E33.T7 large-sample relative efficiency times sigma^3 over 6 (ln 2)^2 sqrt(2 pi), lognormal, large-sample relative efficiency times b over 3, log-Laplace; hypothesis: H40, spread: 200 (lab runs RN16) {hypothesis: H40, spread: 200} | row | large-sample relative efficiency times sigma^3 over 6 (ln 2)^2 sqrt(2 pi), lognormal | large-sample relative efficiency times b over 3, log-Laplace | | r1 | 0.9998 | 0.9906 | E18 [formal; foundation Kolmogorov probability axioms and real analysis] Proof of C1 (section H1 of the proof document A1, rendered in A2). Only the ratio of the actual to the forecast matters, not their size. S1 Since $y$ and $f$ are positive, $|y| + |f| = y + f$. So $s = |y - f|/(y + f)$. (by D1; algebra) S2 Dividing the numerator and the denominator by $f > 0$ gives $\dfrac{|y-f|/f}{(y+f)/f}$. Since $f > 0$, $|y - f|/f = |(y - f)/f| = |r - 1|$. And $(y+f)/f = r + 1$. (by S1; algebra) S3 So $s = |r - 1|/(r + 1)$. The contribution depends only on the ratio of actual to forecast. (by S2; conclusion) E20 [formal; foundation Kolmogorov probability axioms and real analysis] Proof of C3 (section H3 of the proof document A1, rendered in A2). sMAPE measures the error on the log scale. For small errors it is half the absolute log error; for large errors it levels off. S1 Write $z = \ln y - \ln f$, so that $r = y/f = e^z$. By C1, $s = \dfrac{|e^z - 1|}{e^z + 1}$. (by C1; definition) S2 Multiplying the numerator and the denominator of $\dfrac{e^z - 1}{e^z + 1}$ by $e^{-z/2}$ gives $\dfrac{e^{z/2} - e^{-z/2}}{e^{z/2} + e^{-z/2}} = \tanh(z/2)$. The denominator is positive, so $s = |\tanh(z/2)|$. (by S1; algebra) S3 $\tanh$ changes only its sign when its argument changes sign: $\tanh(-x) = -\tanh(x)$, as the definition shows. So $|\tanh(z/2)| = \tanh(|z|/2)$, which is the stated form. (by S2; algebra) S4 For (a): $\tanh$ is 0 at 0. On $[0, \infty)$ it is strictly increasing and concave, and it approaches 1 as its argument grows; its derivative $1 - \tanh^2 x$ is positive and decreasing for $x > 0$. With the argument $|\ln y - \ln f|/2$, $s$ has these properties in $|\ln y - \ln f|$. (by S3; monotonicity) S5 For (b): for every $x \ge 0$, $x - x^3/3 \le \tanh x \le x$. The upper bound holds because $\tanh$ has slope at most 1; the lower one because the derivative of $\tanh x - x + x^3/3$ is $x^2 - \tanh^2 x \ge 0$, and the difference is 0 at $x = 0$. Putting $x = |\ln y - \ln f|/2$ gives the bounds. (by S4; algebra) S6 For (c), the median: the mean of $|\ln y_t - \ln f|$ over a test set is minimised when $\ln f$ is a median of the $\ln y_t$. Since the logarithm is increasing, this is when $f$ is a median $m$ of the $y_t$. (by S5; monotonicity) S7 For (c), the bound: with $z_t = \ln y_t - \ln f$ the log errors of the test set, write $\overline{|z|}(f) = \frac1n\sum_t|\ln y_t - \ln f|$, $\overline{|z|^3}(f) = \frac1n\sum_t|\ln y_t - \ln f|^3$ and $\bar s(f)$ for the mean of the sMAPE terms of the test set. Averaging the bounds of step 5 over the test set gives $\bar s(f) \le \tfrac12\overline{|z|}(f) \le \bar s(f) + \tfrac1{24}\overline{|z|^3}(f)$ for every $f > 0$. Let $f^*$ minimise $\bar s$, so that $\bar s(f^*) \le \bar s(m)$. Then $$\tfrac12\overline{|z|}(f^*) \le \bar s(f^*) + \tfrac1{24}\overline{|z|^3}(f^*) \le \bar s(m) + \tfrac1{24}\overline{|z|^3}(f^*) \le \tfrac12\overline{|z|}(m) + \tfrac1{24}\overline{|z|^3}(f^*).$$ Multiplying by 2 gives $\overline{|z|}(f^*) - \overline{|z|}(m) \le \tfrac1{12}\overline{|z|^3}(f^*)$, the stated inequality. (by S5; algebra) S8 For (c), not the median in general: take the five actuals $1, 1, e, e^4, e^4$, whose median is $e$. With the forecast at $e$ the log errors are $1, 1, 0, 3, 3$ in size and the sMAPE terms sum to $2\tanh(0.5) + 2\tanh(1.5) = 2.734$. With the forecast at 1 the log errors are $0, 0, 1, 4, 4$ and the terms sum to $\tanh(0.5) + 2\tanh(2) = 2.390$, which is lower. So the median does not minimise the sMAPE of this test set, although it minimises its log-MAE (step 6). (by S6; algebra) S9 So $s = \tanh(|\ln y - \ln f|/2)$. It rises from 0 like half the absolute log error and levels off at 1. For small errors sMAPE is close to the log-MAE, and its minimiser is close to the median without in general being the median. (by S8; conclusion) E21 [formal; foundation Kolmogorov probability axioms and real analysis] Proof of C9 (section H4 of the proof document A1, rendered in A2). The expected values of RMSE, MAE and MAPE become infinite for every forecast when the actuals have an infinite moment, such as an infinite mean (a heavy tail, C8). The expected sMAPE never does, and there is always a best forecast. S1 Finite. For positive values, $s = \tanh(|\ln y - \ln f|/2)$ (C3), which lies between 0 and 1. For any signs, $|y - f| \le |y| + |f|$, so again $s \le 1$. A quantity between 0 and 1 has an expectation between 0 and 1, so the expected sMAPE is between 0 and 200, whatever the actuals. (by C3; earlier result) S2 A best forecast. Write $\mathrm E[s(y,f)]$ for the expectation over the random actual $y$. As $f$ approaches 0 or grows without bound, $|\ln y - \ln f|$ grows for every positive value of $y$, so $s$ approaches 1 (C3), and so does its expectation. For every positive $f$ it is below 1. It changes continuously with $f$, because $s$ does and is at most 1 (bounded convergence). A continuous function that is below 1 in between and returns towards 1 at both ends reaches a lowest value at some $f > 0$. A forecast of 0 or a negative forecast scores 1 on every positive actual, so it is never better. (by C3, D1; bounded convergence) S3 So the expected sMAPE is always finite, whatever the tails of the actuals. For positive actuals, a forecast with the lowest expected sMAPE always exists. (by S2; conclusion) E22 [formal; foundation Kolmogorov probability axioms and real analysis] Proof of C4 (section H5 of the proof document A1, rendered in A2). For any distribution, the best forecast balances the actuals below it and above it. Each actual is weighted by how close it is to the forecast by ratio. S1 The derivative of $s$. When $y > f > 0$, $s = (y - f)/(y + f)$. By the quotient rule its derivative in $f$ is $\dfrac{-(y+f) - (y-f)}{(y+f)^2} = -2w(y,f)$. When $f > y > 0$, $s = (f - y)/(y + f)$, and in the same way its derivative is $2w(y,f)$. Together: for $y \ne f$, the derivative is $2w(y,f)(\mathbf 1\{y < f\} - \mathbf 1\{y > f\})$. (by D1; quotient rule) S2 A bound on it. Since $(y+f)^2 \ge 4yf$, $w(y,f) \le 1/(4f)$. So the derivative of $s$ is at most $1/(2f)$ in size, whatever $y$, and at most $1/f$ for every forecast above $f/2$. The only actual at which $s$ has no derivative is $y = f$, which has probability zero. (by S1; algebra) S3 The derivative of the expectation. With this bound, the derivative may be taken inside the expectation (dominated convergence): $$\frac{d}{df}\,\mathrm E[s(y,f)] = 2\,\mathrm E\big[w(y,f)\,(\mathbf 1\{y < f\} - \mathbf 1\{y > f\})\big].$$ (by S2; dominated convergence) S4 The balance. A forecast with the lowest expected sMAPE exists (C9). There this derivative is 0, which is the stated balance: half the total weight lies below $f$ and half above. (by C9; earlier result) S5 The weight. With $r = y/f$, $w(y,f) = \dfrac1f\cdot\dfrac{r}{(1+r)^2} = \dfrac1f\cdot\dfrac{1}{r + 2 + 1/r}$. The sum $r + 1/r$ is the same for $r$ and $1/r$. It is smallest, 2, at $r = 1$, and grows as $r$ moves away from 1. So $w$ is largest at $y = f$ and falls as $y/f$ moves away from 1, equally for $r$ and $1/r$. (by S4; algebra) S6 So sMAPE targets a weighted median of the actuals. Each actual is weighted by how close it is to the forecast by ratio. (by S5; conclusion) E23 [formal; foundation Kolmogorov probability axioms and real analysis] Proof of C5 (section H6 of the proof document A1, rendered in A2). This is the case of C4 in which the weights balance at the median. S1 By C3, $\mathrm E[s(y,f)] = \mathrm E\big[\tanh\big(|\ln y - \ln f|/2\big)\big]$. Take a forecast $f = m\,e^{\tau}$ with $\tau > 0$; the case $\tau < 0$ is the mirror image. We show that its expected sMAPE is larger than that of $m$. (by C3; setup) S2 Write both expectations as integrals over $v$, the distance of $\ln y$ from $\ln f$. For the forecast $m\,e^{\tau}$, $\ln y = \ln m + \tau + v$; for $m$, $\ln y = \ln m + v$. Their difference is $$\Delta = \int \tanh(|v|/2)\,\big[g(\ln m+\tau+v) - g(\ln m+v)\big]\,dv.$$ (by S1; definition) S3 Write $\phi(v) = g(\ln m+\tau+v) - g(\ln m+v)$. By the symmetry of $g$ about $\ln m$, $\phi(-\tau-v) = g(\ln m-v) - g(\ln m-\tau-v) = g(\ln m+v) - g(\ln m+\tau+v) = -\phi(v)$. So reflecting $v$ about $-\tau/2$ reverses the sign of $\phi$. (by S2; definition) S4 Split the integral at $-\tau/2$ and reflect its left part onto the right. This gives $$\Delta = \int_{v > -\tau/2} \big[\tanh(|v|/2) - \tanh(|v+\tau|/2)\big]\,\phi(v)\,dv.$$ (by S3; algebra) S5 For $v > -\tau/2$, $v$ is closer to 0 than $v + \tau$ is: $|v| < |v + \tau|$. So the bracket is negative, because $\tanh$ is increasing (C3). And $\phi(v) \le 0$ by unimodality: $g(\ln m+\tau+v) \le g(\ln m+v)$. Their product is $\ge 0$, and $> 0$ on a set of positive probability. So $\Delta > 0$. (by C3; monotonicity) S6 So every forecast other than the median has a larger expected sMAPE. The median is the only forecast that minimises it. (by S5; conclusion) E29 [formal; foundation Kolmogorov probability axioms and real analysis] Proof of C12 (section H12 of the proof document A1, rendered in A2). The target is above the ordinary median when the actuals above it carry more weight, and below it when those below carry more. When the weights balance and enough actuals lie near the median, the median is the target. This is so when the log of the actual is symmetric with a single peak. S1 The derivative at the median. By C4, step 3, the derivative of the expected sMAPE term in $f$ is $2\,\mathrm E[w(y,f)(\mathbf 1\{y < f\} - \mathbf 1\{y > f\})]$. At $f = m$ it is twice the total weight of the actuals below $m$ minus that of the actuals above $m$. (by C4; earlier result) S2 The shape of expected sMAPE. With a single local minimum, the expected sMAPE decreases up to it and increases after it. Otherwise it would turn again, at a local maximum. After that turn it would have to fall and then rise once more, because it returns towards 1 at both ends (C9). That would give a second local minimum. (by C9; earlier result) S3 (a). If the actuals above $m$ carry more weight, the derivative at the median is negative (step 1): the expected sMAPE is decreasing at the median. By step 2, the median then lies before the minimum, so the target is above the median. The case where the actuals below carry more weight is the mirror image. (by S1, S2; symmetry) S4 The second derivative. Measure the forecast on the log scale, $c = \ln f$, so that the expected contribution is $\mathrm E[\tanh(|\ln y - c|/2)]$ (C3). The derivative of $\tanh(|x - c|/2)$ in $c$ is $-\tfrac12\operatorname{sech}^2\big(\tfrac{x-c}{2}\big)$ for $c < x$ and $+\tfrac12\operatorname{sech}^2\big(\tfrac{x-c}{2}\big)$ for $c > x$: it jumps by 1 at $c = x$. Differentiating again and taking the expectation over $x = \ln y$, the jumps add the density $g$ of $\ln y$ at $c$, and elsewhere the second derivative is $-\tfrac12\operatorname{sech}^2\big(\tfrac{x-c}{2}\big)\tanh\big(\tfrac{|x-c|}{2}\big)$. So the second derivative of the expected contribution is $$g(c) - \tfrac12\,\mathrm E\Big[\operatorname{sech}^2\Big(\frac{\ln y - c}{2}\Big)\tanh\Big(\frac{|\ln y - c|}{2}\Big)\Big].$$ This holds when $g$ is continuous at $c$: since $\operatorname{sech}^2 = 1 - \tanh^2$, the first derivative is $\mathrm P(\ln y \le c) - \tfrac12$, whose derivative at $c$ is $g(c)$, plus the expectation of $-\tfrac12\tanh^2\big(\tfrac{x-c}{2}\big)$ with the sign of $c - x$, a function of $c$ with a bounded continuous derivative, which may be differentiated inside the expectation. (by C3; continuity) S5 (b), the sufficient condition. If the two weights are equal, the first derivative at the median is 0 (step 1): the expected sMAPE is flat there. If, in addition, the stated inequality holds, the second derivative at $c = \ln m$ is positive (step 4), so the median is a strict local minimum. With a single local minimum, it is the target. (by S1, S4; algebra) S6 (b), without the inequality. A point where the first derivative is 0 can be a flat point of inflection, through which the expected sMAPE keeps rising (or keeps falling) without turning. An example, built by the independent checker of this section: let $\ln y - \ln m$ be a mixture of three normal distributions, with mean $-1.5$ and standard deviation $0.3$ (weight 0.406125), mean 0 and standard deviation $0.5$ (weight 0.054825), and mean $1.158664$ and standard deviation 1 (weight 0.539050); these values are rounded from the exact solution of the three conditions. Its median is $m$, the two weights are equal there, and the second derivative of step 4 is 0 at $\ln m$; the expected sMAPE rises through $m$, where it is flat, and its only minimum is near $\ln f = \ln m - 1.25$. So equal weights alone do not make the median the target. (by S4; algebra) S7 (c). By C5 the median is then the only forecast with the lowest expected sMAPE (a density cannot be constant, so a symmetric unimodal density is strictly lower somewhere away from its centre, as C5 asks). For lognormal actuals $\ln y$ has a normal density, and for log-Laplace actuals the Laplace density $e^{-|x - \ln m|/b}/(2b)$; both are symmetric about $\ln m$ and strictly decreasing in the distance from it. (by C5; earlier result) S8 So the target is above the ordinary median when the actuals above it carry more of the weight, and below it when those below carry more. When the two balance, the median is the target if, in addition, enough actuals lie near it, as the stated inequality says; this is the case when the log of the actual is symmetric with a single peak. (by S7; conclusion) E36 [formal; foundation Kolmogorov probability axioms and real analysis] Proof of C19 (section H14 of the proof document A1, rendered in A2). Where the log of the actual is symmetric with a single peak, the forecast that minimises the sMAPE of a test set is asymptotically normal about the median. Compared with the sample median it is asymptotically as precise for narrow distributions and asymptotically loses all of its precision as the actuals spread out. S1 Write $x_t = \ln y_t - \ln m$ for the actuals of a test set and $\theta = \ln f - \ln m$ for a forecast $f$. By C3 the mean sMAPE term of the test set is $M_n(\theta) = \frac1n\sum_t\tanh(|x_t - \theta|/2)$, and its expectation is $M(\theta) = \mathrm E[\tanh(|x - \theta|/2)]$. So $\hat\theta_n = \ln\hat f_n - \ln m$ minimises $M_n$; a minimiser exists, because $M_n$ is continuous, tends to 1 as $|\theta|$ grows, and is below 1 at each $x_t$. (by C3; setup) S2 (a). Write $\kappa(u) = \operatorname{sech}^2(u/2)\tanh(|u|/2) \ge 0$. Since $\kappa(u)$ is the derivative of $\tanh^2(u/2)$ for $u > 0$, $\int\kappa(u)\,du = 2$. By unimodality $g(\ln m + u) \le g(\ln m)$, so $A = \int\kappa(u)\,g(\ln m + u)\,du \le 2g(\ln m)$. A density cannot equal $g(\ln m)$ everywhere, and $g(\ln m + u)$ does not increase as $|u|$ grows; so $g(\ln m + u) < g(\ln m)$ for every $|u|$ beyond some point, where $\kappa > 0$. So the inequality is strict. (by S1; definition) S3 The expected loss is lowest at 0 only. $M(\theta)$ is the expected sMAPE term of the forecast $m\,e^\theta$. By C5, $M(\theta) > M(0)$ for every $\theta \ne 0$. The function $\tanh(|u|/2)$ has slope at most $\tfrac12$, so $M$ and every $M_n$ change by at most $\tfrac12|\theta_1 - \theta_2|$ between $\theta_1$ and $\theta_2$; in particular $M$ is continuous. And $M(\theta) \to 1 > M(0)$ as $|\theta| \to \infty$, by bounded convergence. So for every $\eta > 0$ the lowest value of $M$ over $|\theta| \ge \eta$ is above $M(0)$. (by C5; bounded convergence) S4 $M_n$ is close to $M$ for all $\theta$ at once. For each $\theta$, $M_n(\theta) \to M(\theta)$ with probability 1 (law of large numbers; the terms are bounded). Take a grid of spacing $\gamma$ on $[-R, R]$: by the slope bound of step 3, once $M_n$ is within $\gamma$ of $M$ at the grid points it is within $2\gamma$ of $M$ on all of $[-R, R]$. Outside $[-R, R]$: an actual with $|x_t| \le R/2$ has $\tanh(|x_t - \theta|/2) \ge \tanh(R/4)$ when $|\theta| > R$, so $M_n(\theta)$ is at least $\tanh(R/4)$ times the share of such actuals, which tends to $\mathrm P(|x| \le R/2)$. Since $M(0) < 1$, for $R$ large this is above $M(0)$ by a fixed margin. (by S3; case analysis) S5 Consistency. $M_n(\hat\theta_n) \le M_n(0)$, which tends to $M(0)$. By step 4, with probability tending to 1, $|\hat\theta_n| \le R$ and $M(\hat\theta_n)$ is within any given margin of $M(0)$; by step 3, $|\hat\theta_n| < \eta$. So $\hat\theta_n \to 0$ in probability, which is $\hat f_n \to m$. This is the argument of van der Vaart (1998, Theorem 5.7). (by S4, S3, B15; algebra) S6 The curvature at 0. By C12, step 4, with $c = \ln m$, the second derivative of $M$ at 0 is $g(\ln m) - \tfrac12 A = \tfrac12(2g(\ln m) - A)$, which is positive by step 2. The first derivative at 0 is 0, because 0 is the minimum (step 3). (by S2, S3, C12; earlier result) S7 Asymptotic normality. For every $x \ne 0$, $\theta \mapsto \tanh(|x - \theta|/2)$ has a derivative at $\theta = 0$, $-\psi(x)$ with $\psi(x) = \tfrac12\operatorname{sech}^2(x/2)$ for $x > 0$ and $-\tfrac12\operatorname{sech}^2(x/2)$ for $x < 0$; and $x = 0$ has probability zero. For every $x$ it changes by at most $\tfrac12|\theta_1 - \theta_2|$ (step 3). $M$ has a second-order expansion at its minimum 0 with a positive second derivative (step 6). And $\hat\theta_n$ minimises $M_n$ and tends to 0 in probability (step 5). These are the conditions of the theorem on estimators that minimise a criterion which is Lipschitz in the parameter (van der Vaart, 1998, Theorem 5.23). It gives that $\sqrt n\,\hat\theta_n$ is asymptotically normal with mean 0 and variance $\mathrm E[\psi(x)^2]$ divided by the square of the second derivative of $M$ at 0. (by S3, S6, S5, B15; cited theorem) S8 The variance. $\mathrm E[\psi(x)^2] = \tfrac14\mathrm E[\operatorname{sech}^4(x/2)] = B/4$, and the square of the second derivative is $\tfrac14(2g(\ln m) - A)^2$ (step 6). So the variance is $B/(2g(\ln m) - A)^2$. (by S6; algebra) S9 The sample median. The logarithm of the sample median of the $y_t$ is the sample median of the $\ln y_t$, which minimises the mean of $|x_t - \theta|$. The same theorem applies with $|x - \theta|$ in place of $\tanh(|x - \theta|/2)$: the derivative at 0 is $-1$ for $x > 0$ and $+1$ for $x < 0$, with square 1; the criterion changes by at most $|\theta_1 - \theta_2|$; and the second derivative of $\mathrm E|x - \theta|$ at 0 is $2g(\ln m)$, positive because $g$ is largest at $\ln m$. So the variance is $1/(4g(\ln m)^2)$. (by S8, B15; cited theorem) S10 (b), the ratio. Dividing the variance of step 9 by that of step 8 gives $\mathrm{ARE} = (2g(\ln m) - A)^2/(4g(\ln m)^2 B)$. (by S9, S8; algebra) S11 (c), narrow distributions. Write $\ln y = \ln m + \sigma\varepsilon$ with $\varepsilon$ standard normal, or Laplace with scale 1 and $\sigma = b$; then $g(\ln m) = h(0)/\sigma$ with $h$ the density of $\varepsilon$, and $x = \sigma\varepsilon$. As $\sigma \to 0$, $A \to 0$ and $B \to 1$ by bounded convergence, and $A/(2g(\ln m)) = \sigma A/(2h(0)) \to 0$. So $\mathrm{ARE} = (1 - A/(2g(\ln m)))^2/B \to 1$. (by S10; bounded convergence) S12 (c), wide lognormal distributions. Since $\int\kappa = 2$ (step 2), $2g(\ln m) - A = \int\kappa(u)\,[g(\ln m) - g(\ln m + u)]\,du$. For the normal density, $g(\ln m) - g(\ln m + u) = \big(1 - e^{-u^2/(2\sigma^2)}\big)/(\sigma\sqrt{2\pi})$, and $\tfrac{u^2}{2\sigma^2} - \tfrac{u^4}{8\sigma^4} \le 1 - e^{-u^2/(2\sigma^2)} \le \tfrac{u^2}{2\sigma^2}$. Integrating by parts, $\int u^2\kappa(u)\,du = 4\int_0^\infty u\operatorname{sech}^2(u/2)\,du = 16\ln 2$; and $\int u^4\kappa(u)\,du$ is finite. So $2g(\ln m) - A = 8\ln 2/(\sigma^3\sqrt{2\pi}) + O(\sigma^{-5})$. In the same way, with $\int\operatorname{sech}^4(u/2)\,du = 8/3$, $B = 8/(3\sigma\sqrt{2\pi}) + O(\sigma^{-3})$. With $g(\ln m) = 1/(\sigma\sqrt{2\pi})$, $\mathrm{ARE}\,\sigma^3$ tends to $\dfrac{64(\ln 2)^2}{2\pi}\cdot\dfrac{2\pi}{4}\cdot\dfrac{3\sqrt{2\pi}}{8} = 6(\ln 2)^2\sqrt{2\pi}$. (by S2; algebra) S13 (c), wide log-Laplace distributions. Here $g(\ln m + u) = e^{-|u|/b}/(2b)$, so $g(\ln m) - g(\ln m + u) = (1 - e^{-|u|/b})/(2b)$, which lies between $\big(|u|/b - u^2/(2b^2)\big)/(2b)$ and $|u|/(2b^2)$. Integrating by parts, $\int |u|\,\kappa(u)\,du = 2\int_0^\infty\operatorname{sech}^2(u/2)\,du = 4$. So $2g(\ln m) - A = 2/b^2 + O(b^{-3})$, and in the same way $B = 4/(3b) + O(b^{-2})$. With $g(\ln m) = 1/(2b)$, $\mathrm{ARE}\,b$ tends to $\dfrac{4}{b^4}\cdot b^2\cdot\dfrac{3b}{4}\cdot b = 3$. (by S12; algebra) S14 So the sMAPE minimiser of a test set is asymptotically normal about the median, with a variance that the density at the median and two expectations give. On lognormal and log-Laplace actuals it is asymptotically as precise as the sample median for narrow distributions and asymptotically loses all of its precision relative to it as the actuals spread out, at the rates $\sigma^{-3}$ and $b^{-1}$. (by S13; conclusion) ARTEFACTS A1 [proof] Proofs of the formal claims, version 3, in Markdown with LaTeX: section H3 with part (c); H5 shortened to the weighted median; new sections H12 (the ordinary median, with the checker's example), H13 (signal-to-noise ratios) and H14 (large-sample efficiency). Checked independently by AC6 (CK27 to CK31, CK34 to CK37) and corrected as it proposed; after its checks the notation was made consistent (no second meaning for k, a, G), which AC6 read and found to change nothing. Sections H1 to H14 prove C1, C2, C3, C9, C4, C5, C6, C7, C11, C8, C10, C12, C18 and C19, in that order. Sections H1 to H11 were also checked independently by lab actor AC4 (proof-check-report.md) and by the supervisor. A2 [proof] The same proofs, version 3, rendered as PDF. A6 [data] Every result of the lab record (17182 results of 17 runs) at full precision: the lab result's ID, its run, the quantity as the run's script named it, its conditions and its value (both as JSON). The evidence's tables give the values to four significant digits. REFERENCES B8 10.1016/0169-2070(93)90079-3 (doi) B13 10.3390/forecast8020024 (doi) B14 10.1016/j.ijforecast.2020.06.008 (doi) B15 10.1017/CBO9780511802256 (doi) HOW TO CITE This claim: Svetunkov, I. (guarantor) (2026). Claim C19, in: What does sMAPE reward? An analytical and simulation study. EntryLib. entrylib:2026.000013@v3#C19. https://entrylib.org/p/2026.000013/C19 The claimset it is from: Svetunkov, I. (guarantor) (2026). What does sMAPE reward? An analytical and simulation study. EntryLib. entrylib:2026.000013@v3. https://entrylib.org/p/2026.000013@v3