You are reading a research paper published on EntryLib as separate claims, each with its evidence. How to read this page

corrected · Version 3 · 18 November 2026 · Statistics and Probability

Corrected · 18 November 2026 at 13:08 UTC

Version 3 answers a reader's comments. Corrected: C3 no longer says sMAPE is minimised by the median for small errors, only near it, with a bound; C4 keeps the weighted median, and C12 now holds what was said about the ordinary median; the measure called msMAPE in version 2 is the Suilin loss. Added: how well a test set tells two forecasts apart as the actuals spread out (C18); the large-sample distribution of the sMAPE minimiser (C19); Smyl's msMAPE (C20) and what it rewards on intermittent demand (C21). Also added: sections, the definition of sMAPE first, and a limitation on C6.

By Ivan Svetunkov (ORCID 0000-0001-7826-0281)

What does sMAPE reward? An analytical and simulation study

Responsible researcher: Ivan Svetunkov (0000-0001-7826-0281)

CC-BY-4.0 · Keywords: forecast evaluation, error measures, sMAPE, point forecasts, consistent scoring functions, proof, simulation study, msMAPE, Suilin loss, intermittent demand

Contributions
  • Claude Code · derivations and simulations (AI agent): Methodology, Formal analysis, Software, Investigation, Visualization, Writing – original draft
  • Ivan Svetunkov (0000-0001-7826-0281): Conceptualization, Supervision, Validation, Writing – review & editing
  • Claude Code · version 3: corrections and new derivations (AI agent): Methodology, Formal analysis, Software, Investigation, Visualization, Writing – original draft

Made with claimset-lab.

Text file for AI

Text file: the whole paper as plain text, to attach when your assistant cannot open the address.

The message

Read https://entrylib.org/p/[email protected], a research paper written as claims with their evidence. Give me a brief summary of its main findings and how well each is supported. If you cannot open the link, tell me instead of guessing. Then wait for my questions.

entrylib:2026.000013@v3 · https://entrylib.org/p/2026.000013@v3

Support: 99/100 (EntryLib rubric 0.7; points lost: Evidence 0, Plan 0, Checkable 0, Context 0, Open problems −1). Checks the record, not the science.

Summary

Which forecast does sMAPE reward? sMAPE has been criticised for its asymmetry and its behaviour near zero, but the forecast that minimises it had been found only in special cases. We show that for positive values sMAPE depends only on the ratio of actual to forecast, not on their difference: it is the hyperbolic tangent of half the absolute log error. So it scores a forecast k times too high and one k times too low the same, and is asymmetric only for equal absolute errors. Being bounded, it has a finite expected value for every distribution. Its target is a median of the actuals, each weighted by how close it is to the forecast by ratio. This is the median when the log of the actual is symmetric and unimodal; otherwise it can move to the side of the median that carries more weight. With two well-separated groups of actuals, the median becomes a local maximum. On intermittent demand the target is 0 once the share of zeros passes a threshold below one half. Otherwise sMAPE aims at a weighted median of the non-zero actuals. As the actuals spread out, expected sMAPE approaches its bound for every forecast, so forecasts become hard to tell apart, and the target moves from the median to the mode of the log. Where sMAPE and MAE both target the median, a test set's sMAPE minimiser estimates it less precisely than the sample median, increasingly so as the actuals spread out.

Claims

A term is defined before the claim that first uses it, and what supports a claim stands under its statement.

What sMAPE measures

Terms this claim uses

sMAPE
Symmetric mean absolute percentage error of a point forecast ff over a test set of actuals y1,…,yny_1, \dots, y_n: 200n∑t|yt−f|/(|yt|+|f|)\frac{200}{n}\sum_t |y_t - f|/(|y_t|+|f|), where a term with yt=f=0y_t = f = 0 counts as 0. Introduced as the symmetric MAPE by Makridakis (1993), whose denominator had no absolute values; for positive actuals and forecasts the two agree.
sMAPE term s(y,f)s(y,f)
The contribution of one actual yy and one point forecast ff to sMAPE: s(y,f)=|y−f|/(|y|+|f|)s(y,f) = |y-f|/(|y|+|f|), between 0 and 1, with s=0s = 0 when y=fy = f (including y=f=0y = f = 0, where the fraction is 0/00/0).

Claim C1. sMAPE depends only on the ratio of actual to forecast key claimproved

Only the ratio of the actual to the forecast matters to sMAPE, not their size.

For y>0y > 0 and f>0f > 0, the sMAPE term s(y,f)=|y−f|/(|y|+|f|)s(y,f) = |y-f|/(|y|+|f|) depends on yy and ff only through their ratio r=y/fr = y/f: s=|r−1|/(r+1)s = |r - 1|/(r + 1).

Proof 3 steps
  1. Since yy and ff are positive, |y|+|f|=y+f|y| + |f| = y + f. So s=|y−f|/(y+f)s = |y - f|/(y + f).
  2. Dividing the numerator and the denominator by f>0f > 0 gives |y−f|/f(y+f)/f\dfrac{|y-f|/f}{(y+f)/f}. Since f>0f > 0, |y−f|/f=|(y−f)/f|=|r−1||y - f|/f = |(y - f)/f| = |r - 1|. And (y+f)/f=r+1(y+f)/f = r + 1.

So s=|r−1|/(r+1)s = |r - 1|/(r + 1). The contribution depends only on the ratio of actual to forecast. ∎

Its formulae were also checked on numbers: E1 in the record.

Earlier work and other claims

  • Extends Tofallis (2015). Tofallis (2015) lists the sMAPE term, |y-f|/[(|y|+|f|)/2], among Tornqvist et al.'s measures of relative change; claim C1 shows it depends on the actual and the forecast only through their ratio.
Proved · checked independently by an AI agent · confirmed by Ivan Svetunkov · tested as plannedRecord
Claim
C1 · formal · principal · extends earlier work
Plan
Test planned in lab plan PL9, recorded 17 November 2026
Evidence
E18, E1 in the record
Check: confirmed
By an AI agent (proof checking), full. Derived from the statement alone (first wording, with the factor 200), then compared with the final proofs.md (paper H1), which agrees after rescaling. No conditions needed.
Check: confirmed
By Ivan Svetunkov, full. The supervisor read the proof of paper H1 in proofs.md (AR1.R2; rendered as AR2.R2) and confirmed it: first "Confirmed" (DC6), then, after the last corrections of paper H4, H5 and H9, "All looks fine now. NExt".
Support score
SupportS 100 of 100. The score checks the record, not the science.
Record
This claim on the record page
To cite it
The page of this claim

Claim C2. Too high and too low by the same factor score the same proved

A forecast twice too high and one half too low score the same. For equal absolute errors the higher one scores better. This compares forecasts for one known actual. Which forecast is best when the actual is uncertain is claim claim C4: sMAPE does not in general reward forecasting too high.

For y>0y > 0 and f>0f > 0:

  1. (a)Replacing r=y/fr = y/f by 1/r1/r leaves the sMAPE term unchanged, so a forecast kk times too high (f=kyf = ky) and one kk times too low (f=y/kf = y/k) score the same
  2. (b)Of two forecasts the same absolute distance dd from yy (0<d<y0 < d < y), the higher one scores better: s(y,y+d)=d2y+d<d2y−d=s(y,y−d).s(y, y+d) = \frac{d}{2y+d} < \frac{d}{2y-d} = s(y, y-d).
Proof 4 steps
  1. By claim C1, s=|r−1|/(r+1)s = |r - 1|/(r + 1). Replacing rr by 1/r1/r gives |1/r−1|1/r+1\dfrac{|1/r - 1|}{1/r + 1}. Multiplying the numerator and the denominator by r>0r > 0 gives |1−r|1+r\dfrac{|1 - r|}{1 + r}, the same value.
  2. The forecast f=kyf = ky has r=1/kr = 1/k, and f=y/kf = y/k has r=kr = k. By step 1 they score the same. This is (a).
  3. The forecasts y+dy + d and y−dy - d are both dd away from yy, so both numerators are dd. The denominators are y+(y+d)=2y+dy + (y + d) = 2y + d and y+(y−d)=2y−d>0y + (y - d) = 2y - d > 0. The larger denominator gives the smaller value. This is (b).

So sMAPE treats a forecast kk times too high and one kk times too low alike. It favours the high forecast only when the two are equally wrong in absolute terms. ∎

Its formulae were also checked on numbers: E1 in the record.

Elsewhere in this paper

  • Qualified by claim C4. This claim compares two forecasts of one known actual. For an uncertain actual, the forecast sMAPE rewards is the weighted median of claim C4, not a forecast that is too high.

Earlier work and other claims

  • Qualifies Svetunkov (2026a), C33. C33 says sMAPE penalises a forecast below the outcome more than one above it. That holds for two forecasts the same absolute distance away (claim C2 b); a forecast k times too low and one k times too high score the same (claim C2 a).
  • Refines Goodwin and Lawton (1999). Goodwin and Lawton (1999) showed that sMAPE is asymmetric: for the same absolute error it scores a forecast above the actual better than one below. Claim C2 proves this (b) and shows that by ratio sMAPE is symmetric (a).
  • Refines Hyndman and Koehler (2006). Hyndman and Koehler (2006) note that for the same actual sMAPE penalises low forecasts more heavily than high ones; claim C2 shows this holds for equal absolute errors and not for equal ratios.
  • Refines Hewamalage et al. (2023). Hewamalage et al. (2023) note that sMAPE penalises underestimates more than overestimates for the same actual; claim C2 shows this holds for equal absolute errors and not for equal ratios.
  • Refines Chen et al. (2017). Chen et al. (2017) show a heavier penalty on a forecast 10% under the actual than on one 10% over; claim C2 shows that forecasts the same factor too high and too low score the same.
Proved · checked independently by an AI agent · confirmed by Ivan Svetunkov · tested as plannedRecord
Claim
C2 · formal · secondary · no earlier result found
Plan
Test planned in lab plan PL9, recorded 17 November 2026
Evidence
E19, E1 in the record
Check: confirmed
By an AI agent (proof checking), full. Derived from the statement alone (earlier wording: old H2 and H10), then compared with the final proofs.md (paper H2), which agrees. No conditions needed.
Check: confirmed
By Ivan Svetunkov, full. The supervisor read the proof of paper H2 in proofs.md (AR1.R2; rendered as AR2.R2) and confirmed it: first "Confirmed" (DC6), then, after the last corrections of paper H4, H5 and H9, "All looks fine now. NExt".
Support score
SupportS 100 of 100. The score checks the record, not the science.
Record
This claim on the record page

Terms this claim uses

log error zz
For a positive actual yy and a positive forecast ff, z=ln⁡y−ln⁡fz = \ln y - \ln f.
log-MAE
100 times the mean absolute error of ln⁡f\ln f as a forecast of ln⁡y\ln y: 100n∑t|ln⁡yt−ln⁡f|\frac{100}{n}\sum_t |\ln y_t - \ln f|. Unbounded; sMAPE's small-error limit (claim C3); minimised by the median.

Claim C3. sMAPE is a hyperbolic tangent of the log error proved

sMAPE measures the error on the log scale. For small errors it is half the absolute log error; for large errors it levels off. For small errors its best forecast is close to the median, but not in general the median.

For y>0y > 0 and f>0f > 0, the sMAPE term is s=tanh⁡(|ln⁡y−ln⁡f|/2)s = \tanh(|\ln y - \ln f|/2) (log error ln⁡y−ln⁡f\ln y - \ln f).

  1. (a)It is 0 when y=fy = f, strictly increasing and concave in |ln⁡y−ln⁡f||\ln y - \ln f|, and bounded by 1, which it approaches as |ln⁡y−ln⁡f||\ln y - \ln f| grows.
  2. (b)0≤12|ln⁡y−ln⁡f|−s≤124|ln⁡y−ln⁡f|30 \le \tfrac12|\ln y - \ln f| - s \le \tfrac1{24}|\ln y - \ln f|^3; so when all log errors are small, sMAPE is about 100 times the mean absolute error of ln⁡f\ln f as a forecast of ln⁡y\ln y (log-MAE).
  3. (c)The log-MAE of a test set is minimised by its median; sMAPE is then minimised near the median, not in general at it. For positive actuals y1,…,yny_1, \dots, y_n with a median mm, a forecast f*f^* that minimises their sMAPE satisfies 1n∑t|ln⁡yt−ln⁡f*|−1n∑t|ln⁡yt−ln⁡m|≤112⋅1n∑t|ln⁡yt−ln⁡f*|3:\frac1n\sum_t|\ln y_t - \ln f^*| - \frac1n\sum_t|\ln y_t - \ln m| \le \frac1{12}\cdot\frac1n\sum_t|\ln y_t - \ln f^*|^3: its mean absolute log error exceeds the smallest possible by at most a twelfth of its mean cubed absolute log error. The minimiser is the median itself only under further conditions on the distribution of the actuals.
Proof 9 steps
  1. Write z=ln⁡y−ln⁡fz = \ln y - \ln f, so that r=y/f=ezr = y/f = e^z. By claim C1, s=|ez−1|ez+1s = \dfrac{|e^z - 1|}{e^z + 1}.
  2. Multiplying the numerator and the denominator of ez−1ez+1\dfrac{e^z - 1}{e^z + 1} by e−z/2e^{-z/2} gives ez/2−e−z/2ez/2+e−z/2=tanh⁡(z/2)\dfrac{e^{z/2} - e^{-z/2}}{e^{z/2} + e^{-z/2}} = \tanh(z/2). The denominator is positive, so s=|tanh⁡(z/2)|s = |\tanh(z/2)|.
  3. tanh⁡\tanh changes only its sign when its argument changes sign: tanh⁡(−x)=−tanh⁡(x)\tanh(-x) = -\tanh(x), as the definition shows. So |tanh⁡(z/2)|=tanh⁡(|z|/2)|\tanh(z/2)| = \tanh(|z|/2), which is the stated form.
  4. For (a): tanh⁡\tanh is 0 at 0. On [0,∞)[0, \infty) it is strictly increasing and concave, and it approaches 1 as its argument grows; its derivative 1−tanh2⁡x1 - \tanh^2 x is positive and decreasing for x>0x > 0. With the argument |ln⁡y−ln⁡f|/2|\ln y - \ln f|/2, ss has these properties in |ln⁡y−ln⁡f||\ln y - \ln f|.
  5. For (b): for every x≥0x \ge 0, x−x3/3≤tanh⁡x≤xx - x^3/3 \le \tanh x \le x. The upper bound holds because tanh⁡\tanh has slope at most 1; the lower one because the derivative of tanh⁡x−x+x3/3\tanh x - x + x^3/3 is x2−tanh2⁡x≥0x^2 - \tanh^2 x \ge 0, and the difference is 0 at x=0x = 0. Putting x=|ln⁡y−ln⁡f|/2x = |\ln y - \ln f|/2 gives the bounds.
  6. For (c), the median: the mean of |ln⁡yt−ln⁡f||\ln y_t - \ln f| over a test set is minimised when ln⁡f\ln f is a median of the ln⁡yt\ln y_t. Since the logarithm is increasing, this is when ff is a median mm of the yty_t.
  7. For (c), the bound: with zt=ln⁡yt−ln⁡fz_t = \ln y_t - \ln f the log errors of the test set, write |z|‾(f)=1n∑t|ln⁡yt−ln⁡f|\overline{|z|}(f) = \frac1n\sum_t|\ln y_t - \ln f|, |z|3‾(f)=1n∑t|ln⁡yt−ln⁡f|3\overline{|z|^3}(f) = \frac1n\sum_t|\ln y_t - \ln f|^3 and s¯(f)\bar s(f) for the mean of the sMAPE terms of the test set. Averaging the bounds of step 5 over the test set gives s¯(f)≤12|z|‾(f)≤s¯(f)+124|z|3‾(f)\bar s(f) \le \tfrac12\overline{|z|}(f) \le \bar s(f) + \tfrac1{24}\overline{|z|^3}(f) for every f>0f > 0. Let f*f^* minimise s¯\bar s, so that s¯(f*)≤s¯(m)\bar s(f^*) \le \bar s(m). Then 12|z|‾(f*)≤s¯(f*)+124|z|3‾(f*)≤s¯(m)+124|z|3‾(f*)≤12|z|‾(m)+124|z|3‾(f*).\tfrac12\overline{|z|}(f^*) \le \bar s(f^*) + \tfrac1{24}\overline{|z|^3}(f^*) \le \bar s(m) + \tfrac1{24}\overline{|z|^3}(f^*) \le \tfrac12\overline{|z|}(m) + \tfrac1{24}\overline{|z|^3}(f^*). Multiplying by 2 gives |z|‾(f*)−|z|‾(m)≤112|z|3‾(f*)\overline{|z|}(f^*) - \overline{|z|}(m) \le \tfrac1{12}\overline{|z|^3}(f^*), the stated inequality.
  8. For (c), not the median in general: take the five actuals 1,1,e,e4,e41, 1, e, e^4, e^4, whose median is ee. With the forecast at ee the log errors are 1,1,0,3,31, 1, 0, 3, 3 in size and the sMAPE terms sum to 2tanh⁡(0.5)+2tanh⁡(1.5)=2.7342\tanh(0.5) + 2\tanh(1.5) = 2.734. With the forecast at 1 the log errors are 0,0,1,4,40, 0, 1, 4, 4 and the terms sum to tanh⁡(0.5)+2tanh⁡(2)=2.390\tanh(0.5) + 2\tanh(2) = 2.390, which is lower. So the median does not minimise the sMAPE of this test set, although it minimises its log-MAE (step 6).

So s=tanh⁡(|ln⁡y−ln⁡f|/2)s = \tanh(|\ln y - \ln f|/2). It rises from 0 like half the absolute log error and levels off at 1. For small errors sMAPE is close to the log-MAE, and its minimiser is close to the median without in general being the median. ∎

Its formulae were also checked on numbers: E31 in the record.

Earlier work and other claims

  • Extends Tofallis (2015). Tofallis (2015) relates sMAPE to the log accuracy ratio among relative measures; claim C3 shows the sMAPE term is exactly the hyperbolic tangent of half the absolute log ratio.
Proved · checked independently by an AI agent · confirmed by Ivan Svetunkov · checked independently by an AI agent · tested as plannedRecord
Claim
C3 · formal · principal · extends earlier work
Plan
Test planned in lab plan PL12, recorded 18 November 2026
Evidence
E20, E31 in the record
Check: confirmed
By an AI agent (proof checking), full. Derived from the statement alone (earlier wording: old H3 and H9), then compared with the final proofs.md (paper H3), which agrees; the bound in (b) follows from x - x^3/3 <= tanh x <= x. Concavity is in the absolute log error on [0, inf); 'minimised by the median' is the small-error approximation, as stated.
Check: confirmed
By Ivan Svetunkov, full. The supervisor read the proof of paper H3 in proofs.md (AR1.R2; rendered as AR2.R2) and confirmed it: first "Confirmed" (DC6), then, after the last corrections of paper H4, H5 and H9, "All looks fine now. NExt".
Check: confirmed
By an AI agent (proof checking, version 3), full. Derived independently. With $z=\ln y-\ln f$, $s=|e^z-1|/(e^z+1)=\tanh(|z|/2)$. (a) $\tanh(u/2)$ on $u\ge0$ is 0 at 0, strictly increasing, concave (derivative $\tfrac12\operatorname{sech}^2(u/2)$ positive and decreasing), below 1 with limit 1. (b) $x-x^3/3\le\tanh x\le x$ for $x\ge0$ (the lower bound: $\frac{d}{dx}(\tanh x-x+x^3/3)=x^2-\tanh^2x\ge0$); with $x=|z|/2$ this gives $0\le|z|/2-s\le|z|^3/24$, so $200\cdot\mathrm{mean}(s)\approx100\cdot\mathrm{mean}|z|$. (c) Averaging (b): $\bar s\le a/2\le\bar s+k/24$; with $\bar s(f^*)\le\bar s(m)$, $a(f^*)/2\le\bar s(f^*)+k(f^*)/24\le\bar s(m)+k(f^*)/24\le a(m)/2+k(f^*)/24$, which is the stated bound; the log-MAE is minimised at any median of the $\ln y_t$, i.e. at any median $m$. The proof (proofs.md H3) does exactly this; its only unproved step is the elementary inequality $x-x^3/3\le\tanh x$, closed by the derivative argument above. I recomputed the counterexample of step 8 (actuals $1,1,e,e^4,e^4$: sums 2.734 at the median $e$ against 2.390 at 1), which supports "not in general at it". Every part holds as worded.
Support score
SupportS 100 of 100. The score checks the record, not the science.
Record
This claim on the record page
To cite it
The page of this claim

What sMAPE rewards for positive actuals

Term this claim uses

expected sMAPE
The expected sMAPE of a fixed forecast ff under the distribution of the actual yy: 200E[s(y,f)]200\,\mathrm{E}[s(y, f)], where E\mathrm E is the expectation over the random actual yy. Computed from the distribution (by integration or exact sums), never from a sample.

Claim C9. Expected sMAPE is always finite and has a minimiser proved

The expected sMAPE of a forecast is always finite, even when the actuals have a heavy tail that makes the expected RMSE, MAE or MAPE infinite. And there is always a best forecast.

For actuals and forecasts of any sign, 0≤s≤10 \le s \le 1. So the expected sMAPE of every forecast is finite, between 0 and 200, whatever the distribution of the actuals and without any moment of it being finite (unlike RMSE, MAE and MAPE, claim C8). For a positive random actual yy, at least one forecast has the lowest expected sMAPE.

Proof 3 steps
  1. Finite. For positive values, s=tanh⁡(|ln⁡y−ln⁡f|/2)s = \tanh(|\ln y - \ln f|/2) (claim C3), which lies between 0 and 1. For any signs, |y−f|≤|y|+|f||y - f| \le |y| + |f|, so again s≤1s \le 1. A quantity between 0 and 1 has an expectation between 0 and 1, so the expected sMAPE is between 0 and 200, whatever the actuals.
  2. A best forecast. Write E[s(y,f)]\mathrm E[s(y,f)] for the expectation over the random actual yy. As ff approaches 0 or grows without bound, |ln⁡y−ln⁡f||\ln y - \ln f| grows for every positive value of yy, so ss approaches 1 (claim C3), and so does its expectation. For every positive ff it is below 1. It changes continuously with ff, because ss does and is at most 1 (bounded convergence). A continuous function that is below 1 in between and returns towards 1 at both ends reaches a lowest value at some f>0f > 0. A forecast of 0 or a negative forecast scores 1 on every positive actual, so it is never better.

So the expected sMAPE is always finite, whatever the tails of the actuals. For positive actuals, a forecast with the lowest expected sMAPE always exists. ∎

Its formulae were also checked on numbers: E1 in the record.

Earlier work and other claims

  • Extends Hewamalage et al. (2023). Hewamalage et al. (2023) note that sMAPE is bounded by 200; claim C9 adds that its expected value is therefore finite for every distribution and that a best forecast exists.
Proved · checked independently by an AI agent · confirmed by Ivan Svetunkov · tested as plannedRecord
Claim
C9 · formal · principal · extends earlier work
Plan
Test planned in lab plan PL9, recorded 17 November 2026
Evidence
E21, E1 in the record
Check: confirmed
By an AI agent (proof checking), full. Derived from the statement alone (triangle inequality; bounded convergence gives continuity in f and the limit 1 at both ends, below 1 between), then compared with the final proofs.md (paper H4), which agrees. No condition on the distribution is needed.
Check: confirmed
By Ivan Svetunkov, full. The supervisor read the proof of paper H4 in proofs.md (AR1.R2; rendered as AR2.R2) and confirmed it: first "Confirmed" (DC6), then, after the last corrections of paper H4, H5 and H9, "All looks fine now. NExt".
Support score
SupportS 100 of 100. The score checks the record, not the science.
Record
This claim on the record page
To cite it
The page of this claim

Terms this claim uses

weight w(y,f)w(y,f)
For a positive actual yy and a positive forecast ff, w(y,f)=y/(y+f)2w(y,f) = y/(y+f)^2: how much the actual pulls the forecast that minimises expected sMAPE, since the derivative of E[s(y,f)]\mathrm E[s(y,f)] in ff is 2E[w(y,f)(𝟏{y<f}−𝟏{y>f})]2\,\mathrm E[w(y,f)(\mathbf 1\{y<f\} - \mathbf 1\{y>f\})]. Largest at y=fy = f, the same for y/f=ry/f = r and 1/r1/r.
sMAPE target
The value or values of ff that minimise expected sMAPE for a given distribution of the actual yy, arg⁡minf⁡E[s(y,f)]\arg\min_f \mathrm E[s(y,f)]: the functional sMAPE rewards. It may be a single value or several.

Claim C4. sMAPE targets a weighted median key claimproved

For any distribution, the forecast sMAPE rewards balances the actuals below it and above it. Each actual is weighted by how close it is to the forecast by ratio.

Let the actual yy be positive and take no single value with positive probability. A forecast ff has the lowest expected sMAPE only if the actuals below ff and those above it carry equal total weight w(y,f)=y/(y+f)2w(y,f) = y/(y+f)^2: E[w(y,f)𝟏{y<f}]=E[w(y,f)𝟏{y>f}].\mathrm E[w(y,f)\,\mathbf 1\{y<f\}] = \mathrm E[w(y,f)\,\mathbf 1\{y>f\}]. So the sMAPE target is a median of the actuals weighted by ww, where the ordinary median gives every actual the same weight. An actual's weight is largest when it equals the forecast, is the same for y/f=ry/f = r and 1/r1/r, and falls as y/fy/f moves away from 1 in either direction.

Proof 6 steps
  1. The derivative of ss. When y>f>0y > f > 0, s=(y−f)/(y+f)s = (y - f)/(y + f). By the quotient rule its derivative in ff is −(y+f)−(y−f)(y+f)2=−2w(y,f)\dfrac{-(y+f) - (y-f)}{(y+f)^2} = -2w(y,f). When f>y>0f > y > 0, s=(f−y)/(y+f)s = (f - y)/(y + f), and in the same way its derivative is 2w(y,f)2w(y,f). Together: for y≠fy \ne f, the derivative is 2w(y,f)(𝟏{y<f}−𝟏{y>f})2w(y,f)(\mathbf 1\{y < f\} - \mathbf 1\{y > f\}).
  2. A bound on it. Since (y+f)2≥4yf(y+f)^2 \ge 4yf, w(y,f)≤1/(4f)w(y,f) \le 1/(4f). So the derivative of ss is at most 1/(2f)1/(2f) in size, whatever yy, and at most 1/f1/f for every forecast above f/2f/2. The only actual at which ss has no derivative is y=fy = f, which has probability zero.
  3. The derivative of the expectation. With this bound, the derivative may be taken inside the expectation (dominated convergence): ddfE[s(y,f)]=2E[w(y,f)(𝟏{y<f}−𝟏{y>f})].\frac{d}{df}\,\mathrm E[s(y,f)] = 2\,\mathrm E\big[w(y,f)\,(\mathbf 1\{y < f\} - \mathbf 1\{y > f\})\big].
  4. The balance. A forecast with the lowest expected sMAPE exists (claim C9). There this derivative is 0, which is the stated balance: half the total weight lies below ff and half above.
  5. The weight. With r=y/fr = y/f, w(y,f)=1f⋅r(1+r)2=1f⋅1r+2+1/rw(y,f) = \dfrac1f\cdot\dfrac{r}{(1+r)^2} = \dfrac1f\cdot\dfrac{1}{r + 2 + 1/r}. The sum r+1/rr + 1/r is the same for rr and 1/r1/r. It is smallest, 2, at r=1r = 1, and grows as rr moves away from 1. So ww is largest at y=fy = f and falls as y/fy/f moves away from 1, equally for rr and 1/r1/r.

So sMAPE targets a weighted median of the actuals. Each actual is weighted by how close it is to the forecast by ratio. ∎

Its formulae were also checked on numbers: E31 in the record.

Supported by numerical check C13

Computed on 34 distributions, the target lies on the side of the median that carries more weight, as this claim and claim C12 prove.

Earlier work and other claims

  • Explains Svetunkov (2026b), C5. C5 found sMAPE's preferred candidate close to the median on other distributions; claim C4 shows why: the target is a weighted median, on the side of the median that carries more weight (claim C12).
  • Answers Tofallis (2015). Tofallis (2015) states that minimising sMAPE does not produce an established measure of location; claim C4 identifies the target as a median weighted by y/(y+f)^2.
  • Qualifies Hewamalage et al. (2023). Hewamalage et al. (2023) say sMAPE may tend towards selecting a slightly overestimating model; claim C4 shows the target is a weighted median, which lies on the side of the median that carries more weight, above or below it (claim C12).
  • Qualifies claim C2. Claim C2 compares two forecasts of one known actual. For an uncertain actual, the forecast sMAPE rewards is the weighted median of claim C4, not a forecast that is too high.
Proved · checked independently by an AI agent · confirmed by Ivan Svetunkov · checked independently by an AI agent · tested as plannedRecord
Claim
C4 · formal · principal · no earlier result found
Plan
Test planned in lab plan PL12, recorded 18 November 2026
Evidence
E22, E31 in the record
Check: disputed
By an AI agent (proof checking), full. Derived from the statement alone, then compared with the final proofs.md (paper H5). The balance, the weighted median, the properties of the weight and the 'above'/'below' cases hold. The last clause fails: equal weights at the median make it a stationary point, not necessarily the minimum, and proof step 7 does not show it. Counterexample (numerical): ln Y ~ 0.41408 N(-1.48848, 0.2^2) + 0.03 N(0, 0.3^2) + 0.55592 N(1.13795, 1) has median 1 and equal weights there, a flat inflection at the median, and a single local minimum at ln f = -1.36. Fix: drop 'at the median when the two weights are equal', or add 'if the median is a local minimum'. The statement was corrected, and the checker confirmed the corrected wording in a later check listed here.
Check: confirmed
By an AI agent (proof checking), full. Checked the corrected wording against my own derivation and the revised proofs.md (paper H5, new step 8). The clause disputed in CK5 now says that with equal weights the median is a flat point, the target only if it is a local minimum, and possibly a flat inflection; that is what holds, and my counterexample (single local minimum at ln f = -1.36, median at 1) is an instance. Needs Y positive with no atoms, as stated.
Check: confirmed
By an AI agent (proof checking), full. Checked the added sufficient condition against my own derivation and the revised proofs.md (paper H5, steps 8 to 10). The second derivative of the expected term in ln f is g(c) - E[sech^2 tanh]/2, as I derived independently; zero first derivative plus the stated strict inequality gives a strict local minimum at the median, which is the target when there is a single local minimum. The clause needs ln Y to have a density continuous at ln m, as stated; my counterexample sits at equality, consistent with it.
Check: confirmed
By Ivan Svetunkov, full. The supervisor read the proof of paper H5 in proofs.md (AR1.R2; rendered as AR2.R2) and confirmed it: first "Confirmed" (DC6), then, after the last corrections of paper H4, H5 and H9, "All looks fine now. NExt".
Check: confirmed
By an AI agent (proof checking, version 3), full. Derived independently. For $y\ne f$, $\partial_f s(y,f)=2w(y,f)(\mathbf 1\{y<f\}-\mathbf 1\{y>f\})$ with $w=y/(y+f)^2\le1/(4f)$; since $P(Y=f)=0$ and the difference quotients are bounded by $1/(2(f-\epsilon))$ near $f$, dominated convergence gives $\frac{d}{df}\mathrm E[s(Y,f)]=2\mathrm E[w(\mathbf 1\{Y<f\}-\mathbf 1\{Y>f\})]$ at every $f>0$, so a minimiser (necessarily $f>0$: $f\le0$ scores 1 on every positive actual while $\mathrm E[s]<1$ for $f>0$) satisfies the stated balance. With $r=y/f$, $w=\frac1f\,r/(1+r)^2=\frac1f/(r+2+1/r)$: symmetric under $r\to1/r$, largest at $r=1$, decreasing as $r$ moves away from 1 on either side. The proof (proofs.md H5) matches step by step; the local uniformity of the dominating bound in $f$ is left implicit but is immediate as above. Its use of H4 (existence) is not needed for the "only if" statement but is harmless. Every part holds as worded.
Support score
SupportS 100 of 100. The score checks the record, not the science.
Record
This claim on the record page
To cite it
The page of this claim

Claim C5. Symmetric, unimodal logs: the target is the median proved

When the logarithm of the actual is symmetric with a single peak, the weights balance at the median, and the median is the forecast sMAPE rewards. Lognormal and log-Laplace actuals are examples.

Let ln⁡y\ln y have a density gg symmetric about ln⁡m\ln m, g(ln⁡m+v)=g(ln⁡m−v)g(\ln m+v) = g(\ln m-v), and unimodal: g(ln⁡m+v1)≥g(ln⁡m+v2)g(\ln m+v_1) \ge g(\ln m+v_2) whenever |v1|≤|v2||v_1| \le |v_2|, strictly on a set of positive probability. Then the median mm of yy is the only sMAPE target.

Proof 6 steps
  1. By claim C3, E[s(y,f)]=E[tanh⁡(|ln⁡y−ln⁡f|/2)]\mathrm E[s(y,f)] = \mathrm E\big[\tanh\big(|\ln y - \ln f|/2\big)\big]. Take a forecast f=meτf = m\,e^{\tau} with τ>0\tau > 0; the case τ<0\tau < 0 is the mirror image. We show that its expected sMAPE is larger than that of mm.
  2. Write both expectations as integrals over vv, the distance of ln⁡y\ln y from ln⁡f\ln f. For the forecast meτm\,e^{\tau}, ln⁡y=ln⁡m+τ+v\ln y = \ln m + \tau + v; for mm, ln⁡y=ln⁡m+v\ln y = \ln m + v. Their difference is Δ=∫tanh⁡(|v|/2)[g(ln⁡m+τ+v)−g(ln⁡m+v)]dv.\Delta = \int \tanh(|v|/2)\,\big[g(\ln m+\tau+v) - g(\ln m+v)\big]\,dv.
  3. Write ϕ(v)=g(ln⁡m+τ+v)−g(ln⁡m+v)\phi(v) = g(\ln m+\tau+v) - g(\ln m+v). By the symmetry of gg about ln⁡m\ln m, ϕ(−τ−v)=g(ln⁡m−v)−g(ln⁡m−τ−v)=g(ln⁡m+v)−g(ln⁡m+τ+v)=−ϕ(v)\phi(-\tau-v) = g(\ln m-v) - g(\ln m-\tau-v) = g(\ln m+v) - g(\ln m+\tau+v) = -\phi(v). So reflecting vv about −τ/2-\tau/2 reverses the sign of ϕ\phi.
  4. Split the integral at −τ/2-\tau/2 and reflect its left part onto the right. This gives Δ=∫v>−τ/2[tanh⁡(|v|/2)−tanh⁡(|v+τ|/2)]ϕ(v)dv.\Delta = \int_{v > -\tau/2} \big[\tanh(|v|/2) - \tanh(|v+\tau|/2)\big]\,\phi(v)\,dv.
  5. For v>−τ/2v > -\tau/2, vv is closer to 0 than v+τv + \tau is: |v|<|v+τ||v| < |v + \tau|. So the bracket is negative, because tanh⁡\tanh is increasing (claim C3). And ϕ(v)≤0\phi(v) \le 0 by unimodality: g(ln⁡m+τ+v)≤g(ln⁡m+v)g(\ln m+\tau+v) \le g(\ln m+v). Their product is ≥0\ge 0, and >0> 0 on a set of positive probability. So Δ>0\Delta > 0.

So every forecast other than the median has a larger expected sMAPE. The median is the only forecast that minimises it. ∎

Its formulae were also checked on numbers: E1 in the record.

Supported by numerical check C13

Computed on lognormal outcomes, the target is the median, as this claim proves.

Earlier work and other claims

  • Repeats Anderson (1955). Claim C5 is an elementary case of Anderson's theorem on symmetric unimodal densities, proved here directly for the expected sMAPE.
Proved · checked independently by an AI agent · confirmed by Ivan Svetunkov · tested as plannedRecord
Claim
C5 · formal · supporting · repeats an earlier result
Plan
Test planned in lab plan PL9, recorded 17 November 2026
Evidence
E23, E1 in the record
Check: confirmed
By an AI agent (proof checking), full. Derived from the statement alone (sign of the derivative of expected sMAPE in ln f), then compared with the final proofs.md (paper H6), which reaches it by reflecting about -t/2; both hold under the unimodality as now defined in the statement.
Check: confirmed
By Ivan Svetunkov, full. The supervisor read the proof of paper H6 in proofs.md (AR1.R2; rendered as AR2.R2) and confirmed it: first "Confirmed" (DC6), then, after the last corrections of paper H4, H5 and H9, "All looks fine now. NExt".
Support score
SupportS 100 of 100. The score checks the record, not the science.
Record
This claim on the record page

Claim C12. When sMAPE targets the ordinary median key claimproved

The forecast sMAPE rewards lies above the ordinary median when the actuals above it carry more weight, and below it when those below do. When the weights balance and enough actuals lie near the median, the median is rewarded.

Let the actual yy be positive and take no single value with positive probability, let mm be its ordinary (unweighted) median, and let its expected sMAPE have a single local minimum. With the forecast at mm, compare the total weight w(y,m)=y/(y+m)2w(y,m) = y/(y+m)^2 of the actuals above mm with that of the actuals below it.

  1. (a)If the actuals above mm carry more weight, the sMAPE target lies above mm; if those below carry more, it lies below mm.
  2. (b)If the two weights are equal, mm is a point where expected sMAPE is flat. It is then the target if, in addition, 2g(ln⁡m)>E[sech2⁡(ln⁡y−ln⁡m2)tanh⁡(|ln⁡y−ln⁡m|2)],2g(\ln m) > \mathrm E\Big[\operatorname{sech}^2\Big(\frac{\ln y - \ln m}{2}\Big)\tanh\Big(\frac{|\ln y - \ln m|}{2}\Big)\Big], where gg is the density of ln⁡y\ln y, continuous at ln⁡m\ln m: this makes mm a strict local minimum. Without it, mm can be a flat point of inflection, with the target elsewhere.
  3. (c)When ln⁡y\ln y has a density symmetric about ln⁡m\ln m and unimodal, mm is the only target; lognormal and log-Laplace actuals are examples.
Proof 8 steps
  1. The derivative at the median. By claim C4, step 3, the derivative of the expected sMAPE term in ff is 2E[w(y,f)(𝟏{y<f}−𝟏{y>f})]2\,\mathrm E[w(y,f)(\mathbf 1\{y < f\} - \mathbf 1\{y > f\})]. At f=mf = m it is twice the total weight of the actuals below mm minus that of the actuals above mm.
  2. The shape of expected sMAPE. With a single local minimum, the expected sMAPE decreases up to it and increases after it. Otherwise it would turn again, at a local maximum. After that turn it would have to fall and then rise once more, because it returns towards 1 at both ends (claim C9). That would give a second local minimum.
  3. (a). If the actuals above mm carry more weight, the derivative at the median is negative (step 1): the expected sMAPE is decreasing at the median. By step 2, the median then lies before the minimum, so the target is above the median. The case where the actuals below carry more weight is the mirror image.
  4. The second derivative. Measure the forecast on the log scale, c=ln⁡fc = \ln f, so that the expected contribution is E[tanh⁡(|ln⁡y−c|/2)]\mathrm E[\tanh(|\ln y - c|/2)] (claim C3). The derivative of tanh⁡(|x−c|/2)\tanh(|x - c|/2) in cc is −12sech2⁡(x−c2)-\tfrac12\operatorname{sech}^2\big(\tfrac{x-c}{2}\big) for c<xc < x and +12sech2⁡(x−c2)+\tfrac12\operatorname{sech}^2\big(\tfrac{x-c}{2}\big) for c>xc > x: it jumps by 1 at c=xc = x. Differentiating again and taking the expectation over x=ln⁡yx = \ln y, the jumps add the density gg of ln⁡y\ln y at cc, and elsewhere the second derivative is −12sech2⁡(x−c2)tanh⁡(|x−c|2)-\tfrac12\operatorname{sech}^2\big(\tfrac{x-c}{2}\big)\tanh\big(\tfrac{|x-c|}{2}\big). So the second derivative of the expected contribution is g(c)−12E[sech2⁡(ln⁡y−c2)tanh⁡(|ln⁡y−c|2)].g(c) - \tfrac12\,\mathrm E\Big[\operatorname{sech}^2\Big(\frac{\ln y - c}{2}\Big)\tanh\Big(\frac{|\ln y - c|}{2}\Big)\Big]. This holds when gg is continuous at cc: since sech2⁡=1−tanh2⁡\operatorname{sech}^2 = 1 - \tanh^2, the first derivative is P(ln⁡y≤c)−12\mathrm P(\ln y \le c) - \tfrac12, whose derivative at cc is g(c)g(c), plus the expectation of −12tanh2⁡(x−c2)-\tfrac12\tanh^2\big(\tfrac{x-c}{2}\big) with the sign of c−xc - x, a function of cc with a bounded continuous derivative, which may be differentiated inside the expectation.
  5. (b), the sufficient condition. If the two weights are equal, the first derivative at the median is 0 (step 1): the expected sMAPE is flat there. If, in addition, the stated inequality holds, the second derivative at c=ln⁡mc = \ln m is positive (step 4), so the median is a strict local minimum. With a single local minimum, it is the target.
  6. (b), without the inequality. A point where the first derivative is 0 can be a flat point of inflection, through which the expected sMAPE keeps rising (or keeps falling) without turning. An example, built by the independent checker of this section: let ln⁡y−ln⁡m\ln y - \ln m be a mixture of three normal distributions, with mean −1.5-1.5 and standard deviation 0.30.3 (weight 0.406125), mean 0 and standard deviation 0.50.5 (weight 0.054825), and mean 1.1586641.158664 and standard deviation 1 (weight 0.539050); these values are rounded from the exact solution of the three conditions. Its median is mm, the two weights are equal there, and the second derivative of step 4 is 0 at ln⁡m\ln m; the expected sMAPE rises through mm, where it is flat, and its only minimum is near ln⁡f=ln⁡m−1.25\ln f = \ln m - 1.25. So equal weights alone do not make the median the target.
  7. (c). By claim C5 the median is then the only forecast with the lowest expected sMAPE (a density cannot be constant, so a symmetric unimodal density is strictly lower somewhere away from its centre, as claim C5 asks). For lognormal actuals ln⁡y\ln y has a normal density, and for log-Laplace actuals the Laplace density e−|x−ln⁡m|/b/(2b)e^{-|x - \ln m|/b}/(2b); both are symmetric about ln⁡m\ln m and strictly decreasing in the distance from it.

So the target is above the ordinary median when the actuals above it carry more of the weight, and below it when those below carry more. When the two balance, the median is the target if, in addition, enough actuals lie near it, as the stated inequality says; this is the case when the log of the actual is symmetric with a single peak. ∎

Its formulae were also checked on numbers: E31 in the record.

Qualified by claim C15

Where this claim makes the median the target, the forecast that minimises sMAPE on a test set finds it less precisely than the sample median.

Earlier work and other claims

  • Explains Svetunkov (2026b), C5. C5 found by simulation that sMAPE prefers the median on lognormal and log-Laplace outcomes; claim C12 proves it for every symmetric unimodal ln y.
Proved · checked independently by an AI agent · checked independently by an AI agent · tested as plannedRecord
Claim
C12 · formal · principal · no earlier result found
Plan
Test planned in lab plan PL12, recorded 18 November 2026
Evidence
E29, E31 in the record
Check: confirmed
By an AI agent (proof checking), full. Checked against my own derivations of H5 and H4 and the proofs in proofs.md (paper H6; paper H5 steps 3, 6, 8, 9). (a) holds even with unimodality in the weak sense, since an integrable density cannot be symmetric about two points; (b) holds because a zero first derivative and a positive second derivative in ln f make the median a strict local minimum, which is the target when there is a single local minimum. Caveat: (b) is 'more general' than (a) only where the density in (a) is continuous at ln m.
Check: confirmed
By an AI agent (proof checking, version 3), full. Derived independently, on the log scale $\phi(t)=\mathrm E\tanh(|\ln Y-t|/2)$. (a) $\phi'(\ln m)$ has the sign of (weight below) minus (weight above). With a single local minimum, $\phi$ is decreasing before it and increasing after it: if it were not monotone on one side, a continuous function that is below 1 and tends to 1 at both ends would have an interior local maximum and then a second local minimum. So a negative derivative at $m$ puts the target above $m$, a positive one below. The proof's step 2 says this tersely but correctly. (b) Writing the kernel $-\operatorname{sign}(v)\operatorname{sech}^2(v/2)=-\operatorname{sign}(v)+\operatorname{sign}(v)\tanh^2(v/2)$, the first part gives $\tfrac12(2G(t)-1)$, which is differentiable at $t$ with derivative $g(t)$ when $g$ is continuous at $t$, and the second has a bounded continuous derivative. So $\phi''(\ln m)=g(\ln m)-\tfrac12\mathrm E[\operatorname{sech}^2(x/2)\tanh(|x|/2)]$ exists under continuity of $g$ at $\ln m$ alone. With $\phi'(\ln m)=0$ and $\phi''(\ln m)>0$, $m$ is a strict local minimum, hence the target. This agrees with H12 step 4, which interchanges derivative and expectation without justification; the decomposition above closes that gap. The existence claim "Without it, $m$ can be a flat point of inflection, with the target elsewhere" is supported in the proof only by a reference to an example built by an earlier checker that is not given. I built one myself: $\ln Y-\ln m\sim0.4061\,N(-1.5,0.3^2)+0.0548\,N(0,0.5^2)+0.5390\,N(1.1587,1)$ has median 0, $\phi'(0)=\phi''(0)=0$, $\phi'\ge0$ on both sides of 0 (a flat inflection) and a single local minimum near $t=-1.25$ (numerical grid). Including such an explicit example in the proof would make step 6 self-contained. (Under the single-minimum hypothesis a failing inequality can only mean equality, $\phi''=0$; a strict local maximum is impossible, so "flat point of inflection" is the right description.) (c) holds via H6. Note that H6 assumes unimodality "strictly on a set of positive probability", which H38 (c) does not state; the conclusion holds for every symmetric, weakly unimodal density anyway: $P(|X-t|\le c)<P(|X|\le c)$ for $t\ne0$ on a set of $c$ of positive measure (otherwise $g$ would be constant on $(0,\infty)$), and $\tanh$ is strictly increasing, so no correction to the statement is needed. Lognormal and log-Laplace qualify. Every part holds as worded.
Support score
SupportS 100 of 100. The score checks the record, not the science.
Record
This claim on the record page
To cite it
The page of this claim

Claim C6. Two separate groups: the median is a local maximum key claimproved

When the actuals form two separate groups, the median falls in the gap between them, far from both. So sMAPE scores it worse than forecasts near either group.

Let ln⁡y\ln y have two modes, at ln⁡m−a\ln m - a and ln⁡m+a\ln m + a (a>0a > 0), with the actuals concentrated near them: a mixture of a distribution near ln⁡m−a\ln m - a and its mirror image about ln⁡m\ln m, with weights pp and 1−p1 - p.

  1. (a)With equal weights the median mm lies in the gap between the two groups; when the groups are narrow enough it has a higher expected sMAPE than forecasts near either mode, and it is a local maximum of expected sMAPE; there is a minimiser on each side of the median, and these approach me−am\,e^{-a} and meam\,e^{a} as the groups narrow.
  2. (b)With unequal weights and narrow groups, the best forecast is near the mode of the larger group.
Proof 11 steps
  1. Write forecasts on the log scale as ln⁡f=ln⁡m+c\ln f = \ln m + c. By claim C3, the expected contribution is E[tanh⁡(|ln⁡y−ln⁡m−c|/2)]\mathrm E[\tanh(|\ln y - \ln m - c|/2)], a function of cc.
  2. Why the median is bad. Take the groups to be single points first. A forecast at the median is aa away, on the log scale, from every actual. So its expected ss is tanh⁡(a/2)\tanh(a/2).
  3. A forecast at the mode, ln⁡f=ln⁡m+a\ln f = \ln m + a, is exact for half the actuals and 2a2a away from the other half. So its expected ss is 12tanh⁡(a)\tfrac12\tanh(a). Since tanh⁡(a)=2tanh⁡(a/2)1+tanh2⁡(a/2)\tanh(a) = \dfrac{2\tanh(a/2)}{1 + \tanh^2(a/2)}, this is tanh⁡(a/2)1+tanh2⁡(a/2)<tanh⁡(a/2)\dfrac{\tanh(a/2)}{1 + \tanh^2(a/2)} < \tanh(a/2). The mode is better.
  4. The expected ss changes continuously as the groups widen from points, because ss is bounded and continuous. So the mode is still better for narrow groups.
  5. Why the median is a local maximum. By claim C12, step 4, the second derivative of the expected contribution at the median is g(ln⁡m)−12E[sech2⁡(ln⁡y−ln⁡m2)tanh⁡(|ln⁡y−ln⁡m|2)].g(\ln m) - \tfrac12\,\mathrm E\Big[\operatorname{sech}^2\Big(\frac{\ln y - \ln m}{2}\Big)\tanh\Big(\frac{|\ln y - \ln m|}{2}\Big)\Big].
  6. By symmetry the first derivative at the median is 0. When the groups narrow, few actuals lie near the median, so g(ln⁡m)g(\ln m) tends to 0. Meanwhile the expectation tends to sech2⁡(a/2)tanh⁡(a/2)>0\operatorname{sech}^2(a/2)\tanh(a/2) > 0. So the second derivative becomes negative, and the median is a local maximum.
  7. A minimiser on each side. The expected sMAPE has a lowest point (claim C9). It is not at the local maximum c=0c = 0. By symmetry, if c*c^* is a minimiser, so is −c*-c^*.
  8. Near the modes. Write tanh⁡(|z|/2)=1−2λ(z)\tanh(|z|/2) = 1 - 2\lambda(z), with λ(z)=1/(1+e|z|)\lambda(z) = 1/(1 + e^{|z|}). The function λ\lambda is largest at 0 and falls as |z||z| grows, more slowly the larger |z||z| is. Minimising the expected ss is maximising E[λ(ln⁡y−ln⁡m−c)]\mathrm E[\lambda(\ln y - \ln m - c)]. As the groups narrow, this approaches pλ(c+a)+(1−p)λ(c−a)p\,\lambda(c + a) + (1 - p)\,\lambda(c - a), uniformly in cc.
  9. With equal weights, take 0<c<a0 < c < a. The derivative of this limit is half the rate at which λ\lambda falls at distance a−ca - c, minus half the rate at distance a+ca + c. It is positive, because λ\lambda falls faster nearer 0. For c>ac > a the limit decreases. So on c>0c > 0 its only maximum is at c=ac = a, and the minimisers on that side approach aa. By symmetry, those on the other side approach −a-a. This proves (a).
  10. Unequal weights. The limit increases up to c=−ac = -a and decreases after c=ac = a. In between it is convex, by the argument of step 9. So its maximum is at −a-a or at aa. The difference between these two values is (2p−1)(λ(0)−λ(2a))(2p - 1)\big(\lambda(0) - \lambda(2a)\big), which is positive when p>1/2p > 1/2. So the maximum, and with it the best forecast for narrow groups, is at the mode of the larger group. This proves (b).

So with two separate groups of actuals, the median is a hump of the expected sMAPE. The best forecasts are near the modes, and with unequal groups the larger one wins. As the weights pass through equality, the target jumps from one mode to the other. ∎

Its formulae were also checked on numbers: E1 in the record.

Supported by numerical check C13

Computed on bimodal outcomes, the target is two forecasts, one near each mode, when the modes are equal, and the larger mode's when they are not, as this claim proves.

Proved · checked independently by an AI agent · confirmed by Ivan Svetunkov · tested as plannedRecord
Claim
C6 · formal · principal · no earlier result found
Plan
Test planned in lab plan PL9, recorded 17 November 2026
Evidence
E24, E1 in the record
Check: confirmed
By an AI agent (proof checking), full. Derived from the statement alone (point-mass limit: tanh(a)/2 < tanh(a/2); second derivative at the median g(m) - E[sech^2 tanh]/2 < 0 for narrow groups; limit convex between the modes), then compared with the final proofs.md (paper H7), which agrees. My earlier dispute concerned the words 'exactly when', which the final wording no longer has; the claims are limits as the groups narrow.
Check: confirmed
By Ivan Svetunkov, full. The supervisor read the proof of paper H7 in proofs.md (AR1.R2; rendered as AR2.R2) and confirmed it: first "Confirmed" (DC6), then, after the last corrections of paper H4, H5 and H9, "All looks fine now. NExt".
Support score
SupportS 94 of 100. The score checks the record, not the science.
Record
This claim on the record page
To cite it
The page of this claim

Zeros and negative actuals

Claim C7. Zero actuals: when the target is zero proved

The worst possible score goes to forecasts of the wrong sign and to zero forecasts. With enough zeros among the actuals, sMAPE prefers the forecast 0. The share of zeros needed is below one half.

A forecast of the opposite sign to the actual, a zero forecast of a non-zero actual, and a non-zero forecast of a zero actual all give the sMAPE term its largest value, 1. Let the actual yy be 0 with probability p0p_0 and otherwise positive, and s+s_+ the lowest expected ss a positive forecast reaches on the positive actuals. The forecast 0 is the sMAPE target when p0>(1−s+)/(2−s+)p_0 > (1 - s_+)/(2 - s_+), and a positive forecast does when the inequality is reversed; the threshold is below 1/21/2 whenever s+>0s_+ > 0.

Proof 6 steps
  1. When yy and ff have opposite signs, |y−f|=|y|+|f||y - f| = |y| + |f|, so s=1s = 1. When exactly one of them is 0, the numerator and the denominator both equal the absolute value of the other one, so s=1s = 1.
  2. A positive forecast scores 1 on every zero actual (step 1). So its expected ss is p0p_0 plus (1−p0)(1 - p_0) times its expected ss on the positive actuals. This is at least p0+(1−p0)s+p_0 + (1-p_0)\,s_+, with equality at the best positive forecasts, which exist (claim C9).
  3. The forecast 0 scores 0 on zero actuals and 1 on positive ones. So its expected ss is 1−p01 - p_0. A negative forecast scores 1 on every actual, which is never better than 0.
  4. The forecast 0 is better when 1−p0<p0+(1−p0)s+1 - p_0 < p_0 + (1 - p_0)s_+. Collecting the terms in p0p_0 gives 1−s+<p0(2−s+)1 - s_+ < p_0(2 - s_+). Since 2−s+>02 - s_+ > 0, this is the stated inequality.
  5. Compare the threshold with 1/21/2: 1−s+2−s+<12\dfrac{1 - s_+}{2 - s_+} < \dfrac12 exactly when 2−2s+<2−s+2 - 2s_+ < 2 - s_+, that is, when s+>0s_+ > 0. And s+>0s_+ > 0 unless the positive actuals are all one value.

So sMAPE switches to the forecast 0 once zeros are frequent enough, and the share needed is below one half. At the threshold, the target jumps from a positive forecast to 0. ∎

Its formulae were also checked on numbers: E1 in the record.

Supported by numerical check C13

Computed on lognormal outcomes with zeros, the target switches to 0 at a share of zeros below one half, as this claim proves.

Earlier work and other claims

  • Refines Svetunkov (2026b), C6. C6 found sMAPE preferring zero on counts with more than half zeros; claim C7 gives the exact threshold, which is below one half whenever the positive outcomes vary.
  • Extends Kolassa (2026). Kolassa (2026) shows that sMAPE elicits a zero forecast for sufficiently intermittent actuals; claim C7 gives the share of zeros at which it does, which is below one half.
Proved · checked independently by an AI agent · confirmed by Ivan Svetunkov · tested as plannedRecord
Claim
C7 · formal · principal · extends earlier work
Plan
Test planned in lab plan PL9, recorded 17 November 2026
Evidence
E25, E1 in the record
Check: confirmed
By an AI agent (proof checking), full. Derived from the statement alone (1 - p0 against p0 + (1 - p0) m+), then compared with the final proofs.md (paper H8), which agrees; the threshold (1 - m+)/(2 - m+) is my earlier (200 - m+)/(400 - m+) rescaled. The best positive forecast may be more than one value; at equality both are targets.
Check: confirmed
By Ivan Svetunkov, full. The supervisor read the proof of paper H8 in proofs.md (AR1.R2; rendered as AR2.R2) and confirmed it: first "Confirmed" (DC6), then, after the last corrections of paper H4, H5 and H9, "All looks fine now. NExt".
Support score
SupportS 100 of 100. The score checks the record, not the science.
Record
This claim on the record page
To cite it
The page of this claim

Claim C11. Actuals of both signs: the target jumps across zero proved

When actuals can be negative or positive, the forecast sMAPE rewards is never near zero. It is on the same side of zero as the mean. So a small shift of the mean across zero makes it jump from a negative value to a positive one.

Let the actuals be μ+ε\mu + \varepsilon, where ε\varepsilon is symmetric about 0 and its density decreases away from 0 (for example, normal). If μ>0\mu > 0, every forecast with the lowest expected sMAPE is positive; if μ<0\mu < 0, every one is negative. These forecasts stay away from 0 as μ\mu approaches 0. So as μ\mu moves from slightly below 0 to slightly above, the sMAPE target jumps from a negative forecast to a positive one.

Proof 6 steps
  1. A positive forecast against its mirror image. Take f>0f > 0 and compare it with −f-f. An actual y>0y > 0 has the sign of ff, so it scores less than 1 with ff, and exactly 1 with −f-f (claim C7).
  2. The mirror actual −y-y scores 1 with ff. By symmetry, it scores the same amount less than 1 with −f-f. So each pair of actuals yy and −y-y favours ff when yy is the more likely of the two, and −f-f otherwise.
  3. When μ>0\mu > 0, every positive actual yy is more likely than −y-y, because yy is closer to μ\mu and the density of the actuals falls away from μ\mu. So every pair favours ff, and ff has the lower expected sMAPE. Every best forecast is therefore positive. The case μ<0\mu < 0 is the mirror image.
  4. Best forecasts exist on each side. For a forecast f>0f > 0, the actuals of its sign score below 1 and have positive probability, so its expected ss is below 1. As ff approaches 0 or grows without bound, every non-zero actual scores close to 1, so the expected ss approaches 1. As in claim C9, the expected ss is continuous in ff and so reaches a lowest value for f>0f > 0; the same holds for f<0f < 0.
  5. Away from zero, uniformly. Fix a forecast f1>0f_1 > 0. For every μ\mu in a range around 0, say |μ|≤1|\mu| \le 1, its expected ss is at most some value q<1q < 1, since it changes continuously with μ\mu. For a forecast ff with |f|≤η|f| \le \eta, every actual with |y|≥η|y| \ge \sqrt\eta scores at least (η−η)/(η+η)(\sqrt\eta - \eta)/(\sqrt\eta + \eta), which approaches 1 as η\eta shrinks; and the share of actuals with |y|<η|y| < \sqrt\eta is small for every such μ\mu: since ε\varepsilon is symmetric and its density decreases away from 0, an interval of a given length around 0 holds at least as much of its probability as any other interval of that length, so P(|μ+ε|<η)≤P(|ε|<η)P(|\mu + \varepsilon| < \sqrt\eta) \le P(|\varepsilon| < \sqrt\eta), which tends to 0 as η\eta shrinks. So for η\eta small enough, every forecast within η\eta of 0 has an expected ss above qq, for every μ\mu in the range: it is worse than f1f_1 (or its mirror image). The best forecasts therefore stay at least η\eta from 0.

So the target is always on the same side of zero as the mean, and never near zero. It jumps from negative to positive as the mean crosses zero. ∎

Its formulae were also checked on numbers: E11 in the record.

Supported by numerical check C13

Computed on normal outcomes straddling zero, the target lies away from zero, on the side of the mean, as this claim proves.

Proved · checked independently by an AI agent · confirmed by Ivan Svetunkov · tested as plannedRecord
Claim
C11 · formal · secondary · no earlier result found
Plan
Test planned in lab plan PL7, recorded 16 November 2026
Evidence
E26, E11 in the record
Check: confirmed
By an AI agent (proof checking), full. Derived from the statement alone: for f > 0, s(y,f) - s(y,-f) is odd in y and negative for y > 0, and the density at y exceeds that at -y when mu > 0, so f beats -f; 0 scores 1; near 0 the expected term tends to 1 uniformly in mu near 0 while a fixed forecast stays below 1. Then compared with the final proofs.md (paper H9), which agrees; its steps 4 and 5 leave this uniformity in mu implicit.
Check: confirmed
By Ivan Svetunkov, full. The supervisor read the proof of paper H9 in proofs.md (AR1.R2; rendered as AR2.R2) and confirmed it: first "Confirmed" (DC6), then, after the last corrections of paper H4, H5 and H9, "All looks fine now. NExt".
Support score
SupportS 100 of 100. The score checks the record, not the science.
Record
This claim on the record page

Widely spread actuals

Claim C10. Expected sMAPE flattens as the actuals spread out proved

For widely spread actuals, every forecast scores close to the worst value. The differences between forecasts shrink, and the forecast sMAPE rewards moves from the median to the most likely value on the log scale.

Let ln⁡y=μ+σε\ln y = \mu + \sigma\varepsilon, where ε\varepsilon has a fixed density hh with bounded continuous first and second derivatives, and write a forecast as f=eμ+σcf = e^{\mu + \sigma c} (its quantile level, the probability that ε≤c\varepsilon \le c, does not depend on σ\sigma). As σ\sigma grows, E[s(y,f)]=1−4ln⁡2h(c)/σ+O(σ−3)\mathrm E[s(y,f)] = 1 - 4\ln 2\,h(c)/\sigma + O(\sigma^{-3}) uniformly in cc.

  1. (a)The expected sMAPE of every forecast tends to its bound 200, and the difference between forecasts at two fixed quantile levels shrinks like 1/σ1/\sigma.
  2. (b)When hh has a single highest point c0c_0, the quantile level of the sMAPE target tends to that of c0c_0, the mode of ln⁡y\ln y, which differs from 1/2 when the mode of ε\varepsilon is not its median.
  3. (c)As σ→0\sigma \to 0, when ε\varepsilon has a unique median and a finite E|ε|3\mathrm E|\varepsilon|^3, the quantile level of the target tends to 1/2: sMAPE targets the median for narrow distributions and the mode of ln⁡y\ln y for wide ones.
Proof 10 steps
  1. Rewrite the expectation. By claim C3, and since tanh⁡(|x|/2)=1−21+e|x|\tanh(|x|/2) = 1 - \dfrac{2}{1 + e^{|x|}}, E[s(y,f)]=1−E[2/(1+e|ln⁡y−ln⁡f|)]\mathrm E[s(y,f)] = 1 - \mathrm E\big[2/(1 + e^{|\ln y - \ln f|})\big]. The density of ln⁡y\ln y at xx is h((x−μ)/σ)/σh\big((x - \mu)/\sigma\big)/\sigma. Substituting x=ln⁡f+vx = \ln f + v in the integral gives E[s(y,f)]=1−1σ∫21+e|v|h(c+vσ)dv.\mathrm E[s(y,f)] = 1 - \frac1\sigma\int \frac{2}{1 + e^{|v|}}\,h\Big(c + \frac v\sigma\Big)\,dv.
  2. Expand hh. By Taylor's theorem, h(c+v/σ)=h(c)+h′(c)v/σ+12h′′(ξ)v2/σ2h(c + v/\sigma) = h(c) + h'(c)\,v/\sigma + \tfrac12 h''(\xi)\,v^2/\sigma^2 for some ξ\xi between cc and c+v/σc + v/\sigma.
  3. The first term integrates to 4ln⁡2h(c)4\ln 2\,h(c), since ∫21+e|v|dv=4ln⁡2\int \frac{2}{1+e^{|v|}}\,dv = 4\ln 2. Over v>0v > 0, ∫2/(1+ev)dv=2ln⁡2\int 2/(1 + e^v)\,dv = 2\ln 2.
  4. The second term integrates to 0: its factor vv changes sign with vv, while the rest does not.
  5. The third term is at most 12max⁡|h′′|\tfrac12\max|h''| times ∫2v21+e|v|dv=6ζ(3)≈7.21\int \frac{2v^2}{1+e^{|v|}}\,dv = 6\zeta(3) \approx 7.21, divided by σ2\sigma^2. This integral is a standard Fermi–Dirac integral. With the factor 1/σ1/\sigma of step 1, this gives the expansion.
  6. (a). The expansion tends to 1, so the expected sMAPE tends to 200. Two forecasts at fixed quantile levels have fixed positions c1c_1 and c2c_2. So their difference is 4ln⁡2[h(c2)−h(c1)]/σ+O(σ−3)4\ln 2\,[h(c_2) - h(c_1)]/\sigma + O(\sigma^{-3}).
  7. (b). Minimising the expansion over cc is maximising h(c)h(c), up to a term that shrinks like σ−2\sigma^{-2} for every cc at once. A density with a bounded derivative tends to 0 far out. So its single highest point c0c_0 stands clear of every other point, and the minimisers' positions tend to c0c_0. So do their quantile levels.
  8. (c). By claim C3 (b), 12σ|ε−c|−s\tfrac12\sigma|\varepsilon - c| - s lies between 0 and 124σ3|ε−c|3\tfrac1{24}\sigma^3|\varepsilon - c|^3. So for cc in any bounded range, E[s]/σ=12E|ε−c|+O(σ2)\mathrm E[s]/\sigma = \tfrac12\mathrm E|\varepsilon - c| + O(\sigma^2). This uses E|ε|3<∞\mathrm E|\varepsilon|^3 < \infty. And E|ε−c|\mathrm E|\varepsilon - c| is smallest at the median of ε\varepsilon.
  9. The minimisers also stay in a bounded range. For large |c||c|, most actuals have ε\varepsilon at least |c|/2|c|/2 from cc. And tanh⁡x≥xtanh⁡1\tanh x \ge x\tanh 1 for 0≤x≤10 \le x \le 1. So E[s]/σ\mathrm E[s]/\sigma grows with |c||c|, or exceeds a multiple of 1/σ1/\sigma, beyond its value at the median. So the minimisers' positions tend to the median of ε\varepsilon, whose quantile level is 1/21/2.

So as the actuals spread out, the expected sMAPE flattens towards its bound. Differences between forecasts vanish like 1/σ1/\sigma. The target moves from the median to the mode of ln⁡y\ln y. ∎

Its formulae were also checked on numbers: E1 in the record.

Supported by numerical check C14

Computed for spreads of ln y from 0.1 to 10, expected sMAPE flattens and the target moves to the mode of ln y, as this claim proves for large spreads.

Elsewhere in this paper

  • Refined by claim C18. This claim shows that the scores of forecasts at fixed quantile levels move together like 1/σ. Claim C18 adds the noise and shows that the signal-to-noise ratio falls like σ^(-1/2), while that of the absolute log error does not change.

Earlier work and other claims

  • Supports Svetunkov (2026a), C6. C6 recommends against sMAPE for point forecasts; claim C10 shows that on widely spread actuals every forecast scores close to the worst value and the target moves to the mode of ln y.
  • Supports Kolassa (2026). Kolassa (2026) concludes that sMAPE should not be used; claim C10 shows that on widely spread actuals every forecast scores close to the worst value.
Proved · checked independently by an AI agent · confirmed by Ivan Svetunkov · tested as plannedRecord
Claim
C10 · formal · principal · no earlier result found
Plan
Test planned in lab plan PL9, recorded 17 November 2026
Evidence
E28, E1 in the record
Check: confirmed
By an AI agent (proof checking), full. Derived from the statement alone (integral of 2/(1+e^|v|) is 4 ln 2, odd term vanishes, remainder at most 3 zeta(3) sup|g''|/sigma^3 uniformly in c; checked numerically for the normal), then compared with the final proofs.md (paper H11), which agrees; the constant 4 ln 2 is my earlier 800 ln 2 rescaled. (b) and (c) are claims about the quantile level, which is what holds in general.
Check: confirmed
By Ivan Svetunkov, full. The supervisor read the proof of paper H11 in proofs.md (AR1.R2; rendered as AR2.R2) and confirmed it: first "Confirmed" (DC6), then, after the last corrections of paper H4, H5 and H9, "All looks fine now. NExt".
Support score
SupportS 100 of 100. The score checks the record, not the science.
Record
This claim on the record page
To cite it
The page of this claim

Terms this claim uses

signal-to-noise ratio of a comparison
For a measure with term L(y,f)L(y,f), two forecasts f1f_1 and f2f_2 and a distribution of the actual yy: |E[D]|/sd(D)|\mathrm E[D]|/\mathrm{sd}(D), where D=L(y,f2)−L(y,f1)D = L(y,f_2) - L(y,f_1) is the difference of the two forecasts' scores on one actual. Over a test set of nn independent actuals the mean of DD is n\sqrt n times this many of its standard errors from zero: the larger the ratio, the smaller the test set that tells the two forecasts apart.
MAE
Mean absolute error of a point forecast ff over a test set of actuals y1,…,yny_1, \dots, y_n: 1n∑t|yt−f|\frac1n\sum_t |y_t-f|.
RMSE
Root mean squared error of a point forecast ff over a test set of actuals y1,…,yny_1, \dots, y_n: 1n∑t(yt−f)2\sqrt{\frac1n\sum_t (y_t-f)^2}.

Claim C18. sMAPE loses its power to tell forecasts apart as the actuals spread out proved

For widely spread actuals a test set tells two forecasts apart less and less well under sMAPE: the signal-to-noise ratio of their score difference falls like one over the square root of the spread. The absolute log error and MAE keep their power.

Let ln⁡y=μ+σε\ln y = \mu + \sigma\varepsilon, where ε\varepsilon has a fixed density hh with bounded continuous first and second derivatives and distribution function HH, and compare two forecasts at fixed quantile levels, f1=eμ+σc1f_1 = e^{\mu+\sigma c_1} and f2=eμ+σc2f_2 = e^{\mu+\sigma c_2} with c1<c2c_1 < c_2, by their signal-to-noise ratio of a comparison.

  1. (a)For the sMAPE term, as σ\sigma grows, E[D]=4ln⁡2(h(c1)−h(c2))σ+O(σ−3),Var⁡(D)=4(2ln⁡2−1)(h(c1)+h(c2))σ+O(σ−2).\mathrm E[D] = \frac{4\ln 2\,(h(c_1) - h(c_2))}{\sigma} + O(\sigma^{-3}), \qquad \operatorname{Var}(D) = \frac{4(2\ln 2 - 1)(h(c_1)+h(c_2))}{\sigma} + O(\sigma^{-2}). So when h(c1)≠h(c2)h(c_1) \ne h(c_2) the ratio is 2ln⁡22ln⁡2−1|h(c1)−h(c2)|h(c1)+h(c2)σ−1/2(1+O(1/σ)):\frac{2\ln 2}{\sqrt{2\ln 2 - 1}}\,\frac{|h(c_1)-h(c_2)|}{\sqrt{h(c_1)+h(c_2)}}\,\sigma^{-1/2}\,(1 + O(1/\sigma)): it falls to zero like σ−1/2\sigma^{-1/2}, and the size of the test set that keeps the mean difference a given number of standard errors from zero grows in proportion to σ\sigma.
  2. (b)For the absolute log error |ln⁡y−ln⁡f||\ln y - \ln f| (log-MAE), D=σ(|ε−c2|−|ε−c1|)D = \sigma(|\varepsilon - c_2| - |\varepsilon - c_1|), so the ratio, wherever sd(D)>0\mathrm{sd}(D) > 0, does not depend on σ\sigma.
  3. (c)For the absolute error |y−f||y - f| (MAE), as σ\sigma grows the ratio tends to |2H(c2)−1|2H(c2)(1−H(c2))\frac{|2H(c_2) - 1|}{2\sqrt{H(c_2)(1 - H(c_2))}} when 0<H(c2)<10 < H(c_2) < 1; the limit is positive unless the higher forecast is the median.
  4. (d)For the squared error (y−f)2(y-f)^2 (RMSE), when E[y2]\mathrm E[y^2] is finite the ratio is |2E[y]−f1−f2|/(2sd(y))|2\,\mathrm E[y] - f_1 - f_2|/(2\,\mathrm{sd}(y)); for lognormal actuals, eσ2/2e^{\sigma^2/2} times the ratio tends to 1 as σ\sigma grows. For widely spread actuals, then: the absolute log error keeps its power to tell two forecasts apart wherever its ratio is defined. The absolute error keeps it when 0<H(c2)<10 < H(c_2) < 1 and the higher forecast is not the median. sMAPE loses it at the rate σ−1/2\sigma^{-1/2} when h(c1)≠h(c2)h(c_1) \ne h(c_2). It loses it faster, with a ratio of at most order σ−5/2\sigma^{-5/2}, when h(c1)=h(c2)>0h(c_1) = h(c_2) > 0. The squared error on lognormal actuals loses it faster than any power of σ\sigma.
Proof 10 steps
  1. The sMAPE difference. Write q(x)=2/(1+e|x|)q(x) = 2/(1 + e^{|x|}), so that tanh⁡(|x|/2)=1−q(x)\tanh(|x|/2) = 1 - q(x), and qi=q(σ(ε−ci))q_i = q(\sigma(\varepsilon - c_i)). By claim C3 the sMAPE term of the forecast fif_i is 1−qi1 - q_i, because ln⁡y−ln⁡fi=σ(ε−ci)\ln y - \ln f_i = \sigma(\varepsilon - c_i). So D=q1−q2D = q_1 - q_2.
  2. Its mean. By claim C10, E[qi]=4ln⁡2h(ci)/σ+O(σ−3)\mathrm E[q_i] = 4\ln 2\,h(c_i)/\sigma + O(\sigma^{-3}). So E[D]=4ln⁡2(h(c1)−h(c2))/σ+O(σ−3)\mathrm E[D] = 4\ln 2\,(h(c_1) - h(c_2))/\sigma + O(\sigma^{-3}).
  3. The mean of qi2q_i^2. Substituting v=σ(ε−ci)v = \sigma(\varepsilon - c_i), as in claim C10, step 1, E[qi2]=1σ∫q(v)2h(ci+v/σ)dv\mathrm E[q_i^2] = \frac1\sigma\int q(v)^2\,h(c_i + v/\sigma)\,dv. Expanding hh by Taylor's theorem, as in claim C10, step 2: the first term integrates to h(ci)∫q(v)2dvh(c_i)\int q(v)^2\,dv; the second integrates to 0, because its factor vv changes sign with vv while the rest does not; the third is at most 12max⁡|h′′|∫v2q(v)2dv\tfrac12\max|h''|\int v^2 q(v)^2\,dv divided by σ2\sigma^2, and this integral is finite because q(v)≤2e−|v|q(v) \le 2e^{-|v|}. Since 1(1+ev)2=11+ev−ev(1+ev)2\dfrac{1}{(1+e^v)^2} = \dfrac{1}{1+e^v} - \dfrac{e^v}{(1+e^v)^2}, whose two parts integrate over v>0v > 0 to ln⁡2\ln 2 and 12\tfrac12, ∫q(v)2dv=8(ln⁡2−12)=4(2ln⁡2−1)\int q(v)^2\,dv = 8(\ln 2 - \tfrac12) = 4(2\ln 2 - 1). So E[qi2]=4(2ln⁡2−1)h(ci)/σ+O(σ−3)\mathrm E[q_i^2] = 4(2\ln 2 - 1)\,h(c_i)/\sigma + O(\sigma^{-3}).
  4. The cross term. Since q(x)≤2e−|x|q(x) \le 2e^{-|x|}, q1q2≤4e−σ(|ε−c1|+|ε−c2|)≤4e−σ(c2−c1)q_1 q_2 \le 4e^{-\sigma(|\varepsilon - c_1| + |\varepsilon - c_2|)} \le 4e^{-\sigma(c_2 - c_1)}, because |ε−c1|+|ε−c2|≥c2−c1|\varepsilon - c_1| + |\varepsilon - c_2| \ge c_2 - c_1. So E[q1q2]≤4e−σ(c2−c1)\mathrm E[q_1 q_2] \le 4e^{-\sigma(c_2 - c_1)}, which is smaller than every power of 1/σ1/\sigma.
  5. The variance. Var⁡(D)=E[q12]+E[q22]−2E[q1q2]−(E[D])2\operatorname{Var}(D) = \mathrm E[q_1^2] + \mathrm E[q_2^2] - 2\,\mathrm E[q_1 q_2] - (\mathrm E[D])^2. By steps 2 to 4 this is 4(2ln⁡2−1)(h(c1)+h(c2))/σ+O(σ−2)4(2\ln 2 - 1)(h(c_1) + h(c_2))/\sigma + O(\sigma^{-2}); the term (E[D])2(\mathrm E[D])^2 is of order σ−2\sigma^{-2}.
  6. (a). When h(c1)≠h(c2)h(c_1) \ne h(c_2), h(c1)+h(c2)>0h(c_1) + h(c_2) > 0. Dividing the size of the mean (step 2) by the square root of the variance (step 5) gives the stated ratio. To keep n\sqrt n times the ratio at a given value, nn must be that value squared over the ratio squared, which grows in proportion to σ\sigma. When h(c1)=h(c2)>0h(c_1) = h(c_2) > 0 instead, the mean is O(σ−3)O(\sigma^{-3}) (step 2) while the variance is still of order 1/σ1/\sigma (step 5), so the ratio is O(σ−5/2)O(\sigma^{-5/2}): at most of that order.
  7. (b). For the absolute log error, |ln⁡y−ln⁡fi|=σ|ε−ci||\ln y - \ln f_i| = \sigma|\varepsilon - c_i|, so D=σ(|ε−c2|−|ε−c1|)D = \sigma(|\varepsilon - c_2| - |\varepsilon - c_1|). It is bounded by σ(c2−c1)\sigma(c_2 - c_1), so its mean and standard deviation exist. Both are σ\sigma times those of |ε−c2|−|ε−c1||\varepsilon - c_2| - |\varepsilon - c_1|, whose distribution does not depend on σ\sigma. So their ratio, wherever the standard deviation is not zero, does not depend on σ\sigma.
  8. (c). For the absolute error, D=|y−f2|−|y−f1|D = |y - f_2| - |y - f_1| with f1<f2f_1 < f_2. It equals f2−f1f_2 - f_1 when y≤f1y \le f_1, f1+f2−2yf_1 + f_2 - 2y when f1<y<f2f_1 < y < f_2, and −(f2−f1)-(f_2 - f_1) when y≥f2y \ge f_2; so |D|≤f2|D| \le f_2. Now f1/f2=e−σ(c2−c1)f_1/f_2 = e^{-\sigma(c_2 - c_1)} tends to 0, and for every ε<c2\varepsilon < c_2, y/f2=eσ(ε−c2)y/f_2 = e^{\sigma(\varepsilon - c_2)} tends to 0. So D/f2D/f_2 tends to 1 for ε<c2\varepsilon < c_2 and to −1-1 for ε>c2\varepsilon > c_2. By bounded convergence, E[D/f2]→H(c2)−(1−H(c2))=2H(c2)−1\mathrm E[D/f_2] \to H(c_2) - (1 - H(c_2)) = 2H(c_2) - 1 and E[(D/f2)2]→1\mathrm E[(D/f_2)^2] \to 1. So the variance of D/f2D/f_2 tends to 1−(2H(c2)−1)2=4H(c2)(1−H(c2))1 - (2H(c_2) - 1)^2 = 4H(c_2)(1 - H(c_2)), and the ratio, which is the same for DD and D/f2D/f_2, tends to the stated limit when 0<H(c2)<10 < H(c_2) < 1. The limit is 0 only when H(c2)=1/2H(c_2) = 1/2.
  9. (d). For the squared error, D=(y−f2)2−(y−f1)2=(f2−f1)(f1+f2−2y)D = (y - f_2)^2 - (y - f_1)^2 = (f_2 - f_1)(f_1 + f_2 - 2y). So E[D]=(f2−f1)(f1+f2−2E[y])\mathrm E[D] = (f_2 - f_1)(f_1 + f_2 - 2\,\mathrm E[y]) and sd(D)=2(f2−f1)sd(y)\mathrm{sd}(D) = 2(f_2 - f_1)\,\mathrm{sd}(y), finite when E[y2]\mathrm E[y^2] is. Their ratio is the stated one. For lognormal actuals E[y]=eμ+σ2/2\mathrm E[y] = e^{\mu + \sigma^2/2}, sd(y)=E[y]eσ2−1\mathrm{sd}(y) = \mathrm E[y]\sqrt{e^{\sigma^2} - 1} and fi/E[y]=eσci−σ2/2f_i/\mathrm E[y] = e^{\sigma c_i - \sigma^2/2}, which tends to 0. So eσ2/2e^{\sigma^2/2} times the ratio is |1−(f1+f2)/(2E[y])|eσ2/2/eσ2−1|1 - (f_1 + f_2)/(2\,\mathrm E[y])|\,e^{\sigma^2/2}/\sqrt{e^{\sigma^2} - 1}, which tends to 1.

So for widely spread actuals the absolute log error keeps its power to tell two forecasts apart wherever its ratio is defined. The absolute error keeps it when 0<H(c2)<10 < H(c_2) < 1 and the higher forecast is not the median. sMAPE loses it at the rate σ−1/2\sigma^{-1/2} when h(c1)≠h(c2)h(c_1) \ne h(c_2). It loses it faster, with a ratio of at most order σ−5/2\sigma^{-5/2}, when h(c1)=h(c2)>0h(c_1) = h(c_2) > 0. The squared error on lognormal actuals loses it faster than any power of σ\sigma. ∎

Its formulae were also checked on numbers: E32 in the record.

Earlier work and other claims

  • Supports Svetunkov (2026a), C6. C6 recommends against sMAPE for point forecasts; claim C18 shows that on widely spread actuals a test set tells two forecasts apart less and less well under sMAPE.
  • Supports Kolassa (2026). Kolassa (2026) concludes that sMAPE should not be used; claim C18 shows that on widely spread actuals a test set tells two forecasts apart less and less well under sMAPE.
  • Refines claim C10. Claim C10 shows that the scores of forecasts at fixed quantile levels move together like 1/σ. Claim C18 adds the noise and shows that the signal-to-noise ratio falls like σ^(-1/2), while that of the absolute log error does not change.
Proved · checked independently by an AI agent · tested as plannedRecord
Claim
C18 · formal · principal · no earlier result found
Plan
Test planned in lab plan PL17, recorded 18 November 2026
Evidence
E35, E32 in the record
Check: disputed
By an AI agent (proof checking, version 3), full. Derived independently. (a) With $q(v)=1-\tanh(|v|/2)=2/(1+e^{|v|})$, $\mathrm E[q(\sigma(\varepsilon-c))]=\frac1\sigma\int q(u)h(c+u/\sigma)du=4\ln2\,h(c)/\sigma+O(\sigma^{-3})$ (the $h'$ term vanishes by symmetry; the remainder is bounded by $\tfrac12\sup|h''|\int u^2q/\sigma^3$). Likewise $\mathrm E[q^2]=4(2\ln2-1)h(c)/\sigma+O(\sigma^{-3})$ with $\int q^2=8(\ln2-\tfrac12)$; the cross term is at most $4e^{-\sigma(c_2-c_1)}$; $(\mathrm E D)^2=O(\sigma^{-2})$. So both expansions and their remainder orders hold, and the ratio is as stated with relative error $O(1/\sigma)$ (from the variance), with $h(c_1)+h(c_2)>0$ when $h(c_1)\ne h(c_2)$. A quadrature check (normal $h$, $c_1=-0.3$, $c_2=0.8$, $\sigma$ up to 160) agrees. (b) is immediate; $D$ is bounded by $\sigma(c_2-c_1)$. (c) $D/f_2\to\mathbf 1\{\varepsilon<c_2\}-\mathbf 1\{\varepsilon>c_2\}$ a.s. with $|D/f_2|\le1$, so $\mathrm E[D/f_2]\to2H(c_2)-1$ and $\operatorname{Var}(D/f_2)\to4H(c_2)(1-H(c_2))$. The stated limit therefore holds only when $0<H(c_2)<1$. The statement does not give this condition, and the proof (H13 step 8) adds it. The hypotheses allow it to fail: $h$ may have bounded support, e.g. $h\propto(1-x^2)^3$ on $[-1,1]$, which has bounded continuous $h'$ and $h''$. If $H(c_2)=1$ the formula is $1/0$ (when $c_1$ is inside the support the ratio tends to infinity; when it is not, $D$ is constant and the ratio is undefined for every $\sigma$). If $H(c_2)=0$, $D$ is constant and the ratio is undefined for every $\sigma$. (d) $D=(f_2-f_1)(f_1+f_2-2Y)$ gives the ratio exactly; for lognormal actuals $e^{\sigma^2/2}$ times the ratio is $|1-(f_1+f_2)/(2\mathrm EY)|\,e^{\sigma^2/2}/\sqrt{e^{\sigma^2}-1}\to1$. Every computation in the proof is correct. The dispute concerns only the missing condition in (c). Suggested wording: "(c) ... as $\sigma$ grows the ratio tends to $|2H(c_2)-1|/(2\sqrt{H(c_2)(1-H(c_2))})$ when $0<H(c_2)<1$, which is positive unless the higher forecast is the median." Optionally, also in (b): "wherever $\mathrm{sd}(D)>0$" (it is 0 when $\varepsilon\le c_1$ a.s. or $\varepsilon\ge c_2$ a.s.). The closing sentence is also loose: sMAPE loses power at the rate $\sigma^{-1/2}$ only when $h(c_1)\ne h(c_2)$ (otherwise faster), and the absolute error keeps it only when $H(c_2)\ne1/2$. It would be safer as "for widely spread actuals, comparisons of forecasts with $h(c_1)\ne h(c_2)$ ...". The statement was corrected, and the checker confirmed the corrected wording in a later check listed here.
Check: disputed
By an AI agent (proof checking, version 3), full. Re-checked every part as now worded, against the revised H13. (a) is unchanged from H39 and correct: $\mathrm E[q]=4\ln2\,h(c)/\sigma+O(\sigma^{-3})$ and $\mathrm E[q^2]=4(2\ln2-1)h(c)/\sigma+O(\sigma^{-3})$; the cross term is at most $4e^{-\sigma(c_2-c_1)}$; the ratio is as stated with relative error $O(1/\sigma)$. (b), "wherever $\mathrm{sd}(D)>0$", is correct. (c), now with "when $0<H(c_2)<1$", is correct: $D/f_2\to\mathbf 1\{\varepsilon<c_2\}-\mathbf 1\{\varepsilon>c_2\}$ boundedly, and quadrature agrees. (d) is correct. One point remains, in the closing sentence: "sMAPE loses it, at the rate $\sigma^{-1/2}$ when $h(c_1)\ne h(c_2)$". This asserts that sMAPE loses its power in every case, but neither the statement nor the proof (whose closing paragraph repeats it) shows this when $h(c_1)=h(c_2)$. If $h(c_1)=h(c_2)>0$, it follows from (a) in one line: $|\mathrm E D|=O(\sigma^{-3})$ and $\operatorname{Var}D\sim8(2\ln2-1)h(c_1)/\sigma$, so the ratio is $O(\sigma^{-5/2})$. The proof should say this. If $h(c_1)=h(c_2)=0$ (possible when $h$ vanishes at a point or has bounded support), (a) gives only $\mathrm E D=O(\sigma^{-3})$ and $\operatorname{Var}D=O(\sigma^{-2})$, which do not bound the ratio. I could not derive the claim in that case. It is true in the cases I computed: $h\propto(1-x^2)^3$ with $(c_1,c_2)=(1.5,2)$ or $(-1.5,2)$ gives a ratio falling like $\sigma^{-2}$; $(-1.3,1.3)$ gives 0. The gap is my own doing: my note on H39 suggested "otherwise faster" without having proved the zero-density case. Suggested wording: "sMAPE loses it when $h(c_1)+h(c_2)>0$, at the rate $\sigma^{-1/2}$ when $h(c_1)\ne h(c_2)$ and faster when $h(c_1)=h(c_2)$", with the one-line argument above added to the proof. Alternatively, keep only "sMAPE loses it at the rate $\sigma^{-1/2}$ when $h(c_1)\ne h(c_2)$". Smaller and optional: "the absolute error keeps it unless the higher forecast is the median" omits the condition $0<H(c_2)<1$ of (c) (with $H(c_2)=0$ the ratio is undefined); adding "when $0<H(c_2)<1$" would make it exact. With these changes I would confirm. The statement was corrected, and the checker confirmed the corrected wording in a later check listed here.
Check: disputed
By an AI agent (proof checking, version 3), full. Parts (a)-(d) are unchanged from H43 and correct (checked in CK30 and CK34). I judged each sentence of the new closing against them. (1) The absolute log error keeps its power wherever its ratio is defined: this follows from (b). (2) The absolute error keeps it when $0<H(c_2)<1$ and the higher forecast is not the median: this follows from (c), whose limit is then positive. (3) sMAPE loses it at the rate $\sigma^{-1/2}$ when $h(c_1)\ne h(c_2)$: this follows from (a). (5) The squared error on lognormal actuals: (d) gives $e^{\sigma^2/2}\cdot\text{ratio}\to1$, so the ratio is $e^{-\sigma^2/2}(1+o(1))=o(\sigma^{-k})$ for every $k$, for every $c_1<c_2$; this is fully covered by (d), and lognormal actuals (normal $h$) meet the hypotheses. (4), "It loses it faster, with a ratio of order $\sigma^{-5/2}$, when $h(c_1)=h(c_2)>0$", is not established as worded. (a) gives $\mathrm E[D]=O(\sigma^{-3})$, an upper bound only, and $\operatorname{Var}(D)=8(2\ln2-1)h(c_1)/\sigma+O(\sigma^{-2})$, which is exactly of order $1/\sigma$. So the ratio is $O(\sigma^{-5/2})$: at most of order $\sigma^{-5/2}$, which does establish "loses it faster". But "a ratio of order $\sigma^{-5/2}$" naturally reads as the exact order, and that is false in general. The $\sigma^{-3}$ coefficient of $\mathrm E[D]$ is proportional to $h''(c_1)-h''(c_2)$, which can vanish: for a symmetric $h$ with $c_1=-c_2$, $\mathrm E[D]=0$ exactly and the ratio is 0 for every $\sigma$. The proof's new sentence in step 6 has the same slip: "the mean is of order $\sigma^{-3}$ (step 2)", whereas step 2 gives only $O(\sigma^{-3})$. Suggested wording: "It loses it faster, with a ratio of at most order $\sigma^{-5/2}$ ($O(\sigma^{-5/2})$), when $h(c_1)=h(c_2)>0$", and in proof step 6 "the mean is $O(\sigma^{-3})$ ... so the ratio is $O(\sigma^{-5/2})$". With that change I would confirm. Separately, I checked the revised H12 step 6 (behind H38): the weights 0.406125, 0.054825 and 0.539050 now sum to 1; the parameters are correctly marked as rounded; and "rises through $m$, where it is flat" matches my computation ($\phi'\ge0$ on both sides of 0). It is now right. The statement was corrected, and the checker confirmed the corrected wording in a later check listed here.
Check: confirmed
By an AI agent (proof checking, version 3), full. A word diff against H45 shows that the only change is "at most" in the sentence I disputed in CK36. Parts (a)-(d) are correct, as checked in CK30, CK34 and CK36. I judged each closing sentence against them. The absolute log error keeps its power wherever its ratio is defined: this follows from (b). The absolute error keeps it when $0<H(c_2)<1$ and the higher forecast is not the median: this follows from (c). sMAPE loses it at the rate $\sigma^{-1/2}$ when $h(c_1)\ne h(c_2)$: this follows from (a). "It loses it faster, with a ratio of at most order $\sigma^{-5/2}$, when $h(c_1)=h(c_2)>0$" is now exactly what (a) gives. The mean is $O(\sigma^{-3})$ and the variance is $8(2\ln2-1)h(c_1)/\sigma+O(\sigma^{-2})$, exactly of order $1/\sigma$, so the ratio is $O(\sigma^{-5/2})$; the bound allows the ratio to be smaller, even 0 (symmetric $h$, $c_1=-c_2$). The squared error on lognormal actuals loses its power faster than any power of $\sigma$: by (d) the ratio is $e^{-\sigma^2/2}(1+o(1))$, for every $c_1<c_2$. The revised proof step 6 writes $O(\sigma^{-3})$ and $O(\sigma^{-5/2})$ ("at most of that order"), and the closing paragraph matches the statement. Every part and every closing sentence holds as worded, and the proof establishes them.
Support score
SupportS 100 of 100. The score checks the record, not the science.
Record
This claim on the record page
To cite it
The page of this claim

Other measures

Term this claim uses

MAPE
Mean absolute percentage error of a point forecast ff over a test set of actuals y1,…,yny_1, \dots, y_n: 100n∑t|yt−f|/|yt|\frac{100}{n}\sum_t |y_t-f|/|y_t|; infinite when an actual is zero.

Claim C8. Where RMSE, MAE and MAPE fail, and what they then target proved

RMSE, MAE and MAPE each have an infinite expected value, for every forecast, when a moment they need is infinite. What each then rewards depends on the measure.

For a positive random actual yy:

  1. (a)RMSE: E[(y−f)2]\mathrm E[(y-f)^2] is finite exactly when E[y2]\mathrm E[y^2] is; when E[y2]\mathrm E[y^2] is infinite but E[y]\mathrm E[y] finite, every difference between two forecasts f1f_1 and f2f_2, E[(y−f1)2−(y−f2)2]=(f1−f2)(f1+f2−2E[y]),\mathrm E[(y-f_1)^2 - (y-f_2)^2] = (f_1-f_2)(f_1+f_2-2\,\mathrm E[y]), is finite and the mean is preferred to every other forecast; when E[y]\mathrm E[y] is infinite no forecast is preferred to all others.
  2. (b)MAE: E|y−f|\mathrm E|y-f| is finite exactly when E[y]\mathrm E[y] is; otherwise differences are finite and a median is still preferred.
  3. (c)MAPE: when E[1/y]\mathrm E[1/y] is finite, the expected MAPE is finite and minimised by a median of the actuals weighted by 1/y1/y; when E[1/y]\mathrm E[1/y] is infinite, every positive forecast has infinite expected MAPE while the forecast 0 has MAPE 100, so MAPE targets 0; when yy is 0 with positive probability, MAPE is infinite or undefined for every forecast and targets nothing.
Proof 12 steps
  1. (a) RMSE. If E[y2]\mathrm E[y^2] is finite, so is E[y]\mathrm E[y]. Expanding the square gives E[(y−f)2]=E[y2]−2fE[y]+f2\mathrm E[(y-f)^2] = \mathrm E[y^2] - 2f\,\mathrm E[y] + f^2, which is finite. Conversely, y2≤2(y−f)2+2f2y^2 \le 2(y - f)^2 + 2f^2. So a finite E[(y−f)2]\mathrm E[(y-f)^2] makes E[y2]\mathrm E[y^2] finite.
  2. Expanding both squares gives (y−f1)2−(y−f2)2=(f2−f1)(2y−f1−f2)(y - f_1)^2 - (y - f_2)^2 = (f_2 - f_1)(2y - f_1 - f_2). Its expectation is finite when E[y]\mathrm E[y] is, and equals (f1−f2)(f1+f2−2E[y])=(f1−E[y])2−(f2−E[y])2(f_1 - f_2)(f_1 + f_2 - 2\,\mathrm E[y]) = (f_1 - \mathrm E[y])^2 - (f_2 - \mathrm E[y])^2.
  3. For f1=E[y]f_1 = \mathrm E[y] this is negative for every f2≠E[y]f_2 \ne \mathrm E[y]. So the mean is preferred.
  4. If E[y]\mathrm E[y] is infinite, then for f2>f1f_2 > f_1 the expectation in step 2 is +∞+\infty. Every forecast is beaten by any larger one, so none is preferred to all others.
  5. (b) MAE. Since |y−f|≥y−f|y - f| \ge y - f, E|y−f|\mathrm E|y - f| is infinite when E[y]\mathrm E[y] is. Otherwise it is at most E[y]+|f|\mathrm E[y] + |f|.
  6. The difference |y−f1|−|y−f2||y - f_1| - |y - f_2| is at most |f1−f2||f_1 - f_2| in size, so its expectation is finite.
  7. The derivative of |y−f||y - f| in ff is 𝟏{y<f}−𝟏{y>f}\mathbf 1\{y < f\} - \mathbf 1\{y > f\}. The expectation of this is P(y<f)−P(y>f)P(y < f) - P(y > f), which is negative below a median and positive above it. So a median is preferred.
  8. (c) MAPE. For f>0f > 0, the contribution of an actual to MAPE, divided by 100, is |y−f|/y=|1−f/y||y - f|/y = |1 - f/y|. This is at least f/y−1f/y - 1. So the expected MAPE is infinite when E[1/y]\mathrm E[1/y] is, and at most 1+fE[1/y]1 + f\,\mathrm E[1/y] otherwise.
  9. As a function of ff, |1−f/y||1 - f/y| is convex. Its derivative is (1/y)(𝟏{y<f}−𝟏{y>f})(1/y)(\mathbf 1\{y < f\} - \mathbf 1\{y > f\}). Its expectation is 0 where the actuals below ff and those above it have equal total weight 1/y1/y: at a median of the actuals weighted by 1/y1/y.
  10. The forecast 0 gives |1−0|=1|1 - 0| = 1 for every actual, so MAPE 100. A negative forecast gives more than 1. So when E[1/y]\mathrm E[1/y] is infinite, 0 is the only forecast with a finite expected MAPE.
  11. If y=0y = 0 with positive probability: for f≠0f \ne 0 that actual contributes |f|/0|f|/0, which is infinite. For f=0f = 0 it contributes 0/00/0, which is undefined.

So each of RMSE, MAE and MAPE has an infinite expected value when a moment it needs is infinite. RMSE and MAE still rank forecasts by their differences. MAPE turns to the forecast 0, and with zero actuals it targets nothing. ∎

Its formulae were also checked on numbers: E1 in the record.

Proved · checked independently by an AI agent · confirmed by Ivan Svetunkov · tested as plannedRecord
Claim
C8 · formal · secondary · no earlier result found
Plan
Test planned in lab plan PL9, recorded 17 November 2026
Evidence
E27, E1 in the record
Check: confirmed
By an AI agent (proof checking), full. Derived from the statement alone, then compared with the final proofs.md (paper H10), which agrees in all three parts. When E[1/Y] is infinite, negative forecasts also have infinite expected MAPE, so 0 is the only finite one.
Check: confirmed
By Ivan Svetunkov, full. The supervisor read the proof of paper H10 in proofs.md (AR1.R2; rendered as AR2.R2) and confirmed it: first "Confirmed" (DC6), then, after the last corrections of paper H4, H5 and H9, "All looks fine now. NExt".
Support score
SupportS 100 of 100. The score checks the record, not the science.
Record
This claim on the record page

sMAPE modifications

Term this claim uses

Suilin term s~(y,f)\tilde s(y,f)
The contribution of one actual yy and one forecast ff to the Suilin loss: s~(y,f)=|y−f|max⁡(|y|+|f|+δ,0.5+δ),δ=0.1;\tilde s(y,f) = \frac{|y-f|}{\max(|y|+|f|+\delta,\ 0.5+\delta)}, \qquad \delta = 0.1; the Suilin loss is 200 times its mean over a test set. It replaces the denominator |y|+|f||y|+|f| of the sMAPE term. Proposed by Suilin (2017) in code for a Kaggle competition, as a smoothed variant of sMAPE; defined in print by Hewamalage, Bergmeir and Bandara (2021, Eq. 15), who write the constant δ\delta as ϵ\epsilon and the factor 2 into the measure differently (a constant factor changes no property here). Hewamalage, Ackermann and Bergmeir (2023) call it msMAPE; Smyl (2025) gives that name to a different measure, so this study follows Kolassa (2026) and calls it the Suilin loss. Version 2 of this paper called it msMAPE.

Claim C17. The Suilin loss behaves like sMAPE except near zero proved

The Suilin loss behaves like sMAPE except near zero, where it becomes an absolute error.

The Suilin term has the properties of the sMAPE term except near zero.

  1. (a)Where |y|+|f|≥0.5|y|+|f| \ge 0.5, s~(y,f)=s(y,f)|y|+|f||y|+|f|+δ:\tilde s(y,f) = s(y,f)\,\frac{|y|+|f|}{|y|+|f|+\delta}: the sMAPE term shrunk by a factor between 5/65/6 and 1 that tends to 1 as |y|+|f||y|+|f| grows. So for actuals and forecasts large compared with δ\delta, s~\tilde s is close to ss, though no longer exactly a function of the ratio y/fy/f; on counts, where every actual is 0 or at least 1, this case covers every pair except a zero actual with |f|<0.5|f| < 0.5.
  2. (b)Where |y|+|f|<0.5|y|+|f| < 0.5, s~(y,f)=|y−f|/(0.5+δ)\tilde s(y,f) = |y-f|/(0.5+\delta), an absolute error: a zero forecast of a zero actual scores 0, and a forecast ff with |f|<0.5|f| < 0.5 of a zero actual scores |f|/(0.5+δ)|f|/(0.5+\delta), where the sMAPE term gives every non-zero forecast of a zero actual its largest value, 1.
  3. (c)0≤s~<10 \le \tilde s < 1 everywhere, so the expected Suilin loss of every forecast is finite.
Proof 8 steps
  1. Which branch of the maximum. ℓ+δ≥0.5+δ\ell + \delta \ge 0.5 + \delta exactly when ℓ≥0.5\ell \ge 0.5. So the denominator of s~\tilde s is ℓ+δ\ell + \delta when ℓ≥0.5\ell \ge 0.5, and 0.5+δ0.5 + \delta when ℓ<0.5\ell < 0.5.
  2. (a), the identity. When ℓ≥0.5\ell \ge 0.5, ℓ>0\ell > 0, so s=|y−f|/ℓs = |y-f|/\ell and |y−f|=sℓ|y-f| = s\,\ell. By step 1, s~=|y−f|/(ℓ+δ)=sℓ/(ℓ+δ)\tilde s = |y-f|/(\ell+\delta) = s\,\ell/(\ell+\delta).
  3. (a), the factor. The function ℓ↦ℓ/(ℓ+δ)=1−δ/(ℓ+δ)\ell \mapsto \ell/(\ell+\delta) = 1 - \delta/(\ell+\delta) is increasing in ℓ\ell, below 1, and tends to 1 as ℓ\ell grows. At ℓ=0.5\ell = 0.5 it equals 0.5/0.6=5/60.5/0.6 = 5/6. So for ℓ≥0.5\ell \ge 0.5 it lies in [5/6,1)[5/6, 1).
  4. (a), close to ss, but not a function of the ratio. By steps 2 and 3, 0≤s−s~=sδ/(ℓ+δ)≤δ/(ℓ+δ)0 \le s - \tilde s = s\,\delta/(\ell+\delta) \le \delta/(\ell+\delta), since s≤1s \le 1 (step 7). So s~\tilde s is within δ/(ℓ+δ)\delta/(\ell+\delta) of ss, which shrinks as ℓ\ell grows. But s~\tilde s depends on the scale: (y,f)=(1,2)(y,f) = (1,2) and (10,20)(10,20) have the same ratio and the same s=1/3s = 1/3, while s~=1/3.1\tilde s = 1/3.1 and 10/30.110/30.1. So s~\tilde s is not a function of y/fy/f alone.
  5. (a), counts. If y≥1y \ge 1, then ℓ≥1≥0.5\ell \ge 1 \ge 0.5. If y=0y = 0, then ℓ=|f|\ell = |f|, which is at least 0.5 unless |f|<0.5|f| < 0.5. So on counts only a zero actual with |f|<0.5|f| < 0.5 has ℓ<0.5\ell < 0.5.
  6. (b). When ℓ<0.5\ell < 0.5, by step 1, s~=|y−f|/(0.5+δ)\tilde s = |y-f|/(0.5+\delta). For y=f=0y = f = 0 this is 0. For y=0y = 0 and |f|<0.5|f| < 0.5 it is |f|/(0.5+δ)|f|/(0.5+\delta). And s(0,f)=|f|/|f|=1s(0,f) = |f|/|f| = 1 for f≠0f \ne 0, the largest value of ss (step 7).
  7. (c). |y−f|≤|y|+|f|=ℓ|y - f| \le |y| + |f| = \ell (the triangle inequality), so s≤1s \le 1. When ℓ≥0.5\ell \ge 0.5, s~≤ℓ/(ℓ+δ)<1\tilde s \le \ell/(\ell+\delta) < 1 (step 3). When ℓ<0.5\ell < 0.5, s~≤ℓ/(0.5+δ)<0.5/0.6<1\tilde s \le \ell/(0.5+\delta) < 0.5/0.6 < 1. And s~≥0\tilde s \ge 0 because it is a ratio of a non-negative number to a positive one. A quantity between 0 and 1 has an expectation between 0 and 1, whatever the joint distribution of actuals and forecasts, so the expected Suilin loss is between 0 and 200.

So the Suilin term is the sMAPE term shrunk by a factor between 5/6 and 1 wherever |y|+|f|≥0.5|y| + |f| \ge 0.5, and an absolute error where |y|+|f|<0.5|y| + |f| < 0.5: it differs from sMAPE in kind only near zero. ∎

Its formulae were also checked on numbers: E31 in the record.

Supported by numerical check C16

Computed on actual distributions, the Suilin target stays at the sMAPE target away from zero and moves near zero, as this claim's two cases imply.

Earlier work and other claims

  • Extends Hewamalage et al. (2021). Hewamalage, Bergmeir and Bandara (2021) evaluate forecasts with the Suilin loss, Suilin's modification of sMAPE for series with values near zero; claim C17 shows that it keeps sMAPE's behaviour away from zero and becomes an absolute error near zero.
Proved · checked independently by an AI agent · checked independently by an AI agent · tested as plannedRecord
Claim
C17 · formal · principal · extends earlier work
Plan
Test planned in lab plan PL12, recorded 18 November 2026
Evidence
E30, E31 in the record
Check: confirmed
By an AI agent (proof checking), full. Derived from the statement alone, then compared with proof-msmape.md. Write t = |y|+|f|. The denominator max(t+eps, 0.5+eps) is t+eps exactly when t >= 0.5, and the two branches agree at t = 0.5 (both 0.6), so m is continuous there. (a) For t >= 0.5, t > 0, so |y-f| = s t and m = s t/(t+eps); the factor t/(t+eps) = 1 - eps/(t+eps) is increasing, equals 5/6 at t = 0.5, is strictly below 1 and tends to 1, so it lies in [5/6, 1): the bound 5/6 is attained, 1 is not. s - m = s eps/(t+eps) <= eps/(t+eps), so m is close to s when t is large compared with eps. m is not a function of y/f: (1,2) and (10,20) have the same ratio and the same s = 1/3, but m = 1/3.1 and 10/30.1. On counts, y >= 1 gives t >= 1, and y = 0 gives t = |f|, so only a zero actual with |f| < 0.5 falls outside this case; this holds for forecasts of any sign and any real value. (b) For t < 0.5, m = |y-f|/0.6; m(0,0) = 0; m(0,f) = |f|/0.6 for |f| < 0.5; s(0,f) = |f|/|f| = 1 for every f != 0, and 1 is the largest value of s because |y-f| <= |y|+|f|. (c) m >= 0, and |y-f| <= t < t+eps <= denominator gives m < 1 strictly in both branches, with supremum 1 not attained (opposite signs, t large); a bounded term has a finite expectation under every joint distribution of actuals and forecasts, so expected msMAPE lies in [0, 200). Checked with exact rational arithmetic on 400000 random pairs of mixed signs and scales, including zeros and the boundary t = 0.5: no violation. Every step of proof-msmape.md (1 to 6) holds. Three clauses of the recorded statement are true but have no step in the proof, whose restated Statement omits them: that m is close to s for values large compared with eps (needs the bound s - m <= eps/(t+eps)), that m is no longer exactly a function of y/f (needs one counterexample such as the one above), and that 1 is the largest value of the sMAPE term (follows from the triangle inequality used in step 6 but is not said). The opening sentence, that m has the properties of the sMAPE term except near zero, is a summary and holds only as parts (a) to (c) qualify it: exact dependence on the ratio y/f alone is lost at every scale, not only near zero, and m never reaches the value 1 that s reaches for opposite signs or a zero actual.
Check: confirmed
By an AI agent (proof checking, version 3), full. Derived independently. With $\ell=|y|+|f|$, the denominator $\max(\ell+\delta,0.5+\delta)$ is $\ell+\delta$ exactly when $\ell\ge0.5$. (a) Then $\ell>0$, $|y-f|=s\ell$, $\tilde s=s\,\ell/(\ell+\delta)$, and $\ell/(\ell+\delta)$ is increasing from $0.5/0.6=5/6$ towards 1, never reaching 1. It depends on scale: $(1,2)$ and $(10,20)$ have equal $s$ but different $\tilde s$. On counts, $y\ge1$ gives $\ell\ge1$, and $y=0$ gives $\ell=|f|$, so only a zero actual with $|f|<0.5$ falls outside. (b) For $\ell<0.5$, $\tilde s=|y-f|/0.6$; $(0,0)$ scores 0, $(0,f)$ scores $|f|/0.6$, and $s(0,f)=1=\max s$ for $f\ne0$. (c) In case (a), $\tilde s\le s\cdot\ell/(\ell+\delta)<1$ since $s\le1$. In case (b), $\tilde s\le\ell/0.6<5/6$. And $\tilde s\ge0$, so every expectation is finite. The proof (proof-suilin.md, first proof) matches each step. Every part holds as worded.
Support score
SupportS 100 of 100. The score checks the record, not the science.
Record
This claim on the record page
To cite it
The page of this claim

Term this claim uses

msMAPE term sν(y,f)s_\nu(y,f)
The contribution of one actual y≥0y \ge 0 and one forecast f≥0f \ge 0 to the sparse-proof sMAPE (msMAPE) of Smyl (2025), as Kolassa (2026, Definition 4) states it, for a small constant ν>0\nu > 0 (Kolassa writes it mm, which is the median here): sν(y,f)=|f−y|f+max⁡{y,ν},s_\nu(y,f) = \frac{|f-y|}{f+\max\{y, \nu\}}, which is 0 when y=f=0y = f = 0; msMAPE is 200 times its mean over a test set (Kolassa writes the factor 2 into the term). It replaces the actual in the denominator of the sMAPE term by ν\nu when the actual is below ν\nu.

Claim C20. msMAPE is sMAPE except for small actuals proved

Smyl's msMAPE equals sMAPE for actuals of at least its constant ν. Below ν its denominator stops shrinking. A zero actual then no longer gives every non-zero forecast the worst score, and the expected score has no jump at the zero forecast.

The msMAPE term sνs_\nu is the sMAPE term ss except for actuals below ν\nu. For y≥0y \ge 0, f≥0f \ge 0 and ν>0\nu > 0:

  1. (a)Where y≥νy \ge \nu, sν(y,f)=s(y,f)s_\nu(y,f) = s(y,f); so when every actual is at least ν\nu, msMAPE is sMAPE and has the same target.
  2. (b)Where y<νy < \nu, sν(y,f)=|f−y|/(f+ν)s_\nu(y,f) = |f-y|/(f+\nu): for a zero actual it is f/(f+ν)f/(f+\nu), which is 0 at f=0f = 0 and rises towards 1 as ff grows, where the sMAPE term is 1 for every f>0f > 0.
  3. (c)0≤sν≤10 \le s_\nu \le 1, with sν<1s_\nu < 1 where y<νy < \nu; and for every yy, sν(y,f)s_\nu(y,f) is continuous in ff at f=0f = 0, so the expected msMAPE term has no jump at the zero forecast, where the expected sMAPE term jumps when zero actuals have positive probability.
  4. (d)On counts with ν≤1\nu \le 1, sνs_\nu differs from ss only for zero actuals.
Proof 6 steps
  1. (a). When y≥νy \ge \nu, max⁡{y,ν}=y\max\{y,\nu\} = y, and y>0y > 0 because ν>0\nu > 0. So sν=|f−y|/(f+y)s_\nu = |f-y|/(f+y), which is s(y,f)s(y,f) for non-negative yy and ff. If every actual is at least ν\nu, the two measures are then the same function of the forecast on every test set and under every distribution of such actuals, so they have the same minimisers.
  2. (b). When y<νy < \nu, max⁡{y,ν}=ν\max\{y,\nu\} = \nu, so sν=|f−y|/(f+ν)s_\nu = |f-y|/(f+\nu). For y=0y = 0 this is f/(f+ν)=1−ν/(f+ν)f/(f+\nu) = 1 - \nu/(f+\nu), which is 0 at f=0f = 0, increasing in ff, below 1, and tends to 1 as ff grows. The sMAPE term of a zero actual is |f|/|f|=1|f|/|f| = 1 for every f>0f > 0.
  3. (c), the bounds. sν≥0s_\nu \ge 0 because its numerator is non-negative and its denominator is at least ν>0\nu > 0. Where y≥νy \ge \nu, sν=s≤1s_\nu = s \le 1, since |f−y|≤f+y|f-y| \le f+y. Where y<νy < \nu: if f≥yf \ge y, |f−y|=f−y≤f<f+ν|f-y| = f-y \le f < f+\nu; if f<yf < y, |f−y|=y−f≤y<ν≤f+ν|f-y| = y-f \le y < \nu \le f+\nu. So sν<1s_\nu < 1 there.
  4. (c), no jump at zero. The denominator f+max⁡{y,ν}f+\max\{y,\nu\} is at least ν>0\nu > 0 for every f≥0f \ge 0, so sν(y,f)s_\nu(y,f) is a continuous function of ff on f≥0f \ge 0, at f=0f = 0 too. Since 0≤sν≤10 \le s_\nu \le 1 (step 3), the expected term is continuous in ff as well, by bounded convergence. The sMAPE term of a zero actual is 0 at f=0f = 0 and 1 for f>0f > 0; so when zero actuals have probability p0>0p_0 > 0, the expected sMAPE term jumps by p0p_0 at f=0f = 0.
  5. (d). A count is 0 or at least 1. With ν≤1\nu \le 1, a count of at least 1 is at least ν\nu, so sν=ss_\nu = s by step 1. A zero count is below ν\nu, where sν(0,f)=f/(f+ν)s_\nu(0,f) = f/(f+\nu) and s(0,f)=1s(0,f) = 1 for f>0f > 0 (step 2).

So the msMAPE term is the sMAPE term for actuals of at least ν\nu. Below ν\nu its denominator no longer shrinks with the actual: a zero actual scores f/(f+ν)f/(f+\nu), and the expected msMAPE has no jump at the zero forecast. ∎

Its formulae were also checked on numbers: E31 in the record.

Supported by numerical check C21

Computed on intermittent demand, msMAPE needs a larger share of zeros than sMAPE before its target is 0, in line with this claim (b): a zero actual no longer gives every non-zero forecast the worst score.

Earlier work and other claims

  • Extends Kolassa (2026). Kolassa (2026, Proposition 3) shows that the minimisers of expected msMAPE on counts are integers; claim C20 shows where the msMAPE term equals the sMAPE term and that it removes the jump at the zero forecast.
  • Extends Smyl (2025). Smyl (2025) proposed msMAPE, the sparse-proof sMAPE, as Kolassa (2026) reports; Smyl's article was not read. Claim C20 states properties of its term as Kolassa (2026, Definition 4) defines it.
Proved · checked independently by an AI agent · tested as plannedRecord
Claim
C20 · formal · principal · extends earlier work
Plan
Test planned in lab plan PL12, recorded 18 November 2026
Evidence
E37, E31 in the record
Check: confirmed
By an AI agent (proof checking, version 3), full. Derived independently from the definition of TM38 (I did not check Kolassa's text; the check is of the statement about the term as TM38 defines it). (a) For $y\ge\nu>0$, $\max\{y,\nu\}=y$ and $p=|f-y|/(f+y)=s$ for $y,f\ge0$. So with every actual at least $\nu$ the two expected losses coincide for $f\ge0$; negative forecasts score 1 under sMAPE on positive actuals and cannot be better, so the targets agree. (b) For $y<\nu$, $p=|f-y|/(f+\nu)$; at $y=0$ it is $f/(f+\nu)=1-\nu/(f+\nu)$: 0 at $f=0$, increasing, tending to 1, whereas $s(0,f)=1$ for $f>0$. (c) $p\ge0$; for $y\ge\nu$, $p=s\le1$; for $y<\nu$, $f\ge y$ gives $f-y\le f<f+\nu$ and $f<y$ gives $y-f<\nu\le f+\nu$, so $p<1$. The denominator is at least $\nu>0$, so $p$ is continuous in $f\ge0$, and bounded convergence makes the expectation continuous. For sMAPE, $\mathrm E[s(Y,0)]=1-P(Y=0)$ while $\lim_{f\downarrow0}\mathrm E[s(Y,f)]=1$: a jump of $P(Y=0)$. (d) A count below $\nu\le1$ is 0. The proof (proof-suilin.md, second proof) matches step by step. Every part holds as worded.
Support score
SupportS 99 of 100. The score checks the record, not the science.
Record
This claim on the record page
To cite it
The page of this claim

Terms this claim uses

Suilin target
The value or values of ff that minimise the expected Suilin term for a given distribution of the actual: the functional the Suilin loss rewards, as the sMAPE target is the one sMAPE rewards.
Poisson counts
Poisson outcomes with means 0.3, 0.5, 1, 1.5, 3 and 10: counts whose share of zeros falls from 0.74 to almost 0.

Numerical check C16. The Suilin target beside the sMAPE target evidence

Computed on actual distributions, the forecast the Suilin loss rewards is almost the same as sMAPE's away from zero and on counts. It differs near zero, where the Suilin loss needs a larger share of zeros before it rewards a zero forecast.

Computed from the distributions themselves, the Suilin target differs little from the sMAPE target away from zero and on counts, and differs near zero: it is within 0.3% of the sMAPE target on the six distributions with median about 100, and the same as the sMAPE target on every Poisson counts distribution studied; on lognormal actuals with median 1 and σ=1\sigma = 1 it is 0.985 instead of 1, and the share of zeros at which it switches to the forecast 0 is 0.470 instead of sMAPE's 0.394, while at median 100 the two shares are 0.3937 and 0.3940.

Evidence

The Suilin loss beside sMAPE: near zero, the two targets as a share of the median against the share of zeros, for lognormal outcomes at medians 1 and 100, with…
A53 The Suilin loss beside sMAPE: near zero, the two targets as a share of the median against the share of zeros, for lognormal outcomes at medians 1 and 100, with the zero thresholds at median 1; away from zero, the relative difference of the Suilin target from the sMAPE target for each distribution; on Poisson counts, the two targets against the mean. The figure was drawn when the paper called the Suilin loss msMAPE, and labels it so. Drawn by lab run RN11 from the recorded results of RN10 (lab/runs/RN11/figures/msmape_targets.png).
E16.T1 · The Suilin target beside the sMAPE target, with their quantile levels and the relative difference of the Suilin target from the sMAPE target
Rowsettingtarget (measure: sMAPE)target (measure: Suilin)quantile level of the target (measure: sMAPE)quantile level of the target (measure: Suilin)relative difference of the Suilin target from the sMAPE targetshare of zero outcomes
r1lognormal sigma=0.51001000.50.50.00000
r2lognormal sigma=110099.980.50.4999-0.00020
r3inverse Gaussian shape=1014.0213.980.43870.438-0.00270
r4inverse Gaussian shape=10066.5366.520.49290.4929-0.00010
r5log-skewed left sd=1 skew=-1128.3128.20.53620.5361-0.00020
r6log-skewed right sd=1 skew=+177.9677.960.46380.46380.00000
r7lognormal median=1 sigma=110.98460.50.4938-0.01540
r8lognormal median=1 sigma=1 with zeros p0=0.210.90540.60.5683-0.09460.2
r9lognormal median=1 sigma=1 with zeros p0=0.400.17020.40.4230.4
r10lognormal median=1 sigma=1 with zeros p0=0.6000.60.600.6
r11lognormal median=1 sigma=1 with zeros p0=0.8000.80.800.8
r12lognormal median=100 sigma=110099.980.50.4999-0.00020
r13lognormal median=100 sigma=1 with zeros p0=0.210099.890.60.5997-0.00110.2
r14lognormal median=100 sigma=1 with zeros p0=0.4000.40.400.4
r15lognormal median=100 sigma=1 with zeros p0=0.6000.60.600.6
r16lognormal median=100 sigma=1 with zeros p0=0.8000.80.800.8
r17Poisson mean=0.3000.74080.740800.7408
r18Poisson mean=0.5000.60650.606500.6065
r19Poisson mean=1110.73580.735800.3679
r20Poisson mean=1.5220.80880.808800.2231
r21Poisson mean=3330.64720.647200.0498
r22Poisson mean=1010100.5830.58300.0000
E16.T2 · The share of zeros at which each measure's target switches to the forecast 0, for lognormal outcomes with sigma 1
Rowsettingzero threshold (measure: sMAPE)zero threshold (measure: Suilin)
r1lognormal median=1 sigma=10.3940.47
r2lognormal median=100 sigma=10.3940.3937

Everything recorded for this evidence: E16 in the record.

Earlier work and other claims

  • Supports claim C17. Computed on actual distributions, the Suilin target stays at the sMAPE target away from zero and moves near zero, as claim C17's two cases imply.
estimated as plannedRecord
Claim
C16 · descriptive · secondary · no earlier result found
Plan
Estimate planned in lab plan PL11, recorded 17 November 2026
Evidence
E16 in the record
Scope
data: outcome distributions given exactly (no data); conditions: positive distributions at median about 100, lognormal with zeros at medians 1 and 100, Poisson counts with means 0.3 to 10
Support score
SupportS 99 of 100. The score checks the record, not the science.
Record
This claim on the record page

Terms this claim uses

intermittent demand settings
Demand with many zeros. Counts: Poisson with means 0.1, 0.3, 0.5, 0.7, 1, 1.5 and 3 (share of zeros from 0.90 to 0.05) and, overdispersed, negative binomial with size 0.5 and means 0.5, 1 and 2 (share of zeros 0.71, 0.58 and 0.45). Non-integer demand: lognormal demand sizes (median 1, σ=1\sigma = 1) replaced by 0 with probability 0.2, 0.4, 0.6 or 0.8.
msMAPE target
The value or values of ff that minimise the expected msMAPE term for a given distribution of the actual and a given constant ν\nu: the functional msMAPE rewards, as the sMAPE target is the one sMAPE rewards.

Numerical check C21. What msMAPE rewards on intermittent demand evidence

On counts, msMAPE with ν = 0.1 and the Suilin loss reward the same forecast as sMAPE. With ν = 0.5 or 1 it is 0 less often. On non-integer demand the zero forecast needs a larger share of zeros: sMAPE 0.394; msMAPE 0.419, 0.519, 0.565; Suilin 0.470.

The forecasts that sMAPE, msMAPE with ν=0.1\nu = 0.1, 0.5 and 1, and the Suilin loss reward were computed from the distributions themselves on the 14 intermittent demand settings.

  1. (a)On each of the 10 count distributions, the msMAPE target with ν=0.1\nu = 0.1 and the Suilin target equal the sMAPE target. With ν=0.5\nu = 0.5 the msMAPE target is 1 instead of 0 on Poisson counts with mean 0.7 and on negative binomial counts with mean 2, and 1 instead of 2 on Poisson counts with mean 1.5. With ν=1\nu = 1 it is also 1 instead of 0 on Poisson counts with mean 0.5 and on negative binomial counts with mean 1.
  2. (b)For lognormal demand sizes with zeros, the share of zeros at which the target switches to the forecast 0 is 0.394 for sMAPE, 0.419 for msMAPE with ν=0.1\nu = 0.1, 0.470 for the Suilin loss, 0.519 for msMAPE with ν=0.5\nu = 0.5 and 0.565 with ν=1\nu = 1.
  3. (c)In test sets of 12, 52 and 365 actuals, the forecast with the lowest score is 0 no more often under msMAPE than under sMAPE, and no more often under a larger ν\nu. With 52 actuals from Poisson counts with mean 0.7, it is 0 in 0.6575 of the test sets under sMAPE, 0.523 with ν=0.1\nu = 0.1, 0.1525 with ν=0.5\nu = 0.5 and 0.0265 with ν=1\nu = 1.

Evidence

The forecast with the lowest expected score (the target) against the share of zero actuals, on intermittent demand: one panel per family of settings (Poisson c…
A56 The forecast with the lowest expected score (the target) against the share of zero actuals, on intermittent demand: one panel per family of settings (Poisson counts, negative binomial counts, lognormal demand sizes with zeros), one line per measure (sMAPE, msMAPE with nu = 0.1, 0.5 and 1, the Suilin loss). Drawn by lab run RN15 from its own results (lab/runs/RN15/figures/msmape_intermittent.png).
E34.T1 · The forecast each measure rewards (its target, computed from the distribution itself) on each intermittent demand setting, beside the setting's share of zeros, median, mode and mean; the measures are sMAPE, msMAPE with nu = 0.1, 0.5 and 1, and the Suilin loss
Rowsettingshare of zero outcomesmedian of the outcomesmode of the outcomesmean of the outcomestarget (measure: sMAPE)target (measure: msMAPE nu=0.1)target (measure: msMAPE nu=0.5)target (measure: msMAPE nu=1)target (measure: Suilin)
r1Poisson mean=0.10.9048000.100000
r2Poisson mean=0.30.7408000.300000
r3Poisson mean=0.50.6065000.500010
r4Poisson mean=0.70.4966100.700110
r5Poisson mean=10.367910111111
r6Poisson mean=1.50.2231111.522112
r7Poisson mean=30.049832333333
r8negative binomial mean=0.5 size=0.50.7071000.500000
r9negative binomial mean=1 size=0.50.577400100010
r10negative binomial mean=2 size=0.50.447210200110
r11lognormal sizes with zeros p0=0.20.20.72711.31910.9220.80210.86720.9054
r12lognormal sizes with zeros p0=0.40.40.38010.989200.77910.52710.59320.1702
r13lognormal sizes with zeros p0=0.60.600.659500000
r14lognormal sizes with zeros p0=0.80.800.329700000
E34.T2 · The share of zeros at which each measure's target switches to the forecast 0, for lognormal demand sizes (median 1, sigma 1) with zeros
Rowsettingzero threshold (measure: sMAPE)zero threshold (measure: msMAPE nu=0.1)zero threshold (measure: msMAPE nu=0.5)zero threshold (measure: msMAPE nu=1)zero threshold (measure: Suilin)
r1lognormal sizes with zeros0.3940.41910.51920.56470.47
E34.T4 · Test sets: the share of 2000 test sets of n independent actuals in which the forecast with the lowest score is 0, for each setting, n and measure
Rowsettingnshare of test sets whose minimiser is 0 (measure: sMAPE)share of test sets whose minimiser is 0 (measure: msMAPE nu=0.1)share of test sets whose minimiser is 0 (measure: msMAPE nu=0.5)share of test sets whose minimiser is 0 (measure: msMAPE nu=1)share of test sets whose minimiser is 0 (measure: Suilin)
r1Poisson mean=0.11210.99950.9960.9751
r2Poisson mean=0.15211111
r3Poisson mean=0.136511111
r4Poisson mean=0.3120.9790.9370.8310.71450.979
r5Poisson mean=0.352110.99150.91551
r6Poisson mean=0.33651110.99951
r7Poisson mean=0.5120.8480.75250.5390.3780.848
r8Poisson mean=0.5520.96850.9230.65150.2880.966
r9Poisson mean=0.5365110.84450.0671
r10Poisson mean=0.7120.60450.5150.29150.15550.6045
r11Poisson mean=0.7520.65750.5230.15250.02650.6455
r12Poisson mean=0.73650.8440.560.002500.8295
r13Poisson mean=1120.2560.22650.08950.0370.256
r14Poisson mean=1520.10150.0510.00300.1
r15Poisson mean=136500000
r16Poisson mean=1.5120.03250.030.00750.0020.0325
r17Poisson mean=1.5520.00050.0005000.0005
r18Poisson mean=1.536500000
r19Poisson mean=31200000
r20Poisson mean=35200000
r21Poisson mean=336500000
r22negative binomial mean=0.5 size=0.5120.9710.9520.8570.7210.9705
r23negative binomial mean=0.5 size=0.552110.98950.90951
r24negative binomial mean=0.5 size=0.536511111
r25negative binomial mean=1 size=0.5120.80350.780.6210.44350.808
r26negative binomial mean=1 size=0.5520.9750.96250.76750.4050.974
r27negative binomial mean=1 size=0.5365110.9750.2561
r28negative binomial mean=2 size=0.5120.5670.52650.3840.24050.5765
r29negative binomial mean=2 size=0.5520.7070.6350.30550.0680.7255
r30negative binomial mean=2 size=0.53650.9320.84750.111500.9475
r31lognormal sizes with zeros p0=0.2120.0460.0260.00550.0020.016
r32lognormal sizes with zeros p0=0.25200000
r33lognormal sizes with zeros p0=0.236500000
r34lognormal sizes with zeros p0=0.4120.46550.3740.17350.1190.303
r35lognormal sizes with zeros p0=0.4520.52350.360.0450.0120.172
r36lognormal sizes with zeros p0=0.43650.56650.207000.004
r37lognormal sizes with zeros p0=0.6120.8960.85650.6630.58050.8055
r38lognormal sizes with zeros p0=0.6520.99750.9930.8440.67650.9705
r39lognormal sizes with zeros p0=0.6365110.9990.8921
r40lognormal sizes with zeros p0=0.8120.99750.99550.97250.9590.991
r41lognormal sizes with zeros p0=0.8521110.99951
r42lognormal sizes with zeros p0=0.836511111

Everything recorded for this evidence: E34 in the record.

Earlier work and other claims

  • Supports claim C20. Computed on intermittent demand, msMAPE needs a larger share of zeros than sMAPE before its target is 0, in line with claim C20 (b): a zero actual no longer gives every non-zero forecast the worst score.
estimated as plannedRecord
Claim
C21 · descriptive · secondary · no earlier result found
Plan
Estimate planned in lab plan PL14, recorded 18 November 2026
Evidence
E34 in the record
Scope
data: outcome distributions given exactly, and test sets drawn from them; conditions: 14 settings: 10 count distributions (Poisson with means 0.1 to 3, negative binomial with size 0.5 and means 0.5 to 2) and lognormal demand sizes (median 1, sigma 1) with shares of zeros 0.2 to 0.8, msMAPE with nu = 0.1, 0.5 and 1, test sets of 12, 52 and 365 independent actuals, 2000 test sets per setting and size
Support score
SupportS 97 of 100. The score checks the record, not the science.
Record
This claim on the record page

Precision on a test set

Terms this claim uses

log-symmetric outcomes
Lognormal outcomes with σ\sigma, and log-Laplace outcomes with scale bb, each 0.25, 0.5, 1, 1.5, 2, 3, 4 or 6, median 100: ln⁡y\ln y symmetric and strictly unimodal, so that sMAPE and MAE both target the median (claim C5).
sMAPE minimiser of a test set f^n\hat f_n
The forecast that minimises the sMAPE of one test set y1,…,yny_1, \dots, y_n, f^n=arg⁡minf⁡1n∑ts(yt,f)\hat f_n = \arg\min_f \frac1n\sum_t s(y_t, f), found numerically (the sample counterpart of the sMAPE target).
relative efficiency RE(n)\mathrm{RE}(n)
For a test-set size nn and an outcome distribution where sMAPE and MAE both target the median: RE(n)=Var⁡(ln⁡m^n)/Var⁡(ln⁡f^n)\mathrm{RE}(n) = \operatorname{Var}(\ln \hat m_n) / \operatorname{Var}(\ln \hat f_n) over test sets, where m^n\hat m_n is the sample median (the MAE minimiser) and f^n\hat f_n the sMAPE minimiser of the same test set. Above 1, sMAPE's minimiser varies less; below 1, MAE's does.

Claim C15. sMAPE finds the median less precisely than the sample median key claimevidence

Even where sMAPE and MAE both reward the median, the forecast that minimises sMAPE on a test set varies more than the sample median. The loss of precision grows as the data spread out.

Where sMAPE and MAE both target the median (log-symmetric outcomes), the sMAPE minimiser of a test set varies more than the sample median for every spread and test-set size studied: its relative efficiency is below 1 and falls as the actuals spread out, at n=10000n = 10000 from 0.97 to 0.03 on lognormal actuals as σ\sigma grows from 0.25 to 6, and from 1.00 to 0.34 on log-Laplace actuals as bb grows from 0.25 to 6, close to the large-sample values.

Evidence

lognormal outcomes: relative efficiency of the sMAPE minimiser of a test set against the sample median, against test-set size, one panel per spread, with its 9…
A50 lognormal outcomes: relative efficiency of the sMAPE minimiser of a test set against the sample median, against test-set size, one panel per spread, with its 99% band and the large-sample value as a dashed line; below 1 the sMAPE minimiser varies more than the sample median. Drawn by lab run RN7 from its own results (lab/runs/RN7/figures/efficiency_lognormal.png).
log-Laplace outcomes: relative efficiency of the sMAPE minimiser of a test set against the sample median, against test-set size, one panel per spread, with its…
A51 log-Laplace outcomes: relative efficiency of the sMAPE minimiser of a test set against the sample median, against test-set size, one panel per spread, with its 99% band and the large-sample value as a dashed line; below 1 the sMAPE minimiser varies more than the sample median. Drawn by lab run RN7 from its own results (lab/runs/RN7/figures/efficiency_log-Laplace.png).
E15.T1 · Relative efficiency of the sMAPE minimiser of a test set against the sample median, at test-set sizes 6, 12, 100, 1000 and 10000, beside its large-sample value
Rowsettingrelative efficiency (test-set size: 6)relative efficiency (test-set size: 12)relative efficiency (test-set size: 100)relative efficiency (test-set size: 1000)relative efficiency (test-set size: 10000)asymptotic relative efficiency
r1lognormal sigma=0.250.60590.70260.89960.96620.97010.9696
r2lognormal sigma=0.50.6110.70830.88020.8850.89060.8881
r3lognormal sigma=10.60670.65550.65240.66590.65650.6601
r4lognormal sigma=1.50.54840.51230.45950.46330.450.454
r5lognormal sigma=20.5010.43920.3230.30770.30430.3073
r6lognormal sigma=30.43810.36460.20580.15570.15130.1474
r7lognormal sigma=40.41480.32720.15610.0890.07890.0778
r8lognormal sigma=60.39330.30270.12290.05080.03170.0279
r9log-Laplace b=0.250.70490.82680.97690.99180.99690.9969
r10log-Laplace b=0.50.69890.8190.94480.97790.97530.9765
r11log-Laplace b=10.6670.68870.84790.87630.88530.8953
r12log-Laplace b=1.50.57660.59470.69890.77890.78040.8035
r13log-Laplace b=20.51790.50360.6120.6830.70790.7206
r14log-Laplace b=30.46720.43650.45860.52570.56280.5902
r15log-Laplace b=40.43530.37640.34960.42670.48030.4968
r16log-Laplace b=60.41670.34920.26670.28960.34410.3753

Everything recorded for this evidence: E15 in the record.

Explained by claim C19

This claim found by simulation that the sMAPE minimiser of a test set varies more than the sample median, the more so the wider the spread. Claim C19 gives the limiting variances of both and the rates at which their ratio falls.

Earlier work and other claims

  • Supports Svetunkov (2026b), C11. C11 found sMAPE's choices in short test sets spreading more than MAE's; claim C15 measures the loss of precision for test sets of 6 to 10000 outcomes.
  • Supports Svetunkov (2026a), C6. C6 recommends against sMAPE for point forecasts; claim C15 shows that even where it targets the median it estimates it less precisely than the sample median.
  • Supports Kolassa (2026). Kolassa (2026) concludes that sMAPE should not be used; claim C15 shows that even where it targets the median it estimates it less precisely than the sample median.
  • Qualifies claim C12. Where claim C12 makes the median the target, the forecast that minimises sMAPE on a test set finds it less precisely than the sample median.
estimated as plannedRecord
Claim
C15 · descriptive · principal · no earlier result found
Plan
Estimate planned in lab plan PL2, recorded 16 November 2026
Evidence
E15 in the record
Scope
data: test sets of independent draws from known distributions; conditions: lognormal and log-Laplace outcomes, spread 0.25 to 6, test sets of 6 to 10000 outcomes
Limitations
L2
Support score
SupportS 89 of 100. The score checks the record, not the science.
Record
This claim on the record page
To cite it
The page of this claim

Claim C19. The sMAPE minimiser in large samples proved

Where the log of the actual is symmetric with one peak, the forecast that minimises sMAPE on a test set is asymptotically normal about the median. It is asymptotically as precise as the sample median for narrow distributions and loses its precision as the actuals spread out.

Let ln⁡y\ln y have a density gg that is symmetric about ln⁡m\ln m, unimodal (g(ln⁡m+v1)≥g(ln⁡m+v2)g(\ln m + v_1) \ge g(\ln m + v_2) whenever |v1|≤|v2||v_1| \le |v_2|, strictly on a set of positive probability) and continuous at ln⁡m\ln m. Write x=ln⁡y−ln⁡mx = \ln y - \ln m, A=E[sech2⁡(x/2)tanh⁡(|x|/2)]A = \mathrm E[\operatorname{sech}^2(x/2)\tanh(|x|/2)] and B=E[sech4⁡(x/2)]B = \mathrm E[\operatorname{sech}^4(x/2)].

  1. (a)A<2g(ln⁡m)A < 2g(\ln m).
  2. (b)Over test sets of nn independent actuals, the sMAPE minimiser of a test set f^n\hat f_n (a forecast with the lowest sMAPE of the test set) tends to mm in probability, and n(ln⁡f^n−ln⁡m)\sqrt n(\ln\hat f_n - \ln m) is asymptotically normal with mean 0 and variance B/(2g(ln⁡m)−A)2B/(2g(\ln m) - A)^2; for the sample median the variance is 1/(4g(ln⁡m)2)1/(4g(\ln m)^2). The ratio of these two limiting variances, the asymptotic relative efficiency of the sMAPE minimiser to the sample median, is ARE=(2g(ln⁡m)−A)24g(ln⁡m)2B;\mathrm{ARE} = \frac{(2g(\ln m) - A)^2}{4\,g(\ln m)^2\,B}; that the relative efficiency of finite test sets tends to it is not shown here, since the variances of finite test sets need not tend to those of the limits.
  3. (c)For lognormal actuals with standard deviation σ\sigma of ln⁡y\ln y: ARE→1\mathrm{ARE} \to 1 as σ→0\sigma \to 0 and AREσ3→6(ln⁡2)22π≈7.23\mathrm{ARE}\,\sigma^3 \to 6(\ln 2)^2\sqrt{2\pi} \approx 7.23 as σ→∞\sigma \to \infty. For log-Laplace actuals with scale bb of ln⁡y\ln y: ARE→1\mathrm{ARE} \to 1 as b→0b \to 0 and AREb→3\mathrm{ARE}\,b \to 3 as b→∞b \to \infty. So on both families the sMAPE minimiser is asymptotically as precise as the sample median for narrow distributions and asymptotically loses all of its precision relative to it as the actuals spread out.
Proof 14 steps
  1. Write xt=ln⁡yt−ln⁡mx_t = \ln y_t - \ln m for the actuals of a test set and θ=ln⁡f−ln⁡m\theta = \ln f - \ln m for a forecast ff. By claim C3 the mean sMAPE term of the test set is Mn(θ)=1n∑ttanh⁡(|xt−θ|/2)M_n(\theta) = \frac1n\sum_t\tanh(|x_t - \theta|/2), and its expectation is M(θ)=E[tanh⁡(|x−θ|/2)]M(\theta) = \mathrm E[\tanh(|x - \theta|/2)]. So θ^n=ln⁡f^n−ln⁡m\hat\theta_n = \ln\hat f_n - \ln m minimises MnM_n; a minimiser exists, because MnM_n is continuous, tends to 1 as |θ||\theta| grows, and is below 1 at each xtx_t.
  2. (a). Write κ(u)=sech2⁡(u/2)tanh⁡(|u|/2)≥0\kappa(u) = \operatorname{sech}^2(u/2)\tanh(|u|/2) \ge 0. Since κ(u)\kappa(u) is the derivative of tanh2⁡(u/2)\tanh^2(u/2) for u>0u > 0, ∫κ(u)du=2\int\kappa(u)\,du = 2. By unimodality g(ln⁡m+u)≤g(ln⁡m)g(\ln m + u) \le g(\ln m), so A=∫κ(u)g(ln⁡m+u)du≤2g(ln⁡m)A = \int\kappa(u)\,g(\ln m + u)\,du \le 2g(\ln m). A density cannot equal g(ln⁡m)g(\ln m) everywhere, and g(ln⁡m+u)g(\ln m + u) does not increase as |u||u| grows; so g(ln⁡m+u)<g(ln⁡m)g(\ln m + u) < g(\ln m) for every |u||u| beyond some point, where κ>0\kappa > 0. So the inequality is strict.
  3. The expected loss is lowest at 0 only. M(θ)M(\theta) is the expected sMAPE term of the forecast meθm\,e^\theta. By claim C5, M(θ)>M(0)M(\theta) > M(0) for every θ≠0\theta \ne 0. The function tanh⁡(|u|/2)\tanh(|u|/2) has slope at most 12\tfrac12, so MM and every MnM_n change by at most 12|θ1−θ2|\tfrac12|\theta_1 - \theta_2| between θ1\theta_1 and θ2\theta_2; in particular MM is continuous. And M(θ)→1>M(0)M(\theta) \to 1 > M(0) as |θ|→∞|\theta| \to \infty, by bounded convergence. So for every η>0\eta > 0 the lowest value of MM over |θ|≥η|\theta| \ge \eta is above M(0)M(0).
  4. MnM_n is close to MM for all θ\theta at once. For each θ\theta, Mn(θ)→M(θ)M_n(\theta) \to M(\theta) with probability 1 (law of large numbers; the terms are bounded). Take a grid of spacing γ\gamma on [−R,R][-R, R]: by the slope bound of step 3, once MnM_n is within γ\gamma of MM at the grid points it is within 2γ2\gamma of MM on all of [−R,R][-R, R]. Outside [−R,R][-R, R]: an actual with |xt|≤R/2|x_t| \le R/2 has tanh⁡(|xt−θ|/2)≥tanh⁡(R/4)\tanh(|x_t - \theta|/2) \ge \tanh(R/4) when |θ|>R|\theta| > R, so Mn(θ)M_n(\theta) is at least tanh⁡(R/4)\tanh(R/4) times the share of such actuals, which tends to P(|x|≤R/2)\mathrm P(|x| \le R/2). Since M(0)<1M(0) < 1, for RR large this is above M(0)M(0) by a fixed margin.
  5. Consistency. Mn(θ^n)≤Mn(0)M_n(\hat\theta_n) \le M_n(0), which tends to M(0)M(0). By step 4, with probability tending to 1, |θ^n|≤R|\hat\theta_n| \le R and M(θ^n)M(\hat\theta_n) is within any given margin of M(0)M(0); by step 3, |θ^n|<η|\hat\theta_n| < \eta. So θ^n→0\hat\theta_n \to 0 in probability, which is f^n→m\hat f_n \to m. This is the argument of van der Vaart (1998, Theorem 5.7).
  6. The curvature at 0. By claim C12, step 4, with c=ln⁡mc = \ln m, the second derivative of MM at 0 is g(ln⁡m)−12A=12(2g(ln⁡m)−A)g(\ln m) - \tfrac12 A = \tfrac12(2g(\ln m) - A), which is positive by step 2. The first derivative at 0 is 0, because 0 is the minimum (step 3).
  7. Asymptotic normality. For every x≠0x \ne 0, θ↦tanh⁡(|x−θ|/2)\theta \mapsto \tanh(|x - \theta|/2) has a derivative at θ=0\theta = 0, −ψ(x)-\psi(x) with ψ(x)=12sech2⁡(x/2)\psi(x) = \tfrac12\operatorname{sech}^2(x/2) for x>0x > 0 and −12sech2⁡(x/2)-\tfrac12\operatorname{sech}^2(x/2) for x<0x < 0; and x=0x = 0 has probability zero. For every xx it changes by at most 12|θ1−θ2|\tfrac12|\theta_1 - \theta_2| (step 3). MM has a second-order expansion at its minimum 0 with a positive second derivative (step 6). And θ^n\hat\theta_n minimises MnM_n and tends to 0 in probability (step 5). These are the conditions of the theorem on estimators that minimise a criterion which is Lipschitz in the parameter (van der Vaart, 1998, Theorem 5.23). It gives that nθ^n\sqrt n\,\hat\theta_n is asymptotically normal with mean 0 and variance E[ψ(x)2]\mathrm E[\psi(x)^2] divided by the square of the second derivative of MM at 0.
  8. The variance. E[ψ(x)2]=14E[sech4⁡(x/2)]=B/4\mathrm E[\psi(x)^2] = \tfrac14\mathrm E[\operatorname{sech}^4(x/2)] = B/4, and the square of the second derivative is 14(2g(ln⁡m)−A)2\tfrac14(2g(\ln m) - A)^2 (step 6). So the variance is B/(2g(ln⁡m)−A)2B/(2g(\ln m) - A)^2.
  9. The sample median. The logarithm of the sample median of the yty_t is the sample median of the ln⁡yt\ln y_t, which minimises the mean of |xt−θ||x_t - \theta|. The same theorem applies with |x−θ||x - \theta| in place of tanh⁡(|x−θ|/2)\tanh(|x - \theta|/2): the derivative at 0 is −1-1 for x>0x > 0 and +1+1 for x<0x < 0, with square 1; the criterion changes by at most |θ1−θ2||\theta_1 - \theta_2|; and the second derivative of E|x−θ|\mathrm E|x - \theta| at 0 is 2g(ln⁡m)2g(\ln m), positive because gg is largest at ln⁡m\ln m. So the variance is 1/(4g(ln⁡m)2)1/(4g(\ln m)^2).
  10. (b), the ratio. Dividing the variance of step 9 by that of step 8 gives ARE=(2g(ln⁡m)−A)2/(4g(ln⁡m)2B)\mathrm{ARE} = (2g(\ln m) - A)^2/(4g(\ln m)^2 B).
  11. (c), narrow distributions. Write ln⁡y=ln⁡m+σε\ln y = \ln m + \sigma\varepsilon with ε\varepsilon standard normal, or Laplace with scale 1 and σ=b\sigma = b; then g(ln⁡m)=h(0)/σg(\ln m) = h(0)/\sigma with hh the density of ε\varepsilon, and x=σεx = \sigma\varepsilon. As σ→0\sigma \to 0, A→0A \to 0 and B→1B \to 1 by bounded convergence, and A/(2g(ln⁡m))=σA/(2h(0))→0A/(2g(\ln m)) = \sigma A/(2h(0)) \to 0. So ARE=(1−A/(2g(ln⁡m)))2/B→1\mathrm{ARE} = (1 - A/(2g(\ln m)))^2/B \to 1.
  12. (c), wide lognormal distributions. Since ∫κ=2\int\kappa = 2 (step 2), 2g(ln⁡m)−A=∫κ(u)[g(ln⁡m)−g(ln⁡m+u)]du2g(\ln m) - A = \int\kappa(u)\,[g(\ln m) - g(\ln m + u)]\,du. For the normal density, g(ln⁡m)−g(ln⁡m+u)=(1−e−u2/(2σ2))/(σ2π)g(\ln m) - g(\ln m + u) = \big(1 - e^{-u^2/(2\sigma^2)}\big)/(\sigma\sqrt{2\pi}), and u22σ2−u48σ4≤1−e−u2/(2σ2)≤u22σ2\tfrac{u^2}{2\sigma^2} - \tfrac{u^4}{8\sigma^4} \le 1 - e^{-u^2/(2\sigma^2)} \le \tfrac{u^2}{2\sigma^2}. Integrating by parts, ∫u2κ(u)du=4∫0∞usech2⁡(u/2)du=16ln⁡2\int u^2\kappa(u)\,du = 4\int_0^\infty u\operatorname{sech}^2(u/2)\,du = 16\ln 2; and ∫u4κ(u)du\int u^4\kappa(u)\,du is finite. So 2g(ln⁡m)−A=8ln⁡2/(σ32π)+O(σ−5)2g(\ln m) - A = 8\ln 2/(\sigma^3\sqrt{2\pi}) + O(\sigma^{-5}). In the same way, with ∫sech4⁡(u/2)du=8/3\int\operatorname{sech}^4(u/2)\,du = 8/3, B=8/(3σ2π)+O(σ−3)B = 8/(3\sigma\sqrt{2\pi}) + O(\sigma^{-3}). With g(ln⁡m)=1/(σ2π)g(\ln m) = 1/(\sigma\sqrt{2\pi}), AREσ3\mathrm{ARE}\,\sigma^3 tends to 64(ln⁡2)22π⋅2π4⋅32π8=6(ln⁡2)22π\dfrac{64(\ln 2)^2}{2\pi}\cdot\dfrac{2\pi}{4}\cdot\dfrac{3\sqrt{2\pi}}{8} = 6(\ln 2)^2\sqrt{2\pi}.
  13. (c), wide log-Laplace distributions. Here g(ln⁡m+u)=e−|u|/b/(2b)g(\ln m + u) = e^{-|u|/b}/(2b), so g(ln⁡m)−g(ln⁡m+u)=(1−e−|u|/b)/(2b)g(\ln m) - g(\ln m + u) = (1 - e^{-|u|/b})/(2b), which lies between (|u|/b−u2/(2b2))/(2b)\big(|u|/b - u^2/(2b^2)\big)/(2b) and |u|/(2b2)|u|/(2b^2). Integrating by parts, ∫|u|κ(u)du=2∫0∞sech2⁡(u/2)du=4\int |u|\,\kappa(u)\,du = 2\int_0^\infty\operatorname{sech}^2(u/2)\,du = 4. So 2g(ln⁡m)−A=2/b2+O(b−3)2g(\ln m) - A = 2/b^2 + O(b^{-3}), and in the same way B=4/(3b)+O(b−2)B = 4/(3b) + O(b^{-2}). With g(ln⁡m)=1/(2b)g(\ln m) = 1/(2b), AREb\mathrm{ARE}\,b tends to 4b4⋅b2⋅3b4⋅b=3\dfrac{4}{b^4}\cdot b^2\cdot\dfrac{3b}{4}\cdot b = 3.

So the sMAPE minimiser of a test set is asymptotically normal about the median, with a variance that the density at the median and two expectations give. On lognormal and log-Laplace actuals it is asymptotically as precise as the sample median for narrow distributions and asymptotically loses all of its precision relative to it as the actuals spread out, at the rates σ−3\sigma^{-3} and b−1b^{-1}. ∎

Rests on Vaart (1998), Theorems 5.7 and 5.23

Its formulae were also checked on numbers: E33 in the record.

Earlier work and other claims

  • Explains claim C15. Claim C15 found by simulation that the sMAPE minimiser of a test set varies more than the sample median, the more so the wider the spread. Claim C19 gives the limiting variances of both and the rates at which their ratio falls.
Proved · checked independently by an AI agent · tested as plannedRecord
Claim
C19 · formal · principal · no earlier result found
Plan
Test planned in lab plan PL15, recorded 18 November 2026
Evidence
E36, E33 in the record
Check: disputed
By an AI agent (proof checking, version 3), full. Derived independently, with $x=\ln Y-\ln m$, $\theta=\ln f-\ln m$, $M(\theta)=\mathrm E\tanh(|x-\theta|/2)$. (a) $k(u)=\operatorname{sech}^2(u/2)\tanh(|u|/2)$ is positive for $u\ne0$ with $\int k=2$, and $g(\ln m+u)\le g(\ln m)$ with strict inequality on a set of positive Lebesgue measure (a density cannot be constant on the whole line), so $A<2g(\ln m)$ strictly. This holds even without the "positive probability" clause (e.g. for a uniform $g$), so the hypothesis is stronger than needed. (b) Consistency: the proof's direct argument (step 3: a grid on $[-R,R]$ plus the $\tfrac12$-Lipschitz bound; outside, $M_n\ge\tanh(R/4)P_n(|x|\le R/2)$) is valid. A minimiser of $M_n$ exists because $M_n$ is continuous, tends to 1 at $\pm\infty$ and is below 1 at any data point; this is not stated. van der Vaart Thm 5.23 conditions: $|m_{\theta_1}-m_{\theta_2}|\le\tfrac12|\theta_1-\theta_2|$ (constant, square integrable); differentiable at 0 for every $x\ne0$; second-order expansion $M(\theta)=M(0)+\tfrac12V\theta^2+o(\theta^2)$ with $V=g(\ln m)-A/2>0$. I re-derived H12 step 4: writing the kernel as $-\operatorname{sign}+\operatorname{sign}\cdot\tanh^2$, $M'$ exists everywhere and is differentiable at 0 with $M''(0)=g(\ln m)-A/2$ whenever $g$ is continuous at $\ln m$, and $M(\theta)-M(0)=\int_0^\theta M'$ gives the expansion. So continuity at $\ln m$ suffices. Variance $(B/4)/(V^2)=B/(2g-A)^2$; sample median $1/(4g^2)$; ratio $(2g-A)^2/(4g^2B)$: all correct. (c) Recomputed: $\int u^2k=16\ln2$, $\int\operatorname{sech}^4(u/2)=8/3$, $\int|u|k=4$; lognormal $\mathrm{ARE}\,\sigma^3\to6(\ln2)^2\sqrt{2\pi}\approx7.226$ with relative error $O(\sigma^{-2})$; log-Laplace $2g-A=2/b^2+O(b^{-3})$, $B=4/(3b)+O(b^{-2})$, so $\mathrm{ARE}\,b=3+O(1/b)$ (the proof's orders are right; quadrature gives $b(\mathrm{ARE}\,b-3)\approx-5.6$). Both narrow limits are 1. Quadrature agrees with all four limits. The one statement I cannot confirm as worded is the appositive in (b): "The ratio of the two, the large-sample value of the [[TM9:relative efficiency]], is ARE". TM9 defines $\mathrm{RE}(n)$ as a ratio of finite-sample variances, so its "large-sample value" reads as $\lim_n\mathrm{RE}(n)$. The proof establishes only the ratio of the variances of the limiting normal distributions. Convergence in distribution does not give convergence of $n\operatorname{Var}$, and under the hypotheses of H40 the limit can fail. For example, a symmetric, decreasing, continuous $g$ with $P(|x|>t)\sim1/\ln t$ makes the sample median's variance infinite for every $n$, so $\mathrm{RE}(n)$ never approaches the finite ARE. For lognormal and log-Laplace actuals the identification is very likely true, but it would need a uniform-integrability argument that is not given. Suggested wording: "The ratio of the two asymptotic variances, the asymptotic relative efficiency (the ratio of the variances of the limiting distributions; it is the limit of the relative efficiency only where $n$ times each estimator's variance converges to its asymptotic variance, which is not shown here), is ARE = ...". The closing "as precise as ... loses all of its precision" should then say "asymptotically". With that rewording I would confirm every part. The statement was corrected, and the checker confirmed the corrected wording in a later check listed here.
Check: confirmed
By an AI agent (proof checking, version 3), full. Re-checked every part as now worded, against the revised H14. (a): $\int k=2$, $k>0$ off 0, and $g(\ln m+u)<g(\ln m)$ on a set of positive measure, so $A<2g(\ln m)$ strictly. (b): a minimiser of $M_n$ exists (now stated in the proof: $M_n$ is continuous, tends to 1 at both ends, and is below 1 at each $x_t$). The consistency argument (grid on $[-R,R]$, Lipschitz $\tfrac12$, lower bound outside) is valid. The conditions of van der Vaart Thm 5.23 hold: constant Lipschitz bound $\tfrac12$; differentiability at 0 for $x\ne0$; second-order expansion with $M''(0)=g(\ln m)-A/2>0$, which exists under continuity of $g$ at $\ln m$ by the decomposition now in H12 step 4. The asymptotic variances are $B/(2g-A)^2$ and $1/(4g^2)$, and their ratio is the stated ARE. The corrected sentence now calls this the asymptotic relative efficiency and says explicitly that convergence of the finite-sample relative efficiency (TM9) to it is not shown; that is exactly what the proof establishes. (c): recomputed $\int u^2k=16\ln2$, $\int\operatorname{sech}^4(u/2)=8/3$, $\int|u|k=4$. The limits $\mathrm{ARE}\to1$ (both narrow cases), $\mathrm{ARE}\,\sigma^3\to6(\ln2)^2\sqrt{2\pi}\approx7.226$ and $\mathrm{ARE}\,b\to3$ check by hand and by quadrature. The remainder orders in steps 11–12 are right (the log-Laplace one is a relative $O(1/b)$). The closing sentence, now with "asymptotically", follows from (c). Every part holds as worded and the proof establishes it.
Support score
SupportS 100 of 100. The score checks the record, not the science.
Record
This claim on the record page
To cite it
The page of this claim

Numerical checks

Numerical check C13. The sMAPE target on 34 distributions evidence

Computed on 34 distributions, the forecast sMAPE rewards is the median on lognormal data. It moves away from the median on skewed data, sits near a mode on two-group data, and becomes zero once enough of the data are zeros, always fewer than half.

Computed from the distributions themselves, the sMAPE target on the 34 outcome distributions of plan PL1 is the median on lognormal outcomes, lies below it when ln⁡y\ln y is skewed right and above it when skewed left (quantile levels from 0.439 to 0.556 on the unimodal positive distributions), is two forecasts, one near each mode, on equal bimodal outcomes and the larger mode's on unequal ones, lies away from zero on the side of the mean on normal outcomes straddling zero, and switches to 0 at shares of zero outcomes from 0.343 to 0.468, all below one half. On test sets the forecast sMAPE ranks first spreads widely when they are small: on lognormal outcomes with σ=1\sigma = 1, its quantile level runs from 0.16 to 0.82 (5th to 95th percentile over test sets) with 6 outcomes and from 0.47 to 0.53 with 1000.

Evidence

inverse Gaussian shape=10: for each measure (RMSE, MAE, MAPE, sMAPE), one violin per test-set size showing how the candidate the measure ranks first is spread…
A14 inverse Gaussian shape=10: for each measure (RMSE, MAE, MAPE, sMAPE), one violin per test-set size showing how the candidate the measure ranks first is spread over 1000 test sets, as its quantile level, with a boxplot of its median, quartiles and 5th and 95th percentiles and a diamond at its mean; a dashed line marks the measure's preferred candidate and dotted lines the other measures'. Drawn by lab run RN1 from its own results (lab/runs/RN1/figures/inverse_Gaussian_shape=10.png).
lognormal sigma=1: for each measure (RMSE, MAE, MAPE, sMAPE), one violin per test-set size showing how the candidate the measure ranks first is spread over 100…
A28 lognormal sigma=1: for each measure (RMSE, MAE, MAPE, sMAPE), one violin per test-set size showing how the candidate the measure ranks first is spread over 1000 test sets, as its quantile level, with a boxplot of its median, quartiles and 5th and 95th percentiles and a diamond at its mean; a dashed line marks the measure's preferred candidate and dotted lines the other measures'. Drawn by lab run RN1 from its own results (lab/runs/RN1/figures/lognormal_sigma=1.png).
lognormal sigma=0.5 with zeros p0=0.25: for each measure (RMSE, MAE, MAPE, sMAPE), one violin per test-set size showing how the candidate the measure ranks fir…
A35 lognormal sigma=0.5 with zeros p0=0.25: for each measure (RMSE, MAE, MAPE, sMAPE), one violin per test-set size showing how the candidate the measure ranks first is spread over 1000 test sets, as its quantile level, with a boxplot of its median, quartiles and 5th and 95th percentiles and a diamond at its mean; a dashed line marks the measure's preferred candidate and dotted lines the other measures'. Drawn by lab run RN1 from its own results (lab/runs/RN1/figures/lognormal_sigma=0.5_with_zeros_p0=0.25.png).

View all 34 figures

E13.T1 · The sMAPE target and related values derived from each distribution: the lowest local minimum of expected sMAPE (rank 1), its quantile level and expected sMAPE, its one-step approximation, the pull from above minus the pull from below at the median, the share of zeros at which the target switches to 0, the quantile level of the log-mode, and the MAPE target
Rowsettingderived local minimum of expected sMAPE (rank: 1)quantile level of a derived local minimum of expected sMAPE (rank: 1)expected sMAPE at a derived local minimum (rank: 1)one-step quantile level of the sMAPE targetpull from above minus pull from below at the medianzero threshold of sMAPEquantile level of the log-modemedian of the outcome distributionmean of the outcome distribution
r1inverse Gaussian shape=100095.270.499924.360.4999-0.00010.46760.49895.27100
r2inverse Gaussian shape=20080.140.497848.680.4978-0.00190.43070.482280.43100
r3inverse Gaussian shape=10066.530.492961.970.4929-0.00520.40830.459767.58100
r4inverse Gaussian shape=5048.860.48275.050.482-0.01140.38450.423351.42100
r5inverse Gaussian shape=2025.90.457888.960.457-0.02240.3570.372630.11100
r6inverse Gaussian shape=1014.020.438795.750.4362-0.030.34260.347117.85100
r7log-skewed left sd=0.5 skew=-21190.521132.310.52110.01970.45611116.6109.9
r8log-skewed left sd=0.5 skew=-1110.20.51336.740.5130.01170.44940.6472108.5111.3
r9log-skewed left sd=0.5 skew=-0.5105.10.506837.960.50680.00610.44760.5681104.2112.2
r10log-skewed right sd=0.5 skew=+284.020.478932.310.4789-0.01970.4561085.78121.3
r11log-skewed right sd=0.5 skew=+190.720.48736.740.487-0.01170.44940.352892.13116.3
r12log-skewed right sd=0.5 skew=+0.595.130.493237.960.4932-0.00610.44760.431995.93114.6
r13lognormal sigma=0.51000.538.380.50.00000.44690.5100113.3
r14log-skewed left sd=1 skew=-2151.20.556257.040.55720.04560.41681135.9135.9
r15log-skewed left sd=1 skew=-1128.30.536266.420.53650.02610.40040.6472117.8146
r16log-skewed left sd=1 skew=-0.5113.90.519169.050.51920.01330.39570.5681108.7153.7
r17log-skewed right sd=1 skew=+266.140.443857.040.4428-0.04560.4168073.58
r18log-skewed right sd=1 skew=+177.960.463866.420.4635-0.02610.40040.352884.88216.5
r19log-skewed right sd=1 skew=+0.587.790.480969.050.4808-0.01330.39570.431992.03182.7
r20lognormal sigma=11000.569.940.50.00000.3940.5100164.9
r21bimodal equal modes43.490.374283.840.50.00000.36740.25100159.2
r22bimodal left mode higher39.840.406365.210.4042-0.08390.40260.32544.22122.8
r23bimodal right mode higher2510.593765.210.59580.08390.40260.675226.1195.6
r24normal straddling zero mean=0-0.7530.2257133.40.00000
r25normal straddling zero mean=0.50.98380.6858104.50.50.5
r26normal straddling zero mean=11.2870.612878.5111
r27lognormal sigma=0.5 with zeros p0=0.251000.62578.780.50.00000.44690.62580.6284.99
r28lognormal sigma=0.5 with zeros p0=0.421000.71106.30.50.00000.44690.7157.9965.72
r29lognormal sigma=0.5 with zeros p0=0.4700.471060.50.00000.44690.73545.360.06
r30lognormal sigma=0.5 with zeros p0=0.600.6800.50.00000.44690.8045.33
r31lognormal sigma=1 with zeros p0=0.251000.625102.50.50.00000.3940.62565123.7
r32lognormal sigma=1 with zeros p0=0.371000.685118.10.50.00000.3940.68544.08103.9
r33lognormal sigma=1 with zeros p0=0.4200.421160.50.00000.3940.7133.6395.63
r34lognormal sigma=1 with zeros p0=0.600.6800.50.00000.3940.8065.95
E13.T3 · Each measure's preferred candidate (the candidate forecast with the lowest expected score): its quantile level, and the number of candidates whose expected score is infinite
Rowsettingquantile level of the preferred candidate (measure: RMSE)quantile level of the preferred candidate (measure: MAE)quantile level of the preferred candidate (measure: MAPE)quantile level of the preferred candidate (measure: sMAPE)candidates whose expected score is infinite (measure: RMSE)candidates whose expected score is infinite (measure: MAPE)
r1inverse Gaussian shape=10000.56160.49990.37860.499900
r2inverse Gaussian shape=2000.62770.50.26590.497800
r3inverse Gaussian shape=1000.66810.50.21580.492900
r4inverse Gaussian shape=500.71380.50.17730.48200
r5inverse Gaussian shape=200.77680.50.14740.457800
r6inverse Gaussian shape=100.82190.50.1360.438700
r7log-skewed left sd=0.5 skew=-20.44440.50.250.521100
r8log-skewed left sd=0.5 skew=-10.52150.50.27990.51300
r9log-skewed left sd=0.5 skew=-0.50.56030.50.29410.506800
r10log-skewed right sd=0.5 skew=+20.750.50.370.47891020
r11log-skewed right sd=0.5 skew=+10.67480.50.33880.48700
r12log-skewed right sd=0.5 skew=+0.50.63680.50.32340.493200
r13lognormal sigma=0.50.59870.50.30850.500
r14log-skewed left sd=1 skew=-20.50.500.55620101
r15log-skewed left sd=1 skew=-10.59280.50.06550.536200
r16log-skewed left sd=1 skew=-0.50.64130.50.11550.519100
r17log-skewed right sd=1 skew=+20.50.29290.44381030
r18log-skewed right sd=1 skew=+10.80340.50.23140.46381040
r19log-skewed right sd=1 skew=+0.50.74460.50.19680.480900
r20lognormal sigma=10.69150.50.15870.500
r21bimodal equal modes0.50810.50.23410.374200
r22bimodal left mode higher0.65030.50.28410.406300
r23bimodal right mode higher0.41110.50.18480.593700
r24normal straddling zero mean=00.50.50.50.77430100
r25normal straddling zero mean=0.50.50.50.30850.68580101
r26normal straddling zero mean=10.50.50.15870.61280101
r27lognormal sigma=0.5 with zeros p0=0.250.52930.50.625077
r28lognormal sigma=0.5 with zeros p0=0.420.53640.50.71059
r29lognormal sigma=0.5 with zeros p0=0.470.55160.50.47055
r30lognormal sigma=0.5 with zeros p0=0.60.62270.60.6041
r31lognormal sigma=1 with zeros p0=0.250.68810.50.625077
r32lognormal sigma=1 with zeros p0=0.370.69450.50.685065
r33lognormal sigma=1 with zeros p0=0.420.69970.50.42059
r34lognormal sigma=1 with zeros p0=0.60.73540.60.6041

Everything recorded for this evidence: E13 in the record.

Earlier work and other claims

  • Supports claim C4. Computed on 34 distributions, the target lies on the side of the median that carries more weight, as claim C4 and claim C12 prove.
  • Supports claim C5. Computed on lognormal outcomes, the target is the median, as claim C5 proves.
  • Supports claim C6. Computed on bimodal outcomes, the target is two forecasts, one near each mode, when the modes are equal, and the larger mode's when they are not, as claim C6 proves.
  • Supports claim C7. Computed on lognormal outcomes with zeros, the target switches to 0 at a share of zeros below one half, as claim C7 proves.
  • Supports claim C11. Computed on normal outcomes straddling zero, the target lies away from zero, on the side of the mean, as claim C11 proves.
estimated as plannedRecord
Claim
C13 · descriptive · secondary · no earlier result found
Plan
Estimate planned in lab plan PL1, recorded 16 November 2026
Evidence
E13 in the record
Scope
data: outcome distributions given exactly (no data); conditions: 34 distributions: inverse Gaussian, log-skewed, lognormal, bimodal, normal straddling zero, zero-inflated lognormal
Limitations
L1
Support score
SupportS 93 of 100. The score checks the record, not the science.
Record
This claim on the record page

Term this claim uses

ranking agreement
For one distribution, one measure, a test-set size nn, the measure's target and a forecast at another quantile level: the probability that a test set of nn outcomes gives the target the strictly lower score (ties count as disagreement); 0.5 is a coin toss.

Numerical check C14. Flattening, computed evidence

As the data spread out, the best possible sMAPE rises towards its worst value, the forecast it rewards moves from the median towards the most likely value on the log scale, and sMAPE ranks its own best forecast first less often than MAE does.

As the spread σ\sigma of ln⁡y\ln y grows from 0.1 to 10, the expected sMAPE at the sMAPE target rises towards its bound 200 (from 7.97 to 178.2 on lognormal outcomes), the target's quantile level moves from the median to that of the mode of ln⁡y\ln y (0.372 against the mode's 0.368 at σ=10\sigma = 10 on log-Gumbel outcomes skewed right), and the ranking agreement of sMAPE, for its target against a forecast half a σ\sigma away, is lower than that of MAE: at σ=4\sigma = 4 with test sets of 12 outcomes it ranks its target first in 0.61 of the test sets, MAE in 0.82.

Evidence

The quantile level of the sMAPE target against the spread sigma of ln y, for lognormal and log-Gumbel outcomes, beside the level of the log-mode: the target mo…
A44 The quantile level of the sMAPE target against the spread sigma of ln y, for lognormal and log-Gumbel outcomes, beside the level of the log-mode: the target moves from the median to the mode of ln y as the spread grows. Drawn by lab run RN2 from its own results (lab/runs/RN2/figures/flatness_target.png).
Expected sMAPE against the quantile level of the forecast, one curve per spread sigma, for each family: the curve flattens towards 200 as sigma grows.
A45 Expected sMAPE against the quantile level of the forecast, one curve per spread sigma, for each family: the curve flattens towards 200 as sigma grows. Drawn by lab run RN2 from its own results (lab/runs/RN2/figures/flatness_expected_sMAPE.png).
Ranking agreement: the share of test sets that prefer each measure's target to a forecast away from it, against sigma, for several test-set sizes.
A47 Ranking agreement: the share of test sets that prefer each measure's target to a forecast away from it, against sigma, for several test-set sizes. Drawn by lab run RN2 from its own results (lab/runs/RN2/figures/flatness_agreement.png).

View all 5 figures

E14.T1 · The sMAPE target as the spread of ln y grows: its quantile level beside the log-mode's, the expected sMAPE there and its large-spread approximation
Rowsettingquantile level of the sMAPE targetquantile level of the log-modeexpected sMAPE at the targetlarge-spread approximation of expected sMAPE at the targetlocal minima of expected sMAPE
r1lognormal sigma=0.10.50.57.966-20121
r2log-Gumbel right sigma=0.10.49890.36799.643-17241
r3log-Gumbel left sigma=0.10.50110.63219.643-17241
r4lognormal sigma=0.250.50.519.74-684.91
r5log-Gumbel right sigma=0.250.4940.367923.66-572.71
r6log-Gumbel left sigma=0.250.5060.632123.66-572.71
r7lognormal sigma=0.50.50.538.38-242.41
r8log-Gumbel right sigma=0.50.48110.367944.85-190.41
r9log-Gumbel left sigma=0.50.51890.632144.85-190.41
r10lognormal sigma=10.50.569.94-21.221
r11log-Gumbel right sigma=10.45440.367977.991.2461
r12log-Gumbel left sigma=10.54560.632177.991.2461
r13lognormal sigma=1.50.50.593.6652.521
r14log-Gumbel right sigma=1.50.43370.3679101.266.061
r15log-Gumbel left sigma=1.50.56630.6321101.266.061
r16lognormal sigma=20.50.5111.289.391
r17log-Gumbel right sigma=20.41870.3679117.998.931
r18log-Gumbel left sigma=20.58130.6321117.998.931
r19lognormal sigma=30.50.5134.4126.31
r20log-Gumbel right sigma=30.39990.3679139.5132.31
r21log-Gumbel left sigma=30.60010.6321139.5132.31
r22lognormal sigma=40.50.5148.5144.71
r23log-Gumbel right sigma=40.38940.3679152.5149.11
r24log-Gumbel left sigma=40.61060.6321152.5149.11
r25lognormal sigma=60.50.5164.4163.11
r26log-Gumbel right sigma=60.37920.3679167.11661
r27log-Gumbel left sigma=60.62080.6321167.11661
r28lognormal sigma=100.50.5178.2177.91
r29log-Gumbel right sigma=100.37240.3679179.9179.61
r30log-Gumbel left sigma=100.62760.6321179.9179.61
E14.T2 · Ranking agreement: the share of test sets that give a measure's target a lower score than a forecast half a sigma above it, for three of the settings; every setting, offset and size is in the run's CSV and its figure
Rowsettingtest-set sizeranking agreement (measure: RMSE, offset in sigma: 0.5)ranking agreement (measure: MAE, offset in sigma: 0.5)ranking agreement (measure: MAPE, offset in sigma: 0.5)ranking agreement (measure: sMAPE, offset in sigma: 0.5)
r1lognormal sigma=0.5120.82820.77720.73960.7557
r2lognormal sigma=0.5480.96590.94210.91180.9233
r3lognormal sigma=0.52500.99980.99990.99940.9994
r4lognormal sigma=0.510001111
r5lognormal sigma=4120.98260.82260.00120.605
r6lognormal sigma=4480.97920.97050.00590.6989
r7lognormal sigma=42500.975910.03370.8823
r8lognormal sigma=410000.977110.09280.9909

Everything recorded for this evidence: E14 in the record.

Earlier work and other claims

  • Supports claim C10. Computed for spreads of ln y from 0.1 to 10, expected sMAPE flattens and the target moves to the mode of ln y, as claim C10 proves for large spreads.
estimated as plannedRecord
Claim
C14 · descriptive · secondary · no earlier result found
Plan
Estimate planned in lab plan PL4, recorded 16 November 2026
Evidence
E14 in the record
Scope
data: outcome distributions given exactly, and test sets drawn from them; conditions: lognormal and log-Gumbel outcomes, spread of ln y from 0.1 to 10, test sets of 12 to 1000 outcomes
Support score
SupportS 93 of 100. The score checks the record, not the science.
Record
This claim on the record page

14 more definitions that no claim names (designs, data, cases) are on the record page.

Limitations

L1 · analytic flexibility · could bias magnitude · affects C13 · mitigation: none

Lab run RN1 deviated from plan PL1: RN1 ran at commit 3cb6c7c, not the plan's 92cd9a5. Between them only flatness.py and plot_flatness.py (plan PL3), test_smape.py and measures.py changed; in measures.py the unused function log_mae was removed (the supervisor dropped log-MAE from PL3, DC3). targets.py, theory.py, dists.py and the measures targets.py uses are unchanged. Reason: The commit of the PL3 revision came between PL1's plan and its run; it does not touch what PL1 runs.

L2 · analytic flexibility · could bias magnitude · affects C15 · mitigation: none

Lab run RN7 deviated from plan PL2: RN7 ran efficiency.py at commit fa0ce62, not the plan's 92cd9a5: the script runs its 16 settings in parallel processes (one per core) instead of one after another; theory.py also gained a function (balance_exact) that efficiency.py does not use. Each (setting, test-set size) draws from its own seed as before, and on a subset the parallel code and the plan's code gave byte-identical results.json. Reason: Three earlier attempts on one core (over an hour) were killed by container restarts before the lab recorded them (notes NT2.R2, NT5, NT6); in parallel the run finished in about 20 minutes. The analysis, the settings, the seeds and the results are unchanged.

L3 · sampling · reduces transferability · affects C14, C15, C21 · mitigation: none

The test sets are independent draws from a fixed distribution. Real forecast errors are often correlated and their distribution can change over a test period, so a real test set carries less information than the same number of outcomes here.

L4 · model specification · reduces transferability · affects C13, C14, C15, C16, C21 · mitigation: none

Only the distribution families studied were computed; the formal claims (C4, C10) say what holds beyond them, the simulations only illustrate it.

L5 · model specification · reduces transferability · affects C6 · mitigation: none

C6 is proved for two groups that mirror each other about the median on the log scale. It is not proved for groups of different shape or spread.

L6 · measurement · reduces transferability · affects C20, C21 · mitigation: none

The definition of msMAPE is taken from Kolassa (2026, Definition 4). Smyl's article was not read. Smyl gives no value for the constant; ν = 0.1, 0.5 and 1 are this paper's choices.

L7 · other · reduces transferability · affects C19 · mitigation: partial

That the relative efficiency of finite test sets tends to the asymptotic value is not shown; the simulation of C15 agrees with it at n = 10000.

References

  1. Anderson, T. W., 1955. The integral of a symmetric unimodal function over a symmetric convex set and some probability inequalities. Proceedings of the American Mathematical Society 6, 170–176. https://doi.org/10.1090/S0002-9939-1955-0069229-1 Used by: Context
  2. Chen, C., Twycross, J., Garibaldi, J. M., 2017. A new accuracy measure based on bounded relative error for time series forecasting. PLOS ONE 12, e0174202. https://doi.org/10.1371/journal.pone.0174202 Used by: Context
  3. Goodwin, P., Lawton, R., 1999. On the asymmetry of the symmetric MAPE. International Journal of Forecasting 15, 405–408. https://doi.org/10.1016/S0169-2070(99)00007-2 Used by: Context
  4. Hewamalage, H., Bergmeir, C., Bandara, K., 2021. Recurrent Neural Networks for Time Series Forecasting: Current status and future directions. International Journal of Forecasting 37, 388–427. https://doi.org/10.1016/j.ijforecast.2020.06.008 Used by: Context, D28
  5. Hewamalage, H., Ackermann, K., Bergmeir, C., 2023. Forecast evaluation for data scientists: common pitfalls and best practices. Data Mining and Knowledge Discovery 37, 788–832. https://doi.org/10.1007/s10618-022-00894-5 Used by: Context
  6. Hyndman, R. J., Koehler, A. B., 2006. Another look at measures of forecast accuracy. International Journal of Forecasting 22, 679–688. https://doi.org/10.1016/j.ijforecast.2006.03.001 Used by: Context
  7. Kolassa, S., 2026. The Symmetric Mean Absolute Percentage Error: Unnecessary or Dangerous. Forecasting 8, 24. https://doi.org/10.3390/forecast8020024 Used by: Context, D33
  8. Makridakis, S., 1993. Accuracy measures: theoretical and practical concerns. International Journal of Forecasting 9, 527–529. https://doi.org/10.1016/0169-2070(93)90079-3 Used by: D2
  9. Smyl, S. (2025). Sparse-Proof sMAPE. Foresight: The International Journal of Applied Forecasting, 78, 29–30. https://ideas.repec.org/a/for/ijafaa/y2025i78p29-30.html Used by: Context
  10. Svetunkov, I., 2026a. Evaluating forecasts and classifiers: which measures, scoring rules and tests to use. EntryLib. entrylib:2026.000008@v2 Claims cited: C6, C33. Used by: Context
  11. Svetunkov, I., 2026b. Which forecast does each error measure reward? A simulation study of the functionals targeted by RMSE, MAE, MAPE and sMAPE. EntryLib. entrylib:2026.000009@v1 Claims cited: C5, C6, C11. Used by: Context
  12. Tofallis, C., 2015. A better measure of relative prediction accuracy for model selection and model estimation. Journal of the Operational Research Society 66, 1352–1362. https://doi.org/10.1057/jors.2014.103 Used by: Context
  13. Vaart, A. W. v. d., 1998. Asymptotic Statistics. Cambridge University Press. https://doi.org/10.1017/CBO9780511802256 Used by: C19

Score

Support of the paper: 99, Well supported

Scores are computed by rules from the claimset file (EntryLib rubric 0.7): they check the record, not the science, and may change as the rubric and reviews develop. The rubric. Every issue as text, for an agent preparing the next version.

The mean of its 13 principal claims; the lowest is C15 at 89. Open problems in the principal claims and what they rest on, each counted once, with the most it costs a claim:

  1. −6 Open problems Limitation L2 (could bias magnitude, not mitigated): Lab run RN7 deviated from plan PL2: RN7 ran efficiency.py at commit fa0ce62, not the plan's 92cd9a5: the script runs its 16 settings in parallel processes (one per core) instead of one after another; theory.py also gained a function (balance_exact) that efficiency.py does not use. Each (setting, test-set size) draws from its own seed as before, and on a subset the parallel code and the plan's code gave byte-identical results.json. Reason: Three earlier attempts on one core (over an hour) were killed by container restarts before the lab recorded them (notes NT2.R2, NT5, NT6); in parallel the run finished in about 20 minutes. The analysis, the settings, the seeds and the results are unchanged. In C15. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
  2. −5 Context A principal claim related to no earlier work. In C6. To remove it: Relate the claim to the earlier work it extends, addresses or departs from.
  3. −3 Plan The lab record flags the test: code changed after plan. In C15. To remove it: Rerun the test as planned, or explain the flag in a limitation.
  4. −1 Open problems Limitation L3 (reduces transferability, not mitigated): The test sets are independent draws from a fixed distribution. Real forecast errors are often correlated and their distribution can change over a test period, so a real test set carries less information than the same number of outcomes here. In C14, C15, C21. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
  5. −1 Open problems Limitation L4 (reduces transferability, not mitigated): Only the distribution families studied were computed; the formal claims (C4, C10) say what holds beyond them, the simulations only illustrate it. In C13, C14, C15, C16, C21. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
  6. −1 Open problems Limitation L5 (reduces transferability, not mitigated): C6 is proved for two groups that mirror each other about the median on the log scale. It is not proved for groups of different shape or spread. In C6. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
  7. −1 Open problems Limitation L6 (reduces transferability, not mitigated): The definition of msMAPE is taken from Kolassa (2026, Definition 4). Smyl's article was not read. Smyl gives no value for the constant; ν = 0.1, 0.5 and 1 are this paper's choices. In C20, C21. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.

Claims

C1: 100, Well supported

Novelty: extends earlier work, by the authors

Nothing lost.

C2: 100, Well supported

Novelty: new, by the authors

Nothing lost.

C3: 100, Well supported

Novelty: extends earlier work, by the authors

Nothing lost.

C4: 100, Well supported

Novelty: new, by the authors

Nothing lost.

C5: 100, Well supported

Novelty: replicates earlier work, by the authors

Nothing lost.

C6: 94, Well supported

Novelty: new, by the authors

  1. −5 Context A principal claim related to no earlier work. To remove it: Relate the claim to the earlier work it extends, addresses or departs from.
  2. −1 Open problems Limitation L5 (reduces transferability, not mitigated): C6 is proved for two groups that mirror each other about the median on the log scale. It is not proved for groups of different shape or spread. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.

C7: 100, Well supported

Novelty: extends earlier work, by the authors

Nothing lost.

C8: 100, Well supported

Novelty: new, by the authors

Nothing lost.

C9: 100, Well supported

Novelty: extends earlier work, by the authors

Nothing lost.

C10: 100, Well supported

Novelty: new, by the authors

Nothing lost.

C11: 100, Well supported

Novelty: new, by the authors

Nothing lost.

C12: 100, Well supported

Novelty: new, by the authors

Nothing lost.

C13: 93, Well supported

Novelty: new, by the authors

  1. −6 Open problems Limitation L1 (could bias magnitude, not mitigated): Lab run RN1 deviated from plan PL1: RN1 ran at commit 3cb6c7c, not the plan's 92cd9a5. Between them only flatness.py and plot_flatness.py (plan PL3), test_smape.py and measures.py changed; in measures.py the unused function log_mae was removed (the supervisor dropped log-MAE from PL3, DC3). targets.py, theory.py, dists.py and the measures targets.py uses are unchanged. Reason: The commit of the PL3 revision came between PL1's plan and its run; it does not touch what PL1 runs. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
  2. −1 Open problems Limitation L4 (reduces transferability, not mitigated): Only the distribution families studied were computed; the formal claims (C4, C10) say what holds beyond them, the simulations only illustrate it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.

C14: 93, Well supported

Novelty: new, by the authors

  1. −5 Context Numbers in the statement match no recorded result. To remove it: Record each number as a result or table cell of the evidence the claim cites, and list it in the support's results; or correct the number.
  2. −1 Open problems Limitation L3 (reduces transferability, not mitigated): The test sets are independent draws from a fixed distribution. Real forecast errors are often correlated and their distribution can change over a test period, so a real test set carries less information than the same number of outcomes here. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
  3. −1 Open problems Limitation L4 (reduces transferability, not mitigated): Only the distribution families studied were computed; the formal claims (C4, C10) say what holds beyond them, the simulations only illustrate it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.

C15: 89, Well supported

Novelty: new, by the authors

  1. −6 Open problems Limitation L2 (could bias magnitude, not mitigated): Lab run RN7 deviated from plan PL2: RN7 ran efficiency.py at commit fa0ce62, not the plan's 92cd9a5: the script runs its 16 settings in parallel processes (one per core) instead of one after another; theory.py also gained a function (balance_exact) that efficiency.py does not use. Each (setting, test-set size) draws from its own seed as before, and on a subset the parallel code and the plan's code gave byte-identical results.json. Reason: Three earlier attempts on one core (over an hour) were killed by container restarts before the lab recorded them (notes NT2.R2, NT5, NT6); in parallel the run finished in about 20 minutes. The analysis, the settings, the seeds and the results are unchanged. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
  2. −3 Plan The lab record flags the test: code changed after plan. To remove it: Rerun the test as planned, or explain the flag in a limitation.
  3. −1 Open problems Limitation L3 (reduces transferability, not mitigated): The test sets are independent draws from a fixed distribution. Real forecast errors are often correlated and their distribution can change over a test period, so a real test set carries less information than the same number of outcomes here. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
  4. −1 Open problems Limitation L4 (reduces transferability, not mitigated): Only the distribution families studied were computed; the formal claims (C4, C10) say what holds beyond them, the simulations only illustrate it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.

C16: 99, Well supported

Novelty: new, by the authors

  1. −1 Open problems Limitation L4 (reduces transferability, not mitigated): Only the distribution families studied were computed; the formal claims (C4, C10) say what holds beyond them, the simulations only illustrate it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.

C17: 100, Well supported

Novelty: extends earlier work, by the authors

Nothing lost.

C18: 100, Well supported

Novelty: new, by the authors

Nothing lost.

C19: 100, Well supported

Novelty: new, by the authors

Nothing lost.

C20: 99, Well supported

Novelty: extends earlier work, by the authors

  1. −1 Open problems Limitation L6 (reduces transferability, not mitigated): The definition of msMAPE is taken from Kolassa (2026, Definition 4). Smyl's article was not read. Smyl gives no value for the constant; ν = 0.1, 0.5 and 1 are this paper's choices. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.

C21: 97, Well supported

Novelty: new, by the authors

  1. −1 Open problems Limitation L3 (reduces transferability, not mitigated): The test sets are independent draws from a fixed distribution. Real forecast errors are often correlated and their distribution can change over a test period, so a real test set carries less information than the same number of outcomes here. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
  2. −1 Open problems Limitation L4 (reduces transferability, not mitigated): Only the distribution families studied were computed; the formal claims (C4, C10) say what holds beyond them, the simulations only illustrate it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
  3. −1 Open problems Limitation L6 (reduces transferability, not mitigated): The definition of msMAPE is taken from Kolassa (2026, Definition 4). Smyl's article was not read. Smyl gives no value for the constant; ν = 0.1, 0.5 and 1 are this paper's choices. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.

Not claimed

Planned proofs that give no claim of their own; the lab record has each with its reason.

  • Reworded, then proved again: H1 → C1, H2 → C2, H3 → C3, H23 → C3, H4 → C4, H33 → C4, H5 → C5, H18 → C6, H8 → C7, H12 → C8, H13 → C9, H19 → C10, H34 → C12, H35 → C17, H39 (not supported) → C18, H39 (not supported) → C18, H43 (inconclusive) → C18, H45 (inconclusive) → C18, H40 (not supported) → C19
  • Merged: H16 (inconclusive) → C4, H9 (inconclusive) → C3, H10 (inconclusive) → C2
  • Dropped: H11 (inconclusive)

Versions

  • Version 3 (this page) · 18 November 2026 at 13:08 UTC · updated
  • Version 2 · 17 November 2026 at 22:21 UTC · updated
  • Version 1 · 17 November 2026 at 15:01 UTC · active

Every version is kept in the public registry (github.com/entrylib/registry).