TRAIL OF H39 (claim C18) Statement: Let $\ln Y = \mu + \sigma\varepsilon$, where $\varepsilon$ has a fixed density $h$ with bounded continuous first and second derivatives and distribution function $H$, and compare two forecasts at fixed quantile levels, $f_1 = e^{\mu+\sigma c_1}$ and $f_2 = e^{\mu+\sigma c_2}$ with $c_1 < c_2$, by their [[TM35:signal-to-noise ratio of a comparison]]. (a) For the [[TM1:sMAPE term]], as $\sigma$ grows, $\mathrm E[D] = 4\ln 2\,(h(c_1) - h(c_2))/\sigma + O(\sigma^{-3})$ and $\operatorname{Var}(D) = 4(2\ln 2 - 1)(h(c_1)+h(c_2))/\sigma + O(\sigma^{-2})$. So when $h(c_1) \ne h(c_2)$ the ratio is $\frac{2\ln 2}{\sqrt{2\ln 2 - 1}}\,\frac{|h(c_1)-h(c_2)|}{\sqrt{h(c_1)+h(c_2)}}\,\sigma^{-1/2}\,(1 + O(1/\sigma))$: it falls to zero like $\sigma^{-1/2}$, and the size of the test set that keeps the mean difference a given number of standard errors from zero grows in proportion to $\sigma$. (b) For the absolute log error $|\ln y - \ln f|$ ([[TM19:log-MAE]]), $D = \sigma(|\varepsilon - c_2| - |\varepsilon - c_1|)$, so the ratio, wherever $\mathrm{sd}(D) > 0$, does not depend on $\sigma$. (c) For the absolute error $|y - f|$ ([[TM11:MAE]]), as $\sigma$ grows the ratio tends to $|2H(c_2) - 1|/(2\sqrt{H(c_2)(1 - H(c_2))})$ when $0 < H(c_2) < 1$; the limit is positive unless the higher forecast is the median. (d) For the squared error $(y-f)^2$ ([[TM10:RMSE]]), when $\mathrm E[Y^2]$ is finite the ratio is $|2\,\mathrm E[Y] - f_1 - f_2|/(2\,\mathrm{sd}(Y))$; for lognormal actuals, $e^{\sigma^2/2}$ times the ratio tends to 1 as $\sigma$ grows. For widely spread actuals, then: the absolute log error keeps its power to tell two forecasts apart wherever its ratio is defined. The absolute error keeps it when $0 < H(c_2) < 1$ and the higher forecast is not the median. sMAPE loses it at the rate $\sigma^{-1/2}$ when $h(c_1) \ne h(c_2)$. It loses it faster, with a ratio of at most order $\sigma^{-5/2}$, when $h(c_1) = h(c_2) > 0$. The squared error on lognormal actuals loses it faster than any power of $\sigma$. Status: testing In log order: the question, each version of the hypothesis, the plans that name it with their approvals, deviations and runs, its statuses, and the checks, decisions and notes that refer to them. Results are given by number and file. 2026-11-16 13:20 TM1 term by AC1: sMAPE term [variable] 2026-11-16 13:20 TM10 term by AC1: RMSE [variable] 2026-11-16 13:20 TM11 term by AC1: MAE [variable] 2026-11-16 13:44 TM19 term by AC1: log-MAE [variable] 2026-11-18 11:08 Q3 question by AC5: As the actuals spread out, how well does a test set tell two forecasts apart under sMAPE, compared with the absolute log error, the absolute error and the squa… 2026-11-18 11:08 TM35 term by AC5: signal-to-noise ratio of a comparison [variable] 2026-11-18 11:08 H39 hypothesis by AC5: Let $\ln Y = \mu + \sigma\varepsilon$, where $\varepsilon$ has a fixed density $h$ with bounded continuous first and second derivatives and distribution functi… [a_priori] 2026-11-18 12:02 PL12 plan by AC5: [confirmatory, requires approval] for H36, H37, H38, H41, H39, H40, H42: Version 3 of the paper answers a reader's comments on version 2. Proofs: in proofs.md, section H3 gains part (c); secti… 2026-11-18 12:02 AP9 approval by AC5: PL12 approved by AC2 at 2026-11-18T12:02:11Z, recorded by agent AC5: "1. Opus is fine [...] 5. (2) with 3% is fine 6. We use "m" to denote the median. Use diff… 2026-11-18 12:02 RN13 run by AC5: [confirmatory, success, under PL12, commit 2f0d2fc] python3 claimset-code/smape-target/check_v3.py --seed 8; 82 results in runs/RN13/results.json (RS16073 to RS16154) 2026-11-18 12:14 PL13 plan by AC5: [confirmatory, requires approval] for H39: Written after the deciding run RN13 of PL12, and because of it. H39 states limits as the spread sigma grows; PL12 check… 2026-11-18 12:14 AP10 approval by AC5: PL13 approved by AC2 at 2026-11-18T12:14:44Z, recorded by agent AC5: "2. Agreed to the new short plan" 2026-11-18 12:14 CK30 check by AC6: H39: disputed (full) 2026-11-18 12:15 RN14 run by AC5: [confirmatory, success, under PL13, commit d49b5fd] python3 claimset-code/smape-target/check_h39.py; 42 results in runs/RN14/results.json (RS16155 to RS16196) 2026-11-18 12:19 H43 hypothesis by AC5: Let $\ln Y = \mu + \sigma\varepsilon$, where $\varepsilon$ has a fixed density $h$ with bounded continuous first and second derivatives and distribution functi… [a_priori] 2026-11-18 12:19 PL15 plan by AC5: [confirmatory, requires approval] for H43, H44: The independent checker disputed H39 (CK30) and H40 (CK31) as first worded; both are reworded as it proposed, and this… 2026-11-18 12:19 AP12 approval by AC5: PL15 approved by AC2 at 2026-11-18T12:19:06Z, recorded by agent AC5: "Good" 2026-11-18 12:19 ST38 status by AC5: H39 -> not_supported (plan PL12), evidence RN13, CK30: As worded, H39 does not hold in every case it allows. The independent checker (CK30) gave a counterexample to part (c):… 2026-11-18 12:19 ST39 status by AC5: H39 -> not_supported (plan PL13), evidence RN14, CK30: PL13's numerical tolerances are all met by RN14 (the absolute error's ratio is within 0.0035 of its limit at sigma = 20… 2026-11-18 12:19 ST41 status by AC5: H39 -> revised: Corrected after CK30: the conditions of parts (b) and (c) are stated and the closing sentence narrowed. 2026-11-18 12:20 RN16 run by AC5: [confirmatory, success, under PL15, commit 6aa7a7a] python3 claimset-code/smape-target/check_corrected.py --seed 8; 65 results in runs/RN16/results.json (RS16704 to RS16806) 2026-11-18 12:23 CK34 check by AC6: H43: disputed (full) 2026-11-18 12:23 ST44 status by AC5: H39 -> inconclusive (plan PL15), evidence RN16, CK34: Neither rule is met for H43. The run RN16 meets every tolerance and the checker found parts (a) to (d) right as now wor… 2026-11-18 12:26 H45 hypothesis by AC5: Let $\ln Y = \mu + \sigma\varepsilon$, where $\varepsilon$ has a fixed density $h$ with bounded continuous first and second derivatives and distribution functi… [a_priori] 2026-11-18 12:26 PL16 plan by AC5: [confirmatory, requires approval] for H45: The checker confirmed parts (a) to (d) of H43 and disputed its closing sentence (CK34); this plan decides the wording w… 2026-11-18 12:26 AP13 approval by AC5: PL16 approved by AC2 at 2026-11-18T12:26:20Z, recorded by agent AC5: "Don't produce long sentences. Split to smaller ones." 2026-11-18 12:26 ST45 status by AC5: H39 -> revised: Corrected after CK34: the closing statement is split into short sentences that state only the cases parts (a) and (c) g… 2026-11-18 12:26 RN17 run by AC5: [confirmatory, success, under PL16, commit 6aa7a7a] python3 claimset-code/smape-target/check_h39.py; 42 results in runs/RN17/results.json (RS17099 to RS17140) 2026-11-18 12:27 CK36 check by AC6: H45: disputed (full) 2026-11-18 12:27 ST46 status by AC5: H39 -> inconclusive (plan PL16), evidence RN17, CK36: Neither rule is met for H45. The run RN17 meets every tolerance, and the checker found parts (a) to (d) and four of the… 2026-11-18 12:29 H46 hypothesis by AC5: Let $\ln Y = \mu + \sigma\varepsilon$, where $\varepsilon$ has a fixed density $h$ with bounded continuous first and second derivatives and distribution functi… [a_priori] 2026-11-18 12:29 PL17 plan by AC5: [confirmatory, requires approval] for H46: The checker confirmed H45 except for one closing sentence (CK36); this plan decides the wording with that sentence corr… 2026-11-18 12:29 AP14 approval by AC5: PL17 approved by AC2 at 2026-11-18T12:29:28Z, recorded by agent AC5: "Yes, I agree." 2026-11-18 12:29 ST47 status by AC5: H39 -> revised: Corrected after CK36: the ratio is said to be at most of order sigma^(-5/2) when the two densities are equal and positi… 2026-11-18 12:29 RN18 run by AC5: [confirmatory, success, under PL17, commit 6aa7a7a] python3 claimset-code/smape-target/check_h39.py; 42 results in runs/RN18/results.json (RS17141 to RS17182) 2026-11-18 12:30 CK37 check by AC6: H46: confirmed (full) 2026-11-18 12:31 ST48 status by AC5: H39 -> supported (plan PL17), evidence RN18, CK37: PL17's rule for supported is met for H46, the current wording of H39: the proof (proofs.md, section H13) is complete; t… 2026-11-18 12:33 ST49 status by AC5: H39 -> testing: A correction of ST41, which I recorded in error as 'revised': that status is for a hypothesis replaced by another one.… 2026-11-18 12:33 ST51 status by AC5: H39 -> testing: A correction of ST45, which I recorded in error as 'revised': that status is for a hypothesis replaced by another one.… 2026-11-18 12:33 ST52 status by AC5: H39 -> testing: A correction of ST47, which I recorded in error as 'revised': that status is for a hypothesis replaced by another one.…