{ "bundle_of": "C24", "schema_version": "0.40", "paper": { "id": "entrylib:2026.000002", "version": 3, "title": "Admissibility of lagged state space models through the ARIMA form of the whole model", "licence": "CC-BY-4.0", "status": "corrected", "date": "2026-11-10", "status_notice": { "date": "2026-11-10", "reason_code": "correction", "statement": "Limitations L2 and L4 are answered by a planned independent check (new evidence E25): an exact oracle in rational arithmetic with certified root enclosures, written by a new contributor (P5) without the prototype, compared on the models of the planned tests; it agrees with the PBH reference, the cyclotomic cancellation, the step-down test and the bisection on every model it decides, and its four hypotheses are reported as claims C29-C32, one of them inconclusive by its rule. The assumptions C14 (reworded to the band the tests leave out) and C19 are tested by it; claims C1-C5, C10, C11 and C16 carry its checks; claims C3, C4, C16 and C24-C26 relate to the earlier work they extend, supersede or contradict. The lab record (A9) and the results file (A18) are updated to the record's current version. No earlier claim's conclusion or ID changed.", "issued_by": "Ivan Svetunkov (ORCID 0000-0001-7826-0281)" } }, "claims": [ { "id": "C24", "statement": "On [[D22:1,000 new random ADAM ETS+ARIMA models]] (500 with ARIMA at lag 1, 500 with seasonal ARIMA), [[D13:checking the ETS part and the ARIMA polynomials apart]], as adam() did with bounds=\"admissible\", gave an [[D29:admissibility decision]] different from the [[D5:PBH reference]] of the whole model on about a third of the models in each family, accepting models that are not forecastable and rejecting models that are.", "about": { "subject": "D13", "relation": "gives admissibility decisions different from", "object": [ "D29" ], "comparator": "D5", "conditions": [ "D22" ] }, "type": "descriptive", "role": "principal", "novelty": "no_prior_found", "support": [ { "kind": "simulation", "evidence": "E20", "results": [ "E20.T1.r1", "E20.T1.r3" ], "artefacts": [ "A10", "A1" ] } ], "provenance": "author_stated", "hypothesis": { "lab_event": "H7", "outcome": "supported" }, "planning": "confirmatory", "planned_at": "2026-10-22T15:55:32Z", "planned_at_anchor": "git_remote", "test": { "framework": "deterministic", "parameters": { "sample": "500 models per family, one run, no stopping rule", "threshold": 25 }, "quantities": { "s1": { "variable": "models whose admissibility decision differs from the PBH reference", "conditions": { "family": "ETS+ARIMA", "check": "split" } }, "s2": { "variable": "models whose admissibility decision differs from the PBH reference", "conditions": { "family": "ETS+SARIMA", "check": "split" } } }, "supported": "s1 > threshold and s2 > threshold", "not_supported": "s1 <= 5 and s2 <= 5", "verdict": "computed", "runs": 1, "lab_plan": "PL7", "flags": [ "code_changed_after_plan" ] }, "scope": { "population": "random ADAM models that combine additive ETS with ARIMA", "conditions": [ "ETS: level; no, additive or damped trend; no or one seasonal lag m in {4, 12}; smoothing parameters drawn from wide ranges, some fixed at 0", "ARIMA: AR stationary, differences and MA up to order 2 at lag 1 and 1 at lag m, 15% over-differenced (an MA root at one on the unit circle of a differenced lag)", "ADAM's state space form of ARIMA (one state per lag of the ARI and MA polynomials), checked against the matrices of adam() on 20 of the models" ], "excludes": [ "regressors", "constant", "multiplicative models", "several seasonal lags", "CES", "GUM" ] }, "assumptions": [ "C28" ], "assessments": [ { "by": "author", "strength": "high", "rationale": "Planned test on new models with a fixed rule; the counts are far from the threshold of 25 per family. The share depends on the generator (wide ranges of the smoothing and MA parameters, 15% over-differenced models, some of which are among those rejected by the parts apart); the models accepted by the parts apart and rejected by the reference alone, 104 and 128, exceed the threshold." } ], "limitations": [ "L12" ], "checks": [ { "by": "P4", "outcome": "confirmed", "extent": "full", "notes": "Re-derived and reran. Formal: for ADAM's ARIMA part F = -a 1', g = ma - a, w = 1 (a = ari coefficients per state lag), so phi = det(I + 1' L a) = ari polynomial and D = F - g w' = -ma 1', theta = ma polynomial; checked numerically on 57 and 238 matrices built by adam() itself (AC3.RN3, AC3.RN11: max abs gap of det(I-FL) and det(I-DL) to ari and ma polynomials 1.4e-14). Cross terms: ETS(A,N,N) alpha=1.9 + MA(1) theta=0.1 has whole theta = 1 + B - 0.1 B^2 (exact fractions), the product of the parts' thetas is 1 + B + 0.09 B^2, smallest root modulus 0.916 against 1.111 (AC3.RN3), so the parts-apart check accepts a model that does not forget its states (package modulus 1.0916 whole, 0.9 ETS part alone). Reproduction with new seeds, script unchanged: seed 11 n=200 (AC3.RN1): split check differs from PBH on 65/200 ETS+ARIMA and 71/200 ETS+SARIMA; seed 12 n=200 (AC3.RN2): 67/200 and 58/200; author's RN49 166/500 and 159/500; all far above the 5% rule (25/500). Both directions occur (AC3.RN1: split accepts and PBH rejects 42 and 64; reverse 23 and 7). Review of ets_arima.py: the split baseline matches the old admissible branch of adam_bounds_checker (git 094b76f4~1: smoothEigens on ETS/constant/regressors apart, AR and MA reflections for the ARIMA); generators, AR via reflection coefficients, over-differencing, structure check against adam() (30 models, max gap 4.9e-15, AC3.RN1) are correct. Caveats, not disputes: (1) the percentage depends on the generator (alpha up to 1.9, MA reflection up to 1.5, 15% over-differenced by design), so '>5%' says the parts-apart check is wrong on a non-negligible share of such models, not on real fits; (2) over-differenced MA unit roots that cancel with a difference make the split check reject models PBH accepts, a mechanism other than the ETS-ARIMA cross terms (the cross-term direction alone is 104/500 and 128/500 in RN49 and still above 25); (3) RN49 and RN50 are the same command with the same results.json hash, so they are one data set tested under two plans; (4) the 'code changed after plan' flag is benign: git diff 397f255f..acb23120 touches only lab files." } ], "relations": [ { "type": "contradicts", "reference": "B3", "locator": "adam(), bounds = \"admissible\", smooth 4.6.0", "self": true, "note": "adam() of smooth 4.6.0 with bounds = \"admissible\" accepts an ETS+ARIMA model when its ETS part passes the per-lag eigenvalue check and its ARIMA polynomials pass their own checks, each apart." } ], "verifiability": "open", "verifiability_flags": [] }, { "id": "C28", "statement": "For ETS+ARIMA models with a stationary AR polynomial, $\\theta(B)$ and $\\phi(B)$ of the whole model share no factor with a root in $|B| \\le 1$ that is not cyclotomic, so the [[D5:PBH reference]] and the cancelled MA polynomial decide the same admissibility.", "type": "methodological", "role": "supporting", "support": [ { "kind": "assumption" } ], "assumption_category": "theorem_condition", "check": "tested", "provenance": "author_stated", "check_evidence": "E24", "verifiability": "not_applicable", "verifiability_flags": [] } ], "definitions": [ { "id": "D3", "term": "decaying weights", "definition": "A lagged SSOE model has decaying weights when the weights $\\tilde w' \\tilde D^j \\tilde g$ of the past observations in its fitted values tend to zero, that is, when every eigenvalue $\\lambda$ of its expanded discount matrix that is reachable from $\\tilde g$ and observable through $\\tilde w$ has $|\\lambda| < 1$; the largest such $|\\lambda|$ is its largest mode modulus. A mode that $\\tilde g$ does not reach but $\\tilde w$ sees carries an initial state whose effect is not covered: with $|\\lambda| > 1$ it grows.", "kind": "concept" }, { "id": "D4", "term": "per-lag check", "definition": "The test applied by smoothEigens() of smooth 4.6.0 and of its repository until commit 4a44c1c (src/headers/eigenCalc.h) to models without adaptive regressors: every eigenvalue of every diagonal block of $D$, one block per unique lag, has modulus below 1; reimplemented in numpy as per_lag_max().", "kind": "method" }, { "id": "D5", "term": "PBH reference", "definition": "The largest modulus among the eigenvalues $\\lambda$ of the expanded discount matrix for which the smallest singular values of $[\\lambda I - \\tilde D;\\ \\tilde w']$ and of $[\\lambda I - \\tilde D,\\ \\tilde g]$ both exceed $10^{-7}(\\|\\tilde D\\|_2 + \\|\\tilde g\\| + \\|\\tilde w\\|)$ (Popov-Belevitch-Hautus rank tests), computed with numpy as pbh_max().", "kind": "method" }, { "id": "D6", "term": "cyclotomic cancellation", "definition": "The procedure that computes the coefficients of $$\\begin{aligned} \\theta(B) &= \\det(I - D L(B)) \\\\ \\phi(B) &= \\det(I - F L(B)) \\end{aligned}$$ by an inverse discrete Fourier transform of their values at the $N$-th roots of unity ($N$ the smallest power of 2 above $\\sum_j l_j$), and divides both by every cyclotomic polynomial $\\Phi_n(B)$, $n$ dividing one of the lags, for as long as it divides both with a remainder at most $10^{-9}$ times the sum of absolute coefficients; the quotient of $\\theta$ is the cancelled MA polynomial $\\theta^*(B)$.", "kind": "method" }, { "id": "D7", "term": "step-down test", "definition": "The Levinson-Durbin step-down (Schur-Cohn) recursion on $1 + c_1 B + \\dots + c_n B^n$: for $j = n, \\dots, 1$, $$\\kappa_j = c_j, \\qquad c_i \\leftarrow \\frac{c_i - \\kappa_j c_{j-i}}{1 - \\kappa_j^2};$$ the test passes when every $|\\kappa_j| < 1$, which holds if and only if all roots lie outside the unit circle (the $\\kappa_j$ are the partial autocorrelations of the autoregression with this polynomial).", "source": "B2", "kind": "method" }, { "id": "D10", "term": "check of the whole model", "kind": "method", "definition": "Admissibility of a lagged SSOE model from the MA polynomial $$\\theta(B) = \\det(I - D L(B))$$ of its ARIMA form: $\\theta$ is divided by its common factors with $\\phi(B) = \\det(I - F L(B))$ by [[D6:cyclotomic cancellation]], and the model is accepted when the cancelled polynomial $\\theta^*$ has no root in $|B| \\le 1$ and, for a model with ARIMA components, the AR polynomial of its ARIMA part is stationary; its largest mode modulus is $1/\\min|B|$ over the roots of $\\theta^*$. In the numpy prototype (admissibility.py, confirm.py, ets_arima.py) the roots come from numpy; in smooth the [[D7:step-down test]] decides and bisection gives the modulus." }, { "id": "D13", "term": "check of the parts apart", "kind": "method", "definition": "The check that adam() applied with bounds=\"admissible\" to models combining ETS and ARIMA before this study: the ETS part by the [[D4:per-lag check]], and the AR and MA polynomials of the ARIMA part for stationarity and invertibility, each apart from the others." }, { "id": "D14", "term": "bounds checker of adam()", "kind": "method", "definition": "adam_bounds_checker() of adam(), om() and omg() in R (R/utils-adam.R) and of ADAM, OM and OMG in Python, with bounds=\"admissible\" after this study: smoothEigensCpp() on the whole model, with a shortcut that accepts a pure ARIMA model (without ETS and adaptive regressors) whose MA factors are invertible without running it; the penalty is zero, or 1e100 times the largest root modulus.", "broader": [ "D10" ] }, { "id": "D22", "term": "random ETS+ARIMA models", "kind": "data", "definition": "Random ADAM models combining additive ETS with ARIMA (ets_arima.py), 500 with ARIMA at lag 1 and 500 with seasonal ARIMA per seed (seed 2 for plans PL7 and PL8, seed 3 for plan PL9): ETS with a level; no, additive or damped trend; no or one seasonal lag $m$ in {4, 12}; smoothing parameters drawn from wide ranges, some fixed at 0; ARIMA with a stationary AR polynomial, differences and MA up to order 2 at lag 1 and 1 at lag $m$, 15% over-differenced (an MA root at one on the unit circle of a differenced lag); built in ADAM's state space form (one state per lag of the ARI and MA polynomials)." }, { "id": "D23", "term": "ETS+ARIMA models of the checker's generator", "kind": "data", "definition": "Random ADAM ETS+ARIMA models from the generator of the checker P4 (code/checker/exact_oracle.py with lib.py), wider than the [[D22:random ETS+ARIMA models]]: 250 models for each of seeds 21-23 and 31-34." }, { "id": "D26", "term": "comparison with a reference", "kind": "design", "definition": "Each model's admissibility decision, and its largest mode modulus where both find it inadmissible, computed by the method studied and by a reference on the same model (the [[D5:PBH reference]], the roots by numpy, the numpy prototype of the check of the whole model or an exact oracle); the result is the number of models whose decisions differ and the largest absolute difference of the modulus. In the planned tests, models whose reference modulus lies within 1e-8 of 1 are left out and counted." }, { "id": "D29", "term": "admissibility decision", "kind": "variable", "definition": "Whether a method accepts a model as admissible ([[D3:decaying weights]]); compared between a method and a reference as the number of models on which the two differ." }, { "id": "D39", "term": "exact gcd oracle", "kind": "method", "definition": "Admissibility by rational determinants and the exact greatest common divisor of $\\theta$ and $\\phi$, which cancels any shared factor (code/checker/exact_oracle.py of the checker P4)." } ], "evidence": [ { "id": "E20", "profile": "simulation", "design": "other", "description": "Planned tests (lab plans PL7 and PL8, runs RN49 and RN50, the same command): ets_arima.py --seed 2 --n 500 --adam 20 generated 500 ETS+ARIMA and 500 ETS+SARIMA models (seed 2, new against the exploratory runs RN46-RN48, seed 1), built each in ADAM's form, compared the matrices of 20 of them with adam() (0 lags differed, matrices within 4.4e-15), and decided admissibility by the check of the parts apart, the check of the whole model and the PBH reference.", "setup": { "studied": [ "D13" ], "cases": [ "D22" ], "procedure": [ "D26", "D5" ], "lens": [ "D29" ] }, "availability": { "level": "public" }, "lab_runs": [ "RN49" ], "results": [ { "id": "R17", "variable": "max abs difference between the ARIMA form of the ARIMA part and its ARI and MA polynomials", "value": 2.81719e-15, "conditions": { "models": 1000 } }, { "id": "R18", "variable": "models whose component lags differ from those of adam()", "value": 0, "conditions": { "models": 20 } }, { "id": "R19", "variable": "max abs difference between the matrices built here and those of adam()", "value": 4.44089e-15, "conditions": { "models": 20 } } ], "execution": { "code_version": "acb23120590a35c6b8b19290dbf861f830965b81", "software": { "python": "3.13.16", "platform": "Linux-6.18.44-fc-v140-x86_64-with-glibc2.39" } }, "artefacts": [ "A10", "A1" ], "tables": [ { "id": "T1", "description": "Admissibility decisions of the check of the parts apart ('split') and of the check of the whole model ('joint') against the PBH reference on 500 models of each family: models whose decision differs, models accepted by the check and rejected by the reference, and models rejected by the check and accepted by the reference", "conditions": { "models": 500 }, "row_labels": [ "family", "check" ], "columns": [ "models whose admissibility decision differs from the PBH reference", "models accepted by the check and rejected by the PBH reference", "models rejected by the check and accepted by the PBH reference" ], "rows": [ { "id": "r1", "label": [ "ETS+ARIMA", "split" ], "values": [ 166, 104, 62 ] }, { "id": "r2", "label": [ "ETS+ARIMA", "joint" ], "values": [ 0, 0, 0 ] }, { "id": "r3", "label": [ "ETS+SARIMA", "split" ], "values": [ 159, 128, 31 ] }, { "id": "r4", "label": [ "ETS+SARIMA", "joint" ], "values": [ 0, 0, 0 ] } ], "artefact": "A18" }, { "id": "T2", "description": "Per family of 500 models: models found admissible by the PBH reference, and the largest absolute difference of the largest modulus between the check of the whole model and the PBH reference on the models both find inadmissible, with their number", "row_labels": [ "family" ], "columns": [ "max abs difference of the largest modulus, check of the whole model against the PBH reference, on models both find inadmissible", { "variable": "models found admissible by the PBH reference", "conditions": { "models": 500 } }, "inadmissible" ], "rows": [ { "id": "r1", "label": [ "ETS+ARIMA" ], "values": [ 4.30767e-13, 159, 341 ] }, { "id": "r2", "label": [ "ETS+SARIMA" ], "values": [ 1.62688e-11, 87, 413 ] } ], "artefact": "A18" } ] }, { "id": "E24", "profile": "simulation", "lab_runs": [ "AC3.RN4", "AC3.RN5", "AC3.RN6", "AC3.RN7", "AC3.RN8", "AC3.RN9", "AC3.RN10" ], "artefacts": [ "A15" ], "parameters": "A15", "model": "random ADAM ETS+ARIMA models of a wider generator", "runs": 7, "availability": { "level": "public" }, "description": "Exploratory runs of the checker P4 (exact_oracle.py, seeds 21-23 and 31-34, 250 models each): admissibility by an exact oracle (rational determinants and the exact greatest common divisor of theta and phi, which cancels any shared factor) against the cancellation of cyclotomic factors ('author'), the PBH reference ('pbh') and the package ('pkg'); the number of models whose decision differs from the oracle.", "setup": { "studied": [ "D10", "D5", "D14" ], "cases": [ "D23" ], "procedure": [ "D26", "D39" ], "lens": [ "D29" ] }, "tables": [ { "id": "T1", "description": "Models, of 250 per seed, whose admissibility decision differs from the exact gcd oracle, for the cancellation of cyclotomic factors ('author'), the PBH reference ('pbh') and the package ('pkg'), seeds 21 to 23 and 31 to 34", "conditions": { "models": 250 }, "row_labels": [ "seed" ], "columns": [ { "variable": "models whose admissibility decision differs from the exact gcd oracle", "conditions": { "check": "author" } }, { "variable": "models whose admissibility decision differs from the exact gcd oracle", "conditions": { "check": "pbh" } }, { "variable": "models whose admissibility decision differs from the exact gcd oracle", "conditions": { "check": "pkg" } } ], "rows": [ { "id": "r1", "label": [ 21 ], "values": [ 0.0, 0.0, 1.0 ] }, { "id": "r2", "label": [ 22 ], "values": [ 0.0, 0.0, 0.0 ] }, { "id": "r3", "label": [ 23 ], "values": [ 0.0, 0.0, 0.0 ] }, { "id": "r4", "label": [ 31 ], "values": [ 0.0, 0.0, 0.0 ] }, { "id": "r5", "label": [ 32 ], "values": [ 0.0, 0.0, 0.0 ] }, { "id": "r6", "label": [ 33 ], "values": [ 0.0, 1.0, 1.0 ] }, { "id": "r7", "label": [ 34 ], "values": [ 0.0, 0.0, 1.0 ] } ], "artefact": "A18" } ] } ], "limitations": [ { "id": "L12", "category": "measurement", "statement": "The PBH reference is not exact for a repeated unit root: the checker P4 found a model whose cancelled MA polynomial keeps a double root at -1, which the reference accepted and the cancellation and an exact gcd rejected. The random models of the planned tests have no such root.", "affects": [ "C24", "C25" ], "impact": "reduces_transferability", "mitigation": "none" } ], "artefacts": [ { "id": "A1", "kind": "code", "uri": "https://github.com/openforecast-org/smooth/blob/fe3ecfad42341fb0d2b81d02a7919b43baa5f695/.claude/research/eigenChecker/admissibility.py", "availability": { "level": "public" }, "version": "sha256:f966bd9f065ee5453a2f809cb4e0f3aee938068589540b45ddf36eaa225c8b04", "licence": "LGPL-2.1-only", "description": "numpy prototype of the admissibility check of pure additive lagged SSOE models: lag-1 expansion, the per-lag check of smoothEigens(), the PBH reference, ARIMA-form polynomials by inverse DFT, cyclotomic cancellation, step-down test and bisection; ETS and GUM model builders" }, { "id": "A10", "kind": "code", "uri": "https://github.com/openforecast-org/smooth/blob/604dbcac0e3ca9cb0e8c7c228cb11b1d6161594c/claimset/adam-admissible-bounds/code/ets_arima.py", "availability": { "level": "public" }, "version": "sha256:f3e394d5f6563d9f9218c7519977a0227761bcdbf8428c8decd83811fa80f712", "licence": "LGPL-2.1-only", "description": "test script of Q2: random ETS+ARIMA models, the check of the parts apart and of the whole model against the PBH reference, the structure against adam(), and the bounds checkers of the package in R and Python against the prototype" }, { "id": "A15", "kind": "code", "uri": "https://github.com/openforecast-org/smooth/blob/604dbcac0e3ca9cb0e8c7c228cb11b1d6161594c/claimset/adam-admissible-bounds/code/checker/exact_oracle.py", "availability": { "level": "public" }, "version": "sha256:f07dd7d664eac2995262cbca9170f347dae9ef9c401645ab7a6259fb35730d75", "licence": "LGPL-2.1-only", "description": "the exact oracle of the checker P4 (with lib.py in the same folder): rational determinants and the exact gcd of theta and phi against the cancellation, the PBH reference and the package" }, { "id": "A18", "kind": "data", "uri": "https://entrylib.org/p/2026.000002/v3/files/files/adam-admissible-bounds.results.csv", "version": "sha256:96615c120ab55a010bb7fb0343bd8200729319db710c0e8a329209e385f674ac", "availability": { "level": "public" }, "licence": "CC-BY-4.0", "description": "Every result of the lab record at full precision, one row per result: the lab result, the run that produced it, the quantity, its conditions (JSON) and the value.", "columns": [ { "name": "lab_result", "dimension": "lab record result" }, { "name": "run", "dimension": "lab record run" }, { "name": "quantity", "dimension": "quantity" }, { "name": "conditions", "dimension": "conditions of the result" }, { "name": "value", "variable": "value of the quantity" } ] } ], "references": [ { "id": "B2", "target": "10.1016/0047-259X(73)90030-4", "id_type": "doi" } ] }