{ "bundle_of": "C25", "schema_version": "0.40", "paper": { "id": "entrylib:2026.000002", "version": 3, "title": "Admissibility of lagged state space models through the ARIMA form of the whole model", "licence": "CC-BY-4.0", "status": "corrected", "date": "2026-11-10", "status_notice": { "date": "2026-11-10", "reason_code": "correction", "statement": "Limitations L2 and L4 are answered by a planned independent check (new evidence E25): an exact oracle in rational arithmetic with certified root enclosures, written by a new contributor (P5) without the prototype, compared on the models of the planned tests; it agrees with the PBH reference, the cyclotomic cancellation, the step-down test and the bisection on every model it decides, and its four hypotheses are reported as claims C29-C32, one of them inconclusive by its rule. The assumptions C14 (reworded to the band the tests leave out) and C19 are tested by it; claims C1-C5, C10, C11 and C16 carry its checks; claims C3, C4, C16 and C24-C26 relate to the earlier work they extend, supersede or contradict. The lab record (A9) and the results file (A18) are updated to the record's current version. No earlier claim's conclusion or ID changed.", "issued_by": "Ivan Svetunkov (ORCID 0000-0001-7826-0281)" } }, "claims": [ { "id": "C25", "statement": "On the same [[D22:1,000 random ETS+ARIMA models]], [[D10:the check of the whole model]] (the MA polynomial of the ARIMA form of the whole lagged SSOE after [[D6:cyclotomic cancellation]], with the AR polynomial of the ARIMA part stationary), gave the same [[D29:admissibility decision]] as the [[D5:PBH reference]] on every model, and the same [[D30:largest modulus]] to within rounding error on the models both found inadmissible.", "about": { "subject": "D10", "relation": "gives the same admissibility decision and largest mode modulus as", "object": [ "D29", "D30" ], "comparator": "D5", "conditions": [ "D22" ] }, "type": "methodological", "role": "principal", "novelty": "no_prior_found", "support": [ { "kind": "simulation", "evidence": "E21", "results": [ "E21.T1.r2", "E21.T1.r4", "E21.T2.r1", "E21.T2.r2" ], "artefacts": [ "A10", "A1" ] } ], "provenance": "author_stated", "hypothesis": { "lab_event": "H8", "outcome": "supported" }, "planning": "confirmatory", "planned_at": "2026-10-22T15:55:35Z", "planned_at_anchor": "git_remote", "test": { "framework": "deterministic", "parameters": { "sample": "500 models per family, one run, no stopping rule", "threshold": 1e-06 }, "quantities": { "j1": { "variable": "models whose admissibility decision differs from the PBH reference", "conditions": { "family": "ETS+ARIMA", "check": "joint" } }, "j2": { "variable": "models whose admissibility decision differs from the PBH reference", "conditions": { "family": "ETS+SARIMA", "check": "joint" } }, "d1": { "variable": "max abs difference of the largest modulus, check of the whole model against the PBH reference, on models both find inadmissible", "conditions": { "family": "ETS+ARIMA" } }, "d2": { "variable": "max abs difference of the largest modulus, check of the whole model against the PBH reference, on models both find inadmissible", "conditions": { "family": "ETS+SARIMA" } } }, "supported": "j1 == 0 and j2 == 0 and d1 <= threshold and d2 <= threshold", "not_supported": "j1 > 0 or j2 > 0 or d1 > threshold or d2 > threshold", "verdict": "computed", "runs": 1, "lab_plan": "PL8", "flags": [ "code_changed_after_plan" ] }, "scope": { "population": "random ADAM models that combine additive ETS with ARIMA", "conditions": [ "ETS: level; no, additive or damped trend; no or one seasonal lag m in {4, 12}; smoothing parameters drawn from wide ranges, some fixed at 0", "ARIMA: AR stationary, differences and MA up to order 2 at lag 1 and 1 at lag m, 15% over-differenced (an MA root at one on the unit circle of a differenced lag)", "ADAM's state space form of ARIMA (one state per lag of the ARI and MA polynomials), checked against the matrices of adam() on 20 of the models" ], "excludes": [ "regressors", "constant", "multiplicative models", "several seasonal lags", "CES", "GUM" ] }, "assumptions": [ "C28" ], "assessments": [ { "by": "author", "strength": "high", "rationale": "Planned test on new models with a fixed rule; the reference is exact for these models under the assumption C28. The same run and data as C24, tested under a second plan." } ], "limitations": [ "L12" ], "checks": [ { "by": "P4", "outcome": "confirmed", "extent": "full", "notes": "Reran with new seeds (script unchanged). Whole-model check (cancelled MA polynomial of the ARIMA form, AR stationary) against the PBH reference: seed 11 n=200 (AC3.RN1) 0 differing decisions in 200 ETS+ARIMA and 200 ETS+SARIMA models, max abs difference of the largest modulus 2.9e-13 and 1.5e-12; seed 12 n=200 (AC3.RN2) 0 and 0, max differences 7.9e-13 and 5.7e-9; author's RN50: 0, 0, 4.3e-13, 1.6e-11. Threshold 1e-6 met each time. Stronger test with an independent oracle (AC3.RN4-RN10, 1750 models, 7 seeds): a wider generator (constant state, ETS with two seasonal lags, AR up to 3, I up to 2, MA up to 3 including factors on the unit circle 1+-B, 1+B+B^2, 1+B^2, 1-B+B^2, rational parameters, exact Fraction determinants and exact gcd(theta, phi) instead of cyclotomic-only cancellation): the cyclotomic-cancelled check (the author's functions) disagrees with the exact gcd oracle on 0 of 1750 models, so cancelling only Phi_n is enough here; models whose unit-circle MA factor cancels and which are admissible were 3 to 7 per 250. Limitation of the reference, not of the claim: PBH (tolerance 1e-7 on singular values) accepted one model whose theta* keeps a double root at -1 (AC3.RN9, oracle and cancellation reject it), because repeated unit-circle eigenvalues are defective and the rank tests fail numerically; the author's generator never produces such a case (0 of 2000 differ in RN49/RN50/RN53), but PBH should not be treated as ground truth for repeated unit roots. Formal side (AS1): theta and phi of the whole model share no root inside the circle, so the cyclotomic-only cancellation is complete." } ], "relations": [ { "type": "extends", "reference": "B1", "locator": "Chapter 11", "self": false, "note": "Hyndman, Koehler, Ord and Snyder relate the reduced forms of linear innovations state space models to ARIMA models." } ], "verifiability": "open", "verifiability_flags": [] }, { "id": "C28", "statement": "For ETS+ARIMA models with a stationary AR polynomial, $\\theta(B)$ and $\\phi(B)$ of the whole model share no factor with a root in $|B| \\le 1$ that is not cyclotomic, so the [[D5:PBH reference]] and the cancelled MA polynomial decide the same admissibility.", "type": "methodological", "role": "supporting", "support": [ { "kind": "assumption" } ], "assumption_category": "theorem_condition", "check": "tested", "provenance": "author_stated", "check_evidence": "E24", "verifiability": "not_applicable", "verifiability_flags": [] } ], "definitions": [ { "id": "D2", "term": "expanded discount matrix", "definition": "The transition matrix $\\tilde D$ of size $\\sum_j l_j$ of the lag-1 system equivalent to $v_t = D v_{t-l}$, in which component $j$ carries itself and its $l_j - 1$ lagged copies; $\\tilde F$, $\\tilde g$ and $\\tilde w$ are expanded in the same way, so that $\\tilde D = \\tilde F - \\tilde g \\tilde w'$.", "kind": "concept" }, { "id": "D3", "term": "decaying weights", "definition": "A lagged SSOE model has decaying weights when the weights $\\tilde w' \\tilde D^j \\tilde g$ of the past observations in its fitted values tend to zero, that is, when every eigenvalue $\\lambda$ of its expanded discount matrix that is reachable from $\\tilde g$ and observable through $\\tilde w$ has $|\\lambda| < 1$; the largest such $|\\lambda|$ is its largest mode modulus. A mode that $\\tilde g$ does not reach but $\\tilde w$ sees carries an initial state whose effect is not covered: with $|\\lambda| > 1$ it grows.", "kind": "concept" }, { "id": "D5", "term": "PBH reference", "definition": "The largest modulus among the eigenvalues $\\lambda$ of the expanded discount matrix for which the smallest singular values of $[\\lambda I - \\tilde D;\\ \\tilde w']$ and of $[\\lambda I - \\tilde D,\\ \\tilde g]$ both exceed $10^{-7}(\\|\\tilde D\\|_2 + \\|\\tilde g\\| + \\|\\tilde w\\|)$ (Popov-Belevitch-Hautus rank tests), computed with numpy as pbh_max().", "kind": "method" }, { "id": "D6", "term": "cyclotomic cancellation", "definition": "The procedure that computes the coefficients of $$\\begin{aligned} \\theta(B) &= \\det(I - D L(B)) \\\\ \\phi(B) &= \\det(I - F L(B)) \\end{aligned}$$ by an inverse discrete Fourier transform of their values at the $N$-th roots of unity ($N$ the smallest power of 2 above $\\sum_j l_j$), and divides both by every cyclotomic polynomial $\\Phi_n(B)$, $n$ dividing one of the lags, for as long as it divides both with a remainder at most $10^{-9}$ times the sum of absolute coefficients; the quotient of $\\theta$ is the cancelled MA polynomial $\\theta^*(B)$.", "kind": "method" }, { "id": "D7", "term": "step-down test", "definition": "The Levinson-Durbin step-down (Schur-Cohn) recursion on $1 + c_1 B + \\dots + c_n B^n$: for $j = n, \\dots, 1$, $$\\kappa_j = c_j, \\qquad c_i \\leftarrow \\frac{c_i - \\kappa_j c_{j-i}}{1 - \\kappa_j^2};$$ the test passes when every $|\\kappa_j| < 1$, which holds if and only if all roots lie outside the unit circle (the $\\kappa_j$ are the partial autocorrelations of the autoregression with this polynomial).", "source": "B2", "kind": "method" }, { "id": "D10", "term": "check of the whole model", "kind": "method", "definition": "Admissibility of a lagged SSOE model from the MA polynomial $$\\theta(B) = \\det(I - D L(B))$$ of its ARIMA form: $\\theta$ is divided by its common factors with $\\phi(B) = \\det(I - F L(B))$ by [[D6:cyclotomic cancellation]], and the model is accepted when the cancelled polynomial $\\theta^*$ has no root in $|B| \\le 1$ and, for a model with ARIMA components, the AR polynomial of its ARIMA part is stationary; its largest mode modulus is $1/\\min|B|$ over the roots of $\\theta^*$. In the numpy prototype (admissibility.py, confirm.py, ets_arima.py) the roots come from numpy; in smooth the [[D7:step-down test]] decides and bisection gives the modulus." }, { "id": "D14", "term": "bounds checker of adam()", "kind": "method", "definition": "adam_bounds_checker() of adam(), om() and omg() in R (R/utils-adam.R) and of ADAM, OM and OMG in Python, with bounds=\"admissible\" after this study: smoothEigensCpp() on the whole model, with a shortcut that accepts a pure ARIMA model (without ETS and adaptive regressors) whose MA factors are invertible without running it; the penalty is zero, or 1e100 times the largest root modulus.", "broader": [ "D10" ] }, { "id": "D22", "term": "random ETS+ARIMA models", "kind": "data", "definition": "Random ADAM models combining additive ETS with ARIMA (ets_arima.py), 500 with ARIMA at lag 1 and 500 with seasonal ARIMA per seed (seed 2 for plans PL7 and PL8, seed 3 for plan PL9): ETS with a level; no, additive or damped trend; no or one seasonal lag $m$ in {4, 12}; smoothing parameters drawn from wide ranges, some fixed at 0; ARIMA with a stationary AR polynomial, differences and MA up to order 2 at lag 1 and 1 at lag $m$, 15% over-differenced (an MA root at one on the unit circle of a differenced lag); built in ADAM's state space form (one state per lag of the ARI and MA polynomials)." }, { "id": "D23", "term": "ETS+ARIMA models of the checker's generator", "kind": "data", "definition": "Random ADAM ETS+ARIMA models from the generator of the checker P4 (code/checker/exact_oracle.py with lib.py), wider than the [[D22:random ETS+ARIMA models]]: 250 models for each of seeds 21-23 and 31-34." }, { "id": "D26", "term": "comparison with a reference", "kind": "design", "definition": "Each model's admissibility decision, and its largest mode modulus where both find it inadmissible, computed by the method studied and by a reference on the same model (the [[D5:PBH reference]], the roots by numpy, the numpy prototype of the check of the whole model or an exact oracle); the result is the number of models whose decisions differ and the largest absolute difference of the modulus. In the planned tests, models whose reference modulus lies within 1e-8 of 1 are left out and counted." }, { "id": "D29", "term": "admissibility decision", "kind": "variable", "definition": "Whether a method accepts a model as admissible ([[D3:decaying weights]]); compared between a method and a reference as the number of models on which the two differ." }, { "id": "D30", "term": "largest mode modulus", "kind": "variable", "definition": "The largest modulus of the modes that decide admissibility: for the [[D5:PBH reference]] the largest $|\\lambda|$ of the reachable and observable eigenvalues of the [[D2:expanded discount matrix]], for the check of the whole model $1/\\min|B|$ over the roots $B$ of the cancelled MA polynomial, for the per-lag check the largest $|\\lambda|$ of its blocks; a model is admissible when it is below 1." }, { "id": "D39", "term": "exact gcd oracle", "kind": "method", "definition": "Admissibility by rational determinants and the exact greatest common divisor of $\\theta$ and $\\phi$, which cancels any shared factor (code/checker/exact_oracle.py of the checker P4)." } ], "evidence": [ { "id": "E21", "profile": "simulation", "design": "other", "description": "Planned tests (lab plans PL7 and PL8, runs RN49 and RN50, the same command): ets_arima.py --seed 2 --n 500 --adam 20 generated 500 ETS+ARIMA and 500 ETS+SARIMA models (seed 2, new against the exploratory runs RN46-RN48, seed 1), built each in ADAM's form, compared the matrices of 20 of them with adam() (0 lags differed, matrices within 4.4e-15), and decided admissibility by the check of the parts apart, the check of the whole model and the PBH reference.", "setup": { "studied": [ "D10" ], "cases": [ "D22" ], "procedure": [ "D26", "D5" ], "lens": [ "D29", "D30" ] }, "availability": { "level": "public" }, "lab_runs": [ "RN50" ], "results": [ { "id": "R17", "variable": "max abs difference between the ARIMA form of the ARIMA part and its ARI and MA polynomials", "value": 2.81719e-15, "conditions": { "models": 1000 } }, { "id": "R18", "variable": "models whose component lags differ from those of adam()", "value": 0, "conditions": { "models": 20 } }, { "id": "R19", "variable": "max abs difference between the matrices built here and those of adam()", "value": 4.44089e-15, "conditions": { "models": 20 } } ], "execution": { "code_version": "acb23120590a35c6b8b19290dbf861f830965b81", "software": { "python": "3.13.16", "platform": "Linux-6.18.44-fc-v140-x86_64-with-glibc2.39" } }, "artefacts": [ "A10", "A1" ], "tables": [ { "id": "T1", "description": "Admissibility decisions of the check of the parts apart ('split') and of the check of the whole model ('joint') against the PBH reference on 500 models of each family: models whose decision differs, models accepted by the check and rejected by the reference, and models rejected by the check and accepted by the reference", "conditions": { "models": 500 }, "row_labels": [ "family", "check" ], "columns": [ "models whose admissibility decision differs from the PBH reference", "models accepted by the check and rejected by the PBH reference", "models rejected by the check and accepted by the PBH reference" ], "rows": [ { "id": "r1", "label": [ "ETS+ARIMA", "split" ], "values": [ 166, 104, 62 ] }, { "id": "r2", "label": [ "ETS+ARIMA", "joint" ], "values": [ 0, 0, 0 ] }, { "id": "r3", "label": [ "ETS+SARIMA", "split" ], "values": [ 159, 128, 31 ] }, { "id": "r4", "label": [ "ETS+SARIMA", "joint" ], "values": [ 0, 0, 0 ] } ], "artefact": "A18" }, { "id": "T2", "description": "Per family of 500 models: models found admissible by the PBH reference, and the largest absolute difference of the largest modulus between the check of the whole model and the PBH reference on the models both find inadmissible, with their number", "row_labels": [ "family" ], "columns": [ "max abs difference of the largest modulus, check of the whole model against the PBH reference, on models both find inadmissible", { "variable": "models found admissible by the PBH reference", "conditions": { "models": 500 } }, "inadmissible" ], "rows": [ { "id": "r1", "label": [ "ETS+ARIMA" ], "values": [ 4.30767e-13, 159, 341 ] }, { "id": "r2", "label": [ "ETS+SARIMA" ], "values": [ 1.62688e-11, 87, 413 ] } ], "artefact": "A18" } ] }, { "id": "E24", "profile": "simulation", "lab_runs": [ "AC3.RN4", "AC3.RN5", "AC3.RN6", "AC3.RN7", "AC3.RN8", "AC3.RN9", "AC3.RN10" ], "artefacts": [ "A15" ], "parameters": "A15", "model": "random ADAM ETS+ARIMA models of a wider generator", "runs": 7, "availability": { "level": "public" }, "description": "Exploratory runs of the checker P4 (exact_oracle.py, seeds 21-23 and 31-34, 250 models each): admissibility by an exact oracle (rational determinants and the exact greatest common divisor of theta and phi, which cancels any shared factor) against the cancellation of cyclotomic factors ('author'), the PBH reference ('pbh') and the package ('pkg'); the number of models whose decision differs from the oracle.", "setup": { "studied": [ "D10", "D5", "D14" ], "cases": [ "D23" ], "procedure": [ "D26", "D39" ], "lens": [ "D29" ] }, "tables": [ { "id": "T1", "description": "Models, of 250 per seed, whose admissibility decision differs from the exact gcd oracle, for the cancellation of cyclotomic factors ('author'), the PBH reference ('pbh') and the package ('pkg'), seeds 21 to 23 and 31 to 34", "conditions": { "models": 250 }, "row_labels": [ "seed" ], "columns": [ { "variable": "models whose admissibility decision differs from the exact gcd oracle", "conditions": { "check": "author" } }, { "variable": "models whose admissibility decision differs from the exact gcd oracle", "conditions": { "check": "pbh" } }, { "variable": "models whose admissibility decision differs from the exact gcd oracle", "conditions": { "check": "pkg" } } ], "rows": [ { "id": "r1", "label": [ 21 ], "values": [ 0.0, 0.0, 1.0 ] }, { "id": "r2", "label": [ 22 ], "values": [ 0.0, 0.0, 0.0 ] }, { "id": "r3", "label": [ 23 ], "values": [ 0.0, 0.0, 0.0 ] }, { "id": "r4", "label": [ 31 ], "values": [ 0.0, 0.0, 0.0 ] }, { "id": "r5", "label": [ 32 ], "values": [ 0.0, 0.0, 0.0 ] }, { "id": "r6", "label": [ 33 ], "values": [ 0.0, 1.0, 1.0 ] }, { "id": "r7", "label": [ 34 ], "values": [ 0.0, 0.0, 1.0 ] } ], "artefact": "A18" } ] } ], "limitations": [ { "id": "L12", "category": "measurement", "statement": "The PBH reference is not exact for a repeated unit root: the checker P4 found a model whose cancelled MA polynomial keeps a double root at -1, which the reference accepted and the cancellation and an exact gcd rejected. The random models of the planned tests have no such root.", "affects": [ "C24", "C25" ], "impact": "reduces_transferability", "mitigation": "none" } ], "artefacts": [ { "id": "A1", "kind": "code", "uri": "https://github.com/openforecast-org/smooth/blob/fe3ecfad42341fb0d2b81d02a7919b43baa5f695/.claude/research/eigenChecker/admissibility.py", "availability": { "level": "public" }, "version": "sha256:f966bd9f065ee5453a2f809cb4e0f3aee938068589540b45ddf36eaa225c8b04", "licence": "LGPL-2.1-only", "description": "numpy prototype of the admissibility check of pure additive lagged SSOE models: lag-1 expansion, the per-lag check of smoothEigens(), the PBH reference, ARIMA-form polynomials by inverse DFT, cyclotomic cancellation, step-down test and bisection; ETS and GUM model builders" }, { "id": "A10", "kind": "code", "uri": "https://github.com/openforecast-org/smooth/blob/604dbcac0e3ca9cb0e8c7c228cb11b1d6161594c/claimset/adam-admissible-bounds/code/ets_arima.py", "availability": { "level": "public" }, "version": "sha256:f3e394d5f6563d9f9218c7519977a0227761bcdbf8428c8decd83811fa80f712", "licence": "LGPL-2.1-only", "description": "test script of Q2: random ETS+ARIMA models, the check of the parts apart and of the whole model against the PBH reference, the structure against adam(), and the bounds checkers of the package in R and Python against the prototype" }, { "id": "A15", "kind": "code", "uri": "https://github.com/openforecast-org/smooth/blob/604dbcac0e3ca9cb0e8c7c228cb11b1d6161594c/claimset/adam-admissible-bounds/code/checker/exact_oracle.py", "availability": { "level": "public" }, "version": "sha256:f07dd7d664eac2995262cbca9170f347dae9ef9c401645ab7a6259fb35730d75", "licence": "LGPL-2.1-only", "description": "the exact oracle of the checker P4 (with lib.py in the same folder): rational determinants and the exact gcd of theta and phi against the cancellation, the PBH reference and the package" }, { "id": "A18", "kind": "data", "uri": "https://entrylib.org/p/2026.000002/v3/files/files/adam-admissible-bounds.results.csv", "version": "sha256:96615c120ab55a010bb7fb0343bd8200729319db710c0e8a329209e385f674ac", "availability": { "level": "public" }, "licence": "CC-BY-4.0", "description": "Every result of the lab record at full precision, one row per result: the lab result, the run that produced it, the quantity, its conditions (JSON) and the value.", "columns": [ { "name": "lab_result", "dimension": "lab record result" }, { "name": "run", "dimension": "lab record run" }, { "name": "quantity", "dimension": "quantity" }, { "name": "conditions", "dimension": "conditions of the result" }, { "name": "value", "variable": "value of the quantity" } ] } ], "references": [ { "id": "B2", "target": "10.1016/0047-259X(73)90030-4", "id_type": "doi" } ] }