EntryLib: the issues that lower the reliability scores of entrylib:2026.000005@v1, by the EntryLib rubric 0.4.
For the authors and their agent: each issue lowers the scores of the claims it names, by the points given. Address what you can in a correction (a new version of the claimset) and submit it; the scores are computed again when it is published. Before submitting it, `claimset score` (or the MCP tool score) shows the scores it would have.
Scores are computed by rules from the claimset file (EntryLib rubric 0.4): they check the record, not the science, and may change as the rubric and reviews develop.
Rubric: https://github.com/entrylib/rubric/blob/main/RUBRIC.md
Page: https://entrylib.org/p/2026.000005@v1#score · Scores as JSON: https://entrylib.org/p/2026.000005@v1.score.json

PAPER: reliability 76 (medium), the mean of its 11 principal claims; the lowest is C9 at 66.

ISSUES, those of the principal claims first, most costly first:

P1. Limitation L1 (could reverse, not mitigated): Two runs per agent and phase: differences between the Sonnet and Haiku agents rest on two runs each and could reverse with more runs.
  Check: limitation (group: open); costs up to 10 points.
  Lowers: C2 (-10), C3 (-10), C4 (-10), C8 (-10), C9 (-10), C17 (-10), C18 (-10).
  Concerns: L1.
  To fix: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.

P2. Rests on evidence named without the observations the claim reports.
  Check: support_kind (group: support); costs up to 10 points.
  Lowers: C1 (-10), C4 (-10), C5 (-10), C6 (-10), C7 (-10), C8 (-10), C9 (-10), C10 (-10), C12 (-10), C15 (-10), C16 (-10), C19 (-10), C20 (-10, from C1, C4, C5, C6, C7, C8, C9, C10, C12).
  To fix: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked.

P3. A principal claim reported without a plan recorded beforehand.
  Check: exploratory_principal (group: plan); costs up to 5 points.
  Lowers: C1 (-5), C2 (-5), C3 (-5), C4 (-5), C5 (-5), C6 (-5), C7 (-5), C8 (-5), C9 (-5), C10 (-5), C11 (-5).
  To fix: Plan the measurement in the lab record or a registration before running it, or publish the lab record that shows every run.

P4. A principal claim related to no earlier work.
  Check: no_prior_work (group: context); costs up to 5 points.
  Lowers: C4 (-5), C9 (-5).
  To fix: Relate the claim to the earlier work it extends, addresses or departs from.

P5. Limitation L2 (could bias magnitude, not mitigated): The agents ran as subagents of the orchestrating session: they read the skills as files, the claimset guard hook of their folders was not active, and they received the orchestrating session's own project instructions (which led one Haiku run to ask its questions one at a time); four runs shared one 4-core machine.
  Check: limitation (group: open); costs up to 6 points.
  Lowers: C1 (-6), C2 (-3), C3 (-3), C5 (-6), C9 (-3).
  Concerns: L2.
  To fix: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.

P6. Assumption C21 is not checked: [[D10:The answer key]] states the findings that a correct paper on [[D6:the brief]] would state, with correct numbers.
  Check: unchecked_assumption (group: open); costs up to 3 points.
  Lowers: C1 (-3), C6 (-3), C7 (-3), C8 (-3, from C1, C6, C7).
  Concerns: C21.
  To fix: Test the assumption, or analyse how sensitive the claim is to it.

P7. Limitation L3 (could bias magnitude, mitigation partial): One scorer, the orchestrating agent, wrote the answer key and the protocol and scored every claim, knowing which agent wrote it.
  Check: limitation (group: open); costs up to 1 points.
  Lowers: C6 (-1), C7 (-1), C8 (-1, from C6, C7).
  Concerns: L3.
  To fix: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.

P8. Limitation L4 (could bias magnitude, not mitigated): The answer key was amended once during the lab phase, after a Haiku run reported statsforecast's β cap; the claimsets were scored against the amended key.
  Check: limitation (group: open); costs up to 6 points.
  Lowers: C6 (-6), C7 (-6), C8 (-3, from C6, C7), C20 (-6).
  Concerns: L4.
  To fix: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.

P9. Limitation L5 (reduces transferability, not mitigated): One study task (an ETS simulation comparison in Python), one version of claimset (0.41.6) and of its skills, one harness (Claude Code 2.1.322).
  Check: limitation (group: open); costs up to 1 points.
  Lowers: C1 (-1), C2 (-1), C3 (-1), C4 (-1), C5 (-1), C6 (-1), C7 (-1), C8 (-1), C9 (-1), C10 (-1), C11 (-1), C15 (-1).
  Concerns: L5.
  To fix: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.

P10. Limitation L6 (could bias magnitude, not mitigated): One claimset run (claimset-haiku-2) stopped with its own stated work undone and was sent one message with no content ("Continue.") before it finished.
  Check: limitation (group: open); costs up to 3 points.
  Lowers: C5 (-3).
  Concerns: L6.
  To fix: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.

P11. Assumption C22 is not checked: The [[D5:agents under test]], run as subagents of an orchestrating session, act as they would in sessions of their own started in their run folders.
  Check: unchecked_assumption (group: open); costs up to 3 points.
  Lowers: C2 (-3), C3 (-3), C5 (-3).
  Concerns: C22.
  To fix: Test the assumption, or analyse how sensitive the claim is to it.

CLAIMS: score (band): issues
C1: 75 (medium): P2 (-10), P5 (-6), P3 (-5), P6 (-3), P9 (-1)
C2: 78 (medium): P1 (-10), P3 (-5), P11 (-3), P5 (-3), P9 (-1)
C3: 78 (medium): P1 (-10), P3 (-5), P11 (-3), P5 (-3), P9 (-1)
C4: 69 (medium): P1 (-10), P2 (-10), P3 (-5), P4 (-5), P9 (-1)
C5: 72 (medium): P2 (-10), P5 (-6), P3 (-5), P10 (-3), P11 (-3), P9 (-1)
C6: 74 (medium): P2 (-10), P8 (-6), P3 (-5), P6 (-3), P7 (-1), P9 (-1)
C7: 74 (medium): P2 (-10), P8 (-6), P3 (-5), P6 (-3), P7 (-1), P9 (-1)
C8: 67 (medium): P1 (-10), P2 (-10), P3 (-5), P6 (-3), P8 (-3), P7 (-1), P9 (-1)
C9: 66 (medium): P1 (-10), P2 (-10), P3 (-5), P4 (-5), P5 (-3), P9 (-1)
C10: 84 (high): P2 (-10), P3 (-5), P9 (-1)
C11: 94 (high): P3 (-5), P9 (-1)
C12: 90 (high): P2 (-10)
C13: 100 (high): none
C14: 100 (high): none
C15: 89 (high): P2 (-10), P9 (-1)
C16: 90 (high): P2 (-10)
C17: 90 (high): P1 (-10)
C18: 90 (high): P1 (-10)
C19: 90 (high): P2 (-10)
C20: 84 (high): P2 (-10), P8 (-6)