The claim
Claims it rests on
The AUC is equivalent to averaging the misclassification loss over a distribution of misclassification costs that depends on the classifier.
Scope: data: classifiers that score cases · Cited: Hand (2009) · Limitations: L1, L5 · Used by: C17
Evidence
The AUC averages the misclassification loss over a cost distribution that depends on the classifier, so different classifiers are compared with different weightings of the costs of errors; and it depends only on how the scores rank cases, not on whether the probabilities are right.
Limitations
These claims rest only on works checked by their abstracts and bibliographic records (Brier 1950, Hand 2009: lab notes NT7 and NT8); a full text could qualify what the abstract states.
The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it.
Definitions
Variables and measures 1
D22 AUC: The area under the ROC curve: the probability that a randomly chosen case of the positive class receives a higher score than a randomly chosen case of the negative class. Source: Hand (2009).
References
- Hand, D. J., 2009. Measuring classifier performance: a coherent alternative to the area under the ROC curve. Machine Learning 77, 103–123. https://doi.org/10.1007/s10994-009-5119-5 Used by: C42, D22
Score
Scores are computed by rules from the claimset file (EntryLib rubric 0.7): they check the record, not the science, and may change as the rubric and reviews develop. The rubric.
C17: 64, Partly supported
Novelty: summarises earlier work, by the authors
- −20 Evidence Rests on cited work only; cited work read in full, without the passage it rests on; an argument from other claims. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −10 Open problems Limitation L1 (could reverse, not mitigated): These claims rest only on works checked by their abstracts and bibliographic records (Brier 1950, Hand 2009: lab notes NT7 and NT8); a full text could qualify what the abstract states. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −3 Plan The review's search for the literature: none is recorded; the claims were confirmed against works the authors knew. To remove it: Record the protocol, then the searches and the screening of what they found (claimset lab protocol, search, screen), and export them.
- −3 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
C42: 67, Partly supported
Novelty: summarises earlier work, by the authors
- −20 Evidence Rests on cited work only; cited work read in full, without the passage it rests on; an argument from other claims. To remove it: Support the claim with evidence the claimset records, citing the result entries or table rows it reports, or have its proof checked; for cited work, read the work in full and quote the passage the claim rests on from a captured copy (claimset extension review).
- −10 Open problems Limitation L1 (could reverse, not mitigated): These claims rest only on works checked by their abstracts and bibliographic records (Brier 1950, Hand 2009: lab notes NT7 and NT8); a full text could qualify what the abstract states. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.
- −3 Open problems Limitation L5 (could bias magnitude, not mitigated): The claims restate EntryLib's study-planning guide on prediction, drafted by an agent of the same system that checked them here, and were stated before the works were read; the works were read to confirm what the guide states, and no search was run for work that disagrees with it. To remove it: Mitigate it with evidence, or narrow the claim's scope so that it no longer applies.