Skip to main content
SpringerLink
Log in

Measures Not Directly Related to the 2 × 2 Contingency Table

  • Chapter
  • First Online:
The 2x2 Matrix

Abstract

This chapter considers measures not directly related to the 2 × 2 contingency table but of some relevance to it, in particular the receiver operating characteristic (ROC) plot or curve which may be used to define “optimal” test cut-offs which may then be used in the construction of the 2 × 2 table, so influencing all the outcome measures considered in previous chapters.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Bangdiwala SI, Shankar V. The agreement chart. BMC Med Res Methodol. 2013;13:97.

    Article  Google Scholar 

  2. Bohning D, Holling H, Patilea V. A limitation of the diagnostic-odds ratio in determining an optimal cut-off value for a continuous diagnostic test. Stat Methods Med Res. 2011;20:541–50.

    Article  MathSciNet  Google Scholar 

  3. Choi BC. Slopes of a receiver operating characteristic curve and likelihood ratios for a diagnostic test. Am J Epidemiol. 1998;148:1127–32.

    Article  Google Scholar 

  4. Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. Hillsdale: Lawrence Erlbaum; 1988.

    MATH  Google Scholar 

  5. Cohen J. A power primer. Psychol Bull. 1992;112:155–9.

    Article  Google Scholar 

  6. Coffin M, Sukhatme S. Receiver operating characteristic studies and measurement errors. Biometrics. 1997;53:823–37.

    Article  MATH  Google Scholar 

  7. Cook J, Ramadas V. When to consult precision-recall curves. Stata J. 2020;20:131–48.

    Article  Google Scholar 

  8. Davis J, Goadrich M. The relationship between Precision-Recall and ROC curves. In: ICML ’06: Proceedings of the 23rd International Conference on Machine Learning. New York: ACM; 2006, p. 233–40.

    Google Scholar 

  9. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44:837–45.

    Article  MATH  Google Scholar 

  10. Ellis PD. The essential guide to effect sizes: statistical power, meta-analysis, and the interpretation of research results. Cambridge: Cambridge University Press; 2010.

    Book  Google Scholar 

  11. Fawcett T. An introduction to ROC analysis. Pattern Recogn Lett. 2006;27:861–74.

    Article  Google Scholar 

  12. Habibzadeh F, Yadollahie M. Number needed to misdiagnose: a measure of diagnostic test effectiveness. Epidemiology. 2013;24:170.

    Article  Google Scholar 

  13. Habibzadeh F, Habibzadeh P, Yadollahie M. On determining the most appropriate test cut-off value: the case of tests with continuous results. Biochem Med (Zagreb). 2016;26:297–307.

    Article  Google Scholar 

  14. Hajian-Tilaki K. The choice of methods in determining the optimal cut-off value for quantitative diagnostic test evaluation. Stat Methods Med Res. 2018;27:2374–83.

    Article  MathSciNet  Google Scholar 

  15. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143:29–36.

    Article  Google Scholar 

  16. Hanley JA, McNeil BJ. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology. 1983;148:839–43.

    Article  Google Scholar 

  17. Hilden J, Glasziou P. Regret graphs, diagnostic uncertainty and Youden’s index. Stat Med. 1996;15:969–86.

    Article  Google Scholar 

  18. Hsieh S, McGrory S, Leslie F, Dawson K, Ahmed S, Butler CR, et al. The Mini-Addenbrooke’s Cognitive Examination: a new assessment tool for dementia. Dement Geriatr Cogn Disord. 2015;39:1–11.

    Article  Google Scholar 

  19. Hsu LM. Biases of success rate differences shown in binomial effect size displays. Psychol Methods. 2004;9:183–97.

    Article  Google Scholar 

  20. Johnson NP. Advantages to transforming the receiver operating characteristic (ROC) curve into likelihood ratio co-ordinates. Stat Med. 2004;23:2257–66.

    Article  Google Scholar 

  21. Jones CM, Athanasiou T. Summary receiver operating characteristic curve analysis techniques in the evaluation of diagnostic tests. Ann Thorac Surg. 2005;79:16–20.

    Article  Google Scholar 

  22. Kaivanto K. Maximization of the sum of sensitivity and specificity as a diagnostic cutpoint criterion. J Clin Epidemiol. 2008;61:516–7.

    Article  Google Scholar 

  23. Keilwagen J, Grosse I, Grau J. Area under precision-recall curves for weighted and unweighted data. PLoS One. 2014;9(3):e92209.

    Article  Google Scholar 

  24. Krzanowski WJ, Hand DJ. ROC curves for continuous data. New York: CRC Press; 2009.

    Book  MATH  Google Scholar 

  25. Larner AJ. Effect size (Cohen’s d) of cognitive screening instruments examined in pragmatic diagnostic accuracy studies. Dement Geriatr Cogn Disord Extra. 2014;4:236–41.

    Article  Google Scholar 

  26. Larner AJ. The Q* index: a useful global measure of dementia screening test accuracy? Dement Geriatr Cogn Dis Extra. 2015;5:265–70.

    Article  Google Scholar 

  27. Larner AJ. Cognitive screening instruments for the diagnosis of mild cognitive impairment. Prog Neurol Psychiatry. 2016;20(2):21–6.

    Article  Google Scholar 

  28. Larner AJ. MACE for diagnosis of dementia and MCI: examining cut-offs and predictive values. Diagnostics (Basel). 2019;9:E51.

    Article  Google Scholar 

  29. Larner AJ. What is test accuracy? Comparing unitary accuracy metrics for cognitive screening instruments. Neurodegener Dis Manag. 2019;9:277–81.

    Article  Google Scholar 

  30. Larner AJ. Screening for dementia: Q* index as a global measure of test accuracy revisited. medRxiv. 2020. https://doi.org/10.1101/2020.04.01.20050567.

  31. Larner AJ. Defining “optimal” test cut-off using global test metrics: evidence from a cognitive screening instrument. Neurodegener Dis Manag. 2020;10:223–30.

    Article  Google Scholar 

  32. Larner AJ. The “attended alone” and “attended with” signs in the assessment of cognitive impairment: a revalidation. Postgrad Med. 2020;132:595–600.

    Article  Google Scholar 

  33. Larner AJ. Manual of screeners for dementia: pragmatic test accuracy studies. London: Springer; 2020.

    Book  Google Scholar 

  34. Larner AJ. Mini-Addenbrooke’s Cognitive Examination (MACE): a useful cognitive screening instrument in older people? Can Geriatr J. 2020;23:199–204.

    Article  Google Scholar 

  35. Larner AJ. Assessing cognitive screening instruments with the critical success index. Prog Neurol Psychiatry. 2021;25(3):33–7.

    Google Scholar 

  36. Lee J, Kim KW, Choi SH, Huh J, Park SH. Systematic review and meta-analysis of studies evaluating diagnostic test accuracy: a practical review for clinical researchers – part II. Statistical methods of meta-analysis. Korean J Radiol. 2015;16:1188–96.

    Article  Google Scholar 

  37. Liu X. Classification accuracy and cut point selection. Stat Med. 2012;31:2676–86.

    Article  MathSciNet  Google Scholar 

  38. Lusted L. Introduction to medical decision making. Springfield: Charles Thomas; 1968.

    Google Scholar 

  39. Lusted LB. Signal detectability and medical decision-making. Science. 1971;171:1217–9.

    Article  Google Scholar 

  40. Mallett S, Halligan S, Thompson M, Collins GS, Altman DG. Interpreting diagnostic accuracy studies for patient care. BMJ. 2012;345:e3999.

    Article  Google Scholar 

  41. Mbizvo GK, Larner AJ. Receiver operating characteristic plot and area under the curve with binary classifiers: pragmatic analysis of cognitive screening instruments. Neurodegener Dis Manag. 2021;11:353–60.

    Google Scholar 

  42. Metz CE. Basic principles of ROC analysis. Semin Nucl Med. 1978;8:283–98.

    Article  Google Scholar 

  43. Moses LE, Shapiro D, Littenberg B. Combining independent studies of a diagnostic test into a summary ROC curve: data-analytic approaches and some additional considerations. Stat Med. 1993;12:1293–316.

    Article  Google Scholar 

  44. Muschelli J. ROC and AUC with a binary predictor: a potentially misleading metric. J Classif. 2020;37:696–708.

    Google Scholar 

  45. Ozenne B, Subtil F, Maucort-Boulch D. The precision-recall curve overcame the optimism of the receiver operating characteristic curve in rare diseases. J Clin Epidemiol. 2015;68:855–9.

    Article  Google Scholar 

  46. Randolph JJ, Edmondson RS. Using the binomial effect size display to present the magnitude of effect sizes to the evaluation audience. Pract Assess Res Eval. 2005;10:14.

    Google Scholar 

  47. Remaley AT, Sampson ML, DeLeo JM, Remaley NA, Farsi BD, Zweig MH. Prevalence-value-accuracy plots: a new method for comparing diagnostic tests based on misclassification costs. Clin Chem. 1999;45:934–41.

    Article  Google Scholar 

  48. Richardson ML. The zombie plot: a simple graphic method for visualizing the efficacy of a diagnostic test. AJR Am J Roentgenol. 2016;207:W43–52.

    Article  Google Scholar 

  49. Rosenthal R, Rubin DR. A simple, general purpose display of magnitude of experimental effect. J Educ Psychol. 1982;74:166–9.

    Article  Google Scholar 

  50. Saito T, Rehmsmeier M. The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PLoS One. 2015;10(3):e0118432.

    Article  Google Scholar 

  51. Sawilowsky SS. New effect sizes rules of thumb. J Mod Appl Stat Methods. 2009;8:597–9.

    Article  Google Scholar 

  52. Schisterman EF, Perkins NJ, Liu A, Bondell H. Optimal cut-point and its corresponding Youden Index to discriminate individuals using pooled blood samples. Epidemiology. 2005;16:73–81.

    Article  Google Scholar 

  53. Swets JA. Measuring the accuracy of diagnostic systems. Science. 1988;240:1285–93.

    Article  MathSciNet  MATH  Google Scholar 

  54. Walter SD. Properties of the summary receiver operating characteristic (SROC) curve for diagnostic test data. Stat Med. 2002;21:1237–56.

    Article  Google Scholar 

  55. Weiskrantz L. Blindsight. A case study and implications, Oxford Psychology Series No. 12. Oxford: Clarendon Press; 1986.

    Google Scholar 

  56. Wojtowicz A, Larner AJ. Diagnostic test accuracy of cognitive screeners in older people. Prog Neurol Psychiatry. 2017;21(1):17–21.

    Article  Google Scholar 

  57. Youngstrom EA. A primer on receiver operating characteristic analysis and diagnostic efficiency statistics for pediatric psychology: we are ready to ROC. J Pediatr Psychol. 2014;39:204–21.

    Article  Google Scholar 

  58. Zhou XH, Obuchowski NA, McClish DK. Statistical methods in diagnostic medicine. 2nd ed. Hoboken: Wiley; 2011.

    Book  MATH  Google Scholar 

  59. Zou KH, O’Malley J, Mauri L. Receiver-operating characteristic analysis for evaluating diagnostic tests and predictive models. Circulation. 2007;115:654–7.

    Article  Google Scholar 

  60. Zweig MH, Campbell G. Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine. Clin Chem. 1993;39:561–77.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Cognitive Function Clinic, Walton Centre for Neurology and Neurosurgery, Liverpool, UK

    A. J. Larner

Authors
  1. A. J. Larner
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Larner, A.J. (2021). Measures Not Directly Related to the 2 × 2 Contingency Table. In: The 2x2 Matrix. Springer, Cham. https://doi.org/10.1007/978-3-030-74920-0_6

Download citation

Publish with us

Policies and ethics

Search

Navigation