Indian Pediatrics

, Volume 48, Issue 4, pp 277–287

Receiver operating characteristic (ROC) curve for medical researchers

Perspective

Abstract

Sensitivity and specificity are two components that measure the inherent validity of a diagnostic test for dichotomous outcomes against a gold standard. Receiver operating characteristic (ROC) curve is the plot that depicts the trade-off between the sensitivity and (1-specificity) across a series of cut-off points when the diagnostic test is continuous or on ordinal scale (minimum 5 categories). This is an effective method for assessing the performance of a diagnostic test. The aim of this article is to provide basic conceptual framework and interpretation of ROC analysis to help medical researchers to use it effectively. ROC curve and its important components like area under the curve, sensitivity at specified specificity and vice versa, and partial area under the curve are discussed. Various other issues such as choice between parametric and non-parametric methods, biases that affect the performance of a diagnostic test, sample size for estimating the sensitivity, specificity, and area under ROC curve, and details of commonly used softwares in ROC analysis are also presented.

Key words

Sensitivity Specificity Receiver operating characteristic curve Sample size Optimal cut-off point Partial area under the curve 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Reddy S, Dutta S, Narang A. Evaluation of lactate dehydrogenase, creatine kinase and hepatic enzymes for the retrospective diagnosis of perinatal asphyxia among sick neonates. Indian Pediatr. 2008;45:144–147.PubMedGoogle Scholar
  2. 2.
    Sood SL, Saiprasad GS, Wilson CG. Mid-arm circumference at birth: A screening method for detection of low birth weight. Indian Pediatr. 2002;39:838–842.PubMedGoogle Scholar
  3. 3.
    Randev S, Grover N. Predicting neonatal hyperbilirubinemia using first day serum bilirubin levels. Indian J Pediatr. 2010;77:147–150.PubMedCrossRefGoogle Scholar
  4. 4.
    Vasudevan A, Malhotra A, Lodha R, Kabra SK. Profile of neonates admitted in pediatric ICU and validation of score for neonatal acute physiology (SNAP). Indian Pediatr. 2006;43:344–348.PubMedGoogle Scholar
  5. 5.
    Khanna R, Taneja V, Singh SK, Kumar N, Sreenivas V, Puliyel JM. The clinical risk index of babies (CRIB) score in India. Indian J Pediatr. 2002;69:957–960.PubMedCrossRefGoogle Scholar
  6. 6.
    Indrayan A. Medical Biostatistics (Second Edition). Boca Raton: Chapman & Hall/CRC Press; 2008. p. 263–267.Google Scholar
  7. 7.
    Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143:29–36.PubMedGoogle Scholar
  8. 8.
    Campbell G. Advances in statistical methodology for evaluation of diagnostic and laboratory tests. Stat Med. 1994;13:499–508.PubMedCrossRefGoogle Scholar
  9. 9.
    Bamber D. The area above the ordinal dominance graph and area below the receiver operating characteristic graph. J Math Psychol. 1975;12:387–415.CrossRefGoogle Scholar
  10. 10.
    DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the area under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44:837–845.PubMedCrossRefGoogle Scholar
  11. 11.
    Cleves MA. Comparative assessment of three common algorithms for estimating the variance of the area under the nonparametric receiver operating characteristic curve. Stata J. 2002;3:280–289.Google Scholar
  12. 12.
    Box GEP, Cox DR. An analysis of transformation. J Royal Statistical Society, Series B. 1964;26:211–252.Google Scholar
  13. 13.
    Zhou Xh, Obuchowski NA, McClish DK. Statistical Methods in Diagnostic Medicine. New York: John Wiley and Sons, Inc; 2002.CrossRefGoogle Scholar
  14. 14.
    Metz CE, Herman BA, Shen JH. Maximum likelihood estimation of receiver operating characteristic (ROC) curve from continuously distributed data. Stat Med. 1998;17:1033–1053.PubMedCrossRefGoogle Scholar
  15. 15.
    ROCKIT [Computer program]. Chicago: University of Chicago. Available from: www-radiology.uchicago. edu/krl/KRL_ROC/software_index6.htm. Accessed on February 27, 2007.Google Scholar
  16. 16.
    Faraggi D, Reiser B. Estimating of area under the ROC curve. Stat Med. 2002;21:3093–3106.PubMedCrossRefGoogle Scholar
  17. 17.
    Hajian Tilaki KO, Hanley JA, Joseph L, Collet JP. A comparison of parametric and nonparametric approaches to ROC analysis of quantitative diagnosis tests. Med Decis Making. 1997;17:94–102.PubMedCrossRefGoogle Scholar
  18. 18.
    Pepe M, Longton G, Janes H. Estimation and comparison of receiver operating characteristic curves. Stata J. 2009;9:1.PubMedGoogle Scholar
  19. 19.
    Jiang Y, Metz CE, Nishikawa RM. A receiver operating characteristic partial area index for highly sensitive diagnostic test. Radiology. 1996;201:745–750.PubMedGoogle Scholar
  20. 20.
    Youden WJ. An index for rating diagnostic test. Cancer. 1950;3:32–35.PubMedCrossRefGoogle Scholar
  21. 21.
    Perkins NJ, Schisterman EF. The inconsistency of ‘optimal’ cut points obtained using two criteria based on the receiver operating characteristic curve. Am J Epidemiol. 2006;163:670–675.PubMedCrossRefGoogle Scholar
  22. 22.
    Whiting P, Ruljes AW, Reitsma JB, Glas AS, Bossuyt PM, Kleijnen J. Sources of variation and bias in studies of diagnostic accuracy - a systematic review. Ann Intern Med. 2004;140:189–202.PubMedGoogle Scholar
  23. 23.
    Kelly S, Berry E, Proderick P, Harris KM, Cullingworth J, Gathercale L, et al. The identification of bias in studies of the diagnostic performance of imaging modalities. Br J Radiol. 1997;70:1028–1035.PubMedGoogle Scholar
  24. 24.
    Malaria Site. Peripheral smear study for malaria parasites - Available from: www.malariasite.com/malaria/DiagnosisOfMalaria.htm. Accessed on July 05, 2010.
  25. 25.
    Thomas S, Srivastava A, Jeyaseelan L, Dennison D, Chandy M. NESTROFT as screening test for the detection of thalassaemia & common haemoglobinopathies - an evaluation against a high performance liquid chromatographic method. Indian J Med Res. 1996;104:194–197.PubMedGoogle Scholar
  26. 26.
    Bachmann LM, Puhan MA, ter Riet G, Bossuyt PM. Sample sizes of studies on diagnostic accuracy: literature survey. BMJ. 2006;332:1127–1129.PubMedCrossRefGoogle Scholar
  27. 27.
    Malhotra RK, Indrayan A. A simple nomogram for sample size for estimating the sensitivity and specificity of medical tests. Indion J Ophthalmol. 2010;58:519–522.CrossRefGoogle Scholar
  28. 28.
    Kester AD, Buntinx F. Meta analysis of curves. Med Decis Making. 2000;20:430–439.PubMedCrossRefGoogle Scholar

Copyright information

© Indian Academy of Pediatrics 2011

Authors and Affiliations

  1. 1.Department of Biostatistics and Medical InformaticsUniversity College of Medical SciencesDelhiIndia
  2. 2.Department of Biostatistics and Medical InformaticsUniversity College of Medical SciencesDelhiIndia

Personalised recommendations