Advances in Health Sciences Education

, Volume 15, Issue 5, pp 647–658 | Cite as

Using signal detection theory to model changes in serial learning of radiological image interpretation

  • Kathy Boutis
  • Martin Pecaric
  • Brian Seeto
  • Martin Pusic
Article

Abstract

Signal detection theory (SDT) parameters can describe a learner’s ability to discriminate (d′) normal from abnormal and the learner’s criterion (λ) to under or overcall abnormalities. To examine the serial changes in SDT parameters with serial exposure to radiological cases. 46 participants were recruited for this study: 20 medical students (MED), 6 residents (RES), 12 fellows (FEL), 5 staff pediatric emergency physicians (PEM), and 3 staff radiologists (RAD). Each participant was presented with 234 randomly assigned ankle radiographs using a web-based application. Participants were given a clinical scenario and considered 3 views of the ankle. They classified each case as normal or abnormal. For abnormal cases, they specified the location of the abnormality. Immediate feedback included highlighting on the images and the official radiologist’s report. The low experience group (MED, RES, FEL) showed steady improvement in discrimination ability with each case, while the high experience group (PEM, RAD) had higher and stable discrimination ability throughout the exercise. There was also a difference in the way the high and low experience groups balanced sensitivity and specificity (λ) with the low experience group tending to make more errors calling positive radiographs negative. This tendency was progressively less evident with each increase in expertise level. SDT metrics provide valuable insight on changes associated with learning radiograph interpretation, and may be used to design more effective instructional strategies for a given learner group.

Keywords

Image interpretation Signal detection theory 

References

  1. Abdi, H. (2009). Encylopedia of education. New York: Elsevier.Google Scholar
  2. Anderson, A. (2000). Injury—ankle. In I. G. Fleisher, S. Ludwig, F. Henretig, R. Ruddy, & B. Silverman (Eds.), Textbook of pediatric emergency medicine (pp. 321–329). Philadelphia: Lippincott Williams & Wilkins.Google Scholar
  3. Boutis, K., et al. (2001). Sensitivity of a clinical examination to predict the need for radiography in children with ankle injuries: A prospective study. The Lancet, 358, 2118–2121.CrossRefGoogle Scholar
  4. Chung, S., et al. (2004). Skull radiograph interpretation of children less than age two: How good are pediatric emergency physicians? Annals of Emergency Medicine, 43, 717–722.CrossRefGoogle Scholar
  5. Clarkson, E. (2007). Estimation receiver operating characteristic curve and ideal observers for combined detection/estimation tasks. Journal of the Optical Society of America A: Optics, Image Science, and Vision, 24, B91–B98.CrossRefGoogle Scholar
  6. Colvin, G. (2008). Talent is overrated: What really separates world-class performers from everybody else. New York: Penguin Group.Google Scholar
  7. Doubilet, P. M. (1988). Statistical techniques for medical decision making: Applications to diagnostic radiology. AJR. American Journal of Roentgenology, 150, 745–750.Google Scholar
  8. Dowling, S., et al. (2005). Comparison views to diagnose elbow injuries in children: A survey of Canadian non-pediatric emergency physicians. Canadian Journal of Emergency Medical Care, 7, 237–240.Google Scholar
  9. Ericsson, K. A. (2006). The Cambridge handbook of expertise and expert performance. Cambridge: Cambridge University Press.Google Scholar
  10. Ericsson, K. A., et al. (1993). The role of deliberate practice in the acquisition of expert performance. Psychological Review—New York, 100, 363–406.Google Scholar
  11. Hatala, R. M., et al. (2003). Practice makes perfect: The critical role of mixed practice in the acquisition of ECG interpretation skills. Advances in Health Sciences Education, 8, 17–26.CrossRefGoogle Scholar
  12. McNicol, D. (2004). A primer of signal detection theory. New York: Routledge.Google Scholar
  13. Metz, C. E. (1986). ROC methodology in radiologic imaging. Investigative Radiology, 21, 720–733.CrossRefGoogle Scholar
  14. Minnes, B. G., et al. (2005). Agreement in the interpretation of extremity radiographs of injured children and adolescents. Academic Emergency Medicine, 2, 826–830.CrossRefGoogle Scholar
  15. Nodine, C. F., et al. (1999). How experience and training influence mammography expertise. Academic Radiology, 6, 575–585.CrossRefGoogle Scholar
  16. Norman, G. R., et al. (1992). Expertise in visual diagnosis: A review of the literature. Academic Medicine, 67, S78–S83.CrossRefGoogle Scholar
  17. Obuchowski, N. A. (2003). Receiver operating curves and their use in radiology. Radiology, 229, 3–8.CrossRefGoogle Scholar
  18. Ramsay, C. R., et al. (2001). Statistical assessment of the learning curves of health technologies. Health Technology Assessment, 5, 1–79.Google Scholar
  19. Reeder, B. M., et al. (2004). Referral patterns to a pediatric orthopedic clinic: Implications for education and practice. Pediatrics, 113, 714–719.CrossRefGoogle Scholar
  20. Ryan, L. M., et al. (2004). Recognition and management of pediatric fractures by pediatric residents. Pediatrics, 114, 1530–1533.CrossRefGoogle Scholar
  21. Swensson, R. G. (1996). Unified measurement of observer performance in detecting and localizing target objects on images. Medical Physics, 23, 1709–1725.CrossRefGoogle Scholar
  22. Taras, H. L. (1990). Ten years of graduates evaluate a pediatric residency program. American Journal of Diseases of Children, 144, 1102–1105.Google Scholar
  23. Trainor, J. L., & Krug, S. E. (2000). The training of pediatric residents in the care of acutely ill and injured children. Archives of Pediatrics and Adolescent Medicine, 154, 1154–1159.Google Scholar
  24. Wickens, T. D. (2002). Elementary signal detection theory. New York: Oxford University Press.Google Scholar

Copyright information

© Springer Science+Business Media B.V. 2010

Authors and Affiliations

  • Kathy Boutis
    • 1
  • Martin Pecaric
    • 2
  • Brian Seeto
    • 3
  • Martin Pusic
    • 4
  1. 1.Department of Pediatrics, The Hospital for Sick ChildrenUniversity of TorontoTorontoCanada
  2. 2.Contrail Consulting ServicesTorontoCanada
  3. 3.School of MedicineQueen’s UniversityKingstonCanada
  4. 4.Department of Pediatrics, Morgan Stanley Children’s HospitalColumbia UniversityNew YorkUSA

Personalised recommendations