Abstract
Signal detection theory (SDT) parameters can describe a learner’s ability to discriminate (d′) normal from abnormal and the learner’s criterion (λ) to under or overcall abnormalities. To examine the serial changes in SDT parameters with serial exposure to radiological cases. 46 participants were recruited for this study: 20 medical students (MED), 6 residents (RES), 12 fellows (FEL), 5 staff pediatric emergency physicians (PEM), and 3 staff radiologists (RAD). Each participant was presented with 234 randomly assigned ankle radiographs using a web-based application. Participants were given a clinical scenario and considered 3 views of the ankle. They classified each case as normal or abnormal. For abnormal cases, they specified the location of the abnormality. Immediate feedback included highlighting on the images and the official radiologist’s report. The low experience group (MED, RES, FEL) showed steady improvement in discrimination ability with each case, while the high experience group (PEM, RAD) had higher and stable discrimination ability throughout the exercise. There was also a difference in the way the high and low experience groups balanced sensitivity and specificity (λ) with the low experience group tending to make more errors calling positive radiographs negative. This tendency was progressively less evident with each increase in expertise level. SDT metrics provide valuable insight on changes associated with learning radiograph interpretation, and may be used to design more effective instructional strategies for a given learner group.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abdi, H. (2009). Encylopedia of education. New York: Elsevier.
Anderson, A. (2000). Injury—ankle. In I. G. Fleisher, S. Ludwig, F. Henretig, R. Ruddy, & B. Silverman (Eds.), Textbook of pediatric emergency medicine (pp. 321–329). Philadelphia: Lippincott Williams & Wilkins.
Boutis, K., et al. (2001). Sensitivity of a clinical examination to predict the need for radiography in children with ankle injuries: A prospective study. The Lancet, 358, 2118–2121.
Chung, S., et al. (2004). Skull radiograph interpretation of children less than age two: How good are pediatric emergency physicians? Annals of Emergency Medicine, 43, 717–722.
Clarkson, E. (2007). Estimation receiver operating characteristic curve and ideal observers for combined detection/estimation tasks. Journal of the Optical Society of America A: Optics, Image Science, and Vision, 24, B91–B98.
Colvin, G. (2008). Talent is overrated: What really separates world-class performers from everybody else. New York: Penguin Group.
Doubilet, P. M. (1988). Statistical techniques for medical decision making: Applications to diagnostic radiology. AJR. American Journal of Roentgenology, 150, 745–750.
Dowling, S., et al. (2005). Comparison views to diagnose elbow injuries in children: A survey of Canadian non-pediatric emergency physicians. Canadian Journal of Emergency Medical Care, 7, 237–240.
Ericsson, K. A. (2006). The Cambridge handbook of expertise and expert performance. Cambridge: Cambridge University Press.
Ericsson, K. A., et al. (1993). The role of deliberate practice in the acquisition of expert performance. Psychological Review—New York, 100, 363–406.
Hatala, R. M., et al. (2003). Practice makes perfect: The critical role of mixed practice in the acquisition of ECG interpretation skills. Advances in Health Sciences Education, 8, 17–26.
McNicol, D. (2004). A primer of signal detection theory. New York: Routledge.
Metz, C. E. (1986). ROC methodology in radiologic imaging. Investigative Radiology, 21, 720–733.
Minnes, B. G., et al. (2005). Agreement in the interpretation of extremity radiographs of injured children and adolescents. Academic Emergency Medicine, 2, 826–830.
Nodine, C. F., et al. (1999). How experience and training influence mammography expertise. Academic Radiology, 6, 575–585.
Norman, G. R., et al. (1992). Expertise in visual diagnosis: A review of the literature. Academic Medicine, 67, S78–S83.
Obuchowski, N. A. (2003). Receiver operating curves and their use in radiology. Radiology, 229, 3–8.
Ramsay, C. R., et al. (2001). Statistical assessment of the learning curves of health technologies. Health Technology Assessment, 5, 1–79.
Reeder, B. M., et al. (2004). Referral patterns to a pediatric orthopedic clinic: Implications for education and practice. Pediatrics, 113, 714–719.
Ryan, L. M., et al. (2004). Recognition and management of pediatric fractures by pediatric residents. Pediatrics, 114, 1530–1533.
Swensson, R. G. (1996). Unified measurement of observer performance in detecting and localizing target objects on images. Medical Physics, 23, 1709–1725.
Taras, H. L. (1990). Ten years of graduates evaluate a pediatric residency program. American Journal of Diseases of Children, 144, 1102–1105.
Trainor, J. L., & Krug, S. E. (2000). The training of pediatric residents in the care of acutely ill and injured children. Archives of Pediatrics and Adolescent Medicine, 154, 1154–1159.
Wickens, T. D. (2002). Elementary signal detection theory. New York: Oxford University Press.
Acknowledgments
We would like to thank the Royal College of Physicians and Surgeons of Canada for their grant support of this research.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Boutis, K., Pecaric, M., Seeto, B. et al. Using signal detection theory to model changes in serial learning of radiological image interpretation. Adv in Health Sci Educ 15, 647–658 (2010). https://doi.org/10.1007/s10459-010-9225-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10459-010-9225-8