At graduation from medical school, competency in otoscopy is often insufficient. Simulation-based training can be used to improve technical skills, but the suitability of the training model and assessment must be supported by validity evidence. The purpose of this study was to collect content validity evidence for a simulation-based test of handheld otoscopy skills.
First, a three-round Delphi study was conducted with a panel of nine clinical teachers in otorhinolaryngology (ORL) to determine the content requirements in our educational context. Next, the authenticity of relevant cases in a commercially available technology-enhanced simulator (Earsi, VR Magic, Germany) was evaluated by specialists in ORL. Finally, an integrated course was developed for the simulator based on these results.
The Delphi study resulted in nine essential diagnoses of normal variations and pathologies that all junior doctors should be able to diagnose with a handheld otoscope. Twelve out of 15 tested simulator cases were correctly recognized by at least one ORL specialist. Fifteen cases from the simulator case library matched the essential diagnoses determined by the Delphi study and were integrated into the course.
Content validity evidence for a simulation-based test of handheld otoscopy skills was collected. This informed a simulation-based course that can be used for undergraduate training. The course needs to be further investigated in relation to other aspects of validity and for future self-directed training.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Availability of data and materials
Data is available from the authors upon request.
Jones WS (2006) Video otoscopy: bringing otoscopy out of the “black box”. Int J Pediatr Otorhinolaryngol 70:1875–1883. https://doi.org/10.1016/j.ijporl.2006.06.014
Ringsted CV, Pallisgaard J, Falck G (2002) Physicians’ clinical skills after finishing internship. Ugeskr Læg 164:3211–3215
Oyewumi M, Brandt MG, Carrillo B et al (2016) Objective evaluation of otoscopy skills among family and community medicine, pediatric, and otolaryngology residents. J Surg Educ 73:129–135. https://doi.org/10.1016/j.jsurg.2015.07.011
Fisher EW, Pfleiderer AG (1992) Assessment of the otoscopic skills of general practitioners and medical students: is there room for improvement? Br J Gen Pract 42:65–67
Morris E, Kesser BW, Peirce-Cottler S, Keeley M (2012) Development and validation of a novel ear simulator to teach pneumatic otoscopy. Simul Healthc 7:22–26. https://doi.org/10.1097/SIH.0b013e31822eac39
Kyoto Kagakuku “Ear” Examination simulator 2. https://www.kyotokagaku.com/products/detail01/mw12.html. Accessed 19 May 2020
Wickens B, Lewis J, Morris DP et al (2015) Face and content validity of a novel, web-based otoscopy simulator for medical education. J Otolaryngol Head Neck Surg 44:7. https://doi.org/10.1186/s40463-015-0060-z
Stepniak C, Wickens B, Husein M et al (2017) Blinded randomized controlled study of a web-based otoscopy simulator in undergraduate medical education. Laryngoscope 127:1306–1311. https://doi.org/10.1002/lary.26246
Xu J, Campisi P, Forte V et al (2018) Effectiveness of discovery learning using a mobile otoscopy simulator on knowledge acquisition and retention in medical students: a randomized controlled trial. J Otolaryngol Head Neck Surg 47:1–8. https://doi.org/10.1186/s40463-018-0317-4
Lee DJ, Fu TS, Carrillo B et al (2015) Evaluation of an otoscopy simulator to teach otoscopy and normative anatomy to first year medical students. Laryngoscope 125:2159–2162. https://doi.org/10.1002/lary.25135
Davies J, Djelic L, Campisi P et al (2014) Otoscopy simulation training in a classroom setting: a novel approach to teaching otoscopy to medical students. Laryngoscope 124:2594–2597. https://doi.org/10.1002/lary.24682
Wu V, Sattar J, Cheon S, Beyea JA (2018) Ear disease knowledge and otoscopy skills transfer to real patients: a randomized controlled trial. J Surg Educ 75:1062–1069. https://doi.org/10.1016/j.jsurg.2017.12.011
Dell'Era V, Garzaro M, Carenzo L et al (2020) An innovative and safe way to train novice ear nose and throat residents through simulation: the simorl experience. Acta Otorhinolaryngol Ital 40:19–25. https://doi.org/10.14639/0392-100X-N0128
Wu V, Beyea JA (2017) Evaluation of a web-based module and an otoscopy simulator in teaching ear disease. Otolaryngol Head Neck Surg 156:272–277. https://doi.org/10.1177/0194599816677697
VRmagic Earsi Otoscope. https://www.vrmagic.com/medical-simulators/earsi-otoscope. Accessed 18 May 2020
Wiet GJ, Sørensen MS, Andersen SAW (2017) Otologic skills training. Otolaryngol Clin N Am 50:933–945. https://doi.org/10.1016/j.otc.2017.05.005
Saks K, Leijen Ä (2014) Distinguishing self-directed and self-regulated learning and measuring them in the E-learning context. Procedia Soc Behav Sci 112:190–198. https://doi.org/10.1016/j.sbspro.2014.01.1155
Yudkowsky R, Park YS, Lineberry M et al (2015) Setting mastery learning standards. Acad Med 90:1495–1500. https://doi.org/10.1097/ACM.0000000000000887
Cook DA, Beckman TJ (2006) Current concepts in validity and reliability for psychometric instruments: theory and application. Am J Med 119:166.e7–166.e16. https://doi.org/10.1016/j.amjmed.2005.10.036
Cook DA, Hatala R (2016) Validation of educational assessments: a primer for simulation and beyond. Adv Simul 1:31. https://doi.org/10.1186/s41077-016-0033-y
Borgersen NJ, Naur TMH, Sørensen SMD et al (2018) Gathering validity evidence for surgical simulation: a systematic review. Ann Surg 267:1063–1068. https://doi.org/10.1097/SLA.0000000000002652
Hsu CC, Sandford BA (2007) The Delphi technique: making sense of consensus. Pract Assess Res Eval 12:Article 10. https://doi.org/10.7275/pdz9-th90
Guldager MJ, Melchiors J, Andersen SAW (2020) Development and validation of an assessment tool for technical skills in handheld otoscopy. Ann Otol Rhinol Laryngol 129:715–721. https://doi.org/10.1177/0003489420904734
Downing SM (2006) Face validity of assessments: Faith-based interpretations or evidence-based science? Med Educ 40:7–8. https://doi.org/10.1111/j.1365-2929.2005.02361.x
McMillan SS, King M, Tully MP (2016) How to use the nominal group and Delphi techniques. Int J Clin Pharm 38:655–662. https://doi.org/10.1007/s11096-016-0257-x
Waggoner J, Carline JD, Durning SJ (2016) Is there a consensus on consensus methodology? Descriptions and recommendations for future consensus research. Acad Med 91:663–668. https://doi.org/10.1097/ACM.0000000000001092
The educational quality development fund at the Faculty of Health and Medical Sciences, University of Copenhagen, and the Toyota-Foundation, Denmark, provided funding for the study group to acquire the two Earsi simulators used in the study. SA has received research funding for an international postdoc from the Independent Research Fund Denmark (8026-00003B). The remaining authors have no other sources of funding or support to declare. Funding bodies were not involved in the conception, execution, interpretation, or publication of the study.
Conflicts of interest
The Regional Committee of Health Research Ethics of the Capital Region, Denmark, determined the study to be exempt (ref. 17006972).
Consent to participate
Informed consent was obtained from all participants in the study.
Consent for publication
Additional informed consent was obtained for publication of the article photograph.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
von Buchwald, J.H., Frendø, M., Guldager, M.J. et al. Content validity evidence for a simulation-based test of handheld otoscopy skills. Eur Arch Otorhinolaryngol 278, 2313–2320 (2021). https://doi.org/10.1007/s00405-020-06336-6
- Handheld otoscopy
- Simulation-based training
- Technical skills training
- Evidence-based medical education