Skip to main content

Content validity evidence for a simulation-based test of handheld otoscopy skills



At graduation from medical school, competency in otoscopy is often insufficient. Simulation-based training can be used to improve technical skills, but the suitability of the training model and assessment must be supported by validity evidence. The purpose of this study was to collect content validity evidence for a simulation-based test of handheld otoscopy skills.


First, a three-round Delphi study was conducted with a panel of nine clinical teachers in otorhinolaryngology (ORL) to determine the content requirements in our educational context. Next, the authenticity of relevant cases in a commercially available technology-enhanced simulator (Earsi, VR Magic, Germany) was evaluated by specialists in ORL. Finally, an integrated course was developed for the simulator based on these results.


The Delphi study resulted in nine essential diagnoses of normal variations and pathologies that all junior doctors should be able to diagnose with a handheld otoscope. Twelve out of 15 tested simulator cases were correctly recognized by at least one ORL specialist. Fifteen cases from the simulator case library matched the essential diagnoses determined by the Delphi study and were integrated into the course.


Content validity evidence for a simulation-based test of handheld otoscopy skills was collected. This informed a simulation-based course that can be used for undergraduate training. The course needs to be further investigated in relation to other aspects of validity and for future self-directed training.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2

Availability of data and materials

Data is available from the authors upon request.


  1. 1.

    Jones WS (2006) Video otoscopy: bringing otoscopy out of the “black box”. Int J Pediatr Otorhinolaryngol 70:1875–1883.

    Article  PubMed  Google Scholar 

  2. 2.

    Ringsted CV, Pallisgaard J, Falck G (2002) Physicians’ clinical skills after finishing internship. Ugeskr Læg 164:3211–3215

    PubMed  Google Scholar 

  3. 3.

    Oyewumi M, Brandt MG, Carrillo B et al (2016) Objective evaluation of otoscopy skills among family and community medicine, pediatric, and otolaryngology residents. J Surg Educ 73:129–135.

    Article  PubMed  Google Scholar 

  4. 4.

    Fisher EW, Pfleiderer AG (1992) Assessment of the otoscopic skills of general practitioners and medical students: is there room for improvement? Br J Gen Pract 42:65–67

    CAS  PubMed  PubMed Central  Google Scholar 

  5. 5.

    Morris E, Kesser BW, Peirce-Cottler S, Keeley M (2012) Development and validation of a novel ear simulator to teach pneumatic otoscopy. Simul Healthc 7:22–26.

    Article  PubMed  Google Scholar 

  6. 6.

    Kyoto Kagakuku “Ear” Examination simulator 2. Accessed 19 May 2020

  7. 7.

    Wickens B, Lewis J, Morris DP et al (2015) Face and content validity of a novel, web-based otoscopy simulator for medical education. J Otolaryngol Head Neck Surg 44:7.

    Article  PubMed  PubMed Central  Google Scholar 

  8. 8.

    Stepniak C, Wickens B, Husein M et al (2017) Blinded randomized controlled study of a web-based otoscopy simulator in undergraduate medical education. Laryngoscope 127:1306–1311.

    Article  PubMed  Google Scholar 

  9. 9.

    Xu J, Campisi P, Forte V et al (2018) Effectiveness of discovery learning using a mobile otoscopy simulator on knowledge acquisition and retention in medical students: a randomized controlled trial. J Otolaryngol Head Neck Surg 47:1–8.

    Article  Google Scholar 

  10. 10.

    Lee DJ, Fu TS, Carrillo B et al (2015) Evaluation of an otoscopy simulator to teach otoscopy and normative anatomy to first year medical students. Laryngoscope 125:2159–2162.

    Article  PubMed  Google Scholar 

  11. 11.

    Davies J, Djelic L, Campisi P et al (2014) Otoscopy simulation training in a classroom setting: a novel approach to teaching otoscopy to medical students. Laryngoscope 124:2594–2597.

    Article  PubMed  Google Scholar 

  12. 12.

    Wu V, Sattar J, Cheon S, Beyea JA (2018) Ear disease knowledge and otoscopy skills transfer to real patients: a randomized controlled trial. J Surg Educ 75:1062–1069.

    Article  PubMed  Google Scholar 

  13. 13.

    Dell'Era V, Garzaro M, Carenzo L et al (2020) An innovative and safe way to train novice ear nose and throat residents through simulation: the simorl experience. Acta Otorhinolaryngol Ital 40:19–25.

    Article  Google Scholar 

  14. 14.

    Wu V, Beyea JA (2017) Evaluation of a web-based module and an otoscopy simulator in teaching ear disease. Otolaryngol Head Neck Surg 156:272–277.

    Article  PubMed  Google Scholar 

  15. 15.

    VRmagic Earsi Otoscope. Accessed 18 May 2020

  16. 16.

    Wiet GJ, Sørensen MS, Andersen SAW (2017) Otologic skills training. Otolaryngol Clin N Am 50:933–945.

    Article  Google Scholar 

  17. 17.

    Saks K, Leijen Ä (2014) Distinguishing self-directed and self-regulated learning and measuring them in the E-learning context. Procedia Soc Behav Sci 112:190–198.

    Article  Google Scholar 

  18. 18.

    Yudkowsky R, Park YS, Lineberry M et al (2015) Setting mastery learning standards. Acad Med 90:1495–1500.

    Article  PubMed  Google Scholar 

  19. 19.

    Cook DA, Beckman TJ (2006) Current concepts in validity and reliability for psychometric instruments: theory and application. Am J Med 119:166.e7–166.e16.

    Article  Google Scholar 

  20. 20.

    Cook DA, Hatala R (2016) Validation of educational assessments: a primer for simulation and beyond. Adv Simul 1:31.

    Article  Google Scholar 

  21. 21.

    Borgersen NJ, Naur TMH, Sørensen SMD et al (2018) Gathering validity evidence for surgical simulation: a systematic review. Ann Surg 267:1063–1068.

    Article  PubMed  Google Scholar 

  22. 22.

    Hsu CC, Sandford BA (2007) The Delphi technique: making sense of consensus. Pract Assess Res Eval 12:Article 10.

  23. 23.

    Guldager MJ, Melchiors J, Andersen SAW (2020) Development and validation of an assessment tool for technical skills in handheld otoscopy. Ann Otol Rhinol Laryngol 129:715–721.

  24. 24.

    Downing SM (2006) Face validity of assessments: Faith-based interpretations or evidence-based science? Med Educ 40:7–8.

    Article  PubMed  Google Scholar 

  25. 25.

    McMillan SS, King M, Tully MP (2016) How to use the nominal group and Delphi techniques. Int J Clin Pharm 38:655–662.

    Article  PubMed  PubMed Central  Google Scholar 

  26. 26.

    Waggoner J, Carline JD, Durning SJ (2016) Is there a consensus on consensus methodology? Descriptions and recommendations for future consensus research. Acad Med 91:663–668.

    Article  PubMed  Google Scholar 

Download references


The educational quality development fund at the Faculty of Health and Medical Sciences, University of Copenhagen, and the Toyota-Foundation, Denmark, provided funding for the study group to acquire the two Earsi simulators used in the study. SA has received research funding for an international postdoc from the Independent Research Fund Denmark (8026-00003B). The remaining authors have no other sources of funding or support to declare. Funding bodies were not involved in the conception, execution, interpretation, or publication of the study.

Author information




MG, JM and SA were involved in the conception and design of the study. MG carried out data collection. JB, MF, JM and SA analysed the data. JB drafted the manuscript. MF, MG, JM and SA revised the manuscript critically for important intellectual content. All authors have approved the final version.

Corresponding author

Correspondence to Josefine Hastrup von Buchwald.

Ethics declarations

Conflicts of interest


Ethics approval

The Regional Committee of Health Research Ethics of the Capital Region, Denmark, determined the study to be exempt (ref. 17006972).

Consent to participate

Informed consent was obtained from all participants in the study.

Consent for publication

Additional informed consent was obtained for publication of the article photograph.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

von Buchwald, J.H., Frendø, M., Guldager, M.J. et al. Content validity evidence for a simulation-based test of handheld otoscopy skills. Eur Arch Otorhinolaryngol 278, 2313–2320 (2021).

Download citation


  • Handheld otoscopy
  • Simulation-based training
  • Technical skills training
  • Evidence-based medical education
  • Otology