Skip to main content
SpringerLink
Log in

Development of a reliable simulation-based test for diagnostic abdominal ultrasound with a pass/fail standard usable for mastery learning

  • Ultrasound
  • Published:
European Radiology Aims and scope Submit manuscript

Abstract

Background

This study aimed to develop a test with validity evidence for abdominal diagnostic ultrasound with a pass/fail-standard to facilitate mastery learning.

Method

The simulator had 150 real-life patient abdominal scans of which 15 cases with 44 findings were selected, representing level 1 from The European Federation of Societies for Ultrasound in Medicine and Biology. Four groups of experience levels were constructed: Novices (medical students), trainees (first-year radiology residents), intermediates (third- to fourth-year radiology residents) and advanced (physicians with ultrasound fellowship). Participants were tested in a standardized setup and scored by two blinded reviewers prior to an item analysis.

Results

The item analysis excluded 14 diagnoses. Both internal consistency (Cronbach’s alpha 0.96) and inter-rater reliability (0.99) were good and there were statistically significant differences (p < 0.001) between all four groups, except the intermediate and advanced groups (p = 1.0). There was a statistically significant correlation between experience and test scores (Pearson’s r = 0.82, p < 0.001). The pass/fail-standard failed all novices (no false positives) and passed all advanced (no false negatives). All intermediate participants and six out of 14 trainees passed.

Conclusion

We developed a test for diagnostic abdominal ultrasound with solid validity evidence and a pass/fail-standard without any false-positive or false-negative scores.

Key Points

• Ultrasound training can benefit from competency-based education based on reliable tests.

• This simulation-based test can differentiate between competency levels of ultrasound examiners.

• This test is suitable for competency-based education, e.g. mastery learning.

• We provide a pass/fail standard without false-negative or false-positive scores.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Martínez-Ares D, Martín-Granizo Barrenechea I, Souto-Ruzo J, Yáñez López J, Pallarés Peral A, Vázquez-Iglesias JL (2005) The value of abdominal ultrasound in the diagnosis of colon cancer. Dig Organo Of Soc Esp Patol Dig 97:877–886

    Google Scholar 

  2. Torloni MR, Vedmedovska N, Merialdi M, Betrán AP, Allen T, González R et al (2009) Safety of ultrasonography in pregnancy: WHO systematic review of the literature and meta-analysis. Ultrasound Obstet Gynecol 33:599–608

    Article  CAS  PubMed  Google Scholar 

  3. Anders Ericsson K (2008) Deliberate Practice and Acquisition of Expert Performance: A General Overview. Acad Emerg Med 15:988–994

    Article  PubMed  Google Scholar 

  4. McGaghie WC (2015) Mastery learning: it is time for medical education to join the 21st century. Acad Med J 90:1438–1441

    Article  Google Scholar 

  5. European Society of Radiology (ESR) (2013) Organisation and practice of radiological ultrasound in Europe: a survey by the ESR Working Group on Ultrasound. Insights Imaging 4:401–407

    Article  Google Scholar 

  6. Garg M, Drolet BC, Tammaro D, Fischer SA (2014) Resident Duty Hours: A Survey of Internal Medicine Program Directors. J Gen Intern Med 29:1349–1354

    Article  PubMed  PubMed Central  Google Scholar 

  7. McGaghie WC, Miller GE, Sajid AW, Telder TV (1978) Competency-based curriculum development on medical education: an introduction. Public Health Pap 68:11–91

    Google Scholar 

  8. Downing SM, Yudkowsky R (2009) Assessment in health professions education, 1st edn. Routledge, New York, p 108 and p 143

  9. Ghaderi I, Manji F, Park YS, Juul D, Ott M, Harris I et al (2015) Technical skills assessment toolbox: a review using the unitary framework of validity. Ann Surg 261:251–262

    Article  PubMed  Google Scholar 

  10. Østergaard M, Ewertsen C, Konge L, Albrecht-Beste E, Bachmann Nielsen M (2016) Simulation-Based Abdominal Ultrasound Training – A Systematic Review. Ultraschall Med 37:253–261

    Article  PubMed  Google Scholar 

  11. Schallware (2016) Specifications of simulator. Schallware, Germany. Available via http://www.schallware.com/

  12. European Society of Radiology (2015) Guidelines & recommendations - Appendix 5. EFSUMB. Available via http://www.efsumb.org/guidelines/guidelines01.asp

  13. Thinggaard E, Bjerrum F, Strandbygaard J, Gögenur I, Konge L (2015) Validity of a cross-specialty test in basic laparoscopic techniques (TABLT). Br J Surg 102:1106–1113

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  14. Thomsen ASS, Kiilgaard JF, Kjaerbo H, la Cour M, Konge L (2015) Simulation-based certification for cataract surgery. Acta Ophthalmol 93:416–421

    Article  PubMed  Google Scholar 

  15. Dyre L, Nørgaard LN, Tabor A, Madsen ME, Sørensen JL, Ringsted C et al (2016) Collecting Validity Evidence for the Assessment of Mastery Learning in Simulation-Based Ultrasound Training. Ultraschall Med 37:386–392

    Article  CAS  PubMed  Google Scholar 

  16. Jacobsen ME, Andersen MJ, Hansen CO, Konge L (2015) Testing basic competency in knee arthroscopy using a virtual reality simulator: exploring validity and reliability. J Bone Joint Surg Am 97:775–781

    Article  PubMed  Google Scholar 

  17. Konge L, Clementsen PF, Ringsted C, Minddal V, Larsen KR, Annema JT (2015) Simulator training for endobronchial ultrasound: a randomised controlled trial. Eur Respir J 46:1140–1149

    Article  PubMed  Google Scholar 

  18. Konge L, Albrecht-Beste E, Nielsen MB (2014) Virtual-reality simulation-based training in ultrasound. Ultraschall Med 35:95–97

    Article  CAS  PubMed  Google Scholar 

  19. Ericsson KA (2008) Deliberate practice and acquisition of expert performance: a general overview. Acad Emerg Med Off J Soc Acad Emerg Med 15:988–994

    Article  Google Scholar 

  20. Bransford JD, Schwartz DL (1999) Rethinking Transfer: A Simple Proposal with Multiple Implications. Rev Res Educ 24:61

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Radiology, Copenhagen University Hospital, Rigshospitalet, Blegdamsvej 9, afd. 2023, 2100, Copenhagen O, Denmark

    Mia L. Østergaard, Kristina R. Nielsen & Michael B. Nielsen

  2. Department of Clinical Physiology, Nuclear Medicine and PET, Copenhagen University Hospital, Rigshospitalet, Blegdamsvej 9, 2100, Copenhagen, Denmark

    Elisabeth Albrecht-Beste

  3. Copenhagen Academy for Medical Education and Simulation CAMES, The Capital Region of Denmark, Blegdamsvej 9, 2100, Copenhagen, Denmark

    Lars Konge

Authors
  1. Mia L. Østergaard
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kristina R. Nielsen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Elisabeth Albrecht-Beste
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lars Konge
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Michael B. Nielsen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mia L. Østergaard.

Ethics declarations

Guarantor

The scientific guarantor of this publication is Michael Bachmann Nielsen.

Conflict of interest

The authors of this manuscript declare no relationships with any companies whose products or services may be related to the subject matter of the article.

Funding

The authors state that this work has not received any funding.

Statistics and biometry

One of the authors has significant statistical expertise.

Informed consent

Written informed consent was obtained from all subjects in this study.

Ethical approval

Institutional Review Board approval was obtained.

Methodology

  • Prospective

  • Observational

  • Performed at one institution

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Østergaard, M.L., Nielsen, K.R., Albrecht-Beste, E. et al. Development of a reliable simulation-based test for diagnostic abdominal ultrasound with a pass/fail standard usable for mastery learning. Eur Radiol 28, 51–57 (2018). https://doi.org/10.1007/s00330-017-4913-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00330-017-4913-x

Keywords

Search

Navigation