Advertisement

Surgical Endoscopy

, Volume 28, Issue 3, pp 704–711 | Cite as

Fundamentals of endoscopic surgery: creation and validation of the hands-on test

  • Melina C. Vassiliou
  • Brian J. Dunkin
  • Gerald M. Fried
  • John D. Mellinger
  • Thadeus Trus
  • Pepa Kaneva
  • Calvin Lyons
  • James R. KorndorfferJr.
  • Michael Ujiki
  • Vic Velanovich
  • Michael L. Kochman
  • Shawn Tsuda
  • Jose Martinez
  • Daniel J. Scott
  • Gary Korus
  • Adrian Park
  • Jeffrey M. Marks
Article

Abstract

Background

The Fundamentals of Endoscopic Surgery™ (FES) program consists of online materials and didactic and skills-based tests. All components were designed to measure the skills and knowledge required to perform safe flexible endoscopy. The purpose of this multicenter study was to evaluate the reliability and validity of the hands-on component of the FES examination, and to establish the pass score.

Methods

Expert endoscopists identified the critical skill set required for flexible endoscopy. They were then modeled in a virtual reality simulator (GI Mentor™ II, Simbionix™ Ltd., Airport City, Israel) to create five tasks and metrics. Scores were designed to measure both speed and precision. Validity evidence was assessed by correlating performance with self-reported endoscopic experience (surgeons and gastroenterologists [GIs]). Internal consistency of each test task was assessed using Cronbach’s alpha. Test–retest reliability was determined by having the same participant perform the test a second time and comparing their scores. Passing scores were determined by a contrasting groups methodology and use of receiver operating characteristic curves.

Results

A total of 160 participants (17 % GIs) performed the simulator test. Scores on the five tasks showed good internal consistency reliability and all had significant correlations with endoscopic experience. Total FES scores correlated 0.73, with participants’ level of endoscopic experience providing evidence of their validity, and their internal consistency reliability (Cronbach’s alpha) was 0.82. Test–retest reliability was assessed in 11 participants, and the intraclass correlation was 0.85. The passing score was determined and is estimated to have a sensitivity (true positive rate) of 0.81 and a 1-specificity (false positive rate) of 0.21.

Conclusions

The FES hands-on skills test examines the basic procedural components required to perform safe flexible endoscopy. It meets rigorous standards of reliability and validity required for high-stakes examinations, and, together with the knowledge component, may help contribute to the definition and determination of competence in endoscopy.

Keywords

Flexible endoscopy FES Assessment Education Training Examination 

Notes

Acknowledgments

The authors would like to acknowledge Jessica Mischna, Carla Bryant, and Monique Rose from SAGES for their unrelenting support, and phenomenal expertise, organizational, and administrative skills. In addition, we would also like to acknowledge Gina Adrales, Jake Seagull, Yo Kurashima, Ekaterina Lebedeva, and all of the members of the FES task force for their hard work and contributions. We would also like to thank psychometrician Kaaren Hoffman, PhD, of the University of Southern California, Division of Medical Education, Emerita status for her expertise in analyzing and interpreting the data. Finally, without the generosity of all of the participants, this study and this program would not have been possible.

Disclosures

Melina C Vassiliou—Covidien consultant, and unrestricted Grant to fund research. Brian J Dunkin—Boston Scientific and Covidien, hororaria. Gerald M Fried—Covidien unrestricted grant to fund research, son works for CAE Healthcare. John D Mellinger—Honoraria from Covidien, Boston Scientific, Lifecell and Olympus. Thadeus Trus—Boston Scientific and Olympus consulting. Pepa Kaneva—Covidien unrestricted grant to fund research. Michael Ujiki—Olympus consultant, GORE speaker. Michael L Kochman—BSC consultant, Covidien consultant, Olympus consultant, Cook Medical—patents sold, Merck (spouse): salary and stock, Dark Canyon Laboratory consultant—stock (limited partner). S Tsuda—LifeCell consultant. J Martinez—honoraria from Covidien, Boston Scientific, Lifecell, and Olympus. Daniel J Scott – Karl Storz Endoscopy, Covidien and Ethicon—lab equipment, Accelerated Technologies Inc., and Neat Stitch Inc. Consultant, Magnetically Anchored Instruments—Co-Inventor. Adrian Park—Stryker Endoscopy, honoraria and Advisory Committee—no potential conflict of interest. Jeffrey M Marks—Apollo endosurgery—advisory board member, Olympus consultant, WL GORE consultant, GI Supply consultant. Calvin Lyons, James R Korndorffer, Vic Velanovich and Gary Korus have no conflicts of interest or financial ties to disclose

References

  1. 1.
    Sachdeva AK, Buyske J, Dunnington GL et al (2011) A new paradigm for surgical procedural training. Curr Probl Surg 48:854–968PubMedCrossRefGoogle Scholar
  2. 2.
    Dunkin BJ, Vargo JJ (2008) Measuring procedural competence in endoscopy: what do the numbers really tell us? Gastrointest Endosc 68:1063–1065PubMedCrossRefGoogle Scholar
  3. 3.
    Bittner JG, Coverdill JE, Imam T, Deladisma AM, Edwards MA, Mellinger JD (2008) Do increased training requirements in gastrointestinal endoscopy and advanced laparoscopy necessitate a paradigm shift? A survey of program directors in surgery. J Surg Educ 65:418–430PubMedCrossRefGoogle Scholar
  4. 4.
    Vassiliou MC, Dunkin BJ, Marks JM, Fried GM (2010) FLS and FES: comprehensive models of training and assessment. Surg Clin North Am 90:535–558PubMedCrossRefGoogle Scholar
  5. 5.
    Yudkowsky R, Downing SM, Tekain A (2009) Standard setting. In: Downing SM, Yudkowsky R (eds) Assessment in health professions education. Routledge, New York, pp 119–148Google Scholar
  6. 6.
    Poulose BK, Vassiliou MC, Dunkin BJ, Mellinger JD, Fanelli RD, Martinez JM, Hazey JW, Sillin LF, Delaney CP, Velanovich V, Fried GM, Korndorffer JR Jr, Marks JM (2013) Fundamentals of Endoscopic Surgery cognitive examination: development and validity evidence. Surg Endosc. doi: 10.1007/s00464-013-3220-0
  7. 7.
    Haycock AV, Youd P, Bassett P, Saunders BP, Tekkis P, Thomas-Gibson S (2009) Simulator training improves practical skills in therapeutic GI endoscopy: results from a randomized, blinded, controlled study. Gastrointest Endosc 70:835–845PubMedCrossRefGoogle Scholar
  8. 8.
    Haycock A, Koch AD, Familiari P et al (2010) Training and transfer of colonoscopy skills: a multinational, randomized, blinded, controlled trial of simulator versus bedside training. Gastrointest Endosc 71:298–307PubMedCrossRefGoogle Scholar
  9. 9.
    Cohen J, Cohen SA, Vora KC et al (2006) Multicenter, randomized, controlled trial of virtual-reality simulator training in acquisition of competency in colonoscopy. Gastrointest Endosc 64:361–368PubMedCrossRefGoogle Scholar
  10. 10.
    Walsh CM, Sherlock ME, Ling SC, Carnahan H (2012) Virtual reality simulation training for health professions trainees in gastrointestinal endoscopy. Cochrane Database Syst Rev 6:CD008237. doi: 10.1002/14651858.CD008237.pub2 PubMedGoogle Scholar
  11. 11.
    Cohen J, Bosworth BP, Chak A et al (2012) Preservation and incorporation of valuable endoscopic innovations (PIVI) on the use of endoscopy simulators for training and assessing skill. Gastrointest Endosc 76:471–475PubMedCrossRefGoogle Scholar
  12. 12.
    Vassiliou MC, Kaneva PA, Poulose BK et al (2010) Global Assessment of Gastrointestinal Endoscopic Skills (GAGES): a valid measurement tool for technical skills in flexible endoscopy. Surg Endosc 24:1834–1841PubMedCrossRefGoogle Scholar
  13. 13.
    Vassiliou MC, Kaneva PA, Poulose BK et al (2010) How should we establish the clinical case numbers required to achieve proficiency in flexible endoscopy? Am J Surg 199:121–125PubMedCrossRefGoogle Scholar

Copyright information

© Society of American Gastrointestinal and Endoscopic Surgeons (SAGES) 2013

Authors and Affiliations

  • Melina C. Vassiliou
    • 1
  • Brian J. Dunkin
    • 2
  • Gerald M. Fried
    • 1
  • John D. Mellinger
    • 3
  • Thadeus Trus
    • 4
  • Pepa Kaneva
    • 1
  • Calvin Lyons
    • 2
  • James R. KorndorfferJr.
    • 5
  • Michael Ujiki
    • 6
  • Vic Velanovich
    • 7
  • Michael L. Kochman
    • 8
  • Shawn Tsuda
    • 9
  • Jose Martinez
    • 10
  • Daniel J. Scott
    • 11
  • Gary Korus
    • 8
  • Adrian Park
    • 12
  • Jeffrey M. Marks
    • 13
  1. 1.McGill University Health CentreMontreal General HospitalMontrealCanada
  2. 2.The Methodist HospitalHoustonUSA
  3. 3.Southern Illinois UniversityCarbondaleUSA
  4. 4.Dartmouth Hitchcock Medical CenterLebanonUSA
  5. 5.Tulane University School of MedicineNew OrleansUSA
  6. 6.NorthShore University Health SystemEvanstonUSA
  7. 7.University of South FloridaTampaUSA
  8. 8.University of Pennsylvania Health SystemPhiladelphiaUSA
  9. 9.University of Nevada School of MedicineLas VegasUSA
  10. 10.University of Miami Miller School of MedicineMiamiUSA
  11. 11.UT Southwestern Medical CenterDallasUSA
  12. 12.Anne Arundel Medical CenterAnnapolisUSA
  13. 13.University Hospitals Case Medical CenterClevelandUSA

Personalised recommendations