Fundamentals of endoscopic surgery: creation and validation of the hands-on test
- 749 Downloads
The Fundamentals of Endoscopic Surgery™ (FES) program consists of online materials and didactic and skills-based tests. All components were designed to measure the skills and knowledge required to perform safe flexible endoscopy. The purpose of this multicenter study was to evaluate the reliability and validity of the hands-on component of the FES examination, and to establish the pass score.
Expert endoscopists identified the critical skill set required for flexible endoscopy. They were then modeled in a virtual reality simulator (GI Mentor™ II, Simbionix™ Ltd., Airport City, Israel) to create five tasks and metrics. Scores were designed to measure both speed and precision. Validity evidence was assessed by correlating performance with self-reported endoscopic experience (surgeons and gastroenterologists [GIs]). Internal consistency of each test task was assessed using Cronbach’s alpha. Test–retest reliability was determined by having the same participant perform the test a second time and comparing their scores. Passing scores were determined by a contrasting groups methodology and use of receiver operating characteristic curves.
A total of 160 participants (17 % GIs) performed the simulator test. Scores on the five tasks showed good internal consistency reliability and all had significant correlations with endoscopic experience. Total FES scores correlated 0.73, with participants’ level of endoscopic experience providing evidence of their validity, and their internal consistency reliability (Cronbach’s alpha) was 0.82. Test–retest reliability was assessed in 11 participants, and the intraclass correlation was 0.85. The passing score was determined and is estimated to have a sensitivity (true positive rate) of 0.81 and a 1-specificity (false positive rate) of 0.21.
The FES hands-on skills test examines the basic procedural components required to perform safe flexible endoscopy. It meets rigorous standards of reliability and validity required for high-stakes examinations, and, together with the knowledge component, may help contribute to the definition and determination of competence in endoscopy.
KeywordsFlexible endoscopy FES Assessment Education Training Examination
The authors would like to acknowledge Jessica Mischna, Carla Bryant, and Monique Rose from SAGES for their unrelenting support, and phenomenal expertise, organizational, and administrative skills. In addition, we would also like to acknowledge Gina Adrales, Jake Seagull, Yo Kurashima, Ekaterina Lebedeva, and all of the members of the FES task force for their hard work and contributions. We would also like to thank psychometrician Kaaren Hoffman, PhD, of the University of Southern California, Division of Medical Education, Emerita status for her expertise in analyzing and interpreting the data. Finally, without the generosity of all of the participants, this study and this program would not have been possible.
Melina C Vassiliou—Covidien consultant, and unrestricted Grant to fund research. Brian J Dunkin—Boston Scientific and Covidien, hororaria. Gerald M Fried—Covidien unrestricted grant to fund research, son works for CAE Healthcare. John D Mellinger—Honoraria from Covidien, Boston Scientific, Lifecell and Olympus. Thadeus Trus—Boston Scientific and Olympus consulting. Pepa Kaneva—Covidien unrestricted grant to fund research. Michael Ujiki—Olympus consultant, GORE speaker. Michael L Kochman—BSC consultant, Covidien consultant, Olympus consultant, Cook Medical—patents sold, Merck (spouse): salary and stock, Dark Canyon Laboratory consultant—stock (limited partner). S Tsuda—LifeCell consultant. J Martinez—honoraria from Covidien, Boston Scientific, Lifecell, and Olympus. Daniel J Scott – Karl Storz Endoscopy, Covidien and Ethicon—lab equipment, Accelerated Technologies Inc., and Neat Stitch Inc. Consultant, Magnetically Anchored Instruments—Co-Inventor. Adrian Park—Stryker Endoscopy, honoraria and Advisory Committee—no potential conflict of interest. Jeffrey M Marks—Apollo endosurgery—advisory board member, Olympus consultant, WL GORE consultant, GI Supply consultant. Calvin Lyons, James R Korndorffer, Vic Velanovich and Gary Korus have no conflicts of interest or financial ties to disclose
- 5.Yudkowsky R, Downing SM, Tekain A (2009) Standard setting. In: Downing SM, Yudkowsky R (eds) Assessment in health professions education. Routledge, New York, pp 119–148Google Scholar
- 6.Poulose BK, Vassiliou MC, Dunkin BJ, Mellinger JD, Fanelli RD, Martinez JM, Hazey JW, Sillin LF, Delaney CP, Velanovich V, Fried GM, Korndorffer JR Jr, Marks JM (2013) Fundamentals of Endoscopic Surgery cognitive examination: development and validity evidence. Surg Endosc. doi: 10.1007/s00464-013-3220-0