Advertisement

Surgical Endoscopy

, Volume 31, Issue 12, pp 5219–5227 | Cite as

Psychometric properties of the Fundamentals of Endoscopic Surgery (FES) skills examination

  • Matthew LineberryEmail author
  • E. Matthew Ritter
Article

Abstract

Background

The Fundamentals of Endoscopic Surgery (FES) manual skills examination is a simulation-based assessment of five foundational skills in endoscopic surgery. With the FES skills exam becoming part of the board certification process in general surgery, continual investigation is needed to determine the validity with which the exam is supporting inferences and decision-making about examinees, as well as how it might be improved.

Methods

The present study retrospectively analyzed performance and demographic details for the initial 344 examinees completing the FES skills exam.

Results

The five tasks showed distinct degrees of difficulty, with Loop Reduction being especially difficult for examinees. Tasks related to one another positively but moderately, suggesting that the exam assesses both general and task-specific skills. The number of lower-endoscopic cases completed by an examinee strongly predicted performance, while upper endoscopy experience and career level (e.g., resident vs. fellow vs. practicing) did not. Hand dominance and the type of simulator used were not found to be related to scores. However, three demographic variables that related to one another—gender, glove size, and height—were also related to performance and pass/fail status.

Conclusions

This study’s results generally support the validity argument for the FES skills exam while pointing to additional investigations to be undertaken as the exam is applied more broadly.

Keywords

Psychomotor skills Validity Psychometrics Simulation Surgery 

Notes

Acknowledgments

We would like to gratefully acknowledge Jessica Mischna, Sarah Colon, and Robyann Jumaoas from the SAGES FES Program for their help in providing the de-identified data.

Compliance with ethical standards

Disclosures

Drs. Lineberry and Ritter have no conflicts of interest or financial ties to disclose.

References

  1. 1.
    Hazey JW, Marks JM, Mellinger JD, Trus TL, Chand B, Delaney CP, Dunkin BJ, Fanelli RD, Fried GM, Martinez JM, Pearl JP, Poulose BK, Sillin LF, Vassiliou MC, Melvin WS (2014) Why fundamentals of endoscopic surgery (FES)? Surg Endosc 28:701–703CrossRefPubMedGoogle Scholar
  2. 2.
    Vassiliou MC, Dunkin BJ, Fried GM, Mellinger JD, Trus T, Kaneva P, Lyons C, Korndorffer JR Jr, Ujiki M, Velanovich V, Kochman ML, Tsuda S, Martinez J, Scott DJ, Korus G, Park A, Marks JM (2014) Fundamentals of endoscopic surgery: creation and validation of the hands-on test. Surg Endosc 28:704–711CrossRefPubMedGoogle Scholar
  3. 3.
    Poulose BK, Vassiliou MC, Dunkin BJ, Mellinger JD, Fanelli RD, Martinez JM, Hazey JW, Sillin LF, Delaney CP, Velanovich V, Fried GM, Korndorffer JR Jr, Marks JM (2014) Fundamentals of Endoscopic Surgery cognitive examination: development and validity evidence. Surg Endosc 28:631–638CrossRefPubMedGoogle Scholar
  4. 4.
    Mueller CL, Kaneva P, Fried GM, Feldman LS, Vassiliou MC (2014) Colonoscopy performance correlates with scores on the FES™ manual skills test. Surg Endosc 28(11):3081–3085CrossRefPubMedGoogle Scholar
  5. 5.
    American Board of Surgery (2017) Flexible Endoscopy Curriculum for General Surgery Residents. Available at https://www.absurgery.org/default.jsp?certgsqe_fec Accessed 22 March 2017
  6. 6.
    American Board of Surgery (2017) ABS Establishes New Requirement for Endoscopic Training and Assessment. Available at https://www.absurgery.org/default.jsp?news_fec Accessed 22 March 2017
  7. 7.
    American Educational Research Association, American Psychological Association, National Council on Measurement in Education (2014) Standards for Educational and Psychological Testing. Washington, DC: AERAGoogle Scholar
  8. 8.
    APA Publications and Communications Board Working Group on Journal Article Reporting Standards (2008) Reporting standards for research in psychology: Why do we need them? What might they be? American Psychologist 63(9):839–851Google Scholar
  9. 9.
    Cheng A, Kessler D, Mackinnon R, Chang TP, Nadkarni VM, Hunt EA, Duval-Arnold J, Yiqun L, Cook DA, Pusic M, Hui J, Moher D, Egger M, Auerbach M (2016) Reporting guidelines for health care simulation research: extensions to the CONSORT and STROBE statements. Adv Simul 1:25CrossRefGoogle Scholar
  10. 10.
    Mueller CL, Kaneva P, Fried GM, Mellinger JD, Marks JM, Dunkin BJ, van Sickle K, Vassiliou MC (2016) Validity evidence for a new portable, lower-cost platform for the fundamentals of endoscopic surgery skills test. Surg Endosc 30:1107–1112CrossRefPubMedGoogle Scholar
  11. 11.
    Edwards JR, Bagozzi RP (2000) On the nature and direction of relationships between constructs and measures. Psychol Methods 5(2):155–174CrossRefPubMedGoogle Scholar
  12. 12.
    Cortina JM (1993) What is coefficient alpha? An examination of theory and applications. J Appl Psychol 78:98–104CrossRefGoogle Scholar
  13. 13.
    Cronbach LJ, Shavelson RJ (2004) My current thoughts on coefficient alpha and successor procedures. Educ Psychol Measur 64(3):391–418CrossRefGoogle Scholar
  14. 14.
    Green SB, Yang Y (2009) Commentary on coefficient alpha: a cautionary tale. Psychometrika 74:121–135CrossRefGoogle Scholar
  15. 15.
    Schmitt N (1996) Uses and abuses of coefficient alpha. Psychol Assess 8:350–353CrossRefGoogle Scholar
  16. 16.
    Sijtsma K (2009) On the use, the misuse, and the very limited usefulness of Cronbach’s alpha. Psychometrika 74(1):107CrossRefPubMedGoogle Scholar
  17. 17.
    Yudkowsky R, Park YS, Lineberry M, Knox A, Ritter EM (2015) Setting mastery learning standards. Acad Med 90(11):1495–1500CrossRefPubMedGoogle Scholar
  18. 18.
    Cook DA (2014) Much ado about differences: why expert-novice comparisons add little to the validity argument. Adv Health Sci Educ 27:1–6CrossRefGoogle Scholar
  19. 19.
    Society for Industrial & Organizational Psychology (2003) Principles for the validation and use of personnel selection procedures, 4th edn. SIOP, Bowling GreenGoogle Scholar
  20. 20.
    Campbell SM, Collaer ML (2009) Stereotype threat and gender differences in performance on a novel visuospatial task. Psychol Women Q 33(4):437–444CrossRefGoogle Scholar
  21. 21.
    Kass SJ, Ahlers RH, Dugger M (1998) Eliminating gender differences through practice in an applied visual spatial task. Human Perform 11(4):337–349CrossRefGoogle Scholar
  22. 22.
    Ali A, Subhi Y, Ringsted C, Konge L (2015) Gender differences in the acquisition of surgical skills: a systematic review. Surg Endosc 29(11):3065–3073CrossRefPubMedGoogle Scholar
  23. 23.
    Adams DM, Fenton SJ, Schirmer BD, Mahvi DM, Horvath K, Nichol P (2008) One size does not fit all: current disposable laparoscopic devices do not fit the needs of female laparoscopic surgeons. Surg Endosc 22(10):2310–2313CrossRefPubMedGoogle Scholar
  24. 24.
    Cook DA, Zendejas B, Hamstra SJ, Hatala R, Brydges R (2014) What counts as validity evidence? Examples and prevalence in a systematic review of simulation-based assessment. Adv Health Sci Educ 19(2):233–250CrossRefGoogle Scholar
  25. 25.
    Cook DA, Lineberry M (2016) Consequences validity evidence: evaluating the impact of educational assessments. Acad Med 91(6):785–795CrossRefPubMedGoogle Scholar
  26. 26.
    Tolsgaard MG, Ringsted C (2014) Using equivalence designs to improve methodological rigor in medical education trials. Med Educ 48(2):220–221CrossRefPubMedGoogle Scholar
  27. 27.
    Cook DA, Hatala R (2016) Validation of educational assessments: a primer for simulation and beyond. Adv Simul 1(1):31CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2017

Authors and Affiliations

  1. 1.Zamierowski Institute for Experiential LearningUniversity of Kansas Medical Center and University of Kansas Health SystemKansas CityUSA
  2. 2.Department of SurgeryUniformed Services University of the HealthBethesdaUSA
  3. 3.Department of SurgeryThe Walter Reed National Military Medical CenterBethesdaUSA

Personalised recommendations