Skip to main content
SpringerLink
Log in

Assessment of surgical performance of laparoscopic benign hiatal surgery: a systematic review

  • 2018 SAGES Oral
  • Published:
Surgical Endoscopy Aims and scope Submit manuscript

Abstract

Background

Operative skills correlate with patient outcomes, yet at the completion of training or after learning a new procedure, these skills are rarely formally evaluated. There is interest in the use of summative video assessment of laparoscopic benign foregut and hiatal surgery (LFS). If this is to be used to determine competency, it must meet the robust criteria established for high-stakes assessments. The purpose of this review is to identify tools that have been used to assess performance of LFS and evaluate the available validity evidence for each instrument.

Methods

A systematic search was conducted up to July 2017. Eligible studies reported data on tools used to assess performance in the operating room during LFS. Two independent reviewers considered 1084 citations for eligibility. The characteristics and testing conditions of each assessment tool were recorded. Validity evidence was evaluated using five sources of validity (content, response process, internal structure, relationship to other variables, and consequences).

Results

There were six separate tools identified. Two tools were generic to laparoscopy, and four were specific to LFS [two specific to Nissen fundoplication (NF), one heller myotomy (HM), and one paraesophageal hernia repair (PEH)]. Overall, only one assessment was supported by moderate evidence while the others had limited or unknown evidence. Validity evidence was based mainly on internal structure (all tools reporting reliability and item analysis) and content (two studies referencing previous papers for tool development in the context of clinical assessment, and four listing items without specifying the development procedures). There was little or no evidence supporting test response process (one study reporting rater training), relationship to other variables (two comparing scores in subjects with different clinical experience), and consequences (no studies). Two tools were identified to have evidence for video assessment, specific to NF.

Conclusion

There is limited evidence supporting the validity of assessment tools for laparoscopic foregut surgery. This precludes their use for summative video-based assessment to verify competency. Further research is needed to develop an assessment tool designed for this purpose.

Graphical abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Beard JD, Marriott J, Purdie H, Crossley J (2011) Assessing the surgical skills of trainees in the operating theatre: a prospective observational study of the methodology. Health Technol Assess. https://doi.org/10.3310/hta15010

    Article  PubMed  Google Scholar 

  2. Hamstra SJ, Dubrowski A (2005) Effective training and assessment of surgical skills, and the correlates of performance. Surg Innov 12:71–77

    Article  Google Scholar 

  3. Vassiliou MC, Feldman LS (2011) Objective assessment, selection, and certification in surgery. Surg Oncol 20:140–145

    Article  Google Scholar 

  4. Birkmeyer JD, Finks JF, O’Reilly A, Oerline M, Carlin AM, Nunn AR, Dimick J, Banerjee M, Birkmeyer NJO (2013) Surgical skill and complication rates after bariatric surgery. N Engl J Med 369:1434–1442

    Article  CAS  Google Scholar 

  5. Touchie C, Ten Cate O (2016) The promise, perils, problems and progress of competency-based medical education. Med Educ 50(1):93–100

    Article  Google Scholar 

  6. Feldman LS et al (2004) Relationship between objective assessment of technical skills and subjective in-training evaluations in surgical residents. J Am Coll Surg 198:105–110

    Article  Google Scholar 

  7. Larson JL, William RG, Ketchum J, Boehler ML, Dunnington GL (2005) Feasibility, reliability and validity of an operative performance rating system for evaluating surgery residents. Surgery 138:640–649

    Article  Google Scholar 

  8. Streiner DL (1985) Global rating scales. In: Neufeld VR, Norman GR (eds) Assessing clinical competence. Springer, New York, pp 119–141

    Google Scholar 

  9. Mackenzie H, Ni M, Miskovic D, Motson RW, Gudgeon M, Khan Z, Longman R, Coleman MG, Hanna GB (2015) Clinical validity of consultant technical skills assessment in the English National Training Programme for laparoscopic colorectal surgery. Br J Surg 102:991–997

    Article  CAS  Google Scholar 

  10. Miskovic D, Wyles SM, Carter F, Coleman MG, Hanna GB (2011) Development, validation and implementation of a monitoring tool for training in laparoscopic colorectal surgery in the English National Training Program. Surg Endosc 25:1136–1142

    Article  Google Scholar 

  11. Madani A, Vassiliou MC, Watanabe Y, Al-Halabi B, Al-Rowais MS, Deckelbaum DL, Fried GM, Feldman LS (2017) What are the principles that guide behaviors in the operating room?: creating a framework to define and measure performance. Ann Surg 265:255–267

    Article  Google Scholar 

  12. Medina M (2001) Formidable challenges to teaching advanced laparoscopic skills. J Soc Laparoendosc Surg 5:153–158

    CAS  Google Scholar 

  13. fellowshipcouncil.org

  14. Broeders JAJL, Draaisma WA, Jong HGRd, Smout AJPM, Lanschot JJBv, Broeders IAMJ, Gooszen HG (2011) Impact of surgeon experience on 5-year outcome of laparoscopic Nissen fundoplication. Arch Surg 146:340–346

    Article  Google Scholar 

  15. Birkmeyer JD, Stukel TA, Siewers AE, Goodney PP, Wennberg DE, Lucas FL (2003) Surgeon volume and operative mortality in the United States. N Engl J Med 349:2117–2127

    Article  CAS  Google Scholar 

  16. Vassiliou MC, Feldman LS, Andrew CG, Bergman S, Leffondre K, Stanbridge D, Fried GM (2005) A global assessment tool for evaluation of intraoperative laparoscopic skills. Am J Surg 190:107–113

    Article  Google Scholar 

  17. Downing SM (2003) Validity: on the meaningful interpretation of assessment data. Med Educ 37:830–837

    Article  Google Scholar 

  18. Kogan JR, Holmboe ES, Hauer KE (2009) Tools for direct observation and assessment of clinical skills of medical trainees: a systematic review. JAMA 302:1316–1326

    Article  CAS  Google Scholar 

  19. Ghaderi I, Manji F, Park YS, Juul D, Ott M, Harris I, Farrell TM (2015) Technical skills assessment toolbox: a review using the unitary framework of validity. Ann Surg 261:251–262

    Article  Google Scholar 

  20. Hogle NJ, Liu Y, Ogden RT, Fowler DL (2014) Evaluation of surgical fellows’ laparoscopic performance using Global Operative Assessment of Laparoscopic Skills (GOALS). Surg Endosc 28:1284–1290

    Article  Google Scholar 

  21. Watanabe Y, Madani A, Ito YM, Bilgic E, McKendy KM, Feldman LS, Fried GM, Vassiliou MC (2016) Psychometric properties of the Global Operative Assessment of Laparoscopic Skills (GOALS) using item response theory. Am J Surg. https://doi.org/10.1016/j.amjsurg.2016.09.050

    Article  PubMed  Google Scholar 

  22. Bilgic E, Watanabe Y, McKendy K, Munshi A, Ito YM, Fried GM, Feldman LS, Vassiliou MC (2016) Reliable assessment of operative performance. Am J Surg 211:426–430

    Article  Google Scholar 

  23. Ghaderi I, Auvergne L, Park YS, Farrell TM (2015) Quantitative and qualitative analysis of performance during advanced laparoscopic fellowship: a curriculum based on structured assessment and feedback. Am J Surg 209:71–78

    Article  Google Scholar 

  24. Ahlberg G, Kruuna O, Leijonmarck CE, Ovaska J, Rosseland A, Sandbu R, Stromberg C, Arvidsson D (2005) Is the learning curve for laparoscopic fundoplication determined by the teacher or the pupil? Am J Surg 189:184–189

    Article  Google Scholar 

  25. Peyre SE, Peyre CG, Hagen JA, Sullivan ME (2010) Reliability of a procedural checklist as a high-stakes measurement of advanced technical skill. Am J Surg 199:110–114

    Article  Google Scholar 

  26. Awad M, Awad F, Carter F, Jervis B, Buzink S, Foster J, Jakimowicz J, Francis NK (2018) Consensus views on the optimum training curriculum for advanced minimally invasive surgery: a Delphi study. Int J Surg 53:137–142

    Article  Google Scholar 

Download references

Funding

Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation is funded by an unrestricted educational grant from Medtronic.

Author information

Author notes

  1. Mohammed Al Mahroos is Co-primary author.

Authors and Affiliations

  1. Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre, Montreal, QC, Canada

    Elif Bilgic, Mohammed Al Mahroos, Gerald M. Fried, Melina C. Vassiliou & Liane S. Feldman

  2. Montreal General Hospital Medical Library, McGill University Health Centre, 1650, Cedar Avenue, L9. 309, Montréal, QC, H3G 1A4, Canada

    Tara Landry

Authors
  1. Elif Bilgic
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohammed Al Mahroos
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tara Landry
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gerald M. Fried
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Melina C. Vassiliou
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Liane S. Feldman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Liane S. Feldman.

Ethics declarations

Disclosures

Elif Bilgic, Mohammed Al Mahroos, Tara Landry, Gerald M. Fried, Melina C. Vassiliou, Liane S. Feldman have no relevant conflicts of interest or financial ties to disclose.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

Search strategy for medline

  1. 1.

    exp Esophageal Diseases/su [Surgery] (33036)

  2. 2.

    exp Esophagus/su [Surgery] (9601)

  3. 3.

    Hernia, Hiatal/su [Surgery] (1903)

  4. 4.

    exp Diverticulum, Esophageal/su [Surgery] (1098)

  5. 5.

    (esophag* or oesophag* or gastro?esophag* or para?esophag*).tw,kf. (167048)

  6. 6.

    Fundoplication/ (4057)

  7. 7.

    fundoplicat*.tw,kf. (5356)

  8. 8.

    (nissen adj3 (operat* or procedure*)).tw,kf. (362)

  9. 9.

    (heller adj3 myotom*).tw,kf. (687)

  10. 10.

    epiphrenic-diverticulectom*.tw,kf. (8)

  11. 11.

    mediastinal-dissect*.tw,kf. (283)

  12. 12.

    or/1-11 (175735)

  13. 13.

    exp Laparoscopy/ (84511)

  14. 14.

    laparoscop*.tw,kf. (107840)

  15. 15.

    13 or 14 (120628)

  16. 16.

    12 and 15 (7394)

  17. 17.

    Clinical Competence/ (80679)

  18. 18.

    (competen* or skill or skills).tw,kf. (235018)

  19. 19.

    17 or 18 (287948)

  20. 20.

    Educational Measurement/ (33649)

  21. 21.

    “Task Performance and Analysis”/ (28752)

  22. 22.

    Checklist/(4544)

  23. 23.

    ((operativ* or intraoperativ* or perform*) adj5 (assess* or evaluat* or measur*)).tw,kf. (294777)

  24. 24.

    “Surveys and Questionnaires”/ (386261)

  25. 25.

    ed.fs. (254539)

  26. 26.

    (tool or tools or grade or grading or checklist* or check-list* or questionnaire* or form or rating or score* or scoring or scale or scaling).tw,kf. (2951343)

  27. 27.

    or/20-26 (3530579)

  28. 28.

    19 and 27 (128912)

  29. 29.

    16 and 28 (116)

  30. 30.

    (“24414454” or “20103075” or “18417086”).ui. (3)

  31. 31.

    29 and 30 (3)

  32. 32.

    15 and 28 (2962)

  33. 33.

    (“26679826” or “25454951” or “24414454” or “20103075” or “18417086” or “27776756”).ui. (6)

  34. 34.

    32 and 33 (6)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bilgic, E., Al Mahroos, M., Landry, T. et al. Assessment of surgical performance of laparoscopic benign hiatal surgery: a systematic review. Surg Endosc 33, 3798–3805 (2019). https://doi.org/10.1007/s00464-019-06662-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00464-019-06662-9

Keywords

Search

Navigation