Skip to main content

Advertisement

SpringerLink
Log in

A prospective study demonstrating the reliability and validity of two procedure-specific evaluation tools to assess operative competence in laparoscopic colorectal surgery

  • Published:
Surgical Endoscopy Aims and scope Submit manuscript

Abstract

Background

Laparoscopic colorectal surgery is considered an advanced minimally invasive procedure with a long, variable learning curve. Developing an evaluation tool is essential to ensure that individuals reach a certain level of competence prior to performing this procedure independently. To achieve standardization and wide implementation, an assessment tool must be reflective of practice across many institutions.

Study design

The purpose of this study is to validate two procedure-specific evaluation tools for laparoscopic colorectal surgery that were developed using innovative consensus methodology. Two procedure-specific rating scales for laparoscopic right and sigmoid colectomy were created using the Delphi method. Nine novice and nine expert laparoscopic sigmoid colectomy videos were prospectively collected, and nine novice and ten expert laparoscopic right colectomy videos were recorded. The experts rated the videos using the procedure-specific technical skills evaluation tool for either laparoscopic right colectomy or laparoscopic sigmoid colectomy.

Results

There were statistically significant differences between the expert and novice scores on the laparoscopic right colectomy evaluation tool: the median score of novices was 63.8% and the expert score was 73.1% (p = 0.02). Similarly, there was a significant difference between the median novice score on the sigmoid tool (58.6%) compared with the median expert score (70.7%) (p = 0.003). Cronbach’s alpha was 0.82 for the right colectomy evaluation tool and 0.79 for the sigmoid rating scale.

Conclusions

The procedure-specific evaluation tools for laparoscopic right and sigmoid colectomy demonstrate strong reliability and construct validity, and have the potential to be used for technical skills assessment and feedback.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Elwyn G, Corrigan JM (2005) The patient safety story. BMJ 331:302–304

    Article  PubMed  Google Scholar 

  2. Cuschieri (2003) Medical errors, incidents, accidents and violations. Minim Invasive Ther Allied Technol 12:111–120

    PubMed  Google Scholar 

  3. Baker GR, Norton PG, Flintoft V, Blais R, Brown A, Cox J, Etchells E, Ghali WA, Hebert P, Majumdar SR, O’Beirne M, Palacios-Derflingher L, Reid RJ, Sheps S, Tamblyn R (2004) The Canadian adverse events study: the incidence of adverse events among hospital patients in Canada. CMAJ 170:1678–1686

    Article  PubMed  Google Scholar 

  4. Kim J, Edwards E, Bowne W, Castro A, Moon V, Gadangi P, Ferzli G (2007) Medial-to-lateral laparoscopic colon resection: a view beyond the learning curve. Surg Endosc 21:1503–1507

    Article  PubMed  CAS  Google Scholar 

  5. Schlachta CM, Mamazza J, Seshadri PA, Cadeddu M, Gregoire R, Poulin EC (2001) Defining a learning curve for laparoscopic colorectal resections. Dis Colon Rectum 44:217–222

    Article  PubMed  CAS  Google Scholar 

  6. Wishner JD, Baker JW Jr, Hoffman GC, Hubbard GW 2nd, Gould RJ, Wohlgemuth SD, Ruffin WK, Melick CF (1995) Laparoscopic-assisted colectomy. The learning curve. Surg Endosc 9:1179–1183

    Article  PubMed  CAS  Google Scholar 

  7. Senagore AJ, Luchtefeld MA, Mackeigan JM (1995) What is the learning curve for laparoscopic colectomy? Am Surg 61:681–685

    PubMed  CAS  Google Scholar 

  8. Wexner SD, Latulippe JF (1997) Laparoscopic colorectal surgery and cancer. Swiss Surg 3:266–273

    PubMed  CAS  Google Scholar 

  9. Bennett CL, Stryker SJ, Ferreira MR, Adams J, Beart RW Jr (1997) The learning curve for laparoscopic colorectal surgery. Preliminary results from a prospective analysis of 1,194 laparoscopic-assisted colectomies. Arch Surg 132:41–44 discussion 45

    Article  PubMed  CAS  Google Scholar 

  10. Tekkis PP, Senagore AJ, Delaney CP, Fazio VW (2005) Evaluation of the learning curve in laparoscopic colorectal surgery: comparison of right-sided and left-sided resections. Ann Surg 242:83–91

    Article  PubMed  Google Scholar 

  11. Dincler S, Koller MT, Steurer J, Bachmann LM, Christen D, Buchmann P (2003) Multidimensional analysis of learning curves in laparoscopic sigmoid resection: eight-year results. Dis Colon Rectum 46:1371–1378 discussion 1378–1379

    Article  PubMed  Google Scholar 

  12. Martin JA, Regehr G, Reznick R, MacRae H, Murnaghan J, Hutchison C, Brown M (1997) Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg 84:273–278

    Article  PubMed  CAS  Google Scholar 

  13. Reznick R, Regehr G, MacRae H, Martin J, McCulloch W (1997) Testing technical skill via an innovative “bench station” examination. Am J Surg 173:226–230

    Article  PubMed  CAS  Google Scholar 

  14. Datta V, Bann S, Aggarwal R, Mandalia M, Hance J, Darzi A (2006) Technical skills examination for general surgical trainees. Br J Surg 93:1139–1146

    Article  PubMed  CAS  Google Scholar 

  15. Beard JD, Choksy S, Khan S (2007) Assessment of operative competence during carotid endarterectomy. Br J Surg 94:726–730

    Article  PubMed  CAS  Google Scholar 

  16. Aggarwal R, Grantcharov T, Moorthy K, Milland T, Papasavas P, Dosis A, Bello F, Darzi A (2007) An evaluation of the feasibility, validity, and reliability of laparoscopic skills assessment in the operating room. Ann Surg 245:992–999

    Article  PubMed  Google Scholar 

  17. Ault G, Reznick R, MacRae H, Leadbetter W, DaRosa D, Joehl R, Peters J, Regehr G (2001) Exporting a technical skills evaluation technology to other sites. Am J Surg 182:254–256

    Article  PubMed  CAS  Google Scholar 

  18. Goff BA, Lentz GM, Lee D, Houmard B, Mandel LS (2000) Development of an objective structured assessment of technical skills for obstetric and gynecology residents. Obstet Gynecol 96:146–150

    Article  PubMed  CAS  Google Scholar 

  19. Shah J, Munz Y, Manson J, Moorthy K, Darzi A (2006) Objective assessment of small bowel anastomosis skill in trainee general surgeons and urologists. World J Surg 30:248–251

    Article  PubMed  Google Scholar 

  20. Siddiqui NY, Stepp KJ, Lasch SJ, Mangel JM, Wu JM (2008) Objective structured assessment of technical skills for repair of fourth-degree perineal lacerations. Am J Obstet Gynecol 199(676):e671–e676

    Google Scholar 

  21. Vassiliou MC, Feldman LS, Andrew CG, Bergman S, Leffondre K, Stanbridge D, Fried GM (2005) A global assessment tool for evaluation of intraoperative laparoscopic skills. Am J Surg 190:107–113

    Article  PubMed  Google Scholar 

  22. Grantcharov TP, Kristiansen VB, Bendix J, Bardram L, Rosenberg J, Funch-Jensen P (2004) Randomized clinical trial of virtual reality simulation for laparoscopic skills training. Br J Surg 91:146–150

    Article  PubMed  CAS  Google Scholar 

  23. Dath D, Regehr G, Birch D, Schlachta C, Poulin E, Mamazza J, Reznick R, MacRae HM (2004) Toward reliable operative assessment: the reliability and feasibility of videotaped assessment of laparoscopic technical skills. Surg Endosc 18:1800–1804

    Article  PubMed  CAS  Google Scholar 

  24. Sarker SK, Albrani T, Zaman A, Patel B (2008) Procedural performance in gastrointestinal endoscopy: an assessment and self-appraisal tool. Am J Surg 196:450–455

    Article  PubMed  Google Scholar 

  25. Larsen CR, Grantcharov T, Schouenborg L, Ottosen C, Soerensen JL, Ottesen B (2008) Objective assessment of surgical competence in gynaecological laparoscopy: development and validation of a procedure-specific rating scale. BJOG 115:908–916

    Article  PubMed  CAS  Google Scholar 

  26. Sarker SK, Kumar I, Delaney C (2010) Assessing operative performance in advanced laparoscopic colorectal surgery. World J Surg 34:1594–1603

    Article  PubMed  Google Scholar 

  27. Palter VN, MacRae HM, Grantcharov TP (2011) Development of an objective evaluation tool to assess technical skill in laparoscopic colorectal surgery: a Delphi methodology. Am J Surg 201:251–259

    Article  PubMed  Google Scholar 

  28. Regehr G, MacRae H, Reznick RK, Szalay D (1998) Comparing the psychometric properties of checklists and global rating scales for assessing performance on an OSCE-format examination. Acad Med 73:993–997

    Article  PubMed  CAS  Google Scholar 

  29. Sugden C, Aggarwal R (2010) Assessment and feedback in the skills laboratory and operating room. Surg Clin North Am 90:519–533

    Article  PubMed  Google Scholar 

  30. Gallagher AG, Ritter EM, Satava RM (2003) Fundamental principles of validation, and reliability: rigorous science for the assessment of surgical education and training. Surg Endosc 17:1525–1529

    Article  PubMed  CAS  Google Scholar 

  31. Young-Fadok TM, Nelson H (2000) Laparoscopic right colectomy: five-step procedure. Dis Colon Rectum 43:267–271 discussion 271–263

    Article  PubMed  CAS  Google Scholar 

  32. Senagore AJ, Delaney CP, Brady KM, Fazio VW (2004) Standardized approach to laparoscopic right colectomy: outcomes in 70 consecutive cases. J Am Coll Surg 199:675–679

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

V. Palter is funded by the Canadian Institute of Health Research (CIHR) Master’s Award.

Disclosure

Dr. Teodor Grantcharov has consulted for Covidien Canada and Ethicon Endosurgery Canada. Dr. Vanessa Palter has no conflicts of interest or financial ties to disclose.

Author information

Authors and Affiliations

  1. University of Toronto, 600 University Ave, Rm. 440, Toronto, ON, M5G 1X5, Canada

    Vanessa N. Palter

  2. Department of Surgery, St. Michael’s Hospital, 30 Bond St., 16CC-056, Toronto, ON, M5W 1W8, Canada

    Teodor P. Grantcharov

Authors
  1. Vanessa N. Palter
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Teodor P. Grantcharov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vanessa N. Palter.

Additional information

Accepted for presentation at the Association of Surgical Education (ASE) 2011 meeting.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Palter, V.N., Grantcharov, T.P. A prospective study demonstrating the reliability and validity of two procedure-specific evaluation tools to assess operative competence in laparoscopic colorectal surgery. Surg Endosc 26, 2489–2503 (2012). https://doi.org/10.1007/s00464-012-2220-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00464-012-2220-9

Keywords

Search

Navigation