Skip to main content

Advertisement

SpringerLink
Log in

Development, validation and implementation of a monitoring tool for training in laparoscopic colorectal surgery in the English National Training Program

  • Published:
Surgical Endoscopy Aims and scope Submit manuscript

Abstract

Introduction

The National Training Program for laparoscopic colorectal surgery (LCS) provides supervised training to colorectal surgeons in England. The purpose of this study was to create, validate, and implement a method for monitoring training progression in laparoscopic colorectal surgery that met the requirements of a good assessment tool.

Methods

A generic scale for different tasks in LCS was created under the guidance of a national expert group. The scores were defined by the extent to which the trainees were dependent on support (1 = unable to perform, 5 = unaided (benchmark), 6 = proficient). Trainers were asked to rate their trainees after each supervised case; trainees completed a similar self-assessment form. Construct validity was evaluated comparing scores of trainees at different experience levels (1–5, 6–10, 11–15, 16+) using the Wilcoxon signed-rank test and ANOVA. Internal consistency was determined by Crohnbach’s alpha, interrater reliability by comparing peer- and self-assessment (interclass correlation coefficient, ICC). Proficiency gain curves were plotted using CUSUM charts.

Results

Analysis included 610 assessments (333 by trainers and 277 by trainees). There was high interrater reliability (ICC = 0.867), internal consistency (α = 0.920), and construct validity [F(3,40) = 6.128, p < 0.001]. Detailed analysis of proficiency gain curves demonstrates that theater setup, exposure, and anastomosis were performed independently after 5 to 15 sessions, and the dissection of the vascular pedicle took 24 cases. Mobilization of the colon and of the splenic/hepatic flexure took more than 25 procedures. Median assessment time was 3.3 (interquartile range (IQR) 1–5) minutes and the tool was accepted as useful [median score 5 of 6 (IQR 4–5)].

Discussion

A valid and reliable monitoring tool for surgical training has been implemented successfully into the National Training Program. It provides a description of an individualized proficiency gain curve in terms of both the level of support required and the competency level achieved.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. The National Training Programme in laparoscopic colorectal surgery. Lapco website. www.lapco.nhs.uk

  2. van der Vleuten CP, Schuwirth LW (2005) Assessing professional competence: from methods to programmes. Med Educ 39:309–317

    Article  PubMed  Google Scholar 

  3. Joint Advisory Group on GI Endoscopy (JAG) website. www.thejag.org.uk

  4. Black P, Wiliam D (2009) Developing the theory of formative assessment. Educ Assess Eval Account 21:5–31

    Article  Google Scholar 

  5. Sidhu RS, Vikis E, Cheifetz R, Phang T (2006) Self-assessment during a 2-day laparoscopic colectomy course: can surgeons judge how well they are learning new skills? Am J Surg 191:677–681

    Article  PubMed  Google Scholar 

  6. Wohaibi EM, Earle DB, Ansanitis FE, Wait RB, Fernandez G, Seymour NE (2007) A new web-based operative skills assessment tool effectively tracks progression in surgical resident performance. J Surg Educ 64:333–341

    Article  PubMed  Google Scholar 

  7. Munz Y, Moorthy K, Bann S, Shah J, Ivanova S, Darzi SA (2004) Ceiling effect in technical skills of surgical residents. Am J Surg 188:294–300

    Article  PubMed  Google Scholar 

  8. Martin JA, Regehr G, Reznick R, MacRae H, Murnaghan J, Hutchison C, Brown M (1997) Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg 84:273–278

    Article  PubMed  CAS  Google Scholar 

  9. Vassiliou MC, Feldman LS, Andrew CG, Bergman S, Leffondre K, Stanbridge D, Fried GM (2005) A global assessment tool for evaluation of intraoperative laparoscopic skills. Am J Surg 190:107–113

    Article  PubMed  Google Scholar 

  10. Tekkis PP, Senagore AJ, Delaney CP, Fazio VW (2005) Evaluation of the learning curve in laparoscopic colorectal surgery: comparison of right-sided and left-sided resections. Ann Surg 242:83–91

    Article  PubMed  Google Scholar 

  11. Schlachta CM, Mamazza J, Seshadri PA, Cadeddu M, Gregoire R, Poulin EC (2001) Defining a learning curve for laparoscopic colorectal resections. Dis Colon Rectum 44:217–222

    Article  PubMed  CAS  Google Scholar 

  12. Park IJ, Choi GS, Lim KH, Kang BM, Jun SH (2009) Multidimensional analysis of the learning curve for laparoscopic colorectal surgery: lessons from 1,000 cases of laparoscopic colorectal surgery. Surg Endosc 23:839–846

    Article  PubMed  Google Scholar 

  13. Cuschieri A (2006) Nature of human error: implications for surgical practice. Ann Surg 244:642–648

    Article  PubMed  Google Scholar 

  14. Bull C, Yates R, Sarkar D, Deanfield J, de Leval M (2000) Scientific, ethical, and logistical considerations in introducing a new operation: a retrospective cohort study from paediatric cardiac surgery. BMJ 320:1168–1173

    Article  PubMed  CAS  Google Scholar 

  15. Dalton S, Ghosh A, Zafar N, Riyad K, Dixon A (2009) Competency in laparoscopic colorectal surgery is achievable with appropriate training but takes time: a comparison of 300 elective resections with anastomosis. Colorectal Dis, Jul 6 [Epub ahead of print]

  16. Rijbroek A, Wisselink W, Rauwerda JA (2003) The impact of training in unselected patients on mortality and morbidity in carotid endarterectomy in a vascular training center and the recommendations of the European Board of Surgery Qualification in Vascular Surgery. Eur J Vasc Endovasc Surg 26:256–261

    Article  PubMed  CAS  Google Scholar 

  17. Li JC, Hon SS, Ng SS, Lee JF, Yiu RY, Leung KL (2009) The learning curve for laparoscopic colectomy: experience of a surgical fellow in an university colorectal unit. Surg Endosc 23:1603–1608

    Article  PubMed  Google Scholar 

  18. Miskovic D, Wyles SM, Ni M, Darzi AW, Hanna GB (2010) Mentoring and simulation in laparoscopic colorectal surgery. A systematic review. Ann Surg (in press)

  19. van Mook WN, van Luijk SJ, O’Sullivan H, Wass V, Schuwirth LW, van der Vleuten CP (2009) General considerations regarding assessment of professional behaviour. Eur J Intern Med 20:e90–e95

    Article  PubMed  Google Scholar 

  20. Streiner N, Norman G (2008) Health measurement scales. A practical guide to their development and use, 4th edn. Oxford University Press, Oxford

    Google Scholar 

  21. Evans J, Jenkins I, Kennedy RH (2009) Laparoscopic right hemicolectomy. St. Marks Multimedia, UK (20 mins)

    Google Scholar 

  22. Champault G, Schimmelpenning H (2007) Laparoscopic sigmoidectomy for cancer, operation primer edn, Springer

  23. WebSurg website. http://www.websurg.com

  24. Bland JM, Altman DG (1986) Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1:307–310

    PubMed  CAS  Google Scholar 

  25. Bland JM, Altman DG (1997) Cronbach’s alpha. BMJ 314:572

    PubMed  CAS  Google Scholar 

  26. Grigg OA, Farewell VT, Spiegelhalter DJ (2003) Use of risk-adjusted CUSUM and RSPRT charts for monitoring in medical contexts. Stat Methods Med Res 12:147–170

    PubMed  CAS  Google Scholar 

  27. Hoek J, Gendall P (1993) A new method of predicting voting behaviour. J Market Res Soc 35:361–373

    Google Scholar 

  28. Lapco website (link to download assessment form). http://www.lapco.nhs.uk/userfiles/file/Form%20A%20GAS%20assessment%20sheet.pdf

  29. Sarker SK, Kumar I, Delaney C (2010) Assessing operative performance in advanced laparoscopic colorectal surgery. World J Surg 34(7):1594–1603

    Article  PubMed  Google Scholar 

  30. Dincler S, Koller MT, Steurer J, Bachmann LM, Christen D, Buchmann P (2003) Multidimensional analysis of learning curves in laparoscopic sigmoid resection: eight-year results. Dis Colon Rectum 46:1371–1378 discussion 1378–1379

    Article  PubMed  Google Scholar 

  31. Schlachta CM, Mamazza J, Gregoire R, Burpee SE, Pace KT, Poulin EC (2003) Predicting conversion in laparoscopic colorectal surgery. Fellowship training may be an advantage. Surg Endosc 17:1288–1291

    Article  PubMed  CAS  Google Scholar 

  32. Birch DW, Asiri AH, de Gara CJ (2007) The impact of a formal mentoring program for minimally invasive surgery on surgeon practice and patient outcomes. Am J Surg 193:589–591 discussion 591–582

    Article  PubMed  Google Scholar 

  33. Choi DH, Jeong WK, Lim SW, Chung TS, Park JI, Lim SB, Choi HS, Nam BH, Chang HJ, Jeong SY (2009) Learning curves for laparoscopic sigmoidectomy used to manage curable sigmoid colon cancer: single-institute, three-surgeon experience. Surg Endosc 23:622–628

    Article  PubMed  Google Scholar 

  34. Crossley J, Davies H, Humphris G, Jolly B (2002) Generalisability: a key to unlock professional assessment. Med Educ 36:972–978

    Article  PubMed  Google Scholar 

  35. Joice P, Hanna GB, Cuschieri A (1998) Errors enacted during endoscopic surgery—a human reliability analysis. Appl Ergon 29:409–414

    Article  PubMed  CAS  Google Scholar 

  36. Talebpour M, Alijani A, Hanna GB, Moosa Z, Tang B, Cuschieri A (2009) Proficiency-gain curve for an advanced laparoscopic procedure defined by observation clinical human reliability assessment (OCHRA). Surg Endosc 23:869–875

    Article  PubMed  CAS  Google Scholar 

Download references

Acknowledgments

This work is funded by the National Cancer Action Team as part of the National Training Program in Laparoscopic Colorectal Surgery. We would like to thank all members of the educational committee and the steering group representing the centers participating in the National Training Program for their boundless effort to support the educational activities: Austin Acheson and Charles Maxwell-Armstrong (Nottingham), Tom Cecil (Basingstoke), Chris Cunningham (Oxford), Vivek Datta and Sav Papagrigoriadis (London), Nader Francis (Yeovil), John Griffith (Bradford), James Gunn (Hull), Alan Horgan (Newcastle-upon-Tyne), Robin Kennedy (Harrow, London), Roger Motson (Colchester), Amjad Parvaiz (Portsmouth), and Timothy Rockall (Guildford).

Disclosures

Miskovic, S.M. Wyles, F. Carter, M.G. Coleman, and G.B. Hanna have no conflicts of interest or financial ties to disclose.

Author information

Authors and Affiliations

  1. Department of Surgery and Cancer, St Mary’s Hospital, Imperial College, Praed St., London, W2 1NY, UK

    Danilo Miskovic, Susannah M. Wyles & George B. Hanna

  2. Yeovil District Hospital NHS Foundation Trust, Yeovil, UK

    Fiona Carter

  3. Coordination Office, National Training Program in Laparoscopic Colorectal Surgery, Plymouth, UK

    Mark G. Coleman

Authors
  1. Danilo Miskovic
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Susannah M. Wyles
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fiona Carter
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mark G. Coleman
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. George B. Hanna
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to George B. Hanna.

Additional information

This study is conducted on behalf of the National Training Program in Laparoscopic Colorectal Surgery.

Presented at the 12th WCES, April 14--17, 2010, National Harbor, MD.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Miskovic, D., Wyles, S.M., Carter, F. et al. Development, validation and implementation of a monitoring tool for training in laparoscopic colorectal surgery in the English National Training Program. Surg Endosc 25, 1136–1142 (2011). https://doi.org/10.1007/s00464-010-1329-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00464-010-1329-y

Keywords

Search

Navigation