Skip to main content
SpringerLink
Log in

Inter-rater and intra-rater reliability of the current assessment model and tools for laparoscopic suturing

  • Published:
Surgical Endoscopy Aims and scope Submit manuscript

Abstract

Background

Moorthy checklist (MC) and laparoscopic skill competency assessment tool (LS-CAT) are tools commonly used to evaluate the quality of laparoscopic suturing. The current assessment model is single measurement by multiple raters. Our aim is to examine the reliability of the current assessment model and tools.

Methods

With IRB approval, participants of three different backgrounds, namely medical students, trainees, and surgeons, were enrolled. The participants each accomplished a standardized laparoscopic suturing task. The performances were video-recorded and reviewed with LS-CAT and MC independently by three blinded raters. Intraclass correlation coefficients (ICC) were calculated for inter-rater and intra-rater reliability.

Results

26 participants were enrolled, comprising 10 students, 10 trainees and 6 surgeons. In regard of inter-rater reliability, ICC values (95% CI) were 0.909 (0.768–0.961) and 0.868 (0.608–0.948) in LS-CAP and MC, respectively. For students, ICC values were 0.908 (0.682–0.976) and 0.815 (0.408–0.951) in LS-CAT and MC, respectively. For trainees, ICC values were 0.812 (0.426–0.947) and 0.717 (0.102–0.925), respectively. For surgeons, ICC values were 0.720 (0.064–0.955) and 0.868 (0.608–0.948), respectively. In regard of intra-rater reliability, ICC values of the mean scores from the three raters were 0.956 (0.905–0.980) and 0.925 (0.842–0.966) in LS-CAP and MC, respectively.

Conclusion

LS-CAT and MC are both qualified assessment tools for laparoscopic suturing. LS-CAT is more reliable particularly for medical students and trainees. The current assessment model of single measurement by multiple raters provides excellent reliability.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Szabo Z, Hunter J, Berci G, Sackier J, Cuschieri A (1994) Analysis of surgical movements during suturing in laparoscopy. Endosc Surg Allied Technol 2:55–61

    CAS  PubMed  Google Scholar 

  2. Laguna MP, Schreuders LC, Rassweiler JJ, Abbou CC, van Velthoven R, Janetschek G, Breda G, de la Rosette JJ, European Society of U-T (2005) Development of laparoscopic surgery and training facilities in Europe: results of a survey of the European Society of Uro-Technology (ESUT). Eur Urol 47:346–351

    Article  CAS  Google Scholar 

  3. Stefanidis D, Hope WW, Korndorffer JR Jr, Markley S, Scott DJ (2010) Initial laparoscopic basic skills training shortens the learning curve of laparoscopic suturing and is cost-effective. J Am Coll Surg 210:436–440

    Article  Google Scholar 

  4. Nepomnayshy D, Whitledge J, Fitzgibbons S, Nijjar B, Gardner A, Alseidi A, Birkett R, Deal S, Duvra RR, Anton N, Stefanidis D (2019) Advanced laparoscopic skills: understanding the relationship between simulation-based practice and clinical performance. Am J Surg 218:527–532

    Article  Google Scholar 

  5. Bilgic E, Watanabe Y, Nepomnayshy D, Gardner A, Fitzgibbons S, Ghaderi I, Alseidi A, Stefanidis D, Paige J, Seymour N, McKendy KM, Birkett R, Whitledge J, Kane E, Anton NE, Vassiliou MC, Simulation Committee of the Association for Surgical E (2017) Multicenter proficiency benchmarks for advanced laparoscopic suturing tasks. Am J Surg 213:217–221

    Article  Google Scholar 

  6. Kroeze SG, Mayer EK, Chopra S, Aggarwal R, Darzi A, Patel A (2009) Assessment of laparoscopic suturing skills of urology residents: a pan-European study. Eur Urol 56:865–872

    Article  Google Scholar 

  7. McCluney AL, Vassiliou MC, Kaneva PA, Cao J, Stanbridge DD, Feldman LS, Fried GM (2007) FLS simulator performance predicts intraoperative laparoscopic skill. Surg Endosc 21:1991–1995

    Article  CAS  Google Scholar 

  8. Sroka G, Feldman LS, Vassiliou MC, Kaneva PA, Fayez R, Fried GM (2010) Fundamentals of laparoscopic surgery simulator training to proficiency improves laparoscopic performance in the operating room-a randomized controlled trial. Am J Surg 199:115–120

    Article  Google Scholar 

  9. Moorthy K, Munz Y, Dosis A, Bello F, Chang A, Darzi A (2004) Bimodal assessment of laparoscopic suturing skills: construct and concurrent validity. Surg Endosc 18:1608–1612

    CAS  PubMed  Google Scholar 

  10. Peters JH, Ellison EC, Innes JT, Liss JL, Nichols KE, Lomano JM, Roby SR, Front ME, Carey LC (1991) Safety and efficacy of laparoscopic cholecystectomy. A prospective analysis of 100 initial patients. Ann Surg 213:3–12

    Article  CAS  Google Scholar 

  11. Verdaasdonk EG, Dankelman J, Lange JF, Stassen LP (2008) Transfer validity of laparoscopic knot-tying training on a VR simulator to a realistic environment: a randomized controlled trial. Surg Endosc 22:1636–1642

    Article  CAS  Google Scholar 

  12. Van Sickle KR, Baghai M, Huang IP, Goldenberg A, Smith CD, Ritter EM (2008) Construct validity of an objective assessment method for laparoscopic intracorporeal suturing and knot tying. Am J Surg 196:74–80

    Article  Google Scholar 

  13. Aggarwal R, Hance J, Undre S, Ratnasothy J, Moorthy K, Chang A, Darzi A (2006) Training junior operative residents in laparoscopic suturing skills is feasible and efficacious. Surgery 139:729–734

    Article  Google Scholar 

  14. Ij WM, Leijte E, Ganni S, Luursema JM, Francis NK, Jakimowicz JJ, Botden S (2020) Competency assessment tool for laparoscopic suturing: development and reliability evaluation. Surg Endosc 34:2947–2953

    Article  Google Scholar 

  15. Dubrowski A, Park J, Moulton CA, Larmer J, MacRae H (2007) A comparison of single- and multiple-stage approaches to teaching laparoscopic suturing. Am J Surg 193:269–273

    Article  Google Scholar 

  16. Koo TK, Li MY (2016) A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 15:155–163

    Article  Google Scholar 

Download references

Acknowledgements

The authors appreciate IRCAD-Taiwan and Metronic (Taiwan) Limited for the support of laparoscopic instruments and suturing materials.

Funding

Funding was provided by Taipei Medical University, Shuang Ho Hospital (Grant No. 110YSR-02).

Author information

Author notes

  1. Hsin-An Chen and Kuei‐Yen Tsai have contributed equally to this work as the co-corresponding author.

Authors and Affiliations

  1. Division of Pediatric Surgery, Department of Surgery, Shuang Ho Hospital, New Taipei City, Taiwan

    Chin-Hung Wei

  2. Department of Surgery, School of Medicine, College of Medicine, Taipei Medical University, Taipei, Taiwan

    Chin-Hung Wei, Shih-Chiang Shen, Kuei‐Yen Tsai & Hsin-An Chen

  3. IRCAD/AITS-Asian Institute of TeleSurgery, Show Chwan Health Care System, Changhua, Taiwan

    Chin-Hung Wei

  4. Division of General Surgery, Department of Surgery, Shuang Ho Hospital, New Taipei City, Taiwan

    Shih-Chiang Shen, Kuei‐Yen Tsai & Hsin-An Chen

  5. Division of Pediatric Surgery, Department of Surgery, Mackay Memorial Hospital, Hsinchu, Taiwan

    Yih-Cherng Duh

  6. Mackay Junior College of Medicine, Nursing, and Management, Taipei, Taiwan

    Yih-Cherng Duh

  7. Graduate Institute of Clinical Medicine, College of Medicine, Taipei Medical University, Taipei, Taiwan

    Kuei‐Yen Tsai, Hsin-An Chen & Shih-Wei Huang

  8. Division of General Surgery, Department of Surgery, Show Chwan Memorial Hospital, Changhua, Taiwan

    Shih-Wei Huang

  9. Division of General Surgery, Department of Surgery, Chang Bing Show Chwan Memorial Hospital, Changhua, Taiwan

    Shih-Wei Huang

Authors
  1. Chin-Hung Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shih-Chiang Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yih-Cherng Duh
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kuei‐Yen Tsai
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hsin-An Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Shih-Wei Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Kuei‐Yen Tsai or Hsin-An Chen.

Ethics declarations

Disclosures

Drs. Hsin-An Chen and Chin-Hung Wei are the speakers in IRCAD-Taiwan, and the Grant recipients funded by Shuang Ho Hospital, Grant number 110YSR-02. Dr. Shih-Wei Huang is the director of IRCAD-Taiwan. Drs. Kuei‐Yen Tsai, Shih-Chiang Shen, and Yih-Cherng Duh have no conflicts of interest and financial ties to disclose.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wei, CH., Shen, SC., Duh, YC. et al. Inter-rater and intra-rater reliability of the current assessment model and tools for laparoscopic suturing. Surg Endosc 36, 6586–6591 (2022). https://doi.org/10.1007/s00464-022-09061-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00464-022-09061-9

Keywords

Search

Navigation