Surgical Endoscopy And Other Interventional Techniques

, Volume 20, Issue 5, pp 744–747

The MISTELS program to measure technical skill in laparoscopic surgery

Evidence for reliability
  • M. C. Vassiliou
  • G. A. Ghitulescu
  • L. S. Feldman
  • D. Stanbridge
  • K. Leffondré
  • H. H. Sigman
  • G. M. Fried
Article

DOI: 10.1007/s00464-005-3008-y

Cite this article as:
Vassiliou, M.C., Ghitulescu, G.A., Feldman, L.S. et al. Surg Endosc (2006) 20: 744. doi:10.1007/s00464-005-3008-y

Abstract

Background

The McGill Inanimate System for Training and Evaluation of Laparoscopic Skills (MISTELS) is a series of five tasks with an objective scoring system. The purpose of this study was to estimate the interrater and test–retest reliability of the MISTELS metrics and to assess their internal consistency.

Methods

To determine interrater reliability, two trained observers scored 10 subjects, either live or on tape. Test–retest reliability was assessed by having 12 subjects perform two tests, the second immediately following the first. Interrater and test–retest reliability were assessed using intraclass correlation coefficients. Internal consistency between tasks was estimated using Cronbach’s alpha.

Results

The interrater and test–retest reliabilities for the total scores were both excellent at 0.998 [95% confidence interval (CI), 0.985–1.00] and 0.892 (95% CI, 0.665–0.968), respectively. Cronbach’s alpha for the first assessment of the test–retest was 0.86.

Conclusions

The MISTELS metrics have excellent reliability, which exceeds the threshold level of 0.8 required for high-stakes evaluations. These findings support the use of MISTELS for evaluation in many different settings, including residency training programs.

Keywords

Laparoscopic trainingSimulationEducationEvaluationReliability

Copyright information

© Springer Science+Business Media, Inc. 2006

Authors and Affiliations

  • M. C. Vassiliou
    • 1
  • G. A. Ghitulescu
    • 1
  • L. S. Feldman
    • 1
  • D. Stanbridge
    • 1
  • K. Leffondré
    • 2
  • H. H. Sigman
    • 1
  • G. M. Fried
    • 1
  1. 1.Steinberg–Bernstein Centre for Minimally Invasive Surgery, Department of SurgeryMcGill University Health CentreMontrealCanada
  2. 2.Department of Social and Preventive MedicineUniversity of MontréalMontrealCanada