Advertisement

Surgical Endoscopy

, Volume 18, Issue 12, pp 1800–1804 | Cite as

Toward reliable operative assessment: the reliability and feasibility of videotaped assessment of laparoscopic technical skills

  • D. Dath
  • G. Regehr
  • D. Birch
  • C. Schlachta
  • E. Poulin
  • J. Mamazza
  • R. Reznick
  • H. M. MacRae
Original article

Abstract

Background

Decision making on the competency of surgical trainees to perform laparoscopic procedures has been hampered by the lack of reliable methods to evaluate operative performance. The goal of this study was to develop a feasible and reliable method of evaluation.

Methods

Twenty-nine senior surgical residents were videotaped performing a low anterior resection and a Nissen fundoplication in a pig. Ten blinded laparoscopists rated the videos independently on two scales. Rating time was minimized by allowing raters to fast-forward through the tapes at their discretion. Interrater reliability and the time required to rate a procedure were assessed.

Results

Rating time per procedure was a median of 15 min (range, 6–40). The mean interrater reliability for the two scales was 0.74.

Conclusions

The use of videotapes of operations enabled multiple raters to assess a performance reliably and shortened assessment times by 80%. This assessment technique shows potential as a means of evaluating the performance of advanced laparoscopic procedures by surgical trainees.

Keywords

Technical skills assessment Laparoscopic skills Nissen fundoplication Resident performance Surgical education 

Notes

Acknowledgments

This work was supported by United States Surgical Corporation through a grant from the Center for Excellence in Surgical Education, Research, and Teaching Foundation at the Association for Surgical Education.

References

  1. 1.
    Anastakis, DJ, Regehr, G, Reznick, R, Cusimano, M, Murnaghan, J, Mitchell, B, Hutchinson, C 1999Assessment of technical skills: transfer from the bench training model to the human modelAm J Surg111167170CrossRefGoogle Scholar
  2. 2.
    Beckmann, CR, Lipscomb, GH, Ling, FW, Beckmann, CA, Johnson, H, Barton, L 1995Computer-assisted video evaluation of surgical skillsObstet Gynecol8510391041CrossRefPubMedGoogle Scholar
  3. 3.
    Gorman, PJ, Meier, AH, Rawn, C, Krummel, M 2000The future of medical education is no longer blood and guts, it is bits and bytesAm J Surg180353356CrossRefPubMedGoogle Scholar
  4. 4.
    Hamford, JM, Hall, JC 2000Acquiring surgical skillsBr J Surg872837Google Scholar
  5. 5.
    MacRae, HM, Regehr, G, Leadbetter, W, Reznick, RK 2000A comprehensive examination for senior surgical residentsAm J Surg179190193CrossRefPubMedGoogle Scholar
  6. 6.
    Martin, JA, Regehr, G, Reznick, R, MacRae, H, Murnaghan, J, Hutchison, C, Brown, M 1997An objective structured assessment of technical skills (OSATS) for surgical residentsBr J Surg84273278CrossRefPubMedGoogle Scholar
  7. 7.
    Pietroni, M 1993The assessment of competence in surgical traineesAnn R Coll Surg Engl75200202PubMedGoogle Scholar
  8. 8.
    Regehr, G, MacRae, H, Reznick, RK, Szalay, D 1998Comparing the psychometric properties of checklists and global rating scales for assessing performance on an OSCE-format examinationAcad Med73993997CrossRefPubMedGoogle Scholar
  9. 9.
    Reznick, R, Taylor, B, Maudsley, R, Ruedy, J, Cohen, R, Chalmers, A, Newman, C 1991In-training evaluation—it’s more than just a formAnn R Coll Physicians Surg Can24415420Google Scholar
  10. 10.
    Reznick, RK, Regehr, G, MacRae, HM, Martin, J 1997Testing technical skill via an innovative bench station examinationAm J Surg173226230CrossRefPubMedGoogle Scholar
  11. 11.
    Richards R (1999) Why do we need to assess competence? In: Surgical competence: challenges of assessment in training and practice. (Ed.) Jackson, B.Proceedings of a conference held at the Queen Elizabeth II Conference Centre by the Royal College of Surgeons of England, Nov. 2, 1999Google Scholar
  12. 12.
    Scott, DJ, Bergen, PC, Rege, RV, Laycock, R, Tesfay, ST, Valentine, RJ, Euhus, DM,  et al. 2000Laparoscopic training on bench models: better and more cost effective than operating room experience?J Am Coll Surg191272283CrossRefPubMedGoogle Scholar
  13. 13.
    Scott, DJ, Rege, RV, Bergen, PC, Guo, WA, Laycock, R, Tesfay, ST, Valentine, RJ,  et al. 2000Measuring operative performance after laparoscopic skills training: edited videotape versus direct observationJ Laparoendosc Surg10183190CrossRefGoogle Scholar
  14. 14.
    Strasberg, SM, Hertl, M, Soper, NJ 1995An Analysis of the problem of biliary injury during laparoscopic cholecstectomyJ Am Coll Surg180101125PubMedGoogle Scholar
  15. 15.
    Streiner, DL 1985Global rating scalesNeufeld, VRNorman, GR eds. Assessing clinical competenceSpringerNew York119141Google Scholar
  16. 16.
    Winckel, C, Reznick, RK, Cohen, R, Taylor, B 1994Reliability and construct validity of a structured technical skills assessment formAm J Surg167423427CrossRefPubMedGoogle Scholar

Copyright information

© Springer-Verlag 2004

Authors and Affiliations

  • D. Dath
    • 1
  • G. Regehr
    • 2
  • D. Birch
    • 3
  • C. Schlachta
    • 2
  • E. Poulin
    • 2
  • J. Mamazza
    • 2
  • R. Reznick
    • 2
  • H. M. MacRae
    • 2
  1. 1.Department of SurgeryFaculty of Health Sciences, McMaster UniversityHamiltonCanada
  2. 2.Department of surgery and Centre for Research in EducationFaculty of Medicine, University of TorontoToronto, OntarioCanada
  3. 3.Department of SurgeryFaculty of Medicine and Dentistry, University of AlbertaAlbertaCanada

Personalised recommendations