Abstract
Background
Decision making on the competency of surgical trainees to perform laparoscopic procedures has been hampered by the lack of reliable methods to evaluate operative performance. The goal of this study was to develop a feasible and reliable method of evaluation.
Methods
Twenty-nine senior surgical residents were videotaped performing a low anterior resection and a Nissen fundoplication in a pig. Ten blinded laparoscopists rated the videos independently on two scales. Rating time was minimized by allowing raters to fast-forward through the tapes at their discretion. Interrater reliability and the time required to rate a procedure were assessed.
Results
Rating time per procedure was a median of 15 min (range, 6–40). The mean interrater reliability for the two scales was 0.74.
Conclusions
The use of videotapes of operations enabled multiple raters to assess a performance reliably and shortened assessment times by 80%. This assessment technique shows potential as a means of evaluating the performance of advanced laparoscopic procedures by surgical trainees.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
DJ Anastakis G Regehr R Reznick M Cusimano J Murnaghan B Mitchell C Hutchinson (1999) ArticleTitleAssessment of technical skills: transfer from the bench training model to the human model Am J Surg 111 167–170 Occurrence Handle10.1016/S0002-9610(98)00327-4
CR Beckmann GH Lipscomb FW Ling CA Beckmann H Johnson L Barton (1995) ArticleTitleComputer-assisted video evaluation of surgical skills Obstet Gynecol 85 1039–1041 Occurrence Handle1:STN:280:DyaK2M3ovVSluw%3D%3D Occurrence Handle10.1016/0029-7844(95)00068-3 Occurrence Handle7770251
PJ Gorman AH Meier C Rawn M Krummel (2000) ArticleTitleThe future of medical education is no longer blood and guts, it is bits and bytes Am J Surg 180 353–356 Occurrence Handle1:STN:280:DC%2BD3M7gtF2htA%3D%3D Occurrence Handle10.1016/S0002-9610(00)00514-6 Occurrence Handle11137686
JM Hamford JC Hall (2000) ArticleTitleAcquiring surgical skills Br J Surg 87 28–37
HM MacRae G Regehr W Leadbetter RK Reznick (2000) ArticleTitleA comprehensive examination for senior surgical residents Am J Surg 179 190–193 Occurrence Handle1:STN:280:DC%2BD3c3oslaquw%3D%3D Occurrence Handle10.1016/S0002-9610(00)00304-4 Occurrence Handle10827317
JA Martin G Regehr R Reznick H MacRae J Murnaghan C Hutchison M Brown (1997) ArticleTitleAn objective structured assessment of technical skills (OSATS) for surgical residents Br J Surg 84 273–278 Occurrence Handle1:STN:280:DyaK2s3gsV2ntQ%3D%3D Occurrence Handle10.1002/bjs.1800840237 Occurrence Handle9052454
M Pietroni (1993) ArticleTitleThe assessment of competence in surgical trainees Ann R Coll Surg Engl 75 200–202 Occurrence Handle1:STN:280:DyaK2c3ovVCmug%3D%3D Occurrence Handle8017795
G Regehr H MacRae RK Reznick D Szalay (1998) ArticleTitleComparing the psychometric properties of checklists and global rating scales for assessing performance on an OSCE-format examination Acad Med 73 993–997 Occurrence Handle1:STN:280:DyaK1cvjtFalug%3D%3D Occurrence Handle10.1097/00001888-199809000-00020 Occurrence Handle9759104
R Reznick B Taylor R Maudsley J Ruedy R Cohen A Chalmers C Newman (1991) ArticleTitleIn-training evaluation—it’s more than just a form Ann R Coll Physicians Surg Can 24 415–420
RK Reznick G Regehr HM MacRae J Martin (1997) ArticleTitleTesting technical skill via an innovative bench station examination Am J Surg 173 226–230 Occurrence Handle1:STN:280:DyaK2s3ks1GntQ%3D%3D Occurrence Handle10.1016/S0002-9610(97)89597-9 Occurrence Handle9124632
Richards R (1999) Why do we need to assess competence? In: Surgical competence: challenges of assessment in training and practice. (Ed.) Jackson, B.Proceedings of a conference held at the Queen Elizabeth II Conference Centre by the Royal College of Surgeons of England, Nov. 2, 1999
DJ Scott PC Bergen RV Rege R Laycock ST Tesfay RJ Valentine DM Euhus et al. (2000) ArticleTitleLaparoscopic training on bench models: better and more cost effective than operating room experience? J Am Coll Surg 191 272–283 Occurrence Handle1:STN:280:DC%2BD3cvktF2mtQ%3D%3D Occurrence Handle10.1016/S1072-7515(00)00339-2 Occurrence Handle10989902
DJ Scott RV Rege PC Bergen WA Guo R Laycock ST Tesfay RJ Valentine et al. (2000) ArticleTitleMeasuring operative performance after laparoscopic skills training: edited videotape versus direct observation J Laparoendosc Surg 10 183–190 Occurrence Handle1:STN:280:DC%2BD3cvks1Sltw%3D%3D Occurrence Handle10.1089/109264200421559
SM Strasberg M Hertl NJ Soper (1995) ArticleTitleAn Analysis of the problem of biliary injury during laparoscopic cholecstectomy J Am Coll Surg 180 101–125 Occurrence Handle1:STN:280:DyaK2M%2FptlOhuw%3D%3D Occurrence Handle8000648
DL Streiner (1985) Global rating scales VR Neufeld GR Norman (Eds) Assessing clinical competence Springer New York 119–141
C Winckel RK Reznick R Cohen B Taylor (1994) ArticleTitleReliability and construct validity of a structured technical skills assessment form Am J Surg 167 423–427 Occurrence Handle1:STN:280:DyaK2c3jsVCmsw%3D%3D Occurrence Handle10.1016/0002-9610(94)90128-7 Occurrence Handle8179088
Acknowledgments
This work was supported by United States Surgical Corporation through a grant from the Center for Excellence in Surgical Education, Research, and Teaching Foundation at the Association for Surgical Education.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Dath, D., Regehr, G., Birch, D. et al. Toward reliable operative assessment: the reliability and feasibility of videotaped assessment of laparoscopic technical skills . Surg Endosc 18, 1800–1804 (2004). https://doi.org/10.1007/s00464-003-8157-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00464-003-8157-2