Skip to main content
SpringerLink
Log in

Intraoperative monitoring of laparoscopic skill development based on quantitative measures

  • Published:
Surgical Endoscopy Aims and scope Submit manuscript

Abstract

Background

Methods for evaluating standard skills in the operating room typically are based on direct observation and checklists, but such evaluations are time consuming and can be subject to bias. It often is possible to acquire more objective measurements using surgical simulators. However, motor performance in simulators can differ significantly from that in the operating room. Intraoperative assessment is particularly challenging because of the significant variability between procedures related to differences in the patients, the surgical setup, and the team. This study aimed to evaluate the feasibility of using a new framework for interpreting quantitative measures acquired in the operating room to distinguish between levels of laparoscopic skill development.

Methods

Two levels of surgical skill development were observed, namely, those of three fourth-year residents and three attending surgeons performing three laparoscopic cholecystectomies each. Electromagnetic position sensors were attached by the surgeons to a 5-mm curved dissector and a 5-mm atraumatic grasper. From the tools’ position histories and video recordings, time, kinematics, and movement transition measures were extracted. Various measures such as the Kolmogorov–Smirnov statistic and the Jensen–Shanon Divergence were used to provide intuitive dimensionless difference measures ranging from 0 to 1. These scores were used to compare residents and expert surgeons executing two surgical tasks: exposure of Calot’s triangle and dissection of the cystic duct and artery.

Results

The two groups could be clearly differentiated in both tasks during monitoring for the dominant hand (analysis of variance [ANOVA] and Mann–Whitney; p < 0.05) but not for the nondominant hand.

Conclusions

It is practical to acquire time, kinematic, and movement transition measures intraoperatively using video and electromagnetic position-sensing technologies. Principal component analysis proved to be a useful technique for presenting differences between skill levels based on those measures. The authors conclude that objective assessment of intraoperative surgical motor behavior is feasible and likely practical.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Bridges M, Diamond D (1999) The financial impact of teaching surgical residents in the operating room. Am J Surg 177:28–32

    Article  CAS  PubMed  Google Scholar 

  2. Babineau T, Becker J, Gibbons G, Sentovich S, Hess D, Robertson S, Stone M (2004) The “cost” of operative training for surgical residents. Arch Surg 139:366–370

    Article  PubMed  Google Scholar 

  3. Britt L (2007) Simulation training: what are real questions that must be answered? Am J Surg 194:220

    Article  CAS  PubMed  Google Scholar 

  4. Torkington J, Smith SGT, Rees BI, Darzi A (2001) The role of the basic surgical skills course in the acquisition and retention of laparoscopic skill. Surg Endosc 15:1071–1075

    Article  CAS  PubMed  Google Scholar 

  5. Gallagher AG, Lederman AB, McGlade K, Satava RM, Smith CD (2004) Discriminative validity of the minimally invasive surgical trainer in virtual reality (MIST-VR) using criteria levels based on expert performance. Surg Endosc 24:660–665

    Article  Google Scholar 

  6. Scott D, Rege R, Bergen P, Guo W, Laycock R, Tesfay S, Valentine J, Jones D (2000) Measuring operative performance after laparoscopic skills training: edited videotape versus direct observation. J Laparoendosc Adv Surg Tech A 10:183–190

    Article  CAS  PubMed  Google Scholar 

  7. Derossis AM, Bothwell J, Sigman HH, Fried GM (1998) The effect of practice on performance in a laparoscopic simulator. Surg Endosc 12:1117–1120

    Article  CAS  PubMed  Google Scholar 

  8. Rosser JC Jr, Rosser LE, Savalgi RS (1997) Skill acquisition and assessment for laparoscopic surgery. Arch Surg 132:200–204

    CAS  PubMed  Google Scholar 

  9. Feldman LS, Sherman V, Fried GM (2004) Using simulators to assess laparoscopic competence: ready for widespread use? Surgery 135:28–42

    Article  PubMed  Google Scholar 

  10. Park AE, Witzke D (2002) The surgical competence conundrum. Surg Endosc 16:555–557

    Article  CAS  Google Scholar 

  11. Grantcharov TP, Rosenberg J, Pahle E, Funch-Jensen P (2001) Virtual reality computer simulation. Surg Endosc 15:242–244

    Article  CAS  PubMed  Google Scholar 

  12. Grantcharov TP, Kristiansen VB, Bendix J, Bardram L, Rosenberg J, Funch-Jensen P (2004) Randomized clinical trial of virtual reality simulation for laparoscopic skills training. Br J Surg 91:146–150

    Article  CAS  PubMed  Google Scholar 

  13. Moorthy K, Munz Y, Sarker S, Darzi A (2003) Objective assessment of technical skills in surgery. BMJ 327:1032–1037

    Article  PubMed  Google Scholar 

  14. Wanzel KR, Ward M, Reznick RK (2002) Teaching the surgical craft: from selection to certification. Curr Probl Surg 39:574–659

    Article  Google Scholar 

  15. Alleman A (2005) Have you wondered about your colleague’s surgical skills? Am J Med Qual 20:78–82

    Article  PubMed  Google Scholar 

  16. Hammond I (2006) Training, assessment, and competency in gynaecologic surgery. Best Pract Res Clin Obstet Gynaecol 20:173–187

    Article  PubMed  Google Scholar 

  17. Darzi A, Datta V, Mackay S (2001) The challenge of objective assessment of surgical skill. Am J Surg 181:484–486

    Article  CAS  PubMed  Google Scholar 

  18. Datta V, Bann S, Mandalia M, Darzi A (2006) The surgical efficiency score: a feasible, reliable, and valid method of skills assessment. Am J Surg 192:372–378

    Article  PubMed  Google Scholar 

  19. Aggarwal R, Grantcharov T, Moorthy K, Milland T, Papasavas P, Dosis A, Bello F, Darzi A (2007) An evaluation of the feasibility, validity, and reliability of laparoscopic skills assessment in the operating room. Ann Surg 245:992–999

    Article  PubMed  Google Scholar 

  20. Smith CD, Farrell TM, McNatt SS, Metreveli RE (2001) Assessing laparoscopic manipulative skills. Am J Surg 181:547–550

    Article  CAS  PubMed  Google Scholar 

  21. Hwang H, Lim J, Kinnaird C, Nagy AG, Panton ON, Hodgson AJ, Qayumi KA (2005) Correlating motor performance with surgical error in laparoscopic cholecystectomy. Surg Endosc 20:651–655

    Article  PubMed  Google Scholar 

  22. Cristancho S, Hodgson A, Panton N, Meneghetti A, Qayumi K (2006) Assessing cognitive and motor performance in MIS for training & tool design. Stud Health Technol Informatics 119:108–113

    Google Scholar 

  23. Kologlu M, Tutuncu T, Yuksek YN, Gozalah U, Daglar G, Kama NA (2004) Using a risk score for conversion from laparoscopic to open cholecystectomy in resident training. Surgery 135:282–287

    Article  PubMed  Google Scholar 

  24. Cristancho S, Hodgson A, Panton N, Meneghetti A, Qayumi K (2007) Feasibility of using intraoperatively acquired quantitative kinematic measures to monitor development of laparoscopic skill. Stud Health Technol Informatics 125:85–90

    Google Scholar 

  25. Milne AD, Chess DG, Johnson JA, King GJW (1996) Accuracy of an electromagnetic tracking device: a study of the optimal operating range and metal interference. J Biomech 29:791–793

    Article  CAS  PubMed  Google Scholar 

  26. Polhemus Incorporated (2002) 3Space Fastrak user’s manual. OPM00PI002. Colchester, VT

  27. Dohrmann C, Busby H, Trujillo D (1988) Smoothing noisy data using dynamic programming and generalised cross-validation. J Biomech Eng 110:37–41

    Article  CAS  PubMed  Google Scholar 

  28. Jolliffe IT (2002) Principal component analysis, 2nd edn. Springer, New York

    Google Scholar 

  29. Haverkort B (2001) Markovian models for performance and dependability evaluation: lectures on formal methods and performance analysis, 1st edn. Springer, Berlin, pp 38–83

  30. Drew J, Glen A, Leemis L (2000) Computing the cumulative distribution function of the Kolmogorov–Smirnov statistic. Comput Stat Data Anal 34:1–15

    Article  Google Scholar 

  31. Von Mises R (1964) Mathematical theory of probability and statistics. Academic Press, New York

    Google Scholar 

  32. Hodgson AJ, McBeth PB (2002). Comparing motor performance on similar tasks in different settings: statistical characteristics of a nondimensional difference measure. Internal Document, Neuromotor Control Laboratory, Department of Mechanical Engineering, University of British Columbia

  33. Dagan I, Lee L, Pereira F (1999) Similarity-based models of word cooccurrence probabilities. Mach Learn 34:43–69

    Article  Google Scholar 

  34. Majtey AP, Lamberti PW, Prato DP (2005) Jensen–Shannon divergence as a measure of distinguishability between mixed quantum states. Phys Rev A 72:052310-1–052310-6

    Google Scholar 

  35. Endres D, Schindelin J (2003) A new metric for probability distributions. IEEE Trans Inf Theory 49:1858–1860

    Article  Google Scholar 

  36. de Visser H, Heijnsdijk EA, Herder JL, Pistecky PV (2002) Forces and displacements in colon surgery. Surg Endosc 16:1426–1430

    Article  PubMed  Google Scholar 

  37. Rosen J, Hannaford B, Richards G, Sinanan M (2001) Markov modeling of minimally invasive surgery based on tool/tissue interaction and force/torque signatures for evaluating surgical skills. IEEE Trans Biomed Eng 48:579–591

    Article  CAS  PubMed  Google Scholar 

  38. Rosen J, Brown J, Chang L, Sinanan M, Hannaford B (2006) Generalized approach for modeling minimally invasive surgery as a stochastic process using a discrete Markov model. IEEE Trans Biomed Eng 53:399–413

    Article  PubMed  Google Scholar 

  39. Kinnaird K (2004) A multifaceted quantitative validity assessment of laparoscopic surgical simulators. Master’s Thesis, Department of Mechanical Engineering, University of British Columbia, Vancouver

  40. Murphy T (2004) Towards objective surgical skill evaluation with hidden Markov model-based motion recognition. Master’s Thesis, Johns Hopkins University, Baltimore, MD

Download references

Author information

Authors and Affiliations

  1. Department of Mechanical Engineering, University of British Columbia, Vancouver, Canada

    Sayra M. Cristancho & Antony J. Hodgson

  2. Facultad de Ingenieria Electronica, Universidad Pontificia Bolivariana Seccional Bucaramanga, Santander, Colombia

    Sayra M. Cristancho

  3. Department of Surgery, University of British Columbia, Vancouver, Canada

    O. N. M. Panton, Adam Meneghetti, Garth Warnock & Karim Qayumi

  4. Centre of Excellence for Surgical Education and Innovation, Vancouver General Hospital, Vancouver, Canada

    O. N. M. Panton, Adam Meneghetti, Garth Warnock & Karim Qayumi

Authors
  1. Sayra M. Cristancho
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Antony J. Hodgson
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. O. N. M. Panton
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Adam Meneghetti
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Garth Warnock
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Karim Qayumi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sayra M. Cristancho.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Cristancho, S.M., Hodgson, A.J., Panton, O.N.M. et al. Intraoperative monitoring of laparoscopic skill development based on quantitative measures. Surg Endosc 23, 2181–2190 (2009). https://doi.org/10.1007/s00464-008-0246-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00464-008-0246-9

Keywords

Search

Navigation