Automated objective surgical skill assessment in the operating room from unstructured tool motion in septoplasty

Abstract

Purpose

Previous work on surgical skill assessment using intraoperative tool motion has focused on highly structured surgical tasks such as cholecystectomy and used generic motion metrics such as time and number of movements. Other statistical methods such as hidden Markov models (HMM) and descriptive curve coding (DCC) have been successfully used to assess skill in structured activities on bench-top tasks. Methods to assess skill and provide effective feedback to trainees for unstructured surgical tasks in the operating room, such as tissue dissection in septoplasty, have yet to be developed.

Methods

We proposed a method that provides a descriptive structure for septoplasty by automatically segmenting it into higher-level meaningful activities called strokes. These activities characterize the surgeon’s tool motion pattern. We constructed a spatial graph from the sequence of strokes in each procedure and used its properties to train a classifier to distinguish between expert and novice surgeons. We compared the results from our method with those from HMM, DCC, and generic metric-based approaches.

Results

We showed that our method—with an average accuracy of 91 %—performs better or equal than these state-of-the-art methods, while simultaneously providing surgeons with an intuitive understanding of the procedure.

Conclusions

In this study, we developed and evaluated an automated approach to objectively assess surgical skill during unstructured task of tissue dissection in nasal septoplasty.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

References

  1. 1.

    Sugden C, Aggarwal R (2010) Assessment and feedback in the skills laboratory and operating room. Surg Clin North Am 90:519–533

    Article  PubMed  Google Scholar 

  2. 2.

    Champagne BJ (2013) Effective teaching and feedback strategies in the or and beyond. Clin Colon Rectal Surg 26:244–249

    Article  PubMed Central  PubMed  Google Scholar 

  3. 3.

    Fettman N, Sanford T, Sindwani R (2009) Surgical management of the deviated septum: techniques in septoplasty. Otolaryngol Clin North Am 42:241–253

    Article  PubMed  Google Scholar 

  4. 4.

    Hwang PH, McLaughlin RB, Lanza DC, Kennedy DW (1999) Endoscopic septoplasty: indications, technique, and results. Otolaryngol Head Neck Surg 120:78–82

    Article  Google Scholar 

  5. 5.

    Bhattacharyya N (2006) Ambulatory sinus and nasal surgery in the United States: demographics and perioperative outcomes. Laryngoscope 120:635–638

    Article  Google Scholar 

  6. 6.

    Stewart MG, Witsell DL, Smith DL, Weaver EM, Yueh B, Hannley MT (2014) Development and validation of the Nasal Obstruction Symptom Evaluation (NOSE) scale. Otolaryngol Head Neck Surg 130:157–163

    Article  Google Scholar 

  7. 7.

    Mason JD, Ansell J, Warren N, Torkington J (2013) Is motion analysis a valid tool for assessing laparoscopic skill? Surg Endosc 27:1468–1477

  8. 8.

    Moorthy K, Munz Y, Sarkar SK, Darzi A (2003) Objective assessment of technical skills in surgery. BMJ 327:1032–1037

    Article  PubMed Central  PubMed  Google Scholar 

  9. 9.

    Van Hove PD, Tuijthof GJ, Verdaasdonk EG, Stassen LP, Dankelman J (2010) Objective assessment of technical surgical skills. Br J Surg 97:972–987

    Article  PubMed  Google Scholar 

  10. 10.

    Aggarwal R, Grantcharov T, Moorthy K, Milland T, Papasavas P, Dosis A, Bello F, Darzi A (2007) An evaluation of the feasibility, validity, and reliability of laparoscopic skills assessment in the operating room. Ann Surg 245:992–999

    Article  PubMed Central  PubMed  Google Scholar 

  11. 11.

    Dosis A, Aggarwal R, Bello F, Moorthy K, Munz Y, Gillies D, Darzi A (2005) Synchronized video and motion analysis for the assessment of procedures in the operating theater. Arch Surg 140:293–299

    Article  PubMed  Google Scholar 

  12. 12.

    Oropesa I, Sanchez-Gonzalez P, Lamat P, Chmarra MK, Pagador JB, Sanchez-Margallo JA, Sanchez-Margallo FM, Gmez EJ (2011) Methods and tools for objective assessment of psychomotor skills in laparoscopic surgery. J Surg Res 171:e81–e95

    Article  PubMed  Google Scholar 

  13. 13.

    Reiley CE, Lin HC, Yuh DD, Hager GD (2011) Review of methods for objective surgical skill evaluation. Surg Endosc 25:356–366

    Article  PubMed  Google Scholar 

  14. 14.

    Varadarajan B, Reiley C, Lin H, Khudanpur S, Hager GD (2009) Data-derived models for segmentation with application to surgical assessment and training. Med Image Comput Comput Assist Interv 12:426–434

    PubMed  Google Scholar 

  15. 15.

    Tao L, Elhamifar E, Khudanpur S, Hager GD, Vidal R (2012) Sparse Hidden Markov Models for Surgical Gesture Classification and Skill Evaluation. IPCAI 2012. LNCS, vol 7330, Springer, Heidelberg, pp 167–177

  16. 16.

    Ahmidi N, Gao Y, Bejar B, Vedula SS, Khudanpur S, Vidal R, Hager GD (2013) String motif-based description of tool motion for detecting skill and gestures in robotic surgery. MICCAI 16:26–33

  17. 17.

    Ahmidi N, Hager GD, Ishii L, Gallia GL, Ishii M (2012) Robotic path planning for surgeon skill evaluation in minimally-invasive sinus surgery. Springer, Berlin

    Google Scholar 

Download references

Acknowledgments

We gratefully acknowledge support from NIH 5R21DE022656-02. We would also like to thank participating attending surgeons—Drs. Kofi Boahene, Patrick Byrne, Ira Papel, and Theda Kontis, and trainee surgeons—Drs. Sun Ahn, Amit Kocchar, Linda Lee, Ryan Li, Myriam Loyo, Sofia Lyford-Pike, Peter Revenaugh, David Smith, Babar Sultan, David Mener, Samuel Oyer, Daniel Sun, at the Johns Hopkins Medical Institutions.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Narges Ahmidi.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Ahmidi, N., Poddar, P., Jones, J.D. et al. Automated objective surgical skill assessment in the operating room from unstructured tool motion in septoplasty. Int J CARS 10, 981–991 (2015). https://doi.org/10.1007/s11548-015-1194-1

Download citation

Keywords

  • Unstructured activities
  • Partially observed time series
  • Surgical skill assessment
  • Feature extraction
  • Septoplasty
  • Feedback