Optimal Sub-Sequence Matching for the Automatic Prediction of Surgical Tasks

  • Germain ForestierEmail author
  • François Petitjean
  • Laurent Riffaud
  • Pierre Jannin
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9105)


Surgery is one of the riskiest and most important medical acts that is performed today. The desires to improve patient outcomes, surgeon training, and also to reduce the costs of surgery, have motivated surgeons to equip their Operating Rooms with sensors that describe the surgical intervention. The richness and complexity of the data that is collected calls for new machine learning methods to support pre-, peri- and post-surgery (before, during and after).

This paper introduces a new method for the prediction of the next task that the surgeon is going to perform during the surgery (peri). Our method bases its prediction on the optimal matching of the current surgery to a set of pre-recorded surgeries.

We assess our method on a set of neurosurgeries (lumbar disc herniation removal) and show that our method outperforms the state of the art by providing a prediction (of the next task that is going to be performed by the surgeon) more than 85% of the time with a 95% accuracy.


Lumbar Disc Herniation Dynamic Time Warping Agreement Rate Surgical Task Surgical Activity 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Haynes, A.B., Weiser, T.G., Berry, W.R., Lipsitz, S.R., Breizat, A.H.S., Dellinger, E.P., Herbosa, T., Joseph, S., Kibatala, P.L., Lapitan, M.C.M., et al.: A surgical safety checklist to reduce morbidity and mortality in a global population. New England Journal of Medicine 360(5), 491–499 (2009)CrossRefGoogle Scholar
  2. 2.
    Lalys, F., Riffaud, L., Morandi, X., Jannin, P.: Automatic phases recognition in pituitary surgeries by microscope images classification. In: Navab, N., Jannin, P. (eds.) IPCAI 2010. LNCS, vol. 6135, pp. 34–44. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  3. 3.
    Lalys, F., Riffaud, L., Bouget, D., Jannin, P.: An application-dependent framework for the recognition of high-level surgical tasks in the OR. In: Fichtinger, G., Martel, A., Peters, T. (eds.) MICCAI 2011, Part I. LNCS, vol. 6891, pp. 331–338. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  4. 4.
    Meißner, C., Meixensberger, J., Pretschner, A., Neumuth, T.: Sensor-based surgical activity recognition in unconstrained environments. Minimally Invasive Therapy & Allied Technologies, 1–8 (2014)Google Scholar
  5. 5.
    Lalys, F., Jannin, P.: Surgical process modelling: a review. International Journal of Computer Assisted Radiology and Surgery 8(5), 1–17 (2013)Google Scholar
  6. 6.
    Forestier, G., Petitjean, F., Riffaud, L., Jannin, P.: Non-linear temporal scaling of surgical processes. Artificial Intelligence in Medicine 62(3), 143–152 (2014)CrossRefGoogle Scholar
  7. 7.
    Liu, Z., Hauskrecht, M.: Clinical time series prediction with a hierarchical dynamical system. In: Peek, N., Marín Morales, R., Peleg, M. (eds.) AIME 2013. LNCS(LNAI), vol. 7885, pp. 227–237. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  8. 8.
    Ruda, K., Beekman, D., White, L.W., Lendvay, T.S., Kowalewski, T.M.: Surgtrak – a universal platform for quantitative surgical data capture. Journal of Medical Devices 7(3), 030923 (2013)Google Scholar
  9. 9.
    Ahmidi, N., Gao, Y., Béjar, B., Vedula, S.S., Khudanpur, S., Vidal, R., Hager, G.D.: String motif-based description of tool motion for detecting skill and gestures in robotic surgery. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013, Part I. LNCS, vol. 8149, pp. 26–33. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  10. 10.
    Mehta, N., Haluck, R., Frecker, M., Snyder, A.: Sequence and task analysis of instrument use in common laparoscopic procedures. Surgical Endoscopy 16(2), 280–285 (2002)CrossRefGoogle Scholar
  11. 11.
    Sakoe, H., Chiba, S.: Dynamic programming algorithm optimization for spoken word recognition. IEEE Transactions on Acoustics, Speech and Signal Processing 26(1), 43–49 (1978)zbMATHCrossRefGoogle Scholar
  12. 12.
    Forestier, G., Lalys, F., Riffaud, L., Collins, D.L., Meixensberger, J., Wassef, S.N., Neumuth, T., Goulet, B., Jannin, P.: Multi-site study of surgical practice in neurosurgery based on surgical process models. Journal of Biomedical Informatics 46(5), 822–829 (2013)CrossRefGoogle Scholar
  13. 13.
    Forestier, G., Lalys, F., Riffaud, L., Trelhu, B., Jannin, P.: Classification of surgical processes using Dynamic Time Warping. Journal of Biomedical Informatics 45(2), 255–264 (2012)CrossRefGoogle Scholar
  14. 14.
    Kittler, J., Hatef, M., Duin, R.P., Matas, J.: On combining classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(3), 226–239 (1998)CrossRefGoogle Scholar
  15. 15.
    Yankov, D., Keogh, E., Medina, J., Chiu, B., Zordan, V.: Detecting time series motifs under uniform scaling. In: International Conference on Knowledge Discovery and Data Mining, pp. 844–853. ACM (2007)Google Scholar
  16. 16.
    Zhou, Y., Bailey, J., Ioannou, I., Wijewickrema, S., O’Leary, S., Kennedy, G.: Pattern-based real-time feedback for a temporal bone simulator. In: Symposium on Virtual Reality Software and Technology, pp. 7–16. ACM (2013)Google Scholar
  17. 17.
    Petitjean, F., Forestier, G., Webb, G., Nicholson, A., Chen, Y., Keogh, E.: Dynamic Time Warping averaging of time series allows faster and more accurate classification. In: IEEE International Conference on Data Mining (2014)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Germain Forestier
    • 1
    Email author
  • François Petitjean
    • 2
  • Laurent Riffaud
    • 3
    • 4
  • Pierre Jannin
    • 3
  1. 1.MIPSUniversity of Haute-AlsaceMulhouseFrance
  2. 2.Faculty of Information TechnologyMonash UniversityMelbourneAustralia
  3. 3.INSERM MediCIS, Unit U1099 LTSIUniversity of Rennes 1RennesFrance
  4. 4.Department of NeurosurgeryPontchaillou University HospitalRennesFrance

Personalised recommendations