Automatic phase prediction from low-level surgical activities

  • Germain Forestier
  • Laurent Riffaud
  • Pierre Jannin
Original Article



Analyzing surgical activities has received a growing interest in recent years. Several methods have been proposed to identify surgical activities and surgical phases from data acquired in operating rooms. These context-aware systems have multiple applications including: supporting the surgical team during the intervention, improving the automatic monitoring, designing new teaching paradigms.


In this paper, we use low-level recordings of the activities that are performed by a surgeon to automatically predict the current (high-level) phase of the surgery. We augment a decision tree algorithm with the ability to consider the local context of the surgical activities and a hierarchical clustering algorithm.


Experiments were performed on 22 surgeries of lumbar disk herniation. We obtained an overall precision of 0.843 in detecting phases of 51,489 single activities. We also assess the robustness of the method with regard to noise.


We show that using the local context allows us to improve the results compared with methods only considering single activity. Experiments show that the use of the local context makes our method very robust to noise and that clustering the input data first improves the predictions.


Surgical process Temporal analysis Prediction Surgery 



The authors would like to thanks all the surgeons of the Neurosurgery Department of the Leipzig University Hospital, Germany involved in this work. The authors would like to thank the anonymous reviewers for their valuable comments and suggestions to improve the manuscript.


  1. 1.
    Bardram JE, Doryab A, Jensen RM, Lange PM, Nielsen KL, Petersen ST (2011) Phase recognition during surgical procedures using embedded and body-worn sensors. In: IEEE international conference on pervasive computing and communications, pp 45–53Google Scholar
  2. 2.
    Blamoutier A (2013) Surgical discectomy for lumbar disc herniation: surgical techniques. Orthop Traumatol Surg Res 99(1):S187–S196CrossRefPubMedGoogle Scholar
  3. 3.
    Bouarfa L, Jonker PP, Dankelman J (2011) Discovery of high-level tasks in the operating room. J Biomed Inform 44(3):455–462CrossRefPubMedGoogle Scholar
  4. 4.
    Bricon-Souf N, Newman CR (2007) Context awareness in health care: a review. Int J Med Inform 76(1):2–12CrossRefPubMedGoogle Scholar
  5. 5.
    Forestier G, Lalys F, Riffaud L, Collins DL, Meixensberger J, Wassef SN, Neumuth T, Goulet B, Jannin P (2013) Multi-site study of surgical practice in neurosurgery based on surgical process models. J Biomed Inform 46(5):822–829Google Scholar
  6. 6.
    Forestier G, Lalys F, Riffaud L, Trelhu B, Jannin P (2012) Classification of surgical processes using dynamic time warping. J Biomed Inform 45(2):255–264Google Scholar
  7. 7.
    Forestier G, Petitjean F, Riffaud L, Jannin P (2014) Non-linear temporal scaling of surgical processes. Artif Intell Med 62(3):143–152CrossRefPubMedGoogle Scholar
  8. 8.
    Franke S, Meixensberger J, Neumuth T (2013) Intervention time prediction from surgical low-level tasks. J Biomed Inform 46(1):152–159CrossRefPubMedGoogle Scholar
  9. 9.
    Hall M, Frank E, Holmes G, Pfahringer B, Reutemann P, Witten IH (2009) The WEKA data mining software: an update. ACM SIGKDD Explor Newsl 11(1):10–18CrossRefGoogle Scholar
  10. 10.
    Lalys F, Bouget D, Riffaud L, Jannin P (2013) Automatic knowledge-based recognition of low-level tasks in ophthalmological procedures. Int J Comput Assist Radiol Surg 8(1):39–49CrossRefPubMedGoogle Scholar
  11. 11.
    Lalys F, Jannin P (2013) Surgical process modelling: a review. Int J Comput Assist Radiol Surg 8(5):1–17Google Scholar
  12. 12.
    Lalys F, Riffaud L, Morandi X, Jannin P (2010) Automatic phases recognition in pituitary surgeries by microscope images classification. In: Information processing in computer-assisted interventions, vol 6135. Springer, pp 34–44Google Scholar
  13. 13.
    Mehta N, Haluck R, Frecker M, Snyder A (2002) Sequence and task analysis of instrument use in common laparoscopic procedures. Surg Endosc 16(2):280–285CrossRefPubMedGoogle Scholar
  14. 14.
    Meißner C, Meixensberger J, Pretschner A, Neumuth T (2014) Sensor-based surgical activity recognition in unconstrained environments. Minimally Invasive Therapy & Allied TechnologiesGoogle Scholar
  15. 15.
    Padoy N, Blum T, Feussner H, Berger MO, Navab N (2008) On-line recognition of surgical activity for monitoring in the operating room. In: AAAI, pp 1718–1724Google Scholar
  16. 16.
    Padoy N, Mateus D, Weinland D, Berger MO, Navab N (2009) Workflow monitoring based on 3d motion features. In: IEEE international conference on computer vision workshops, pp 585–592 (2009)Google Scholar
  17. 17.
    Quinlan R (1993) C4.5: programs for machine learning. Morgan Kaufmann Publishers, San MateoGoogle Scholar
  18. 18.
    Riffaud L, Neumuth T, Morandi X, Trantakis C, Meixensberger J, Burgert O, Trelhu B, Jannin P (2010) Recording of surgical processes: a study comparing senior and junior neurosurgeons during lumbar disc herniation surgery. Neurosurgery 67:325–332CrossRefPubMedGoogle Scholar
  19. 19.
    Sebban M, NockO R, Chauchat J, Rakotomalala R (2000) Impact of learning set quality and size on decision tree performances. IJCSS 1(1):85Google Scholar
  20. 20.
    Shi Y, Bobick A, Essa I (2006) Learning temporal sequence model from partially labeled data. In: 2006 IEEE computer society conference on computer vision and pattern recognition, vol 2. IEEE, pp 1631–1638Google Scholar
  21. 21.
    Stauder R, Okur A, Peter L, Schneider A, Kranzfelder M, Feussner H, Navab N (2014) Random forests for phase detection in surgical workflow analysis. In: Information processing in computer-assisted interventions. Springer, pp 148–157Google Scholar
  22. 22.
    Varadarajan B, Reiley C, Lin H, Khudanpur S, Hager G (2009) Data-derived models for segmentation with application to surgical assessment and training. In: Medical image computing and computer-assisted intervention-MICCAI 2009. Springer, pp 426–434Google Scholar
  23. 23.
    Ženko B (2008) Learning predictive clustering rules. Informatica 32:95–96Google Scholar

Copyright information

© CARS 2015

Authors and Affiliations

  1. 1.MIPS, University of Haute-AlsaceMulhouseFrance
  2. 2.INSERM, UMR 1099RennesFrance
  3. 3.Université de Rennes 1, LTSIRennesFrance
  4. 4.Neurosurgery DepartmentRennes University HospitalRennesFrance

Personalised recommendations