Deep Neural Networks Predict Remaining Surgery Duration from Cholecystectomy Videos

  • Ivan AksamentovEmail author
  • Andru Putra Twinanda
  • Didier Mutter
  • Jacques Marescaux
  • Nicolas Padoy
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10434)


For every hospital, it is desirable to fully utilize its operating room (OR) capacity. Inaccurate planning of OR occupancy impacts patient comfort, safety and financial turnover of the hospital. A source of suboptimal scheduling often lies in the incorrect estimation of the surgery duration, which may vary significantly due to the diversity of patient conditions, surgeon skills and intraoperative situations. We propose automatic methods to estimate the remaining surgery duration in real-time by using only the image feed from the endoscopic camera and no other sensor. These approaches are based on neural networks designed to learn the workflow of an endoscopic procedure. We train and evaluate our models on a large dataset of 120 endoscopic cholecystectomies. Results show the strong benefits of these approaches when surgeries last longer than usual and promise practical improvements in OR management.


Remaining duration prediction Surgical workflow analysis Operating room management Deep learning Recurrent neural networks 



This work was supported by French state funds managed by the ANR within the Investissements d’Avenir program under references ANR-11-LABX-0004 (Labex CAMI) and ANR-10-IAHU-02 (IHU Strasbourg). The authors would also like to acknowledge the support of NVIDIA with the donation of the GPU used in this research.


  1. 1.
    Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., Devin, M., et al.: Tensorflow: large-scale machine learning on heterogeneous distributed systems. arXiv preprint arXiv:1603.04467 (2016)
  2. 2.
    Ammori, B.J., Larvin, M., McMahon, M.J.: Elective laparoscopic cholecystectomy. Surg. Endosc. 15(3), 297–300 (2001)CrossRefGoogle Scholar
  3. 3.
    Dexter, F., Epstein, R.H., Lee, J.D., Ledolter, J.: Automatic updating of times remaining in surgical cases using bayesian analysis of historical case duration data and instant messaging updates from anesthesia providers. Anesth. Analg. 108(3), 929–940 (2009)CrossRefGoogle Scholar
  4. 4.
    Franke, S., Meixensberger, J., Neumuth, T.: Intervention time prediction from surgical low-level tasks. J. Biomed. Inform. 46(1), 152–159 (2013)CrossRefGoogle Scholar
  5. 5.
    Guédon, A.C.P., Paalvast, M., Meeuwsen, F.C., Tax, D.M.J., van Dijke, A.P., Wauben, L., van der Elst, M., Dankelman, J., van den Dobbelsteen, J.: Real-time estimation of surgical procedure duration. In: International Conference on E-health Networking, Application & Services, pp. 6–10 (2015)Google Scholar
  6. 6.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR, pp. 770–778 (2016)Google Scholar
  7. 7.
    Kayış, E., Khaniyev, T.T., Suermondt, J., Sylvester, K.: A robust estimation model for surgery durations with temporal, operational, and surgery team effects. Health Care Manag. Sci. 18(3), 222–233 (2015)CrossRefGoogle Scholar
  8. 8.
    Kayis, E., Wang, H., Patel, M., Gonzalez, T., Jain, S., Ramamurthi, R.J., Santos, C.A., Singhal, S., Suermondt, J., Sylvester, K.: Improving prediction of surgery duration using operational and temporal factors. In: AMIA (2012)Google Scholar
  9. 9.
    Macario, A., Dexter, F.: Estimating the duration of a case when the surgeon has not recently scheduled the procedure at the surgical suite. Anesth. Analg. 89, 1241–1245 (1999)CrossRefGoogle Scholar
  10. 10.
    Maktabi, M., Neumuth, T.: Online time and resource management based on surgical workflow time series analysis. IJCARS 12(2), 325–338 (2017)Google Scholar
  11. 11.
    Padoy, N., Blum, T., Feussner, H., Berger, M.O., Navab, N.: On-line recognition of surgical activity for monitoring in the operating room. In: IAAI, pp. 1718–1724 (2008)Google Scholar
  12. 12.
    Travis, E., Woodhouse, S., Tan, R., Patel, S., Donovan, J., Brogan, K.: Operating theatre time, where does it all go? A prospective observational study. BMJ 349, g7182 (2014)CrossRefGoogle Scholar
  13. 13.
    Twinanda, A.P., Shehata, S., Mutter, D., Marescaux, J., de Mathelin, M., Padoy, N.: Endonet: a deep architecture for recognition tasks on laparoscopic videos. IEEE Trans. Med. Imaging 36(1), 86–97 (2017)CrossRefGoogle Scholar
  14. 14.
    Wiegmann, D.A., ElBardissi, A.W., Dearani, J.A., Daly III, R.C., Sundt, T.M.: Disruptions in surgical flow and their relationship to surgical errors: an exploratory investigation. Surgery 142(5), 658–665 (2007)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Ivan Aksamentov
    • 1
    Email author
  • Andru Putra Twinanda
    • 1
  • Didier Mutter
    • 2
  • Jacques Marescaux
    • 2
  • Nicolas Padoy
    • 1
  1. 1.CNRS, IHU Strasbourg, ICubeUniversity of StrasbourgStrasbourgFrance
  2. 2.IRCAD, IHU StrasbourgUniversity Hospital of StrasbourgStrasbourgFrance

Personalised recommendations