Skip to main content

Video content analysis of surgical procedures

Surgical Endoscopy volume 32, pages 553–568 (2018)Cite this article

Abstract

Background

In addition to its therapeutic benefits, minimally invasive surgery offers the potential for video recording of the operation. The videos may be archived and used later for reasons such as cognitive training, skills assessment, and workflow analysis. Methods from the major field of video content analysis and representation are increasingly applied in the surgical domain. In this paper, we review recent developments and analyze future directions in the field of content-based video analysis of surgical operations.

Methods

The review was obtained from PubMed and Google Scholar search on combinations of the following keywords: ‘surgery’, ‘video’, ‘phase’, ‘task’, ‘skills’, ‘event’, ‘shot’, ‘analysis’, ‘retrieval’, ‘detection’, ‘classification’, and ‘recognition’. The collected articles were categorized and reviewed based on the technical goal sought, type of surgery performed, and structure of the operation.

Results

A total of 81 articles were included. The publication activity is constantly increasing; more than 50% of these articles were published in the last 3 years. Significant research has been performed for video task detection and retrieval in eye surgery. In endoscopic surgery, the research activity is more diverse: gesture/task classification, skills assessment, tool type recognition, shot/event detection and retrieval. Recent works employ deep neural networks for phase and tool recognition as well as shot detection.

Conclusions

Content-based video analysis of surgical operations is a rapidly expanding field. Several future prospects for research exist including, inter alia, shot boundary detection, keyframe extraction, video summarization, pattern discovery, and video annotation. The development of publicly available benchmark datasets to evaluate and compare task-specific algorithms is essential.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

43,34 €

Price includes VAT (Finland)
Tax calculation will be finalised during checkout.

Fig. 1

Notes

  1. https://cirl.lcsr.jhu.edu/research/hmm/datasets/jigsaws_release/

  2. http://grand-challenge.org/site/endovissub-workflow/data/

  3. http://camma.u-strasbg.fr/m2cai2016/index.php/program-challenge/

  4. http://camma.u-strasbg.fr/datasets

References

  1. Henken KR, Jansen FW, Klein J, Stassen LPS, Dankelman J, van den Dobbelsteen JJ (2012) Implications of the law on video recording in clinical practice. Surg Endosc 26:2909–2916. doi: 10.1007/s00464-012-2284-6

    Article  PubMed  Google Scholar 

  2. Turnbull AMJ, Emsley ES (2014) Video recording of ophthalmic surgery-ethical and legal considerations. Surv Ophthalmol 59:553–558. doi: 10.1016/j.survophthal.2014.01.006

    Article  PubMed  Google Scholar 

  3. Dimick JB, Varban OA (2015) Surgical video analysis: an emerging tool for improving surgeon performance. BMJ Qual Saf 24:490–491. doi: 10.1136/bmjqs-2015-004439

    Article  PubMed  Google Scholar 

  4. Bonrath EM, Gordon LE, Grantcharov TP (2015) Characterising “near miss” events in complex laparoscopic surgery through video analysis. BMJ Qual Saf 24:516–521. doi: 10.1136/bmjqs-2014-003816

    Article  PubMed  Google Scholar 

  5. Snoek CGM, Worring M (2007) Concept-based video retrieval. Found Trends Inf Retr 2:215–322. doi: 10.1561/1500000014

    Article  Google Scholar 

  6. Hu W, Xie N, Li L, Zeng X, Maybank S (2011) A survey on visual content-based video indexing and retrieval. IEEE Trans Syst Man Cybern C 41:797–819. doi: 10.1109/TSMCC.2011.2109710

    Article  Google Scholar 

  7. Sermanet P, Eigen D, Zhang X, Mathieu M, Fergus R, LeCun Y (2013) OverFeat: integrated recognition, localization and detection using convolutional networks. In: International conference on learning representations (ICLR 2014), Banff, pp 1–16

  8. Reiley CE, Lin HC, Yuh DD, Hager GD (2011) Review of methods for objective surgical skill evaluation. Surg Endosc 25:356–366. doi: 10.1007/s00464-010-1190-z

    Article  PubMed  Google Scholar 

  9. Dosis A, Bello F, Gillies D, Undre S, Aggarwal R, Darzi A (2005) Laparoscopic task recognition using hidden Markov Models. Stud Health Technol Inform 111:115–122

    PubMed  Google Scholar 

  10. Weede O, Dittrich F, Worn H, Jensen B, Knoll A, Wilhelm D, Kranzfelder M, Schneider A, Feussner H (2012) Workflow analysis and surgical phase recognition in minimally invasive surgery. In: 2012 IEEE international conference on robotics and biomimetics (ROBIO)—conference digest, pp 1068–1074. doi: 10.1109/ROBIO.2012.6491111

  11. Padoy N, Blum T, Ahmadi S-A, Feussner H, Berger M-O, Navab N (2012) Statistical modeling and recognition of surgical workflow. Med Image Anal 16:632–641. doi: 10.1016/j.media.2010.10.001

    Article  PubMed  Google Scholar 

  12. Lin B, Sun Y, Qian X, Goldgof D, Gitlin R, You Y (2016) Video-based 3D reconstruction, laparoscope localization and deformation recovery for abdominal minimally invasive surgery: a survey. Int J Med Robot Comput Assist Surg 12:158–178. doi: 10.1002/rcs.1661

    Article  Google Scholar 

  13. Bouget D, Allan M, Stoyanov D, Jannin P (2017) Vision-based and marker-less surgical tool detection and tracking: a review of the literature. Med Image Anal 35:633–654. doi: 10.1016/j.media.2016.09.003

    Article  PubMed  Google Scholar 

  14. Foster JD, Miskovic D, Allison AS, Conti JA, Ockrim J, Cooper EJ, Hanna GB, Francis NK (2016) Application of objective clinical human reliability analysis (OCHRA) in assessment of technical performance in laparoscopic rectal cancer surgery. Tech Coloproctol 20:361–367. doi: 10.1007/s10151-016-1444-4

    CAS  Article  PubMed  Google Scholar 

  15. Miskovic D, Ni M, Wyles SM, Parvaiz A, Hanna GB (2012) Observational clinical human reliability analysis (OCHRA) for competency assessment in laparoscopic colorectal surgery at the specialist level. Surg Endosc 26:796–803. doi: 10.1007/s00464-011-1955-z

    Article  PubMed  Google Scholar 

  16. Bardram JE, Doryab A, Jensen RM, Lange PM, Nielsen KLG, Petersen ST (2011) Phase recognition during surgical procedures using embedded and body-worn sensors. In: IEEE international conference on pervasive computing and communications, pp 45–53. doi: 10.1109/PERCOM.2011.5767594

  17. Bouarfa L, Jonker PP, Dankelman J (2011) Discovery of high-level tasks in the operating room. J Biomed Inform 44:455–462. doi: 10.1016/j.jbi.2010.01.004

    CAS  Article  PubMed  Google Scholar 

  18. Klank U, Padoy N, Feussner H, Navab N (2008) Automatic feature generation in endoscopic images. Int J Comput Assist Radiol Surg 3:331–339. doi: 10.1007/s11548-008-0223-8

    Article  Google Scholar 

  19. Blum T, Feussner H, Navab N (2010) Modeling and segmentation of surgical workflow from laparoscopic video. Lect Notes Comput Sci 6363:400–407

    Article  Google Scholar 

  20. Dergachyova O, Bouget D, Huaulmé A, Morandi X, Jannin P (2016) Automatic data-driven real-time segmentation and recognition of surgical workflow. Int J Comput Assist Radiol Surg 11:1081–1089. doi: 10.1007/s11548-016-1371-x

    Article  PubMed  Google Scholar 

  21. Primus MJ, Schoeffmann K, Böszörmenyi L (2016) Temporal segmentation of laparoscopic videos into surgical phases. In: 14th international workshop on content-based multimedia indexing, pp 1–6

  22. Twinanda AP, Shehata S, Mutter D, Marescaux J, de Mathelin M, Padoy N (2017) EndoNet: a deep architecture for recognition tasks on laparoscopic videos. IEEE Trans Med Imaging 36:86–97. doi: 10.1109/TMI.2016.2593957

    Article  PubMed  Google Scholar 

  23. Lea C, Choi JH, Reiter A, Hager GD (2016) Surgical phase recognition: from instrumented ORs to hospitals around the world. In: Medical image computing and computer-assisted intervention M2CAI—MICCAI workshop, pp 45–54

  24. Lalys F, Riffaud L, Morandi X, Jannin P (2010) Automatic phases recognition in pituitary surgeries by microscope images classification. Lect Notes Comput Sci 6135:34–44. doi: 10.1007/978-3-642-13711-2

    Article  Google Scholar 

  25. Lalys F, Riffaud L, Morandi X, Jannin P (2011) Surgical phases detection from microscope videos by combining SVM and HMM. Lect Notes Comput Sci 6533:54–62

    Article  Google Scholar 

  26. Megali G, Sinigaglia S, Tonet O, Dario P (2006) Modelling and evaluation of surgical performance using hidden Markov models. IEEE Trans Biomed Eng 53:1911–1919. doi: 10.1109/TBME.2006.881784

    Article  PubMed  Google Scholar 

  27. Loukas C, Georgiou E (2011) Multivariate autoregressive modeling of hand kinematics for laparoscopic skills assessment of surgical trainees. IEEE Trans Biomed Eng 58:3289–3297. doi: 10.1109/TBME.2011.2167324

    Article  PubMed  Google Scholar 

  28. Zappella L, Béjar B, Hager G, Vidal R (2013) Surgical gesture classification from video and kinematic data. Med Image Anal 17:732–745. doi: 10.1016/j.media.2013.04.007

    Article  PubMed  Google Scholar 

  29. Haro BB, Zappella L, Vidal R (2012) Surgical gesture classification from video data. Lect Notes Comput Sci 7510:34–41

    Article  Google Scholar 

  30. Tao L, Zappella L, Hager GD, Vidal R (2013) Surgical gesture segmentation and recognition. Lect Notes Comput Sci 8151:339–346

    Article  Google Scholar 

  31. Lea C, Hager GD, Vidal R (2015) An improved model for segmentation and recognition of fine-grained activities with application to surgical training tasks. In: IEEE winter conference on applications of computer vision, Waikoloa, pp 1123–1129

  32. Lea C, Reiter A, Vidal R, Hager GD (2016) Segmental spatiotemporal CNNs for fine-grained action segmentation. Lect Notes Comput Sci 9907:36–52. doi: 10.1007/978-3-319-46487-9_3

    Article  Google Scholar 

  33. Gao Y, Vedula SS, Reiley CE, Ahmidi N, Varadarajan B, Lin HC, Tao L, Zappella L, Bejar B, Yuh DD, Chen CCG, Vidal R, Khudanpur S, Hager GD (2014) The JHU-ISI gesture and skill assessment dataset (JIGSAWS): A surgical activity working set for human motion modeling. In: Medical image computing and computer-assisted intervention M2CAI—MICCAI workshop

  34. Krishnan S, Garg A, Patil S, Lea C, Hager G, Abbeel P, Goldberg K (2016) Transition state clustering: unsupervised trajectory segmentation of multi-modal demonstrations with deep learning. In: IEEE international conference on robotics and automation, Genova, Italy, pp 1–8

  35. Murali A, Garg A, Krishnan S, Pokorny FT, Abbeel P, Darrell T, Goldberg K (2016) TSC-DL: Unsupervised trajectory segmentation of multi-modal surgical demonstrations with deep learning. In: IEEE international conference on robotics and automation, Stockholm, Sweden, pp 1–8

  36. Rupprecht C, Lea C, Tombari F, Navab N, Hager GD (2016) Sensor substitution for video-based action recognition. In: 2016 IEEE/RSJ international conference on intelligent robots and systems, pp 5230–5237. IEEE

  37. Lo BPL, Darzi A, Yang G (2003) Episode classification for the analysis of tissue/instrument interaction with multiple visual cues. In: 6th international conference on medical imaging and computer-assisted intervention, Montréal, pp 230–237

  38. Lahane A, Yesha Y, Grasso M, Joshi A, Park A, Lo J (2012) Detection of unsafe action from laparoscopic cholecystectomy video. In: Proceedings of the 2nd ACM SIGHIT international health informatics symposium—IHI 2012. ACM Press, New York, pp 315–322

  39. Giannarou S, Yang G (2010) Content-based surgical workflow representation using probabilistic motion modeling. Lect Notes Comput Sci 6326:314–323

    Article  Google Scholar 

  40. Loukas C, Georgiou E (2015) Smoke detection in endoscopic surgery videos: a first step towards retrieval of semantic events. Int J Med Robot Comput Assist Surg 11:80–94. doi: 10.1002/rcs.1578

    Article  Google Scholar 

  41. Twinanda AP, Marescaux J, de Mathelin M, Padoy N (2015) Classification approach for automatic laparoscopic video database organization. Int J Comput Assist Radiol Surg 10:1449–1460. doi: 10.1007/s11548-015-1183-4

    Article  PubMed  Google Scholar 

  42. Munzer B, Schoeffmann K, Boszormenyi L (2013) Relevance segmentation of laparoscopic videos. In: IEEE international symposium on multimedia. IEEE, Anaheim, pp 84–91

  43. Twinanda AP, Marescaux J, de Mathelin M, Padoy N (2014) Towards better laparoscopic video database organization by automatic surgery classification. Lect Notes Comput Sci 8498:186–195

    Article  Google Scholar 

  44. Padoy N, Blum T, Feußner H, Berger M-O, Navab N (2008) On-line recognition of surgical activity for monitoring in the operating room. In: 20th national conference on innovative applications of artificial intelligence (IAAI 2008), pp 1718–1724

  45. Bhatia B, Oates T, Xiao Y, Hu P (2007) Real-time identification of operating room state from video. In: 19th international conference on innovative applications of artificial intelligence. Vancouver, pp 1761–1766

  46. Sakabe F, Murakawa M, Kobayashi T, Higuchi T, Otsu N (2009) Anomalousness detection for surgery videos using CHLAC feature. In: Symposium on bio-inspired, learning, and intelligent systems for security (BLISS 2009). IEEE, Edinburgh, pp 66–68

  47. Suzuki T, Sakurai Y, Yoshimitsu K, Nambu K, Muragaki Y, Iseki H (2010) Intraoperative multichannel audio-visual information recording and automatic surgical phase and incident detection. In: International conference of the IEEE engineering in medicine and biological society, pp 1190–1193

  48. Twinanda AP, Alkan EO, Gangi A, de Mathelin M, Padoy N (2015) Data-driven spatio-temporal RGBD feature encoding for action recognition in operating rooms. Int J Comput Assist Radiol Surg 10:737–747. doi: 10.1007/s11548-015-1186-1

    Article  PubMed  Google Scholar 

  49. Twinanda AP, Winata P, Gangi A, De M (2016) Multi-stream deep architecture for surgical phase recognition on multi-view RGBD videos. Medical image computing and computer-assisted intervention M2CAI—MICCAI workshop, pp 25–34

  50. Tran D, Sakurai R, Lee J (2015) An improvement of surgical phase detection using latent dirichlet allocation and hidden Markov model. In: Innovation in medicine healthcare. Springer, Cham, pp 249–261

  51. Unger M, Chalopin C, Neumuth T (2014) Vision-based online recognition of surgical activities. Int J Comput Assist Radiol Surg 9:979–986. doi: 10.1007/s11548-014-0994-z

    Article  PubMed  Google Scholar 

  52. Droueche Z, Lamard M, Cazuguel G, Quellec G, Roux C, Cochener B (2011) Content-based medical video retrieval based on region motion trajectories. In: 5th european conference of the international federation for medical and biological engineering, Budapest, pp 622–625

  53. Droueche Z, Lamard M, Cazuguel G, Quellec G, Roux C, Cochener B (2012) Motion-based video retrieval with application to computer-assisted retinal surgery. In: IEEE engineering in medicine and biology society, San Diego, pp 4962–4965

  54. Quellec G, Lamard M, Cazuguel G, Droueche Z, Roux C, Cochener B (2011) Real-time retrieval of similar videos with application to computer-aided retinal surgery. In: International conference of the IEEE engineering in medicine and biological society, Boston, pp 4465–4468

  55. Quellec G, Charrière K, Lamard M, Droueche Z, Roux C, Cochener B, Cazuguel G (2014) Real-time recognition of surgical tasks in eye surgery videos. Med Image Anal 18:579–590. doi: 10.1016/j.media.2014.02.007

    Article  PubMed  Google Scholar 

  56. Quellec G, Lamard M, Droueche Z, Cochener B, Roux C, Cazuguel G (2013) A polynomial model of surgical gestures for real-time retrieval of surgery videos. Lect Notes Comput Sci 7723:10–20. doi: 10.1007/978-3-642-36678-9_2

    Article  Google Scholar 

  57. Quellec G, Lamard M, Cochener B, Cazuguel G (2015) Real-time task recognition in cataract surgery videos using adaptive spatiotemporal polynomials. IEEE Trans Med Imaging 34:877–887

    Article  PubMed  Google Scholar 

  58. Charriere K, Quellec G, Lamard M, Martiano D, Cazuguel G, Coatrieux G, Cochener B (2016) Real-time multilevel sequencing of cataract surgery videos. In: 14th international workshop on content-based multimedia indexing, pp 1–6

  59. Charrière K, Quellec G, Lamard M, Martiano D, Cazuguel G, Coatrieux G, Cochener B (2016) Real-time analysis of cataract surgery videos using statistical models. arXiv:1610.05465

  60. Charriere K, Quellec G, Lamard M, Coatrieux G, Cochener B, Cazuguel G (2014) Automated surgical step recognition in normalized cataract surgery videos. In: International conference of the IEEE engineering in medicine and biology society, Chicago, pp 4647–4650

  61. Quellec G, Charriére K, Lamard M, Cochener B, Cazuguel G (2014) Normalizing videos of anterior eye segment surgeries. In: International conference of the IEEE engineering in medicine and biology society, pp 122–125

  62. Lalys F, Riffaud L, Bouget D, Jannin P (2012) A framework for the recognition of high-level surgical tasks from video images for cataract surgeries. IEEE Trans Biomed Eng 59:966–976. doi: 10.1109/TBME.2011.2181168

    CAS  Article  PubMed  Google Scholar 

  63. Lalys F, Bouget D, Riffaud L, Jannin P (2013) Automatic knowledge-based recognition of low-level tasks in ophthalmological procedures. Int J Comput Assist Radiol Surg 8:39–49. doi: 10.1007/s11548-012-0685-6

    Article  PubMed  Google Scholar 

  64. Lalys F, Riffaud L, Bouget D, Jannin P (2011) An application-dependent framework for the recognition of high-level surgical tasks in the OR. Int Conf Med Image Comput Comput Interv 14:331–338

    Google Scholar 

  65. Mendi E, Cecen S, Ermisoglu E, Bayrak C (2010) Automated neurosurgical video segmentation and retrieval system. J Biomed Sci Eng 3:618–624. doi: 10.4236/jbise.2010.36084

    Article  Google Scholar 

  66. Mendi E, Bayrak C (2011) Shot boundary detection and key-frame extraction from neurosurgical video sequences. Imaging Sci J 60:90–96. doi: 10.1179/1743131X11Y.0000000005

    Article  Google Scholar 

  67. Primus MJ, Schoeffmann K, Böszörmenyi L (2013) Segmentation of recorded endoscopic videos by detecting significant motion changes. In: 11th international workshop on content-based multimedia indexing, Veszprem, pp 223–228

  68. Loukas C, Nikiteas N, Schizas D, Georgiou E (2016) Shot boundary detection in endoscopic surgery videos using a variational Bayesian framework. Int J Comput Assist Radiol Surg 11:1937–1949. doi: 10.1007/s11548-016-1431-2

    Article  PubMed  Google Scholar 

  69. Varytimidis C, Rapantzikos K, Loukas C, Kollias S (2016) Surgical video retrieval using deep neural networks. In: Medical image computing and computer-assisted intervention M2CAI—MICCAI workshop, pp 4–14

  70. Lux M, Marques O, Schöffmann K, Böszörmenyi L, Lajtai G (2009) A novel tool for summarization of arthroscopic videos. Multimed Tools Appl 46:521–544. doi: 10.1007/s11042-009-0353-1

    Article  Google Scholar 

  71. Schoeffmann K, Del Fabro M, Szkaliczki T, Böszörmenyi L, Keckstein J (2014) Keyframe extraction in endoscopic video. Multimed Tools Appl 74:11187–11206. doi: 10.1007/s11042-014-2224-7

    Article  Google Scholar 

  72. Mendi E, Bayrak C, Cecen S, Ermisoglu E (2013) Content-based management service for medical videos. Telemed e-Health 19:36–41. doi: 10.1089/tmj.2011.0239

    Article  Google Scholar 

  73. Lokoc J, Schoeffmann K, del Fabro M (2015) Dynamic hierarchical visualization of keyframes in endoscopic video. Lect Notes Comput Sci 8936:291–294

    Article  Google Scholar 

  74. Roldan-Carlos J, Lux M, Giró-i-Nieto X, Muñoz P, Anagnostopoulos N (2015) Visual information retrieval in endoscopic video archives. In: International workshop on content-based multimedia indexing, Prague, pp 109–114

  75. Beecks C, Schoeffmann K, Lux M, Uysal MS, Seidl T (2015) Endoscopic video retrieval: a signature-based approach for linking endoscopic images with video segments. In: Del Bimbo A, Chen S-C, Wang H, Yu H, Zimmermann R (eds) IEEE proceedings of the international symposium on multimedia, Miami, pp 1–6

  76. Schoeffmann K, Beecks C, Lux M, Seran M, Seidl T (2016) Content-based retrieval in videos from laparoscopic surgery. In: SPIE medical imaging: image-guided procedures, robotic interventions, and modeling, San Diego, pp 1–10

  77. Twinanda AP, de Mathelin M, Padoy N (2014) Fisher kernel based task boundary retrieval in laparoscopic database with single video query. Med Image Comput Comput Interv 17(3):409–416. doi: 10.1007/978-3-319-10443-0_52

    Google Scholar 

  78. Chen L, Zhang P, Li B (2014) Instructive video retrieval based on hybrid ranking and attribute learning a case study on surgical skill training. In: 22nd ACM international conference on multimedia (ACM MM), Orlando, pp 1045–1048

  79. Speidel S, Benzko J, Krappe S, Sudra G, Azad P, Müller-Stich BP, Gutt C, Dillmann R (2009) Automatic classification of minimally invasive instruments based on endoscopic image sequences. In: SPIE medical imaging: image-guided procedures, robotic interventions, and modeling, pp 72610A–72610A1

  80. Kumar S, Narayanan MS, Misra S, Garimella S, Singhal P, Corso JJ, Krovi V (2013) Vision based decision-support and safety systems for robotic surgery. In: Proceedings of workshop on medical cyber physical systems

  81. Primus MJ, Schoeffmann K, Böszörmenyi L (2015) Instrument classification in laparoscopic videos. In: International workshop on content-based multimedia indexing, Prague, pp 1–6

  82. Beecks C, Schoeffmann K, Lux M, Uysal MS, Seidl T (2014) Segmentation and indexing of endoscopic videos. In: ACM international conference on multimedia (ACM MM), Orlando, pp 659–662

  83. Bouget D, Lalys F, Jannin P (2012) Surgical tools recognition and pupil segmentation for cataract surgical process modeling. Stud Health Technol Inform 173:78–84

    PubMed  Google Scholar 

  84. Alsheakhali M, Eslami A, Navab N (2015) Microscopic surgical tool type detection. In: Proceedings of MICCAI workshop on interventional microscopy, Munich, pp 1–8

  85. Bouget D, Benenson R, Omran M, Riffaud L, Schiele B, Jannin P (2015) Detecting surgical tools by modelling local appearance and global shape. IEEE Trans Med Imaging 34:2603–2617. doi: 10.1109/TMI.2015.2450831

    Article  PubMed  Google Scholar 

  86. Loukas C, Georgiou E (2016) Performance comparison of various feature detector-descriptors and temporal models for video-based assessment of laparoscopic skills. Int J Med Robot Comput Assist Surg 12:387–398. doi: 10.1002/rcs.1702

    Article  Google Scholar 

  87. Zhang Q, Li B (2011) Video-based motion expertise analysis in simulation-based surgical training using hierarchical dirichlet process hidden markov model. In: International ACM workshop on medical multimedia analysis and retrieval. ACM Press, New York, pp 19–24

  88. Zhang Q, Li B (2015) Relative hidden Markov models for video-based evaluation of motion skills in surgical training. IEEE Trans Pattern Anal Mach Intell 37:1206–1218. doi: 10.1109/TPAMI.2014.2361121

    Article  PubMed  Google Scholar 

  89. Zhang Q, Chen L, Tian Q, Li B (2013) Video-based analysis of motion skills in simulation-based surgical training. In: SPIE, multimedia content and mobile devices, Burlingame, pp 86670A–86770A

  90. Gray RJ, Kahol K, Islam G, Smith M, Chapital A, Ferrara J (2012) High-fidelity, low-cost, automated method to assess laparoscopic skills objectively. J Surg Educ 69:335–339. doi: 10.1016/j.jsurg.2011.10.014

    Article  PubMed  Google Scholar 

  91. Suzuki T, Egi H, Hattori M, Tokunaga M, Sawada H, Ohdan H (2015) An evaluation of the endoscopic surgical skills assessment using a video analysis software program. Surg Endosc 29:1804–1808. doi: 10.1007/s00464-014-3863-5

    Article  PubMed  Google Scholar 

  92. Islam G, Kahol K, Li B, Smith M, Patel VL (2016) Affordable, web-based surgical skill training and evaluation tool. J Biomed Inform 59:102–114. doi: 10.1016/j.jbi.2015.11.002

    Article  PubMed  Google Scholar 

  93. Bettadapura V, Schindler G, Ploetz T, Essa I (2013) Augmenting bag-of-words: data-driven discovery of temporal and structural information for activity recognition. In: IEEE conference on computer vision and pattern recognition. IEEE, pp 2619–2626

  94. Sharma Y, Bettadapura V, Ploetz T, Hammerla N, Mellor S, McNaney R, Olivier P, Deshmukh S, Mccaskie A, Essa I (2014) Video based assessment of OSATS using sequential motion textures. In: Proceedings of M2CAI, 2014

  95. Zia A, Sharma Y, Bettadapura V, Sarin EL, Clements MA, Essa I (2015) Automated assessment of surgical skills using frequency analysis. In: Navab N, Hornegger J, Wells WM, Frangi AF (eds) Lecture notes in computer science (MICCAI 2015). Springer, Cham, pp 430–438

    Google Scholar 

  96. Zia A, Sharma Y, Bettadapura V, Sarin EL, Ploetz T, Clements MA, Essa I (2016) Automated video-based assessment of surgical skills for training and evaluation in medical schools. Int J Comput Assist Radiol Surg 11:1623–1636. doi: 10.1007/s11548-016-1468-2

    Article  PubMed  Google Scholar 

  97. Zhu J, Luo J, Soh JM, Khalifa YM (2015) A computer vision-based approach to grade simulated cataract surgeries. Mach Vis Appl 26:115–125. doi: 10.1007/s00138-014-0646-x

    Article  Google Scholar 

  98. Kononowicz AA, Wiśniowski Z (2008) MPEG-7 as a metadata standard for indexing of surgery videos in medical e-learning. Lect Notes Comput Sci 5103:188–197

    Article  Google Scholar 

  99. Guggenberger M, Lux M, Riegler M, Halvorsen P (2014) Event understanding in endoscopic surgery videos. In: 1st ACM international workshop on human centered event understanding from multimedia, Orlando, pp 17–22

  100. Xhura D (2014) Learning recognition of semantically relevant video segments from endoscopy videos contributed and edited in a private social network categories and subject descriptors. In: ACM international workshop on multimedia, Orlando, pp 663–666

  101. Lalys F, Jannin P (2014) Surgical process modelling: a review. Int J Comput Assist Radiol Surg 9:495–511. doi: 10.1007/s11548-013-0940-5

    Article  PubMed  Google Scholar 

Download references

Author information

Affiliations

  1. Laboratory of Medical Physics, Medical School, National and Kapodistrian University of Athens, Mikras Asias 75 str., 11527, Athens, Greece

    Constantinos Loukas

Authors
  1. Constantinos Loukas
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Constantinos Loukas.

Ethics declarations

Disclosures

Dr. Constantinos Loukas has no conflicts of interest or financial ties to disclose.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Loukas, C. Video content analysis of surgical procedures. Surg Endosc 32, 553–568 (2018). https://doi.org/10.1007/s00464-017-5878-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00464-017-5878-1

Keywords

  • Surgery
  • Content analysis
  • Video analysis
  • Video retrieval
  • Video annotation