Skip to main content
SpringerLink
Log in

Online recognition of surgical instruments by information fusion

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

Automatic online recognition of surgical instruments is required to monitor instrument use for surgical process modeling. A system was developed and tested using available technologies.

Methods

A recognition system was developed using RFID technology to identify surgical activities. Information fusion for online recognition of surgical process models was conceived as a layer model to abstract information from specific sensor technologies. Redundant, complementary, and cooperative sensor signal fusion was used in the layer model to increase the surgical instrument recognition rate. Several different information fusion strategies were evaluated for situation recognition abilities in a mock-up environment based on simulations of surgical processes.

Results

This information fusion system was able to reliably detect, identify, and localize surgical instruments in an interventional suite. A combination of information fusion strategies was able to achieve a correct classification rate of 97% and was as effective as observer-based acquisition methods.

Conclusion

Different information fusion strategies for the recognition of surgical instruments were evaluated, showing that redundant, complementary, and cooperative information fusion is feasible for recognition of surgical work steps. A combination of sensor- and observer-based modeling strategies provides the most robust solution for surgical process models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Cleary K, Kinsella A, Mun SK (2005) OR2020 workshop report: operating room of the future. In: Lemke HU, Inamura K, Doi K, Vannier MW, Farmann AG (eds) CARS 2005 computer assisted radiology and surgery. Elsevier, Amsterdam, pp 832–838

    Google Scholar 

  2. Lemke HU, Vannier MW (2006) The operating room and the need for an IT infrastructure and standards. Int J Comput Assist Radiol Surg 1: 117–121

    Article  Google Scholar 

  3. Neumuth T, Jannin P, Schlomberg J, Meixensberger J, Wiedemann P, Burgert O (2011) Analysis of surgical intervention populations using generic surgical process models. Int J Comput Assist Radiol Surg 6: 59–71

    Article  PubMed  Google Scholar 

  4. Blum T, Padoy N, Feußner H, Navab N (2008) Workflow mining for visualization and analysis of surgeries. Int J Comput Assist Radiol Surg 3: 379–386

    Article  Google Scholar 

  5. Padoy N, Blum T, Ahmadi SA, Feussner H, Berger MO, Navab N (2010) Statistical modeling and recognition of surgical workflow. Med Image Anal. doi:10.1016/j.media.2010.10.001

  6. Sudra G, Becker A, Braun M, Speidel S, Mueller-Stich BP, Dillmann R (2009) Estimating similarity of surgical situations with case-retrieval-nets. Stud Health Technol Inform 142: 358–363

    PubMed  Google Scholar 

  7. Lalys F, Riffaud L, Morandi X, Jannin P (2010) Automatic phases recognition in pituitary surgeries by microscope images classification. Presented at IPCAI 2010

  8. Bouarfa L, Jonker PP, Dankelman J (2011) Discovery of high-level tasks in the operating room. J Biomed Inform 44: 455–462

    Article  PubMed  CAS  Google Scholar 

  9. Lin H, Hager G (2009) User-independent models of manipulation using video contextual cues. Presented at M2CAI-Workshop 2009

  10. Varadarajan B, Reiley C, Lin H, Khudanpur S, Hager G (2009) Data-derived models for segmentation with application to surgical assessment and training. Med Image Comput Comput Assist Interv 12: 426–434

    PubMed  Google Scholar 

  11. Darzi A, Mackay S (2002) Skills assessment of surgeons. Surgery 131: 121–124

    Article  PubMed  Google Scholar 

  12. Rosen J, Hannaford B, Richards CG, Sinanan MN (2001) Markov modeling of minimally invasive surgery based on tool/tissue interaction and force/torque signatures for evaluating surgical skills. IEEE Trans Biomed Eng 48: 579–591

    Article  PubMed  CAS  Google Scholar 

  13. Ahmadi A, Padoy N, Rybachuk K, Feußner H, Heining S, Navab N (2009) Motif discovery in OR sensor data with application to surgical workflow analysis and activity detection. Presented at M2CAI-Workshop 2009

  14. Xiao Y, Hu P, Hu H, Ho D, Dexter F, Mackenzie CF, Seagull FJ, Dutton RP (2005) An algorithm for processing vital sign monitoring data to remotely identify operating room occupancy in real-time. Anesth Analg 101: 823–829

    Article  PubMed  Google Scholar 

  15. Lin HC, Shafran I, Yuh D, Hager GD (2006) Towards automatic skill evaluation: detection and segmentation of robot-assisted surgical motions. Comput Aided Surg 11: 220–230

    PubMed  Google Scholar 

  16. James A, Vieira D, Lo B, Darzi A, Yang GZ (2007) Eye-gaze driven surgical workflow segmentation. Med Image Comput Comput Assist Interv 10: 110–117

    PubMed  CAS  Google Scholar 

  17. Nara A, Izumi K, Iseki H, Suzuki T, Nambu K, Sakurai Y (2010) Surgical workflow analysis based on staff’s trajectory patterns. Presented at M2CAI-Workshop 2009

  18. Bhatia B, Oates T, Xiao Y, Hu P (2007) Real-time identification of operating room state from video. In: Holte C, Howe A (eds) Proceedings of the 21st AAAI conference on artificial intelligence. AAAI Organization. pp 1761–1766

  19. MacKenzie CL, Ibbotson JA, Cao CGL, Lomax AJ (2001) Hierarchical decomposition of laparoscopic surgery: a human factors approach to investigating the operating room environment. Min Invas Ther Allied Technol 10: 121–127

    Article  Google Scholar 

  20. Jannin P, Raimbault M, Morandi X, Riffaud L, Gibaud B (2003) Model of surgical procedures for multimodal image-guided neurosurgery. Comput Aided Surg 8: 98–106

    Article  PubMed  CAS  Google Scholar 

  21. Jannin P, Morandi X (2007) Surgical models for computer-assisted neurosurgery. NeuroImage 37: 783–791

    Article  PubMed  CAS  Google Scholar 

  22. Neumuth T, Jannin P, Strauss G, Meixensberger J, Burgert O (2009) Validation of knowledge acquisition for surgical process models. J Am Med Inform Assoc 16: 72–80

    Article  PubMed  Google Scholar 

  23. Neumuth T, Trantakis C, Riffaud L, Strauss G, Meixensberger J, Burgert O (2009) Assessment of technical needs for surgical equipment by surgical process models. Min Invas Ther Allied Technol 18: 841–849

    Article  Google Scholar 

  24. Neumuth T, Krauss A, Meixensberger J, Muensterer O (2011) Impact quantification of the DaVinci Telemanipulator system on the surgical workflow using resource impact profiles. Int J Med Robot 7: 156–164

    Article  PubMed  CAS  Google Scholar 

  25. Hall D, LLinas J (1997) An introduction to multisensor data fusion. Proc IEEE 85: 6–23

    Article  Google Scholar 

  26. Xiong N, Svensson P (2002) Multi-sensor management for information fusion: issues and approaches. Inf. Fusion 3: 163–186

    Article  Google Scholar 

  27. Llinas J, Bowman C, Rogova G, Steinberg A, Waltz E, White F (2004) Revisiting the JDL data fusion model II. In: Svensson P, Schubert J (eds) Proceedings of the 7th international conference on information fusion. pp 1218–1230

  28. Kokar MM, Tomasik JA, Weyman J (2004) Formalizing classes of information fusion systems. Inf Fusion 5: 189–202

    Article  Google Scholar 

  29. Durrant-Whyte HF (1988) Sensor models and multisensor integration. Int J Robot Res 7: 97–113

    Article  Google Scholar 

  30. Luo RC, Yih CC, Su KL (2002) Multisensor fusion and integration: approaches, applications, and future research directions. IEEE Sens J 2: 107–119

    Article  Google Scholar 

  31. MathWorks, MATLAB. Natick, MA, 2009

  32. SPSS Inc., SPSS 15.0. Chicago, 2009

  33. van der Togt R, van Lieshout ER, Hensbroek R, Beinat E, Binnekade JM, Bakker PJM (2008) Electromagnetic interference from radio frequency identification inducing potentially hazardous incidents in critical care medical equipment. J Am Med Assoc 299: 2884–2890

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Universität Leipzig, Innovation Center Computer Assisted Surgery (ICCAS), Semmelweisstr. 14, 04103, Leipzig, Germany

    Thomas Neumuth & Christian Meißner

Authors
  1. Thomas Neumuth
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christian Meißner
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Thomas Neumuth.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Neumuth, T., Meißner, C. Online recognition of surgical instruments by information fusion. Int J CARS 7, 297–304 (2012). https://doi.org/10.1007/s11548-011-0662-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-011-0662-5

Keywords

Search

Navigation