Augmented Visualization and Touchless Interaction with Virtual Organs

  • Lucio Tommaso De PaolisEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10814)


The actual trend in surgery is the transition from open procedures to minimally invasive interventions and revolutionary changes in computer-aided surgery have been obtained thanks to the introduction of the augmented reality technology that can support the doctor during the surgery. A realistic three-dimensional visualization of the patient’s organs can be obtained by means of specific algorithms of segmentation and classification of medical images and advanced modalities of touchless interaction and innovative gesture-control devices permit surgeon to have a natural and simple way of fruition of the patient’s data preserving the surgical environment from the danger of contamination.


Touchless interaction Computer aided surgery Interactive 3D modelling Augmented reality 


  1. 1.
    Maad, S.: Augmented reality. In: The Horizon of Virtual and Augmented Reality: The Reality of the Global Digital Age. Intech, January 2010. ISBN 978-953-7619-69-5Google Scholar
  2. 2.
    De Paolis, L.T., Aloisio, G.: Augmented reality in minimally invasive surgery. In: Mukhopadhyay, S.C., Lay-Ekuakille, A. (eds.) Advances in Biomedical Sensing, Measurements. LNEE, vol. 55. Instrumentation and Systems. Springer, Heidelberg (2010). CrossRefGoogle Scholar
  3. 3.
    Samset, E., Schmalstieg, D., Vander Sloten, J., Freudenthal, A., Declerck, J., Casciaro, S., Rideng, Ø., Gersak, B.: Augmented reality in surgical procedures. In: SPIE Human Vision and Electronic Imaging XIII (2008)Google Scholar
  4. 4.
    Bichlmeier, C., Wimmer, F., Michael, H.S., Nassir, N.: Contextual anatomic mimesis: hybrid in-situ visualization method for improving multi-sensory depth perception in medical augmented reality. In: Proceedings of Sixth IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR 2007), Nara, Japan, pp. 129–138 (2007)Google Scholar
  5. 5.
    Nicolau, S.A., Pennec, X., Soler, L., Buy, X., Gangi, A., Ayache, N., Marescaux, J.: An augmented reality system for liver thermal ablation: design and evaluation on clinical cases. Med. Image Anal. 13(3), 494–506 (2009). ElsevierCrossRefGoogle Scholar
  6. 6.
    De Mauro, A., Mazars, J., Manco, L., Mataj, T., Fernandez, A.H., Cortes, C., De Paolis, L.T.: Intraoperative navigation system for image guided surgery. In: 6th International Conference on Complex, Intelligent, and Software Intensive Systems, pp. 486–490 (2012)Google Scholar
  7. 7.
    De Paolis, L.T., Pulimeno, M., Aloisio, G.: An augmented reality application for minimally invasive surgery. In: Katashev, A., Dekhtyar, Y., Spigulis, J. (eds.) 14th Nordic-Baltic Conference on Biomedical Engineering and Medical Physics. IFMBE, vol. 20, pp. 489–492. Springer, Heidelberg (2008). CrossRefGoogle Scholar
  8. 8.
    Ricciardi, F., Copelli, C., De Paolis, L.T.: A pre-operative planning module for an augmented reality application in maxillo-facial surgery. In: De Paolis, L.T., Mongelli, A. (eds.) AVR 2015. LNCS, vol. 9254, pp. 244–254. Springer, Cham (2015). CrossRefGoogle Scholar
  9. 9.
    De Paolis, L.T., Ricciardi, F., Dragoni, A.F., Aloisio, G.: An augmented reality application for the radio frequency ablation of the liver tumors. In: Murgante, B., Gervasi, O., Iglesias, A., Taniar, D., Apduhan, B.O. (eds.) ICCSA 2011. LNCS, vol. 6785, pp. 572–581. Springer, Heidelberg (2011). CrossRefGoogle Scholar
  10. 10.
    Abe, Y., Sato, S., Kato, K., Hyakumachi, T., Yanagibashi, Y., Ito, M., Abumi, K.: A novel 3D guidance system using augmented reality for percutaneous vertebroplasty. J. Neurosurg. Spine 19, 492–501 (2013)CrossRefGoogle Scholar
  11. 11.
    Baker, D.K., Fryberger, C.T., Ponce, B.A.: The emergence of augmented reality in orthopaedic surgery and education. Orthop. J. Harvard Med. School 16, 8–16 (2015)Google Scholar
  12. 12.
    Houts, P.S., Doak, C.C., Doak, L.G., Loscalzo, M.J.: The role of pictures in improving health communication: a review of research on attention, comprehension, recall, and adherence. Patient Educ. Counsel 61, 173–190 (2006)CrossRefGoogle Scholar
  13. 13.
    König, T.S.: Usability issues in 3D medical visualization. Ph.D thesis, Technische Universitat Wien, Wien (2001)Google Scholar
  14. 14.
    Livatino, S., De Paolis, L.T., D’Agostino, M., Zocco, A., Agrimi, A., De Santis, A., Bruno, L.V., Lapresa, M.: Stereoscopic visualization and 3-D technologies in medical endoscopic teleoperation. IEEE Trans. Ind. Electron. 62(1), 525–535 (2015)CrossRefGoogle Scholar
  15. 15.
    De Paolis, L.T., De Mauro, A., Raczkowsky, J., Aloisio, G.: Virtual model of the human brain for neurosurgical simulation. Stud. Health Technol. Inform. 150, 811–815 (2009)Google Scholar
  16. 16.
    Yoo, T.S.: Insight into Images. A K Peters, Natick (2004)CrossRefGoogle Scholar
  17. 17.
    Garber, L.: Gestural technology: moving interfaces in a new direction [technology news]. Computer 46, 22–25 (2013)CrossRefGoogle Scholar
  18. 18.
    Joslin, C., El-Sawah, A., Chen, Q., Georganas, N.: Dynamic gesture recognition. In: Proceedings of the IEEE Instrumentation and Measurement Technology Conference, Ottawa, ON, Canada, 16–19 May 2005, vol. 3, pp. 1706–1711 (2005)Google Scholar
  19. 19.
    Bellarbi, A., Benbelkacem, S., Zenati-Henda, N., Belhocine, M.: Hand gesture interaction using color-based method for tabletop interfaces. In: Proceedings of the 7th International Symposium on Intelligent Signal Processing (WISP), Floriana, Malta, 19–21 September 2011, pp. 1–6 (2011)Google Scholar
  20. 20.
    Fiala, M.: ARTag, a fiducial marker system using digital techniques. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, San Diego, CA, USA, 20–25 June 2005, vol. 2, pp. 590–596 (2005)Google Scholar
  21. 21.
    Invitto, S., Faggiano, C., Sammarco, S., De Luca, V., De Paolis, L.T.: Haptic, virtual interaction and motor imagery: entertainment tools and psychophysiological testing. Sensors 16, 394 (2016)CrossRefGoogle Scholar
  22. 22.
    Nintendo Wii System.
  23. 23.
    The leap motion controller.
  24. 24.
    Sony PlayStation.
  25. 25.
  26. 26.
    Myo Armband.

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.AVR Lab, Department of Engineering for InnovationUniversity of SalentoLecceItaly

Personalised recommendations