Floating autostereoscopic 3D display with multidimensional images for telesurgical visualization

  • Dong Zhao
  • Longfei Ma
  • Cong Ma
  • Jie Tang
  • Hongen LiaoEmail author
Original Article



We propose a combined floating autostereoscopic three-dimensional (3D) display approach for telesurgical visualization, which could reproduce live surgical scene in a realistic and intuitive manner.


A polyhedron-shaped 3D display device is developed for spatially floating autostereoscopic 3D image. Integral videography (IV) technique is adopted to generate real-time 3D images. Combined two-dimensional (2D) and 3D displays are presented floatingly around the center of the display device through reflection of semitransparent mirrors. Intra-operative surgery information is fused and updated in the 3D display, so that telesurgical visualization could be enhanced remotely.


The experimental results showed that our approach can achieve a combined floating autostereoscopic display that presents 2D and 3D fusion images. The glasses-free IV 3D display has full parallax and can be observed by multiple persons from surrounding areas at the same time. Furthermore, real-time surgical scene could be presented and updated in a realistic and intuitive visualization platform. It is shown that the proposed method is feasible for facilitating telesurgical visualization.


The proposed floating autostereoscopic display device presents surgical information in an efficient form, so as to enhance operative cooperation and efficiency during operation. Combined presentation of imaging information is promising for medical applications.


Autostereoscopic display Floating display Integral videography Telesurgery Combined display 



This work was supported in part by National Natural Science Foundation of China (Grant Nos. 81427803, 61361160417, 81271735), Grant-in-Aid of Project 985, and Beijing Municipal Science & Technology Commission (Z151100003915079).

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflicts of interest in the research.


  1. 1.
    Guthart G, Salisbury JK (2000) The intuitive\(^{TM}\) telesurgery system: overview and application. In: IEEE international conference on robotics and automation (ICRA), vol 1, pp 618–621Google Scholar
  2. 2.
    Newman JG, Kuppersmith RB (2011) Robotics and telesurgery in otolaryngology. Otolaryngol Clin N Am 44(6):1317–1331CrossRefGoogle Scholar
  3. 3.
    Antoniou SA, Antoniou GA, Franzen J, Bollmann S, Koch OO, Pointner R, Granderath FA (2012) A comprehensive review of telementoring applications in laparoscopic general surgery. Surg Endosc 26(8):2111–2116CrossRefPubMedGoogle Scholar
  4. 4.
    Marescaux J, Leroy J, Gagner M, Rubino F, Mutter D, Vix M, Smith MK (2001) Transatlantic robot-assisted telesurgery. Nature 413(6854):379–380CrossRefPubMedGoogle Scholar
  5. 5.
    Lum MJ, Friedman DC, Sankaranarayanan G, King H, Fodero K, Leuschke R, Hannaford B, Rosen J, Sinanan MN (2009) The RAVEN: design and validation of a telesurgery system. Int J Robot Res 28(9):1183–1197CrossRefGoogle Scholar
  6. 6.
    Bucholz RD, Laycock KA, McDurmont L (2011) Operating room integration and telehealth. Springer, ViennaCrossRefGoogle Scholar
  7. 7.
    Shekelle PG, Wachter RM, Pronovost PJ, Lucas S, Noble M, Reston JT, Vaiana ME (2013) Making health care safer II: an updated critical analysis of the evidence for patient safety practices. Evidence report/technology assessment 211, pp 1–945Google Scholar
  8. 8.
    Irion KM, Novak P (2000) Systems workplace for endoscopic surgery. Minim Invasive Ther Allied Technol 9(3–4):193–197CrossRefPubMedGoogle Scholar
  9. 9.
    Zindel C (2000) Systems solutions for integration of devices in the OR. Minim Invasive Ther Allied Technol 9(3–4):199–205CrossRefPubMedGoogle Scholar
  10. 10.
    Abildgaard A, Witwit AK, Karlsen JS, Jacobsen EA, Tennøe B, Ringstad G, Due-Tønnessen P (2010) An autostereoscopic 3D display can improve visualization of 3D models from intracranial MR angiography. Int J Comput Assist Radiol Surg 5:549–554CrossRefPubMedGoogle Scholar
  11. 11.
    Hinata N, Miyake H, Kurahashi T, Ando M, Furukawa J, Ishimura T, Fujisawa M (2014) Novel telementoring system for robot-assisted radical prostatectomy: impact on the learning curve. Urology 83(5):1088–1092CrossRefPubMedGoogle Scholar
  12. 12.
    Narita Y, Tsukagoshi S, Suzuki M et al (2014) Usefulness of a glasses-free medical three-dimensional autostereoscopic display in neurosurgery. Int J Comput Assist Radiol Surg 9(5):905–911CrossRefPubMedGoogle Scholar
  13. 13.
    Liao H, Iwahara M, Hata N, Dohi T (2004) High-quality integral videography using a multiprojector. Opt Express 12(6):1067–1076CrossRefPubMedGoogle Scholar
  14. 14.
    Liao H, Hata N, Nakajima S, Iwahara M, Sakuma I, Dohi T (2004) Surgical navigation by autostereoscopic image overlay of integral videography. IEEE Trans Inf Technol Biomed 8(2):114–121Google Scholar
  15. 15.
    Wang J, Suenaga H, Hoshi K, Yang L, Kobayashi E, Sakuma I, Liao H (2014) Augmented reality navigation with automatic marker-free image registration using 3D image overlay for dental surgery. IEEE Trans Biomed Eng 61(4):1295–1304CrossRefPubMedGoogle Scholar
  16. 16.
  17. 17.
    Zhao D, Su B, Chen G, Liao H (2015) 360 degrees viewable floating autostereoscopic display using integral photography and multiple semitransparent mirrors. Opt Express 23(8):9812–9823CrossRefPubMedGoogle Scholar
  18. 18.
    Liao H, Iwahara M, Koike T, Hata N, Sakuma I, Dohi T (2005) Scalable high-resolution integral videography autostereoscopic display with a seamless multiprojection system. Appl Opt 44(3):305–315CrossRefPubMedGoogle Scholar
  19. 19.
    Oh Y, Shin D, Lee BG, Jeong SI, Choi HJ (2014) Resolution-enhanced integral imaging in focal mode with a time-multiplexed electrical mask array. Opt Express 22(15):17620–17629Google Scholar
  20. 20.
    Lin B, Sun, Y, Qian X, Goldgof D, Gitlin R, You Y (2015) Video-based 3D reconstruction, laparoscope localization and deformation recovery for abdominal minimally invasive surgery: a survey. Int J Med Robot Comput Assist Surg. doi: 10.1002/rcs.1661
  21. 21.
    Liao H, Tsuzuki M, Mochizuki T, Kobayashi E, Chiba T, Sakuma I (2009) Fast image mapping of endoscopic image mosaics with three-dimensional ultrasound image for intrauterine fetal surgery. Minim Invasive Ther Allied Technol 18(6):332–340CrossRefPubMedGoogle Scholar
  22. 22.
    Wren SM, Curet MJ (2011) Single-port robotic cholecystectomy: results from a first human use clinical study of the new da Vinci single-site surgical platform. Arch Surg 146(10):1122–1127CrossRefPubMedGoogle Scholar

Copyright information

© CARS 2015

Authors and Affiliations

  1. 1.Department of Biomedical Engineering, School of MedicineTsinghua UniversityBeijingChina
  2. 2.Department of Neurosurgery, Beijing Tiantan HospitalCapital Medical UniversityBeijingChina

Personalised recommendations