Dense soft tissue 3D reconstruction refined with super-pixel segmentation for robotic abdominal surgery

  • Veronica PenzaEmail author
  • Jesús Ortiz
  • Leonardo S. Mattos
  • Antonello Forgione
  • Elena De Momi
Original Article



Single-incision laparoscopic surgery decreases postoperative infections, but introduces limitations in the surgeon’s maneuverability and in the surgical field of view. This work aims at enhancing intra-operative surgical visualization by exploiting the 3D information about the surgical site. An interactive guidance system is proposed wherein the pose of preoperative tissue models is updated online. A critical process involves the intra-operative acquisition of tissue surfaces. It can be achieved using stereoscopic imaging and 3D reconstruction techniques. This work contributes to this process by proposing new methods for improved dense 3D reconstruction of soft tissues, which allows a more accurate deformation identification and facilitates the registration process.


Two methods for soft tissue 3D reconstruction are proposed: Method 1 follows the traditional approach of the block matching algorithm. Method 2 performs a nonparametric modified census transform to be more robust to illumination variation. The simple linear iterative clustering (SLIC) super-pixel algorithm is exploited for disparity refinement by filling holes in the disparity images.


The methods were validated using two video datasets from the Hamlyn Centre, achieving an accuracy of 2.95 and 1.66 mm, respectively. A comparison with ground-truth data demonstrated the disparity refinement procedure: (1) increases the number of reconstructed points by up to 43 % and (2) does not affect the accuracy of the 3D reconstructions significantly.


Both methods give results that compare favorably with the state-of-the-art methods. The computational time constraints their applicability in real time, but can be greatly improved by using a GPU implementation.


Surface reconstruction Super-pixel segmentation Robotic surgery Census transform Depth estimation 



Authors would like to thank Nikhil Deshpande for his kind suggestions and support during the elaboration of this paper.

Compliance with ethical standards

Conflict of interest

All authors declare that they have no conflict of interest.


  1. 1.
    Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Susstrunk S (2012) Slic superpixels compared to state-of-the-art superpixel methods. IEEE Trans Pattern Anal Mach Intell 34(11):2274–2282CrossRefPubMedGoogle Scholar
  2. 2.
    Banks J, Bennamoun M, Corke P (1997) Non-parametric techniques for fast and robust stereo matching. In: TENCON’97. IEEE region 10 annual conference. Speech and image technologies for computing and telecommunications. Proceedings of IEEE, IEEE, vol 1, pp 365–368Google Scholar
  3. 3.
    Barbalata C, Mattos L (2014) Laryngeal tumor detection and classification in endoscopic video. IEEE J Biomed Health Inform. doi: 10.1109/JBHI.2014.2374975
  4. 4.
    Bergen T, Wittenberg T (2014) Stitching and surface reconstruction from endoscopic image sequences: a review of applications and methods. IEEE J Biomed Health Inform. doi: 10.1109/JBHI.2014.2384134
  5. 5.
    Chum O, Matas J, Kittler J (2003) Locally optimized ransac. In: Pattern recognition, vol 2781. Springer, Heidelberg, pp 236–243. doi: 10.1007/978-3-540-45243-0_31
  6. 6.
    Ee WWG, Lau WLJ, Yeo W, Von Bing Y, Yue WM (2014) Does minimally invasive surgery have a lower risk of surgical site infections compared with open spinal surgery? Clin Orthop Relat Res 472(6):1718–1724PubMedCentralPubMedGoogle Scholar
  7. 7.
    Giannarou S, Visentini-Scarzanella M, Yang GZ (2013) Probabilistic tracking of affine-invariant anisotropic regions. IEEE Trans Pattern Anal Mach Intell 35(1):130–143. doi: 10.1109/TPAMI.2012.81 CrossRefPubMedGoogle Scholar
  8. 8.
    Guthart G, Salisbury Jr JK (2000) The intuitive TM telesurgery system: overview and application. In: ICRA, pp 618–621Google Scholar
  9. 9.
    Hartley R, Zisserman A (2003) Multiple view geometry in computer vision. Cambridge University Press, CambridgeGoogle Scholar
  10. 10.
    Kanade T, Okutomi M (1994) A stereo matching algorithm with an adaptive window: theory and experiment. IEEE Trans Pattern Anal Mach Intell 16(9):920–932CrossRefGoogle Scholar
  11. 11.
    Klaus A, Sormann M, Karner K (2006) Segment-based stereo matching using belief propagation and a self-adapting dissimilarity measure. In: 18th International conference on pattern recognition, 2006. ICPR 2006, IEEE, vol 3, pp 15–18Google Scholar
  12. 12.
    Ma L, Li J, Ma J, Zhang H (2013) A modified census transform based on the neighborhood information for stereo matching algorithm. In: 2013 Seventh international conference on image and graphics (ICIG), IEEE, pp 533–538Google Scholar
  13. 13.
    Mountney P, Stoyanov D, Yang GZ (2010) Three-dimensional tissue deformation recovery and tracking. IEEE Signal Process Mag 27(4):14–24CrossRefGoogle Scholar
  14. 14.
    Musialski P, Wonka P, Aliaga DG, Wimmer M, Gool L, Purgathofer W (2013) A survey of urban reconstruction. In: Computer graphics forum, Wiley Online Library, vol 32, pp 146–177Google Scholar
  15. 15.
    Nicolau S, Goffin L, Soler L (2005) A low cost and accurate guidance system for laparoscopic surgery: validation on an abdominal phantom. In: Proceedings of the ACM symposium on virtual reality software and technology, ACM, pp 124–133Google Scholar
  16. 16.
    Nicolau S, Soler L, Mutter D, Marescaux J (2011) Augmented reality in laparoscopic surgical oncology. Surg Oncol 20(3):189–201CrossRefPubMedGoogle Scholar
  17. 17.
    OpenCV (2015).
  18. 18.
    Ortiz J, Calderon H, Fontaine J (2010) Disparity map computation on scalable computing. In: 2010 IEEE/ASME international conference on mechatronics and embedded systems and applications (MESA), IEEE, pp 301–306Google Scholar
  19. 19.
    Parchami M, Cadeddu JA, Mariottini GL (2014) Endoscopic stereo reconstruction: a comparative study. In: 2014 36th annual international conference of the IEEE Engineering in Medicine and Biology Society (EMBC), IEEE, pp 2440–2443Google Scholar
  20. 20.
    Penza V, Ortiz J, De Momi E, Forgione A, Mattos L (2014) Virtual assistive system for robotic single incision laparoscopic surgery. In: 4th Joint workshop on computer/robot assisted surgery, pp 52–55Google Scholar
  21. 21.
    Quigley M, Conley K, Gerkey BP, Faust J, Foote T, Leibs J, Wheeler R, Ng AY (2009) Ros: an open-source robot operating system. In: ICRA workshop on open source softwareGoogle Scholar
  22. 22.
    Röhl S, Bodenstedt S, Suwelack S, Kenngott H, Müller-Stich BP, Dillmann R, Speidel S (2012) Dense gpu-enhanced surface reconstruction from stereo endoscopic images for intraoperative registration. Med Phys 39(3):1632–1645CrossRefPubMedGoogle Scholar
  23. 23.
    Scharstein D, Szeliski R (2002) A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int J Comput Vis 47(1–3):7–42CrossRefGoogle Scholar
  24. 24.
    Schoob A, Kundrat D, Kleingrothe L, Kahrs LA, Andreff N, Ortmaier T (2014) Tissue surface information for intraoperative incision planning and focus adjustment in laser surgery. Int J Comput Assist Radiol Surg 10(2):171–181Google Scholar
  25. 25.
    Stoyanov D (2012) Surgical vision. Ann Biomed Eng 40(2):332–345CrossRefPubMedGoogle Scholar
  26. 26.
    Stoyanov D, Scarzanella MV, Pratt P, Yang GZ (2010) Real-time stereo reconstruction in robotically assisted minimally invasive surgery. In: Medical image computing and computer-assisted intervention–MICCAI 2010. Springer, pp 275–282Google Scholar
  27. 27.
    Tan HJ, Wolf JS, Ye Z, Hafez KS, Miller DC (2014) Population level assessment of hospital based outcomes following laparoscopic versus open partial nephrectomy during the adoption of minimally invasive surgery. J Urol 191(5):1231–1237CrossRefPubMedGoogle Scholar
  28. 28.
    Taylor G, Barrie J, Hood A, Culmer P, Neville A, Jayne D (2013) Surgical innovations: addressing the technology gaps in minimally invasive surgery. Trends Anaesth Crit Care 3(2):56–61CrossRefGoogle Scholar
  29. 29.
    Tian Q, Huhns MN (1986) Algorithms for subpixel registration. Comput Vis Graph Image Process 35(2):220–233CrossRefGoogle Scholar
  30. 30.
    Varela JE, Wilson SE, Nguyen NT (2010) Laparoscopic surgery significantly reduces surgical-site infections compared with open surgery. Surg Endosc 24(2):270–276CrossRefPubMedGoogle Scholar
  31. 31.
    Wichmann MW, Hüttl TP, Winter H, Spelsberg F, Angele MK, Heiss MM, Jauch KW (2005) Immunological effects of laparoscopic vs open colorectal surgery: a prospective clinical study. Arch Surg 140(7):692–697CrossRefPubMedGoogle Scholar
  32. 32.
    Zitnick CL, Kang SB (2007) Stereo for image-based rendering using image over-segmentation. Int J Comput Vis 75(1):49–65CrossRefGoogle Scholar
  33. 33.
    Zitova B, Flusser J (2003) Image registration methods: a survey. Image Vis Comput 21(11):977–1000CrossRefGoogle Scholar

Copyright information

© CARS 2015

Authors and Affiliations

  • Veronica Penza
    • 1
    • 5
    Email author
  • Jesús Ortiz
    • 1
  • Leonardo S. Mattos
    • 1
  • Antonello Forgione
    • 2
    • 3
    • 4
  • Elena De Momi
    • 5
  1. 1.Department of Advanced RoboticsIstituto Italiano di TecnologiaGenoaItaly
  2. 2.Ospedale Niguarda Ca’ GrandaMilanItaly
  3. 3.AIMS AcademyMilanItaly
  4. 4.Valuebiotech s.r.l.MilanItaly
  5. 5.Department of Electronics, Information and BioengineeringPolitecnico di MilanoMilanItaly

Personalised recommendations