Dense soft tissue 3D reconstruction refined with super-pixel segmentation for robotic abdominal surgery
Single-incision laparoscopic surgery decreases postoperative infections, but introduces limitations in the surgeon’s maneuverability and in the surgical field of view. This work aims at enhancing intra-operative surgical visualization by exploiting the 3D information about the surgical site. An interactive guidance system is proposed wherein the pose of preoperative tissue models is updated online. A critical process involves the intra-operative acquisition of tissue surfaces. It can be achieved using stereoscopic imaging and 3D reconstruction techniques. This work contributes to this process by proposing new methods for improved dense 3D reconstruction of soft tissues, which allows a more accurate deformation identification and facilitates the registration process.
Two methods for soft tissue 3D reconstruction are proposed: Method 1 follows the traditional approach of the block matching algorithm. Method 2 performs a nonparametric modified census transform to be more robust to illumination variation. The simple linear iterative clustering (SLIC) super-pixel algorithm is exploited for disparity refinement by filling holes in the disparity images.
The methods were validated using two video datasets from the Hamlyn Centre, achieving an accuracy of 2.95 and 1.66 mm, respectively. A comparison with ground-truth data demonstrated the disparity refinement procedure: (1) increases the number of reconstructed points by up to 43 % and (2) does not affect the accuracy of the 3D reconstructions significantly.
Both methods give results that compare favorably with the state-of-the-art methods. The computational time constraints their applicability in real time, but can be greatly improved by using a GPU implementation.
KeywordsSurface reconstruction Super-pixel segmentation Robotic surgery Census transform Depth estimation
Authors would like to thank Nikhil Deshpande for his kind suggestions and support during the elaboration of this paper.
Compliance with ethical standards
Conflict of interest
All authors declare that they have no conflict of interest.
- 2.Banks J, Bennamoun M, Corke P (1997) Non-parametric techniques for fast and robust stereo matching. In: TENCON’97. IEEE region 10 annual conference. Speech and image technologies for computing and telecommunications. Proceedings of IEEE, IEEE, vol 1, pp 365–368Google Scholar
- 3.Barbalata C, Mattos L (2014) Laryngeal tumor detection and classification in endoscopic video. IEEE J Biomed Health Inform. doi: 10.1109/JBHI.2014.2374975
- 4.Bergen T, Wittenberg T (2014) Stitching and surface reconstruction from endoscopic image sequences: a review of applications and methods. IEEE J Biomed Health Inform. doi: 10.1109/JBHI.2014.2384134
- 5.Chum O, Matas J, Kittler J (2003) Locally optimized ransac. In: Pattern recognition, vol 2781. Springer, Heidelberg, pp 236–243. doi: 10.1007/978-3-540-45243-0_31
- 8.Guthart G, Salisbury Jr JK (2000) The intuitive TM telesurgery system: overview and application. In: ICRA, pp 618–621Google Scholar
- 9.Hartley R, Zisserman A (2003) Multiple view geometry in computer vision. Cambridge University Press, CambridgeGoogle Scholar
- 11.Klaus A, Sormann M, Karner K (2006) Segment-based stereo matching using belief propagation and a self-adapting dissimilarity measure. In: 18th International conference on pattern recognition, 2006. ICPR 2006, IEEE, vol 3, pp 15–18Google Scholar
- 12.Ma L, Li J, Ma J, Zhang H (2013) A modified census transform based on the neighborhood information for stereo matching algorithm. In: 2013 Seventh international conference on image and graphics (ICIG), IEEE, pp 533–538Google Scholar
- 14.Musialski P, Wonka P, Aliaga DG, Wimmer M, Gool L, Purgathofer W (2013) A survey of urban reconstruction. In: Computer graphics forum, Wiley Online Library, vol 32, pp 146–177Google Scholar
- 15.Nicolau S, Goffin L, Soler L (2005) A low cost and accurate guidance system for laparoscopic surgery: validation on an abdominal phantom. In: Proceedings of the ACM symposium on virtual reality software and technology, ACM, pp 124–133Google Scholar
- 17.OpenCV (2015). http://opencv.org/
- 18.Ortiz J, Calderon H, Fontaine J (2010) Disparity map computation on scalable computing. In: 2010 IEEE/ASME international conference on mechatronics and embedded systems and applications (MESA), IEEE, pp 301–306Google Scholar
- 19.Parchami M, Cadeddu JA, Mariottini GL (2014) Endoscopic stereo reconstruction: a comparative study. In: 2014 36th annual international conference of the IEEE Engineering in Medicine and Biology Society (EMBC), IEEE, pp 2440–2443Google Scholar
- 20.Penza V, Ortiz J, De Momi E, Forgione A, Mattos L (2014) Virtual assistive system for robotic single incision laparoscopic surgery. In: 4th Joint workshop on computer/robot assisted surgery, pp 52–55Google Scholar
- 21.Quigley M, Conley K, Gerkey BP, Faust J, Foote T, Leibs J, Wheeler R, Ng AY (2009) Ros: an open-source robot operating system. In: ICRA workshop on open source softwareGoogle Scholar
- 24.Schoob A, Kundrat D, Kleingrothe L, Kahrs LA, Andreff N, Ortmaier T (2014) Tissue surface information for intraoperative incision planning and focus adjustment in laser surgery. Int J Comput Assist Radiol Surg 10(2):171–181Google Scholar
- 26.Stoyanov D, Scarzanella MV, Pratt P, Yang GZ (2010) Real-time stereo reconstruction in robotically assisted minimally invasive surgery. In: Medical image computing and computer-assisted intervention–MICCAI 2010. Springer, pp 275–282Google Scholar