Skip to main content

Advertisement

SpringerLink
Log in

Dense soft tissue 3D reconstruction refined with super-pixel segmentation for robotic abdominal surgery

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

Single-incision laparoscopic surgery decreases postoperative infections, but introduces limitations in the surgeon’s maneuverability and in the surgical field of view. This work aims at enhancing intra-operative surgical visualization by exploiting the 3D information about the surgical site. An interactive guidance system is proposed wherein the pose of preoperative tissue models is updated online. A critical process involves the intra-operative acquisition of tissue surfaces. It can be achieved using stereoscopic imaging and 3D reconstruction techniques. This work contributes to this process by proposing new methods for improved dense 3D reconstruction of soft tissues, which allows a more accurate deformation identification and facilitates the registration process.

Methods

Two methods for soft tissue 3D reconstruction are proposed: Method 1 follows the traditional approach of the block matching algorithm. Method 2 performs a nonparametric modified census transform to be more robust to illumination variation. The simple linear iterative clustering (SLIC) super-pixel algorithm is exploited for disparity refinement by filling holes in the disparity images.

Results

The methods were validated using two video datasets from the Hamlyn Centre, achieving an accuracy of 2.95 and 1.66 mm, respectively. A comparison with ground-truth data demonstrated the disparity refinement procedure: (1) increases the number of reconstructed points by up to 43 % and (2) does not affect the accuracy of the 3D reconstructions significantly.

Conclusion

Both methods give results that compare favorably with the state-of-the-art methods. The computational time constraints their applicability in real time, but can be greatly improved by using a GPU implementation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Susstrunk S (2012) Slic superpixels compared to state-of-the-art superpixel methods. IEEE Trans Pattern Anal Mach Intell 34(11):2274–2282

    Article  PubMed  Google Scholar 

  2. Banks J, Bennamoun M, Corke P (1997) Non-parametric techniques for fast and robust stereo matching. In: TENCON’97. IEEE region 10 annual conference. Speech and image technologies for computing and telecommunications. Proceedings of IEEE, IEEE, vol 1, pp 365–368

  3. Barbalata C, Mattos L (2014) Laryngeal tumor detection and classification in endoscopic video. IEEE J Biomed Health Inform. doi:10.1109/JBHI.2014.2374975

  4. Bergen T, Wittenberg T (2014) Stitching and surface reconstruction from endoscopic image sequences: a review of applications and methods. IEEE J Biomed Health Inform. doi:10.1109/JBHI.2014.2384134

  5. Chum O, Matas J, Kittler J (2003) Locally optimized ransac. In: Pattern recognition, vol 2781. Springer, Heidelberg, pp 236–243. doi:10.1007/978-3-540-45243-0_31

  6. Ee WWG, Lau WLJ, Yeo W, Von Bing Y, Yue WM (2014) Does minimally invasive surgery have a lower risk of surgical site infections compared with open spinal surgery? Clin Orthop Relat Res 472(6):1718–1724

    PubMed Central  PubMed  Google Scholar 

  7. Giannarou S, Visentini-Scarzanella M, Yang GZ (2013) Probabilistic tracking of affine-invariant anisotropic regions. IEEE Trans Pattern Anal Mach Intell 35(1):130–143. doi:10.1109/TPAMI.2012.81

    Article  PubMed  Google Scholar 

  8. Guthart G, Salisbury Jr JK (2000) The intuitive TM telesurgery system: overview and application. In: ICRA, pp 618–621

  9. Hartley R, Zisserman A (2003) Multiple view geometry in computer vision. Cambridge University Press, Cambridge

    Google Scholar 

  10. Kanade T, Okutomi M (1994) A stereo matching algorithm with an adaptive window: theory and experiment. IEEE Trans Pattern Anal Mach Intell 16(9):920–932

    Article  Google Scholar 

  11. Klaus A, Sormann M, Karner K (2006) Segment-based stereo matching using belief propagation and a self-adapting dissimilarity measure. In: 18th International conference on pattern recognition, 2006. ICPR 2006, IEEE, vol 3, pp 15–18

  12. Ma L, Li J, Ma J, Zhang H (2013) A modified census transform based on the neighborhood information for stereo matching algorithm. In: 2013 Seventh international conference on image and graphics (ICIG), IEEE, pp 533–538

  13. Mountney P, Stoyanov D, Yang GZ (2010) Three-dimensional tissue deformation recovery and tracking. IEEE Signal Process Mag 27(4):14–24

    Article  Google Scholar 

  14. Musialski P, Wonka P, Aliaga DG, Wimmer M, Gool L, Purgathofer W (2013) A survey of urban reconstruction. In: Computer graphics forum, Wiley Online Library, vol 32, pp 146–177

  15. Nicolau S, Goffin L, Soler L (2005) A low cost and accurate guidance system for laparoscopic surgery: validation on an abdominal phantom. In: Proceedings of the ACM symposium on virtual reality software and technology, ACM, pp 124–133

  16. Nicolau S, Soler L, Mutter D, Marescaux J (2011) Augmented reality in laparoscopic surgical oncology. Surg Oncol 20(3):189–201

    Article  PubMed  Google Scholar 

  17. OpenCV (2015). http://opencv.org/

  18. Ortiz J, Calderon H, Fontaine J (2010) Disparity map computation on scalable computing. In: 2010 IEEE/ASME international conference on mechatronics and embedded systems and applications (MESA), IEEE, pp 301–306

  19. Parchami M, Cadeddu JA, Mariottini GL (2014) Endoscopic stereo reconstruction: a comparative study. In: 2014 36th annual international conference of the IEEE Engineering in Medicine and Biology Society (EMBC), IEEE, pp 2440–2443

  20. Penza V, Ortiz J, De Momi E, Forgione A, Mattos L (2014) Virtual assistive system for robotic single incision laparoscopic surgery. In: 4th Joint workshop on computer/robot assisted surgery, pp 52–55

  21. Quigley M, Conley K, Gerkey BP, Faust J, Foote T, Leibs J, Wheeler R, Ng AY (2009) Ros: an open-source robot operating system. In: ICRA workshop on open source software

  22. Röhl S, Bodenstedt S, Suwelack S, Kenngott H, Müller-Stich BP, Dillmann R, Speidel S (2012) Dense gpu-enhanced surface reconstruction from stereo endoscopic images for intraoperative registration. Med Phys 39(3):1632–1645

    Article  PubMed  Google Scholar 

  23. Scharstein D, Szeliski R (2002) A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int J Comput Vis 47(1–3):7–42

    Article  Google Scholar 

  24. Schoob A, Kundrat D, Kleingrothe L, Kahrs LA, Andreff N, Ortmaier T (2014) Tissue surface information for intraoperative incision planning and focus adjustment in laser surgery. Int J Comput Assist Radiol Surg 10(2):171–181

  25. Stoyanov D (2012) Surgical vision. Ann Biomed Eng 40(2):332–345

    Article  PubMed  Google Scholar 

  26. Stoyanov D, Scarzanella MV, Pratt P, Yang GZ (2010) Real-time stereo reconstruction in robotically assisted minimally invasive surgery. In: Medical image computing and computer-assisted intervention–MICCAI 2010. Springer, pp 275–282

  27. Tan HJ, Wolf JS, Ye Z, Hafez KS, Miller DC (2014) Population level assessment of hospital based outcomes following laparoscopic versus open partial nephrectomy during the adoption of minimally invasive surgery. J Urol 191(5):1231–1237

    Article  PubMed  Google Scholar 

  28. Taylor G, Barrie J, Hood A, Culmer P, Neville A, Jayne D (2013) Surgical innovations: addressing the technology gaps in minimally invasive surgery. Trends Anaesth Crit Care 3(2):56–61

    Article  Google Scholar 

  29. Tian Q, Huhns MN (1986) Algorithms for subpixel registration. Comput Vis Graph Image Process 35(2):220–233

    Article  Google Scholar 

  30. Varela JE, Wilson SE, Nguyen NT (2010) Laparoscopic surgery significantly reduces surgical-site infections compared with open surgery. Surg Endosc 24(2):270–276

    Article  PubMed  Google Scholar 

  31. Wichmann MW, Hüttl TP, Winter H, Spelsberg F, Angele MK, Heiss MM, Jauch KW (2005) Immunological effects of laparoscopic vs open colorectal surgery: a prospective clinical study. Arch Surg 140(7):692–697

    Article  PubMed  Google Scholar 

  32. Zitnick CL, Kang SB (2007) Stereo for image-based rendering using image over-segmentation. Int J Comput Vis 75(1):49–65

    Article  Google Scholar 

  33. Zitova B, Flusser J (2003) Image registration methods: a survey. Image Vis Comput 21(11):977–1000

    Article  Google Scholar 

Download references

Acknowledgments

Authors would like to thank Nikhil Deshpande for his kind suggestions and support during the elaboration of this paper.

Author information

Authors and Affiliations

  1. Department of Advanced Robotics, Istituto Italiano di Tecnologia, via Morego, 30, 16163, Genoa, Italy

    Veronica Penza, Jesús Ortiz & Leonardo S. Mattos

  2. Ospedale Niguarda Ca’ Granda, P.zza Dell’Ospedale Maggiore, 3, 20162, Milan, Italy

    Antonello Forgione

  3. AIMS Academy, P.zza Dell’Ospedale Maggiore, 3, 20162, Milan, Italy

    Antonello Forgione

  4. Valuebiotech s.r.l., P.zza Dell’Ospedale Maggiore, 3, 20162, Milan, Italy

    Antonello Forgione

  5. Department of Electronics, Information and Bioengineering, Politecnico di Milano, P.zza L. Da Vinci, 32, 20133, Milan, Italy

    Veronica Penza & Elena De Momi

Authors
  1. Veronica Penza
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jesús Ortiz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Leonardo S. Mattos
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Antonello Forgione
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Elena De Momi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Veronica Penza.

Ethics declarations

Conflict of interest

All authors declare that they have no conflict of interest.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Penza, V., Ortiz, J., Mattos, L.S. et al. Dense soft tissue 3D reconstruction refined with super-pixel segmentation for robotic abdominal surgery. Int J CARS 11, 197–206 (2016). https://doi.org/10.1007/s11548-015-1276-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-015-1276-0

Keywords

Search

Navigation