Biomedical Engineering Letters

, Volume 3, Issue 3, pp 149–157 | Cite as

3D surface reconstruction of stereo endoscopic images for minimally invasive surgery

  • Xishi Huang
  • Anwar Abdalbari
  • Jing RenEmail author
Original Article



Surface to surface registration of MR and endoscopic images is the key to MR and endoscopic image fusion, which will provide the surgeon with better 3D context of the surgical site in minimally invasive procedures. However, accurate reconstruction of 3D surface from stereo endoscopic images is still a challenging task especially for the surgical site with few features. In this paper, we propose a new method to reconstruct 3D surface from stereo endoscopic images.


We project a gridline light pattern onto the surgical site and then use a stereo endoscope to acquire two stereo images. The major steps in the surface reconstruction process include 1) applying an automatic method of detecting region of interest, 2) applying an image intensity correction algorithm, and 3) applying a novel automatic method to match the intersection points of the gridline pattern.


We have validated our proposed technique on a liver phantom and compared our method with an existing method of similar scope. Our experiment results show that our method outperforms the existing method in terms of correct matching rate (98% vs. 47%) which is an indicator of the surface reconstruction accuracy.


The proposed technique has the potential to be used in clinical practice to improve image guidance in endoscope based minimally invasive procedures. This technique may also be applied to the endoscopic procedures of other organs in the abdomen, chest cavity and pelvis such as kidneys and lungs.


Surface reconstruction Stereo endoscope Image guidance Gridline light pattern 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    Baumhauer M, Feuerstein M, Meinzer HP, Rassweiler J. Navigation in endoscopic soft tissue surgery: perspectives and limitations. J Endourol. 2008; 22(4): 751–766.CrossRefGoogle Scholar
  2. [2]
    Feuerstein M, Mussack T, Heining SM, Navab N. Intraoperative laparoscope augmentation for port placement and resection planning in minimally invasive liver resection. IEEE T Med Imaging. 2008; 27(3):355–369.CrossRefGoogle Scholar
  3. [3]
    Sonka M, Hlavac V, Boyle R. Image Processing Analysis and Machine Vision, 3rd Edition, Cengage Learning; 2008.Google Scholar
  4. [4]
    Prince JL, Links JM. Medical Imaging Signals and Systems, Prentice Hall; 2006.Google Scholar
  5. [5]
    Clements LW, Chapman WC, Dawant BM, Galloway RL Jr, Miga MI. Robust surface registration using salient anatomical features for image-guided liver surgery: Algorithm and validation. Med Phys. 2008; 35(6):2528–2540.CrossRefGoogle Scholar
  6. [6]
    Maier-Hein L, Schmidt M, Franz AM, dos Santos RT, Seitel A, Jähne B, Fitzpatrick JM, Meinzer HP. Accounting for anisotropic noise in fine registration of time-of-flight range data with high-resolution surface data. Med Image Comput Comput Assist Interv. 2010; 13(Pt 1):251–258.Google Scholar
  7. [7]
    Groch A, Seitel A, Hempel S, Speidel S, Engelbrecht R, Penne J, Höller K, Röhl S, Yung K, Bodenstedt S, Pflaum F, dos Santos TR, Mersmann S, Meinzer HP, Hornegger J, Maier-Hein L 3D surface reconstruction for laparo-scopic computer-assisted interventions: comparison of state-of-the-art methods. Medical Imaging 2011: Visualization, Image-Guided Procedures, and Modeling, Proc SPIE 7964, 2011; 796415.Google Scholar
  8. [8]
    Hayashibe M, Suzuki N, Nakamura Y. Laser-scan endoscope system for intraoperative geometry acquisition and surgical robot safety management. Med Image Anal. 2006; 10(4):509–519.CrossRefGoogle Scholar
  9. [9]
    Proesmans M, van Gool LJ, Oosterlinck AJ. Active acquisition of 3D shape for moving objects. Int Conf Image Process. 1996; 3:647–650.Google Scholar
  10. [10]
    Hu G, Stockman G. 3-D surface solution using structured light and constraint propagation. IEEE T Pattern Anal. 1989; 11(4):390–402.CrossRefGoogle Scholar
  11. [11]
    Kawasaki H, Furukawa R, Sagawa R, Yagi Y. Dynamic scene shape reconstruction using a single structured light pattern. IEEE Conf Comput Vis Pattern Recogn (CVPR). 2008. 1–8.Google Scholar
  12. [12]
    Mountney P, Stoyanov D, Yang GZ. Three-dimensional tissue deformation recovery and tracking. IEEE Signal Proc Mag. 2010; 27(4):14–24.CrossRefGoogle Scholar
  13. [13]
    Lowe DG. Distinctive image features from scale-invariant keypoints. Int J Comput Vision. 2004; 60(2):91–110.CrossRefGoogle Scholar
  14. [14]
    Bouguet JY. Camera Calibration Toolbox for Matlab. accessed date: May 20, 2010.
  15. [15]
    Bevilacqua V, Cambò S, Cariello L, Mastronardi G. A combined method to detect retinal fundus features. Eur Conf Emergent Aspects Clin Data Anal. 2005. 1-6.Google Scholar
  16. [16]
    Dey D, Gobbi DG, Slomka PJ, Surry KJ, Peters TM. Automatic fusion of freehand endoscopic brain images to threedimensional surfaces: creating stereoscopic panoramas. IEEE T Med Imaging. 2002; 21(1):23–30.CrossRefGoogle Scholar

Copyright information

© Korean Society of Medical and Biological Engineering and Springer 2013

Authors and Affiliations

  1. 1.Department of Medical ImagingUniversity of TorontoTorontoCanada
  2. 2.Faculty of Engineering and Applied ScienceUniv. of Ontario Institute of TechnologyOshawaCanada

Personalised recommendations