Skip to main content
SpringerLink
Log in

Comparative study on surface reconstruction accuracy of stereo imaging devices for microsurgery

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

Processing stereoscopic image data is an emerging field. Especially in microsurgery that requires sub-millimeter accuracy, application of stereo-based methods on endoscopic and microscopic scenarios is of major interest. In this context, direct comparison of stereo-based surface reconstruction applied to several camera settings is presented.

Methods

A method for stereo matching is proposed and validated on in-vitro data. Demonstrating suitability for surgical scenarios, this method is applied to two custom-made stereo cameras, a miniaturized, bendable surgical endoscope and an operating microscope. Reconstruction accuracy is assessed on a custom-made reference sample. Subsequent to its fabrication, a coordinate measuring arm is used to acquire ground truth. Next, the sample is positioned by a robot at varying distances to each camera. Surface estimation is performed, while the specimen is localized based on. markers. Finally, the error between estimated surface and ground truth is computed.

Results

Sample measurement with the coordinate measuring arm yields reliable ground truth data with a root-mean-square error of \(11.2\,\upmu \hbox {m}\). Overall surface reconstruction with analyzed cameras is quantified by a root-mean-square error of less than 0.18 mm. Microscope setting with the highest magnification yields the most accurate measurement, while the maximum deviation does not exceed 0.5 mm. Custom-made stereo cameras perform similar but with outliers of increased magnitude. Miniaturized, bendable surgical endoscope produces the maximum error of approximately \(1.2\,\hbox {mm}\).

Conclusions

Reconstruction results reveal that microscopic imaging outperforms investigated chip-on-the-tip solutions, i.e., at higher magnification. Nonetheless, custom-made cameras are suitable for application in microsurgery. Although reconstruction with the miniaturized endoscope is more inaccurate, it provides a good trade-off between accuracy, outer dimensions and accessibility to hard-to-reach surgical sites.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Barkhoudarian G, Romero ADCB, Laws ER (2013) Evaluation of the 3-dimensional endoscope in transsphenoidal surgery. Neurosurgery 73:74–79. doi:10.1227/NEU.0b013e31828ba962

    Article  Google Scholar 

  2. Bergen T, Wittenberg T (2014) Stitching and surface reconstruction from endoscopic image sequences: a review of applications and methods. IEEE J Biomed Health Inform. doi:10.1109/JBHI.2014.2384134

    PubMed  Google Scholar 

  3. Bergmeier J, Kundrat D, Schoob A, Kahrs LA, Ortmaier T (2015) Methods for a fusion of optical coherence tomography and stereo camera image data. Proc SPIE 9415: doi:10.1117/12.2082511

  4. Bilgic B, Horn B, Masaki I (2010) Efficient integral image computation on the GPU. IEEE intelligent vehicles symposium (IV) pp 528–533. doi:10.1109/IVS.2010.5548142

  5. Bouguet JY (2004) The camera calibration toolbox for matlab. http://www.vision.caltech.edu/bouguetj/

  6. Bradski G (2000) The opencv library. Dobb’s J Softw Tools 25(11):120–126

    Google Scholar 

  7. Brown MB, Forsythe AB (1974) Robust tests for the equality of variances. J Am Stat Assoc 69(346):364–367

    Article  Google Scholar 

  8. Catmull EE (1974) A subdivision algorithm for computer display of curved surfaces. Ph.D thesis, The University of Utah, AAI7504786

  9. Cignoni P, Callieri M, Corsini M, Dellepiane M, Ganovelli F, Ranzuglia G (2008) Meshlab: an open-source mesh processing tool. Eurographics Italian chapter conference, pp 129–136

  10. Crause LA, O’Donoghue DE, O’Connor JE, Strümpfer F (2010) Use of a faro arm for optical alignment. Proc SPIE 7739: doi:10.1117/12.856810

  11. Danuser G (1999) Photogrammetric calibration of a stereo light microscope. J Microsc 193(1):62–83. doi:10.1046/j.1365-2818.1999.00425.x

    Article  PubMed  Google Scholar 

  12. Eisemann E, Durand F (2004) Flash photography enhancement via intrinsic relighting. ACM Trans Graph 23(3):673–678. doi:10.1145/1015706.1015778

    Article  Google Scholar 

  13. Harris M, Sengupta S, Owens JD (2007) Parallel prefix sum (scan) with cuda. GPU Gems 3(39):851–876

    Google Scholar 

  14. Hartley RI, Sturm P (1997) Triangulation. Comput Vision Image Underst 68(2):146–157

    Article  Google Scholar 

  15. Hartley RI, Zisserman A (2004) Multiple view geometry in computer vision, 2nd edn. Cambridge University Press, New York, NY, USA. ISBN 0521540518

  16. Hirschmuller H, Scharstein D (2009) Evaluation of stereo matching costs on images with radiometric differences. IEEE Trans Pattern Anal Mach Intell 31(9):1582–1599. doi:10.1109/TPAMI.2008.221

    Article  PubMed  Google Scholar 

  17. Illingworth J, Kittler J (1988) A survey of the hough transform. Comput Vis Graph Image Process 44(1):87–116. doi:10.1016/S0734-189X(88)80033-1

    Article  Google Scholar 

  18. Kalentev O, Rai A, Kemnitz S, Schneider R (2011) Connected component labeling on a 2D grid using CUDA. J. Parallel Distrib Comput 71(4):615–620. doi:10.1016/j.jpdc.2010.10.012

    Article  Google Scholar 

  19. Martin KLS (2015) MCO 25plus and MCO 50plus–product brochure. KLS Martin Group, Tuttlingen, Germany. http://www.klsmartin.com/

  20. Kobler JP, Diaz Diaz J, Fitzpatrick JM, Lexow GJ, Majdani O, Ortmaier T (2014) Localization accuracy of sphere fiducials in computed tomography images. Proc SPIE 9036: doi:10.1117/12.2043472

  21. Kundrat D, Schoob A, Kahrs LA, Ortmaier T (2015) Flexible robot for laser phonomicrosurgery. In: Soft Robotics, pp 265–271. doi:10.1007/978-3-662-44506-8_22

  22. Maier-Hein L, Groch A, Bartoli A, Bodenstedt S, Boissonnat G, Chang PL, Clancy N, Elson D, Haase S, Heim E, Hornegger J, Jannin P, Kenngott H, Kilgus T, Muller-Stich B, Oladokun D, Rohl S, dos Santos T, Schlemmer HP, Seitel A, Speidel S, Wagner M, Stoyanov D (2014) Comparative validation of single-shot optical techniques for laparoscopic 3-d surface reconstruction. IEEE Trans Med Imaging 33(10):1913–1930. doi:10.1109/TMI.2014.2325607

    Article  CAS  PubMed  Google Scholar 

  23. Maier-Hein L, Mountney P, Bartoli A, Elhawary H, Elson D, Groch A, Kolb A, Rodrigues M, Sorger J, Speidel S, Stoyanov D (2013) Optical techniques for 3d surface reconstruction in computer-assisted laparoscopic surgery. Med Image Anal 17(8):974–996. doi:10.1016/j.media.2013.04.003

    Article  CAS  PubMed  Google Scholar 

  24. Marcus HJ, Hughes-Hallett A, Cundy TP, Di Marco A, Pratt P, Nandi D, Darzi A, Yang GZ (2014) Comparative effectiveness of 3-dimensional vs 2-dimensional and high-definition vs standard-definition neuroendoscopy: a preclinical randomized crossover study. Neurosurgery 74(4):375–381

    Article  PubMed Central  PubMed  Google Scholar 

  25. Mattos LS, Deshpande N, Barresi G, Guastini L, Peretti G (2014) A novel computerized surgeon-machine interface for robot-assisted laser phonomicrosurgery. Laryngoscope 124(8):1887–1894. doi:10.1002/lary.24566

    Article  PubMed  Google Scholar 

  26. McGill R, Tukey JW, Larsen WA (1978) Variations of box plots. Am Stat 32(1):12–16

    Google Scholar 

  27. Mountney P, Stoyanov D, Yang GZ (2010) Three-dimensional tissue deformation recovery and tracking. Signal Process Mag IEEE 27(4):14–24. doi:10.1109/MSP.2010.936728

    Article  Google Scholar 

  28. Pantilie C, Nedevschi S (2012) Optimizing the census transform on CUDA enabled GPUs. IEEE international conference on intelligent computer communication and processing (ICCP) pp 201–207. doi:10.1109/ICCP.2012.6356186

  29. Patel S, Rajadhyaksha M, Kirov S, Li Y, Toledo-Crow R (2012) Endoscopic laser scalpel for head and neck cancer surgery. Proc SPIE 8207: doi:10.1117/12.909172

  30. Petschnigg G, Szeliski R, Agrawala M, Cohen M, Hoppe H, Toyama K (2004) Digital photography with flash and no-flash image pairs. ACM Trans Graph 23(3):664–672. doi:10.1145/1015706.1015777

    Article  Google Scholar 

  31. Röhl S, Bodenstedt S, Suwelack S, Kenngott H, Muller-Stich BP, Dillmann R, Speidel S (2012) Dense GPU-enhanced surface reconstruction from stereo endoscopic images for intraoperative registration. Med Phys 39:1632. doi:10.1118/1.3681017

    Article  PubMed  Google Scholar 

  32. Rubinstein M, Armstrong W (2011) Transoral laser microsurgery for laryngeal cancer: a primer and review of laser dosimetry. Lasers Med Sci 26(1):113–124. doi:10.1007/s10103-010-0834-5

    Article  PubMed Central  PubMed  Google Scholar 

  33. Scharstein D, Szeliski R (2002) A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int J Comput Vis 47(1–3):7–42. doi:10.1023/A:1014573219977

  34. Schoob A, Kundrat D, Kleingrothe L, Kahrs L, Andreff N, Ortmaier T (2015) Tissue surface information for intraoperative incision planning and focus adjustment in laser surgery. Int J Comput Assist Radiol Surg 10(2):171–181. doi:10.1007/s11548-014-1077-x

  35. Schoob A, Podszus F, Kundrat D, Kahrs L, Ortmaier T (2013) Stereoscopic surface reconstruction in minimally invasive surgery using efficient non-parametric image transforms. Proc 3rd joint workshop on new technologies for computer/robot assisted surgery (CRAS), pp 26–29

  36. Schreier H, Garcia D, Sutton M (2004) Advances in light microscope stereo vision. Exp Mech 44(3):278–288. doi:10.1007/BF02427894

    Article  Google Scholar 

  37. Steiner W, Ambrosch P (2000) Endoscopic laser surgery of the upper aerodigestive tract: with special emphasis on cancer surgery, 1st edn. Thieme, Stuttgart, Germany. ISBN 9783131252715

  38. Stoyanov D, Scarzanella M, Pratt P, Yang GZ (2010) Real-time stereo reconstruction in robotically assisted minimally invasive surgery. Proc of medical image computing and computer-assisted intervention (MICCAI) 6361:275–282. doi:10.1007/978-3-642-15705-9_34

  39. Tamadazte B, Andreff N (2014) Weakly calibrated stereoscopic visual servoing for laser steering: application to phonomicrosurgery. IEEE/RSJ international conference on intelligent robots and systems (IROS), pp 743–748. doi:10.1109/IROS.2014.6942641

  40. Tang HW, Brussel HV, Sloten JV, Reynaerts D, De Win G, Cleynenbreugel BV, Koninckx PR (2006) Evaluation of an intuitive writing interface in robot-aided laser laparoscopic surgery. Comput Aided Surg 11(1):21–30. doi:10.3109/10929080500450886

    Article  PubMed  Google Scholar 

  41. Zhang Z (2000) A flexible new technique for camera calibration. IEEE Trans Pattern Anal Mach Intell 22(11):1330–1334. doi:10.1109/34.888718

    Article  Google Scholar 

  42. Zinner C, Humenberger M, Ambrosch K, Kubinger W (2008) An optimized software-based implementation of a census-based stereo matching algorithm. Adv Vis Comput 5358:216–227. doi:10.1007/978-3-540-89639-5_21

Download references

Acknowledgments

The research leading to the presented results has received funding from the European Union Seventh Framework Programme FP7/2007–2013 Challenge 2 Cognitive Systems, Interaction, Robotics under Grant agreement \(\upmu \)RALP-No. 288663. We furthermore thank Prof. Giorgio Peretti from the Department of Otorhinolaryngology, University of Genoa, Italy providing in vivo laryngeal image data.

Conflict of interest

All authors declare that they have no conflict of interest.

Author information

Authors and Affiliations

  1. Institute of Mechatronic Systems, Leibniz Universität Hannover, 30167, Hanover, Germany

    Andreas Schoob, Dennis Kundrat, Lüder A. Kahrs & Tobias Ortmaier

Authors
  1. Andreas Schoob
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dennis Kundrat
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lüder A. Kahrs
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tobias Ortmaier
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Andreas Schoob.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Schoob, A., Kundrat, D., Kahrs, L.A. et al. Comparative study on surface reconstruction accuracy of stereo imaging devices for microsurgery. Int J CARS 11, 145–156 (2016). https://doi.org/10.1007/s11548-015-1240-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-015-1240-z

Keywords

Search

Navigation