Advertisement

Tissue Shape Acquisition with a Hybrid Structured Light and Photometric Stereo Endoscopic System

  • Marco Visentini-ScarzanellaEmail author
  • Tatsuya Hanayama
  • Ryunosuke Masutani
  • Shigeto Yoshida
  • Yoko Kominami
  • Yoji Sanomura
  • Shinji Tanaka
  • Ryo Furukawa
  • Hiroshi Kawasaki
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9515)

Abstract

In situ 3D reconstruction from endoscopic images is important to determine the correct course of action for, e.g., treatment of abnormal growths. Currently, the endoscopist has to rely solely on visual cues in order to infer the growth’s shape and size and determine an appropriate treatment. However, tissue uniformity and scale ambiguity from traditional monocular endoscopes make this visual assessment prone to errors and time consuming. We propose a practical system to densely reconstruct both shape and size of tissues with minimal modifications to a standard endoscope. We present a custom single-fiber structured light probe projecting a wave pattern on the tissue surface that allows semi-dense reconstruction with few ambiguities. Based on the coarse reconstruction, we retrieve the surface reflectance parameters according to a hybrid diffuse/specular model which are used to initialise a close-range Photometric Stereo reconstruction. By taking into account the tissue characteristics and the light fall-off, our Photometric Stereo formulation provides dense metric 3D shape information without the need for surface normal integration. A preliminary study was carried out both on phantoms and ex vivo samples of human tissue.

Keywords

Minimally Invasive Surgery Structure Light Human Tissue Sample Photometric Stereo Camera Perspective 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Notes

Acknowledgments

This work was supported by The Japanese Foundation for the Promotion of Science, Grant-in-Aid for JSPS Fellows no. 26.04041.

References

  1. 1.
    Chadebecq, F., Tilmant, C., Bartoli, A.: How big is this neoplasia? live colonoscopic size measurement using the infocus-breakpoint. Med. Image Anal. 19, 58–74 (2015)CrossRefGoogle Scholar
  2. 2.
    Maier-Hein, L., Mountney, P., Bartoli, A., Elhawary, H., Elson, D., Groch, A., Kolb, A., Rodrigues, M., Sorger, J., Speidel, S., Stoyanov, D.: Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery. Med. Image Anal. 17, 974–996 (2013)CrossRefGoogle Scholar
  3. 3.
    Maier-Hein, L., Groch, A., Bartoli, A., Bodenstedt, S., Boissonnat, G., Chang, P.L., Clancy, N., Elson, D., Haase, S., Heim, E., Hornegger, J., Jannin, P., Kenngott, H., Kilgus, T., Muller-Stich, B., Oladokun, D., Rohl, S., dos Santos, T., Schlemmer, H.P., Seitel, A., Speidel, S., Wagner, M., Stoyanov, D.: Comparative validation of single-shot optical techniques for laparoscopic 3-D surface reconstruction. IEEE Trans. Med. Imaging 33, 1913–1930 (2014)CrossRefGoogle Scholar
  4. 4.
    Furukawa, R., Aoyama, M., Hiura, S., Aoki, H., Kominami, Y., Sanomura, Y., Yoshida, S., Tanaka, S., Sagawa, R., Kawasaki, H.: Calibration of a 3D endoscopic system based on active stereo method for shape measurement of biological tissues and specimen. In: IEEE International Engineering in Medicine and Biology Conference (EMBC), pp. 4991–4994 (2014)Google Scholar
  5. 5.
    Schmalz, C., Forster, F., Schick, A., Angelopoulou, E.: An endoscopic 3D scanner based on structured light. Med. Image Anal. 16, 1063–1072 (2012)CrossRefGoogle Scholar
  6. 6.
    Clancy, N.T., Stoyanov, D., Maier-Hein, L., Groch, A., Yang, G.Z., Elson, D.: Spectrally encoded fiber-based structured lighting probe for intraoperative 3D imaging. Biomed. Opt. Express 2, 3119–3128 (2011)CrossRefGoogle Scholar
  7. 7.
    Visentini-Scarzanella, M., Mylonas, G.P., Stoyanov, D., Yang, G.-Z.: i-BRUSH: a gaze-contingent virtual paintbrush for dense 3D reconstruction in robotic assisted surgery. In: Yang, G.-Z., Hawkes, D., Rueckert, D., Noble, A., Taylor, C. (eds.) MICCAI 2009, Part I. LNCS, vol. 5761, pp. 353–360. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  8. 8.
    Wu, C., Narasimhan, S., Jaramaz, B.: A multi-image shape-from-shading framework for near-lighting perspective endoscopes. Int. J. Comput. Vis. 86, 211–228 (2010)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Malti, A., Bartoli, A.: Estimating the cook-torrance BRDF parameters in-vivo from laparoscopic images. In: Workshop on Augmented Environment in Medical Image Computing and Computer Assisted Intervention (MICCAI), Nice, France (2012)Google Scholar
  10. 10.
    Ciuti, G., Visentini-Scarzanella, M., Dore, A., Menciassi, A., Dario, P., Yang, G.Z.: Intra-operative monocular 3D reconstruction for image-guided navigation in active locomotion capsule endoscopy. In: IEEE RAS EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob), pp. 768–774 (2012)Google Scholar
  11. 11.
    Collins, T., Bartoli, A.: 3D reconstruction in laparoscopy with close-range photometric stereo. In: Ayache, N., Delingette, H., Golland, P., Mori, K. (eds.) MICCAI 2012, Part II. LNCS, vol. 7511, pp. 634–642. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  12. 12.
    Mecca, R., Wetzler, A., Bruckstein, A.M., Kimmel, R.: Near field photometric stereo with point light sources. SIAM J. Imaging Sci. 7, 2732–2770 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Sagawa, R., Sakashita, K., Kasuya, N., Kawasaki, H., Furukawa, R., Yagi, Y.: Grid-based active stereo with single-colored wave pattern for dense one-shot 3D scan. In: 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission (3DIMPVT), pp. 363–370 (2012)Google Scholar
  14. 14.
    Furukawa, R., Masutani, R., Miyazaki, D., Baba, M., Hiura, S., Visentini-Scarzanella, M., Morinaga, H., Kawasaki, H., Sagawa, R.: 2-DOF auto-calibration for a 3D endoscope system based on active stereo. In: IEEE International Engineering in Medicine and Biology Conference (EMBC) (2015)Google Scholar
  15. 15.
    Visentini-Scarzanella, M., Kawasaki, H.: Simultaneous camera, light position and radiant intensity distribution calibration. In: Pacific Rim Symposium on Image and Video Technology (PSIVT) (2015)Google Scholar
  16. 16.
    Kawasaki, H., Masuyama, H., Sagawa, R., Furukawa, R.: Single colour one-shot scan using modified penrose tiling pattern. IET Comput. Vis. 7, 293–301 (2013)CrossRefGoogle Scholar
  17. 17.
    Vogiatzis, G., Hernandez, C.: Self-calibrated, multi-spectral photometric stereo for 3D face capture. Int. J. Comput. Vis. 97, 91–103 (2012)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Marco Visentini-Scarzanella
    • 1
    Email author
  • Tatsuya Hanayama
    • 1
  • Ryunosuke Masutani
    • 2
  • Shigeto Yoshida
    • 3
  • Yoko Kominami
    • 3
  • Yoji Sanomura
    • 3
  • Shinji Tanaka
    • 3
  • Ryo Furukawa
    • 2
  • Hiroshi Kawasaki
    • 1
  1. 1.Department of Information Systems and Biomedical EngineeringKagoshima UniversityKagoshimaJapan
  2. 2.Department of Endoscopy and MedicineHiroshima UniversityHiroshimaJapan
  3. 3.Department of Intelligent SystemsHiroshima City UniversityHiroshimaJapan

Personalised recommendations