The Endoscopogram: A 3D Model Reconstructed from Endoscopic Video Frames

  • Qingyu Zhao
  • True Price
  • Stephen Pizer
  • Marc Niethammer
  • Ron Alterovitz
  • Julian Rosenman
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9900)

Abstract

Endoscopy enables high resolution visualization of tissue texture and is a critical step in many clinical workflows, including diagnosis and radiation therapy treatment planning for cancers in the nasopharynx. However, an endoscopic video does not provide explicit 3D spatial information, making it difficult to use in tumor localization, and it is inefficient to review. We introduce a pipeline for automatically reconstructing a textured 3D surface model, which we call an endoscopogram, from multiple 2D endoscopic video frames. Our pipeline first reconstructs a partial 3D surface model for each input individual 2D frame. In the next step (which is the focus of this paper), we generate a single high-quality 3D surface model using a groupwise registration approach that fuses multiple, partially overlapping, incomplete, and deformed surface models together. We generate endoscopograms from synthetic, phantom, and patient data and show that our registration approach can account for tissue deformations and reconstruction inconsistency across endoscopic video frames.

Notes

Acknowledgements

This work was supported by NIH grant R01 CA158925.

References

  1. 1.
    Hong, D., Tavanapong, W., Wong, J., Oh, J., de Groen, P.C.: 3D reconstruction of virtual colon structures from colonoscopy images. Comput. Med. Imaging Graph. 38(1), 22–23 (2014)CrossRefGoogle Scholar
  2. 2.
    Maier-Hein, L., Mountney, P., Bartoli, A., Elhawary, H., Elson, D., Groch, A., Kolb, A., Rodrigues, M., Sorger, J., Speidel, S., Stoyanov, D.: Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery. Med. Image Anal. 17(8), 974–996 (2013)CrossRefGoogle Scholar
  3. 3.
    Wu, C., Narasimhan, S.G., Jaramaz, B.: A multi-image shape-from-shading framework for near-lighting perspective endoscopes. Int. J. Comput. Vis. 86(2), 211–228 (2010)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Price, T., Zhao, Q., Rosenman, J., Pizer, S., Frahm, J.M.: Shape from motion and shading in uncontrolled environments. Under submission, To appear. http://midag.cs.unc.edu/
  5. 5.
    Durrleman, S., Prastawa, M., Korenberg, J.R., Joshi, S., Trouvé, A., Gerig, G.: Topology preserving atlas construction from shape data without correspondence using sparse parameters. In: Ayache, N., Delingette, H., Golland, P., Mori, K. (eds.) MICCAI 2012, Part III. LNCS, vol. 7512, pp. 223–230. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  6. 6.
    Durrleman, S., Pennec, X., Trouvé, A., Ayache, N.: Statistical models of sets of curves and surfaces based on currents. Med. Image Anal. 13(5), 793–808 (2009)CrossRefGoogle Scholar
  7. 7.
    Balci, S.K., Golland, P., Shenton, M., Wells, W.M.: Free-form B-spline deformation model for groupwise registration. In: MICCAI, pp. 23–30 (2007)Google Scholar
  8. 8.
    Arslan, S., Parisot, S., Rueckert, D.: Joint spectral decomposition for the parcellation of the human cerebral cortex using resting-state fMRI. In: Ourselin, S., Alexander, D.C., Westin, C.-F., Cardoso, M.J. (eds.) IPMI 2015. LNCS, vol. 9123, pp. 85–97. Springer, Heidelberg (2015)CrossRefGoogle Scholar
  9. 9.
    Zhao, Q., Price, J.T., Pizer, S., Niethammer, M., Alterovitz, R., Rosenman, J.: Surface registration in the presence of topology changes and missing patches. In: Medical Image Understanding and Analysis, pp. 8–13 (2015)Google Scholar
  10. 10.
    Zhao, Q., Pizer, S., Niethammer, M., Rosenman, J.: Geometric-feature-based spectral graph matching in pharyngeal surface registration. In: Golland, P., Hata, N., Barillot, C., Hornegger, J., Howe, R. (eds.) MICCAI 2014, Part I. LNCS, vol. 8673, pp. 259–266. Springer, Heidelberg (2014)Google Scholar
  11. 11.
    Curless, B., Levoy, M.: A volumetric method for building complex models from range images. In: SIGGRAPH, pp. 303–312 (1996)Google Scholar
  12. 12.
    Huber, D.F., Hebert, M.: Fully automatic registration of multiple 3D data sets. Image Vis. Comput. 21(7), 637–650 (2003)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Qingyu Zhao
    • 1
  • True Price
    • 1
  • Stephen Pizer
    • 1
    • 2
  • Marc Niethammer
    • 1
  • Ron Alterovitz
    • 1
  • Julian Rosenman
    • 1
    • 2
  1. 1.Computer ScienceUniversity of North Carolina at Chapel HillChapel HillUSA
  2. 2.Radiation OncologyUniversity of North Carolina at Chapel HillChapel HillUSA

Personalised recommendations