Advertisement

3D Transparent Visualization of Relief-Type Cultural Heritage Assets Based on Depth Reconstruction of Old Monocular Photos

  • Jiao PanEmail author
  • Liang Li
  • Hiroshi Yamaguchi
  • Kyoko Hasegawa
  • Fadjar I. Thufail
  • Bramantara
  • Satoshi Tanaka
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 1094)

Abstract

We propose an efficient method to achieve 3D visualization directly from a single monocular 2D image for relief-type cultural heritages. To achieve a proper depth feel of 3D visualization, we first reconstruct the 3D point clouds by estimating the depth from the monocular image using a depth estimation network. We then apply our stochastic point-based rendering mechanism to achieve a 3D transparent visualization of the reconstructed point clouds. Herein, we apply our method to the Buddhist temple heritage of Borobudur Temple, in Indonesia, a UNESCO World Heritage Site with the most complete collection of Buddhist reliefs. The proposed method achieved 90% accuracy of the reconstructed point cloud on average and a promising visualization result with an intuitive understanding.

Keywords

Digital archives Visualization Neural networks 

References

  1. 1.
    Besl, P., McKay, N.D.: A method for registration of 3-D shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14(2), 239–256 (1992).  https://doi.org/10.1109/34.121791. http://ieeexplore.ieee.org/document/121791/CrossRefGoogle Scholar
  2. 2.
    Eigen, D., Puhrsch, C., Fergus, R.: Depth map prediction from a single image using a multi-scale deep network (2014). https://papers.nips.cc/paper/5539-depth-map-prediction-from-a-single-image-using-a-multi-scale-deep-network
  3. 3.
    Gregor, R., Sipiran, I., Papaioannou, G., Schreck, T., Andreadis, A., Mavridis, P.: Towards automated 3D reconstruction of defective cultural heritage objects. In: Eurographics Workshop on Graphics and Cultural Heritage, pp. 135–144 (2014).  https://doi.org/10.2312/gch.20141311, http://diglib.eg.org/EG/DL/WS/GCH/GCH2014/135-144.pdf
  4. 4.
    Hermoza, R., Sipiran, I.: 3D Reconstruction of Incomplete Archaeological Objects Using a Generative Adversarial Network (2017). http://arxiv.org/abs/1711.06363
  5. 5.
    Ioannides, M., et al.: Online 4D reconstruction using multi-images available under open access. ISPRS Ann. Photogram. Remote Sens. Spat. Inf. Sci. 2(5/W1), 169–174 (2013). https://doi.org/10.5194/isprsannals-II-5-W1-169-2013CrossRefGoogle Scholar
  6. 6.
    Kersten, T.P., Lindstaedt, M.: Automatic 3D object reconstruction from multiple images for architectural, cultural heritage and archaeological applications using open-source software and web services automatische 3D-objektrekonstruktion aus digitalen Bilddaten für Anwendungen in Archit. Photogrammetrie - Fernerkundung - Geoinformation 2012(6), 727–740 (2013).  https://doi.org/10.1127/1432-8364/2012/0152CrossRefGoogle Scholar
  7. 7.
    Kyriakaki, G., et al.: 4D reconstruction of tangible cultural heritage objects from web-retrieved images. Int. J. Heritage Digit. Era 3(2), 431–451 (2014).  https://doi.org/10.1260/2047-4970.3.2.431. http://journals.sagepub.com/doi/10.1260/2047-4970.3.2.431CrossRefGoogle Scholar
  8. 8.
    Ladicky, L., Shi, J., Pollefeys, M.: Pulling things out of perspective. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 89–96. IEEE (2014).  https://doi.org/10.1109/CVPR.2014.19, http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=6909413
  9. 9.
    Lu, M., Zheng, B., Takamatsu, J., Nishino, K., Ikeuchi, K.: Preserving the Khmer smile: classifying and restoring the faces of Bayon. In: Proceedings of the 12th International Conference on Virtual Reality, Archaeology and Cultural Heritage, pp. 161–168 (2011).  https://doi.org/10.2312/vast/vast11/161-168, https://dl.acm.org/citation.cfm?id=2384521
  10. 10.
    Pan, J., Li, L., Yamaguchi, H., Hasegawa, K., Thufail, F.I., Tanaka, S.: 3D Reconstruction and Transparent Visualization of Indonesian Cultural Heritage from a Single Image, pp. 3–6 (2018).  https://doi.org/10.2312/gch.20181363
  11. 11.
    Park, J.H., Muhammad, T., Jae-Hong, A.: The 3D reconstruction and visualization of Seokguram Grotto World Heritage Site. In: Proceedings of the 2014 International Conference on Virtual Systems and Multimedia, VSMM 2014, pp. 180–183 (2014).  https://doi.org/10.1109/VSMM.2014.7136646
  12. 12.
    Saxena, A., Chung, S.H., Ng, A.Y.: Learning Depth from Single Monocular Images (2006). https://papers.nips.cc/paper/2921-learning-depth-from-single-monocular-images
  13. 13.
    Tanaka, S., et al.: SEE-through imaging of laser-scanned 3D cultural heritage objects based on stochastic rendering of large-scale point clouds. ISPRS Ann. Photogram. Remote Sens. Spat. Inf. Sci. 3(July), 73–80 (2016).  https://doi.org/10.5194/isprs-annals-III-5-73-2016CrossRefGoogle Scholar
  14. 14.
    Tanaka, S., et al.: Application of stochastic point-based rendering to transparent visualization of large-scale laser-scanned data of 3D cultural assets. In: 2014 IEEE Pacific Visualization Symposium, pp. 267–271. IEEE (2014).  https://doi.org/10.1109/PacificVis.2014.25, http://ieeexplore.ieee.org/document/6787179/

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  • Jiao Pan
    • 1
    Email author
  • Liang Li
    • 2
  • Hiroshi Yamaguchi
    • 3
  • Kyoko Hasegawa
    • 2
  • Fadjar I. Thufail
    • 4
  • Bramantara
    • 5
  • Satoshi Tanaka
    • 2
  1. 1.Graduate School of Information Science and EngineeringRitsumeikan UniversityKyotoJapan
  2. 2.College of Information Science and EngineeringRitsumeikan UniversityKyotoJapan
  3. 3.Nara National Research Institute for Cultural PropertiesNaraJapan
  4. 4.Research Center for Area Studies (P2W)Indonesian Institute of Sciences (LIPI)JakartaIndonesia
  5. 5.Borobudur Conservation OfficeMagelangIndonesia

Personalised recommendations