A Methodology to Reconstruct Large Damaged Regions in Heritage Structures



While it is important to digitize heritage sites “as is”, building 3D models of damaged archaeological structures can be visually unpleasant due to the presence of large missing regions. In this chapter, we discuss geometric reconstruction of such large damaged regions (or holes) in 3D digital models. Without constraining the size or complexity of the damaged region, the missing 3D geometry inference problem is solved by making use of geometric prior from self-similar structures which provide a salient cue about the missing surface characteristics that may be unique to an object class. The underlying surface is then recovered by adaptively propagating 3D surface smoothness from local geometric information around the boundary of the hole and appropriately using the cue provided by the available self-similar examples. We have used two methodologies to effectively harness the geometric prior: (i) a non-iterative framework based on tensor voting when multiple self-similar examples are available and (ii) a dictionary learning-based method when only a single self-similar example is available. We showcase the relevance of our method in the archaeological domain which warrants “filling-in” missing information in damaged heritage sites. We show several examples from Hampi which is a UNESCO heritage site located in Northern Karnataka in India.


  1. 1.
    Aharon M, Elad M, Bruckstein A (2006) K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans Signal Process 54:4311–4322CrossRefzbMATHGoogle Scholar
  2. 2.
    Andriy M, Xubo S (2010) Point set registration: Coherent point drift. IEEE Trans Pattern Anal Mach Intell 32:2262–2275.
  3. 3.
    Breckon TP, Fisher RB (2008) Three-dimensional surface relief completion via nonparametric techniques. IEEE Trans Pattern Anal Mach Intell 30:2249–2255CrossRefGoogle Scholar
  4. 4.
    Callieri M, Cignoni P, Ganovelli F, Impoco G, Montani C, Pingi P, Ponchio F, Scopigno R (2004) Visualization and 3D data processing in the David restoration. IEEE Comput Gr Appl 24:16–21CrossRefGoogle Scholar
  5. 5.
    Cignoni P, Corsini M, Ranzuglia G (2008) MeshLab: an Open-Source 3D Mesh Processing System. ERCIM News. 45–46Google Scholar
  6. 6.
    Davis J, Marschner S.R, Garr M, Levoy M (2002) Filling holes in complex surfaces using volumetric diffusion. 3D Data processing visualization and transmission. In: First International Symposium on. 428–438.
  7. 7.
    Geuzaine C, Remacle JF (2009) Gmsh: A 3-D finite element mesh generator with built-in pre- and post-processing facilities. Int J Numer Method Eng 79:1309–1331MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Gupta S, Castleman K.R, Markey M.K, Bovik A.C (2010) Texas 3D Face Recognition Database. In: Proc. 2010 SSIAI, TX2010, Austin, pp 97–100.
  9. 9.
    In: Google-Art-Project (2011)
  10. 10.
    In: INTACH (1984)
  11. 11.
  12. 12.
    In: Photosynth (2010)
  13. 13.
    In: p3d (2013)
  14. 14.
    Jia J, Tang CK (2004) Inference of segmented color and texture description by tensor voting. IEEE Trans Pattern Anal Mach Intell 26:771–786CrossRefGoogle Scholar
  15. 15.
    Julien, M, Francis, B, Jean, P, Guillermo, S (2009) Online dictionary learning for sparse coding. ICML. pp 689–696Google Scholar
  16. 16.
    Kulkarni M, Rajagopalan AN, Rigoll G (2012) Depth Inpainting with Tensor Voting Using Local Geometry. In Proceedings, VISAPPGoogle Scholar
  17. 17.
    Kulkarni M, Rajagopalan AN (2013) Depth inpainting by tensor voting. J Opt Soc Am A opt Image Sci 30:1155–1165CrossRefGoogle Scholar
  18. 18.
    Lai K, Bo L, Ren X, Fox D (2013) RGB-D Object Recognition: Features, Algorithms, and a Large Scale Benchmark. Consumer Depth Cameras for Computer Vision: Research Topics and Applications, pp 167–192Google Scholar
  19. 19.
    Levoy M, Pulli K, Curless B, Rusinkiewicz S, Koller D, Pereira L, Ginzton M, Anderson S, Davis J, Ginsberg J, Shade J, Fulk D (2000) The digital Michelangelo project: 3D scanning of large statues. ACM SIGGRAPH 2000:131–144Google Scholar
  20. 20.
    Liepa, P (2003) Filling holes in meshes. In: Proceedings of the 2003 Eurographics/ACM SIGGRAPH symposium on Geometry processing. pp 200–205Google Scholar
  21. 21.
    Mairal J, Bach F, Ponce J, Sapiro G (2010) Online learning for matrix factorization and sparse coding. J Mach Learn Res 11:19–60MathSciNetzbMATHGoogle Scholar
  22. 22.
    Padalkar MG, Joshi MV (2015) Auto-inpainting heritage scenes: a complete framework for detecting and infilling cracks in images and videos with quantitative assessment. Mach Vis Appl 26:317–337CrossRefGoogle Scholar
  23. 23.
    Patrick P, Michel G, Andrew B (2003) Poisson image editing. ACM Trans Graph 22:313–318CrossRefGoogle Scholar
  24. 24.
    Pauly M, Mitra NJ, Giesen J, Gross M, Guibas LJ (2005) Example-based 3D scan completion. In: Proceedings of the third Eurographics symposium on Geometry processingGoogle Scholar
  25. 25.
    Philippos M, Grard M (2006) Tensor Voting: A Perceptual Organization Approach to Computer Vision and Machine Learning. Synth Lect Image Video Multimed Process. 2:1–136Google Scholar
  26. 26.
    Scharstein D, Szeliski R (2002) A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms. Int J Comput Vision 47:7–42CrossRefzbMATHGoogle Scholar
  27. 27.
    Shailaja, T (2013) Monuments gone missing, In: The Hindu.
  28. 28.
    Silberman N, Hoiem D, Kohli P, Fergus R (2012) Indoor segmentation and support inference from RGBD images. ECCV. pp 746–760Google Scholar
  29. 29.
    Sturm, J, Engelhard, N, Endres, F, Burgard, W, Cremers, D (2012) A benchmark for the evaluation of RGB-D SLAM systems. In: Proceeding of the international conference on intelligent robot systems (IROS)Google Scholar
  30. 30.
    Szeliski R (2010) Computer Vision: Algorithms and Applications. Springer, New YorkGoogle Scholar
  31. 31.
    Tosic, I, Olshausen, B.A, Culpepper, B.J (2010) Learning sparse representations of depth. arXiv:1011.6656
  32. 32.
    UNESCO (2003) In: Draft charter on the preservation of the digital heritage. UNESCO General Conference 32nd sessionGoogle Scholar
  33. 33.
    Verdera, J, Caselles, V, Bertalmio, M, Sapiro, G (2003) Inpainting surface holes. In: Proceedings ICIP 2003, vol 2 II– pp 903–906Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2017

Authors and Affiliations

  1. 1.Indian Institute of Technology MadrasChennaiIndia

Personalised recommendations