Automatic Detection and Inpainting of Defaced Regions and Cracks in Heritage Monuments



Historical monuments are considered as one of the key aspects of our cultural heritage. Unfortunately, due to a variety of factors, the monuments get damaged. The need for preservation of cultural heritage has desiderated research on digitally repairing the photographs of damaged monuments. One may think of digitally undoing the damage to the monuments by inpainting, a process to fill-in missing regions in an image. For images of historic monuments, in particular, there is a consensus to fill-up the defaced regions and cracks so that one can view these in their undamaged form. Thus, we are not talking about image restoration, but about object completion by digitally repairing defaced regions/cracks that the physical objects have. In this chapter, we discuss techniques for automatically detecting the damaged facial regions and cracks in photographs of monuments. Unlike the usual practice of manually selecting the mask for inpainting, the regions to be inpainted are automatically selected and inpainting is done using the existing algorithm. Thus, the process of digital repair using inpainting is completely automated. We also extend our work on crack detection to perform auto-inpainting in videos by making use of scale invariant feature transform (SIFT) and homography. Finally, we provide a temporal consistency measure to quantify the quality of the inpainted video.



This work is a part of project sponsored by Department of Science and Technology (DST), Govt. of India (Grant No: NRDMS/11/1586/2009/Phase-II). The authors would like to thank the co-authors of [23] for their help in developing the contents of Sect. 2. The authors are also grateful to Prof. Toshiyuki Amano, Faculty of Systems Engineering, Wakayama University, for his valuable inputs and sharing the code of his work in [1].


  1. 1.
    Amano T (2006) Correlation based image defect detection. In: Proceedings of the 18th International Conference on Pattern Recognition, vol 01. IEEE Computer Society, Washington, DC, USA, ICPR ’06, pp 163–166Google Scholar
  2. 2.
    Bertalmio M, Sapiro G, Caselles V, Ballester C (2000) Image inpainting. In: Proceedings of the 27th annual conference computer graphics and interactive techniques. ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, pp 417–424Google Scholar
  3. 3.
    Chan T, Vese L (2001) Active contours without edges. IEEE Trans Image Process 10(2):266–277Google Scholar
  4. 4.
    Criminisi A, Pérez P, Toyama K (2004) Region filling and object removal by exemplar-based image inpainting. IEEE Trans Image Process 13:1200–1212CrossRefGoogle Scholar
  5. 5.
    Emerson C, Lam N, Quattrochi D (1999) Multi-scale fractal analysis of image texture and patterns. Photogramm Eng Remote Sens 65(1):51–62Google Scholar
  6. 6.
    Farias M, Mitra S (2005) No-reference video quality metric based on artifact measurements. In: Proceedings of the international conference on image processing, vol 3, pp III–141–4Google Scholar
  7. 7.
    Faugeras O, Lustman F (1988) Motion and structure from motion in a piecewise planar environment. Technical report RR-0856, INRIAGoogle Scholar
  8. 8.
    Fischler MA, Bolles RC (1981) Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM 24(6):381–395MathSciNetCrossRefGoogle Scholar
  9. 9.
    Google images (2012).
  10. 10.
    Guillemot C, Meur OL (2014) Image inpainting: overview and recent advances. IEEE Signal Process Mag 31(1):127–144CrossRefGoogle Scholar
  11. 11.
    Hartley R, Zisserman A (2003) Multiple view geometry in computer vision, 2nd edn. Cambridge University Press, New York, NY, USAzbMATHGoogle Scholar
  12. 12.
    Jobson DJ, Rahman Z, Woodell GA (1997) Properties and performance of a center/surround retinex. IEEE Trans Image Process 6(3):451–462CrossRefGoogle Scholar
  13. 13.
    Katahara S, Aoki M (1999) Face parts extraction windows based on bilateral symmetry of gradient direction. Computer analysis of images and patterns, vol 1689. Springer, Heidelberg, pp 834–834CrossRefGoogle Scholar
  14. 14.
    Kenney JF (1954) : Mathematicals of statistics. Van NostrandGoogle Scholar
  15. 15.
    Kovesi PD (2005) MATLAB and Octave: functions for computer vision and image processing. Centre for Exploration Targeting, School of Earth and Environment, The University of Western Australia.
  16. 16.
    Legrand P (2009) Local regularity and multifractal methods for image and signal analysis. In: Scaling, fractals and wavelets. WileyGoogle Scholar
  17. 17.
    Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110CrossRefGoogle Scholar
  18. 18.
    Lucas BD, Kanade T (1981) An iterative image registration technique with an application to stereo vision. In: Proceedings of the 7th international joint conference on Artificial intelligence, pp 674–679Google Scholar
  19. 19.
    Ma Y, Soatto S, Kosecka J, Sastry SS (2003) An invitation to 3-D vision: from images to geometric models. SpringerGoogle Scholar
  20. 20.
    Masnou S, Morel JM (1998) Level lines based disocclusion. Proceedings of the International Conference on Image Processing 3:259–263Google Scholar
  21. 21.
    Padalkar MG, Joshi MV (2015) Auto-inpainting heritage scenes: a complete framework for detecting and infilling cracks in images and videos with quantitative assessment. Mach Vis Appl 26(2):317–337CrossRefGoogle Scholar
  22. 22.
    Padalkar MG, Joshi MV, Khatri NL (2016) Digital heritage reconstruction using super-resolution and inpainting. In: Synthesis lectures on visual computing. Morgan & Claypool PublishersGoogle Scholar
  23. 23.
    Padalkar MG, Vora MV, Joshi MV, Zaveri MA, Raval MS (2013) Identifying vandalized regions in facial images of statues for inpainting. New trends in image analysis and processing-iciap 2013, vol 8158. Lecture notes in computer science. Springer, Heidelberg, pp 208–217CrossRefGoogle Scholar
  24. 24.
    Padalkar MG, Zaveri MA, Joshi MV (2013) SVD based automatic detection of target regions for image inpainting. In: Park JI, Kim J (eds) Computer vision–ACCV 2012 workshops, vol 7729. Lecture notes in computer science. Springer, Heidelberg, pp 61–71CrossRefGoogle Scholar
  25. 25.
    Patwardhan KA, Sapiro G, Bertalmío M (2007) Video inpainting under constrained camera motion. IEEE Trans Image Process 16(2):545–553MathSciNetCrossRefGoogle Scholar
  26. 26.
    Pérez P, Gangnet M, Blake A (2003) Poisson image editing. ACM Trans Gr 22(3):313–318Google Scholar
  27. 27.
    Perona P, Malik J (1990) Scale-space and edge detection using anisotropic diffusion. IEEE Trans Pattern Anal Mach Intell 12:629–639CrossRefGoogle Scholar
  28. 28.
    Rother C, Kolmogorov V, Blake A (2004) "grabcut": Interactive foreground extraction using iterated graph cuts. ACM Trans Gr 23(3):309–314CrossRefGoogle Scholar
  29. 29.
    Saad M, Bovik A, Charrier C (2014) Blind prediction of natural video quality. IEEE Trans Image Process 23(3):1352–1365MathSciNetCrossRefzbMATHGoogle Scholar
  30. 30.
    Saad MA, Bovik AC (2012) Blind quality assessment of videos using a model of natural scene statistics and motion coherency. In: Asilomar conference on signals, systems, and computers, pp 332–336Google Scholar
  31. 31.
    Scholes S (2011) Mcconkie ranch petroglyphs near vernal, utah. Accessed 01 Sept 2014
  32. 32.
    Štruc V, Pavešić N (2011) Photometric normalization techniques for illumination invariance. IGI-Global, pp 279–300Google Scholar
  33. 33.
    Tibshirani R, Walther G, Hastie T (2001) Estimating the number of clusters in a data set via the gap statistic. J Roy Stat Soc B 63(2):411–423MathSciNetCrossRefzbMATHGoogle Scholar
  34. 34.
    Turakhia N, Shah R, Joshi M (2012) Automatic crack detection in heritage site images for image inpainting. In: Eighth indian conference on computer vision, graphics and image processing (ICVGIP), p 68Google Scholar
  35. 35.
    Varma M, Zisserman A (2002) Classifying images of materials: achieving viewpoint and illumination independence. In: 7th European conference on computer vision (ECCV 2002), pp 255–271Google Scholar
  36. 36.
    Wagner RA, Fischer MJ (1974) The string-to-string correction problem. J ACM 21(1):168–173Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2017

Authors and Affiliations

  1. 1.Dhirubhai Ambani Institute of Information and Communication TechnologyGandhinagarIndia

Personalised recommendations