A Patch-Based Constrained Inpainting for Damaged Mural Images



Heritage artefacts and monuments are important components of social science. Those are under constant threat of decaying and degrading due to exposition to unfriendly natural environment and hooliganism. Restoration of heritage artefacts such as murals and paintings is an important task for preservation of social, cultural and political history of a nation. As being in the temples in India, a significant share of murals and paintings are not accessible for physical restoration. This motivates many researchers to put effort in restoration of such priceless paintings and reliefs digitally in augmented reality domain. In this work, we have proposed an exemplar based coherent texture synthesis technique to inpaint the digital image of damaged portion of murals and paintings. Inpainting method, while maintaining the spatial coherency, usually introduces blurring as well as structured noise to the inpainted regions. To overcome this problem, we have combined the proposed patch-based diffusion technique with a novel technique for high-frequency generation that leads to edge sharpening and denoising simultaneously. Finally, the proposed constraint and interactive nature of the method is found efficient to handle rich variety of such paintings. The experimental results with empirical evaluation show the efficacy of the proposed method.


Patch matching Mural Restoration Texture synthesis 



This work is partially supported by DST, GOI (Grant no. NRDMS/11/1586/09/Phase-I/Project No. 9) under the Indian Digital Heritage-Hampi Project. Authors are grateful to Dr. V. Chandru and Dr. M. Rao and their team for providing the images of mural paintings used in the experiment. We also gratefully acknowledge the computer vision group of the University of California, Berkeley for providing the images shown in the first column of Fig. 10.


  1. 1.
    Arora N, Kumar A, Kalra P (2012) Digital restoration of old paintings. In: International conference in central Europe on computer graphics, visualization and computer vision (WSCG)Google Scholar
  2. 2.
    Arya S, Mount DM, Netanyahu NS, Silverman R, Wu AY (1998) An optimal algorithm for approximate nearest neighbor searching fixed dimensions. J ACM 45(6):891–923.
  3. 3.
    Aswatha SM, Mukherjee J, Bhowmick P (2016) An integrated repainting system for digital restoration of vijayanagara murals. Int J Image Gr16(01):1650,005Google Scholar
  4. 4.
    Barnes C, Shechtman E, Finkelstein A, Goldman DB(2009) Patchmatch: a randomized correspondence algorithm for structural image editing. In: ACM Transactions on Graphics, SIGGRAPH. ACM, New York, NY, USA, pp 24.1–24.11.
  5. 5.
    Biemond J, Lagendijk R, Mersereau R (1990) Iterative methods for image deblurring. Proc IEEE 78(5):856–883.
  6. 6.
    Buades A, Coll B, Morel JM (2005) A non-local algorithm for image denoising. IEEE Conf Comput Vis Pattern Recogn (CVPR) 2:60–65zbMATHGoogle Scholar
  7. 7.
    Criminisi A, Perez P, Toyama K (2004) Region filling and object removal by exemplar-based image inpainting. IEEE Trans Image Process TIP) 13(9):1200–1212CrossRefGoogle Scholar
  8. 8.
    Darabi S, Shechtman E, Barnes C, Goldman DB, Sen P (2012) Image melding: combining inconsistent images using patch-based synthesis. ACM Trans Gr (TOG) 31(4):82:1–82:10Google Scholar
  9. 9.
    Daugman J (2004) How iris recognition works. IEEE Trans Circuits Syst Video Technol (TCSVT) 14(1):21–30CrossRefGoogle Scholar
  10. 10.
    Efros AA, Freeman WT (2001) Image quilting for texture synthesis and transfer. In: Proceedings of the 28th annual conference on Computer graphics and interactive techniques. ACM, New York, NY, USA, pp 341–346.
  11. 11.
    Efros AA, Leung T (1999) Texture synthesis by non-parametric sampling. IEEE Int Conf Comput Vis (ICCV) 2:1033–1038Google Scholar
  12. 12.
    Freedman G, Fattal R (2010) Image and video upscaling from local self-examples. ACM Trans Gr SIGGRAPH 28(3):1–10.
  13. 13.
    Ghorai M, Chanda B (2015) An image inpainting method using plsa-based search space estimation. Mach Vis Appl 26(1):69–87CrossRefGoogle Scholar
  14. 14.
    Giakoumis I, Nikolaidis N, Pitas I (2006) Digital image processing techniques for the detection and removal of cracks in digitized paintings. IEEE Trans Image Process (TIP) 15(1):178–188.
  15. 15.
    Giakoumis I, Pitas I (1998) Digital restoration of painting cracks. IEEE Int Symp Circuits Syst (ISCAS) 4:269–272.
  16. 16.
    Glasner D, Bagon S, Irani M (2009) Super-resolution from a single image. In: IEEE international conference on computer vision (ICCV), pp 349–356Google Scholar
  17. 17.
    Indyk P, Motwani R (1998) Approximate nearest neighbors: towards removing the curse of dimensionality. In: Proceedings of the thirtieth annual ACM symposium on theory of computing. ACM, New York, NY, USA, pp 604–613.
  18. 18.
    Karianakis N, Maragos P (2013) An integrated system for digital restoration of prehistoric theran wall paintings. In: 2013 18th International Conference on Digital Signal Processing (DSP). IEEE, pp 1–6Google Scholar
  19. 19.
  20. 20.
    Perona P, Malik J (1990) Scale-space and edge detection using anisotropic diffusion. IEEE Trans Pattern Anal Mach Intell (PAMI) 12(7):629–639.
  21. 21.
    Purkait P, Chanda B (2012) Digital restoration of damaged mural images. In: Proceedings of the eighth Indian conference on computer vision, graphics and image processing, ICVGIP ’12, pp 49:1–49:8Google Scholar
  22. 22.
    Roussos A, Maragos P (2010) Tensor-based image diffusions derived from generalizations of the total variation and beltrami functionals. In: IEEE international conference on image processing (ICIP), pp 4141–4144Google Scholar
  23. 23.
    Simakov D, Caspi Y, Shechtman E, Irani M (2008) Summarizing visual data using bidirectional similarity. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 1–8Google Scholar
  24. 24.
    Tschumperle D, Brun L (2009) Non-local image smoothing by applying anisotropic diffusion pde’s in the space of patches. In: IEEE international conference on image processing (ICIP), pp 2957–2960Google Scholar
  25. 25.
    Wang Q, Lu D, Zhang H (2011) Virtual completion of facial image in ancient murals. In: 2011 workshop on digital media and digital content management (DMDCM). IEEE, pp 203–209Google Scholar
  26. 26.
    yi Wei L, Levoy M (2000) Fast texture synthesis using tree-structured vector quantization, pp 479–488Google Scholar
  27. 27.
    Weickert J, Scharr H (2000) A scheme for coherence-enhancing diffusion filtering with optimized rotation invarianceGoogle Scholar
  28. 28.
    Welch G, Bishop G (1995) An introduction to the kalman filterGoogle Scholar
  29. 29.
    Wetzler A, Kimmel R (2012) Efficient beltrami flow in patch-space. In: Scale space and variational methods in computer vision, Lecture notes in computer science, vol 6667. Springer, Heidelberg, pp 134–143Google Scholar
  30. 30.
    Wexler Y, Shechtman E, Irani M (2004) Space-time video completion. In: IEEE conference on computer vision and pattern recognition (CVPR), vol. 1, pp I.120–I.127.
  31. 31.
    Xu Z, Sun J (2010) Image inpainting by patch propagation using patch sparsity. IEEE Trans Image Process (TIP) 19(5):1153–1165MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2017

Authors and Affiliations

  1. 1.University of BirminghamBirminghamUK
  2. 2.ECSU ISI KolkataKolkataIndia

Personalised recommendations