Multimedia Tools and Applications

, Volume 76, Issue 5, pp 7213–7234 | Cite as

Exemplar-based image inpainting using svd-based approximation matrix and multi-scale analysis



Reconstruction of images by digital inpainting is an active field of research and such algorithms are, in fact, now widely used. In conventional methods, a texture synthesis algorithm is used for filling the unknown regions of the image. However, due to the lack of global analysis on the image, the result may contain undesirable artifacts especially when we have images with relatively large missing regions. Here we propose a new inpainting technique to overcome this limitation by using an approximation matrix. The basic idea is to first make an approximation matrix using singular value decomposition, and then reconstruct the target region by using this matrix. Approximation matrix here, is in fact a gray-scale copy of the original image in which the target region is approximated throughout the process of rank lowering. Experiments are performed on a variety of input images ranging from purely synthetic images to full-color photographs. The results demonstrate the effectiveness of the proposed approach.


Image inpainting Image completion Object removal Singular value decomposition Image pyramids 


  1. 1.
    Adelson EH, Anderson CH, Bergen JR, Burt PJ, Ogden JM (1984) Pyramid methods in image processing. RCA engineer 29(6):33–41Google Scholar
  2. 2.
    Ahmed N, Natarajan T, Rao KR (1974) Discrete cosine transform. IEEE Trans Comput 100(1):90–93MathSciNetCrossRefMATHGoogle Scholar
  3. 3.
    Alilou VK, Yaghmaee F (2015) Application of grnn neural network in non-texture image inpainting and restoration. Pattern Recogn Lett 62:24–31CrossRefGoogle Scholar
  4. 4.
    Alexandru T (2004) An image inpainting technique based on the fast marching method. Journal of Graphics Tools 9(1):23–34CrossRefGoogle Scholar
  5. 5.
    Andrews H, Patterson C (1976) Singular value decompositions and digital image processing. IEEE Trans Acoustics, Speech and Signal Processing 24(1):26–53CrossRefGoogle Scholar
  6. 6.
    Ballester C, Bertalmio M, Caselles V, Sapiro G, Verdera J (2001) Filling-in by joint interpolation of vector fields and gray levels. IEEE Trans Image Process 10 (8):1200–1211MathSciNetCrossRefMATHGoogle Scholar
  7. 7.
    Bertalmio M, Bertozzi AL, Sapiro G (2001) Navier-stokes, fluid dynamics, and image and video inpainting. In: Proceedings of the 2001 IEEE computer society conference on Computer vision and pattern recognition, 2001. CVPR 2001, vol 1. IEEE, pp i–355Google Scholar
  8. 8.
    Bertalmio M, Sapiro G, Caselles V, Ballester C (2000) Image inpainting. In: Proceedings of the 27th annual conference on computer graphics and interactive techniques, pp 417–424. ACM press/addison-wesley publishing co.Google Scholar
  9. 9.
    Brigham EO (1988) The fast Fourier transform and its applications, vol 1. Prentice Hall, Englewood CliffsGoogle Scholar
  10. 10.
    Bugeau A, Bertalmío M, Caselles V, Sapiro G (2010) A comprehensive framework for image inpainting. IEEE Trans Image Process 19(10):2634–2645MathSciNetCrossRefGoogle Scholar
  11. 11.
    Casaca W, Boaventura M, De Almeida MP, Nonato LG (2014) Combining anisotropic diffusion, transport equation and texture synthesis for inpainting textured images. Pattern Recogn Lett 36:36– 45CrossRefGoogle Scholar
  12. 12.
    Chan T, Shen J (2001) Local inpainting models and tv inpainting. SIAM J Appl Math 62(3):1019– 1043MathSciNetGoogle Scholar
  13. 13.
    Chan TF, Kang SH (2006) Error analysis for image inpainting. Journal of Mathematical imaging and Vision 26(1–2):85–103MathSciNetCrossRefGoogle Scholar
  14. 14.
    Criminisi A, Pérez P, Toyama K (2004) Region filling and object removal by exemplar-based image inpainting. IEEE Trans Image Process 13(9):1200–1212CrossRefGoogle Scholar
  15. 15.
    Drori I, Cohen-Or D, Yeshurun H (2003) Fragment-based image completion. In: ACM Transactions on graphics (TOG), vol. 22, pp. 303–312. ACMGoogle Scholar
  16. 16.
    Efros AA, Leung TK (1999) Texture synthesis by non-parametric sampling. In: proceedings of the seventh IEEE international conference on Computer vision, 1999, vol 2, pp 1033–1038 . IEEEGoogle Scholar
  17. 17.
    Gepshtein S, Keller Y (2013) Image completion by diffusion maps and spectral relaxation. IEEE Trans Image Process 22(8):2983–2994MathSciNetCrossRefGoogle Scholar
  18. 18.
    Gerbrands JJ (1981) On the relationships between svd, klt and pca. Pattern Recog 14(1):375– 381MathSciNetCrossRefMATHGoogle Scholar
  19. 19.
    Ghorai M, Mandal S, Chanda B A two-step image inpainting algorithm using tensor svd. In: Computer Vision-ACCV 2014 Workshops, pp 63–77. Springer, p 2014Google Scholar
  20. 20.
    Golub GH, Van Loan CF (2012) Matrix computations, vol 3, JHU PressGoogle Scholar
  21. 21.
    Guleryuz OG (2005) On missing data prediction using sparse signal models: a comparison of atomic decompositions with iterated denoising. In: International society for optics and photonics. 2005, pp 59141g–59141gGoogle Scholar
  22. 22.
    Heeger DJ, Bergen JR (1995) Pyramid-based texture analysis/synthesis. In: Proceedings of the 22nd annual conference on computer graphics and interactive techniques, pp 229–238. ACMGoogle Scholar
  23. 23.
    Holtzman-Gazit M, Yavneh I (2006) Scale consistent image completion. In: Advances in visual computing, pp 648–659. SpringerGoogle Scholar
  24. 24.
    Huan X, Murali B, Ali AL (2010) Image restoration based on the fast marching method and block based sampling. Comput Vis Image Underst 114(8):847–856CrossRefGoogle Scholar
  25. 25.
    Huang H, Xiao N (2012) Color image inpainting based on multichannel-mca and k-svd. In: Communications and information processing, pp 666–674. SpringerGoogle Scholar
  26. 26.
    Jia J, Tang C-K (2003) Image repairing: Robust image synthesis by adaptive nd tensor voting. In: Proceedings of the 2003 IEEE computer society conference on Computer vision and pattern recognition, 2003, vol 1, pp i–643. IEEEGoogle Scholar
  27. 27.
    Jiying W, Ruan Q (2006) Object removal by cross isophotes exemplar-based inpainting. In: 18th international conference on Pattern recognition, 2006. ICPR 2006, vol 3, pp 810–813 . IEEEGoogle Scholar
  28. 28.
    Jolliffe I (2002) Principal component analysis, Wiley Online LibraryGoogle Scholar
  29. 29.
    Kamm JL (1998) Singular value decomposition-based methods for signal and image restorationGoogle Scholar
  30. 30.
    Kawai N, Sato T, Yokoya N (2009) Image inpainting considering brightness change and spatial locality of textures and its evaluation. In: Advances in image and video technology, pp 271–282. SpringerGoogle Scholar
  31. 31.
    Koh M-S, Rodriguez-Marek E (2009) Turbo inpainting: Iterative k-svd with a new dictionary. In: IEEE International Workshop on Multimedia Signal Processing, 2009. MMSP’09, pp 1–6. IEEEGoogle Scholar
  32. 32.
    Komodakis N, Tziritas G (2007) Image completion using efficient belief propagation via priority scheduling and dynamic pruning. IEEE Trans Image Process 16(11):2649–2661MathSciNetCrossRefGoogle Scholar
  33. 33.
    Lee J, Lee D-K, Park R-H (2012) Robust exemplar-based inpainting algorithm using region segmentation. IEEE Trans Consum Electron 58(2):553–561CrossRefGoogle Scholar
  34. 34.
    Levin A, Zomet A, Weiss Y (2003) Learning how to inpaint from global image statistics. In: Proceedings of the 9th IEEE international conference on Computer vision, 2003, pp 305–312. IEEEGoogle Scholar
  35. 35.
    Liu Y, Caselles V (2013) Exemplar-based image inpainting using multiscale graph cuts. IEEE Trans Image Process 22(5):1699–1711MathSciNetCrossRefGoogle Scholar
  36. 36.
    Minqin W (2011) A novel image inpainting method based on image decomposition. Procedia Engineering 15:3733–3738CrossRefGoogle Scholar
  37. 37.
    Moonen M, Van Dooren P, Vandewalle J (1992) A singular value decomposition updating algorithm for subspace tracking. SIAM Journal on Matrix Analysis and Applications 13(4):1015–1038MathSciNetCrossRefMATHGoogle Scholar
  38. 38.
    Padalkar MG, Zaveri MA, Joshi MV (2013) Svd based automatic detection of target regions for image inpainting. In: Computer vision-ACCV 2012 workshops, pp 61–71. SpringerGoogle Scholar
  39. 39.
    Papandreou G, Maragos P, Kokaram A Image inpainting with a wavelet domain hidden markov tree model. IEEEGoogle Scholar
  40. 40.
    Pereira T, Leme RP, Velho L, Lewiner T (2009) Symmetry-based completion. In: GRAPP, pp 39–45. CiteseerGoogle Scholar
  41. 41.
    Pritch Y, Kav-Venaki E, Peleg S (2009) Shift-map image editing. In: IEEE 12th international conference on Computer vision, 2009, pp 151–158. IEEEGoogle Scholar
  42. 42.
    Saad MA, Bovik AC, Charrier C (2011) Dct statistics model-based blind image quality assessment. In: 2011 18th IEEE International Conference on Image Processing (ICIP), pp 3093–3096. IEEEGoogle Scholar
  43. 43.
    Stollnitz EJ, DeRose TD (1996) Wavelets for computer graphics: theory and applications, Morgan KaufmannGoogle Scholar
  44. 44.
    Sun J, Yuan L, Jia J, Shum H-Y (2005) Image completion with structure propagation. In: ACM Transactions on graphics (tog), vol 24, pp 861–868. ACMGoogle Scholar
  45. 45.
    Wall ME, Rechtsteiner A, Rocha LM (2003) Singular value decomposition and principal component analysis. In: A practical approach to microarray data analysis, pp 91–109. SpringerGoogle Scholar
  46. 46.
    Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13 (4):600–612CrossRefGoogle Scholar
  47. 47.
    Wexler Y, Shechtman E, Irani M (2007) Space-time completion of video. IEEE Trans Pattern Analysis and Machine Intelligence 29(3):463–476CrossRefGoogle Scholar
  48. 48.
    Wong A, Orchard J (2008) A nonlocal-means approach to exemplar-based inpainting. In: 15th IEEE International Conference on Image Processing, 2008. ICIP 2008, pp 2600–2603 . IEEEGoogle Scholar
  49. 49.
    Yim C, Bovik AC (2011) Quality assessment of deblocked images. IEEE Trans Image Process 20(1):88–98MathSciNetCrossRefGoogle Scholar
  50. 50.
    Zheng C, Li G, Liu Y, Wang X (2012) Subspace weighted 2,1 minimization for sparse signal recovery. EURASIP Journal on Advances in Signal Processing 2012(1):1–11CrossRefGoogle Scholar
  51. 51.
    Zongben X, Sun J (2010) Image inpainting by patch propagation using patch sparsity. IEEE Trans Image Process 19(5):1153–1165MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  1. 1.Department of Computer EngineeringSemnan UniversitySemnanIran
  2. 2.Faculty of Computer EngineeringSemnan UniversitySemnanIran

Personalised recommendations