Skip to main content
Log in

Exemplar-based image inpainting using svd-based approximation matrix and multi-scale analysis

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Reconstruction of images by digital inpainting is an active field of research and such algorithms are, in fact, now widely used. In conventional methods, a texture synthesis algorithm is used for filling the unknown regions of the image. However, due to the lack of global analysis on the image, the result may contain undesirable artifacts especially when we have images with relatively large missing regions. Here we propose a new inpainting technique to overcome this limitation by using an approximation matrix. The basic idea is to first make an approximation matrix using singular value decomposition, and then reconstruct the target region by using this matrix. Approximation matrix here, is in fact a gray-scale copy of the original image in which the target region is approximated throughout the process of rank lowering. Experiments are performed on a variety of input images ranging from purely synthetic images to full-color photographs. The results demonstrate the effectiveness of the proposed approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

References

  1. Adelson EH, Anderson CH, Bergen JR, Burt PJ, Ogden JM (1984) Pyramid methods in image processing. RCA engineer 29(6):33–41

    Google Scholar 

  2. Ahmed N, Natarajan T, Rao KR (1974) Discrete cosine transform. IEEE Trans Comput 100(1):90–93

    Article  MathSciNet  MATH  Google Scholar 

  3. Alilou VK, Yaghmaee F (2015) Application of grnn neural network in non-texture image inpainting and restoration. Pattern Recogn Lett 62:24–31

    Article  Google Scholar 

  4. Alexandru T (2004) An image inpainting technique based on the fast marching method. Journal of Graphics Tools 9(1):23–34

    Article  Google Scholar 

  5. Andrews H, Patterson C (1976) Singular value decompositions and digital image processing. IEEE Trans Acoustics, Speech and Signal Processing 24(1):26–53

    Article  Google Scholar 

  6. Ballester C, Bertalmio M, Caselles V, Sapiro G, Verdera J (2001) Filling-in by joint interpolation of vector fields and gray levels. IEEE Trans Image Process 10 (8):1200–1211

    Article  MathSciNet  MATH  Google Scholar 

  7. Bertalmio M, Bertozzi AL, Sapiro G (2001) Navier-stokes, fluid dynamics, and image and video inpainting. In: Proceedings of the 2001 IEEE computer society conference on Computer vision and pattern recognition, 2001. CVPR 2001, vol 1. IEEE, pp i–355

  8. Bertalmio M, Sapiro G, Caselles V, Ballester C (2000) Image inpainting. In: Proceedings of the 27th annual conference on computer graphics and interactive techniques, pp 417–424. ACM press/addison-wesley publishing co.

  9. Brigham EO (1988) The fast Fourier transform and its applications, vol 1. Prentice Hall, Englewood Cliffs

    Google Scholar 

  10. Bugeau A, Bertalmío M, Caselles V, Sapiro G (2010) A comprehensive framework for image inpainting. IEEE Trans Image Process 19(10):2634–2645

    Article  MathSciNet  Google Scholar 

  11. Casaca W, Boaventura M, De Almeida MP, Nonato LG (2014) Combining anisotropic diffusion, transport equation and texture synthesis for inpainting textured images. Pattern Recogn Lett 36:36– 45

    Article  Google Scholar 

  12. Chan T, Shen J (2001) Local inpainting models and tv inpainting. SIAM J Appl Math 62(3):1019– 1043

    MathSciNet  Google Scholar 

  13. Chan TF, Kang SH (2006) Error analysis for image inpainting. Journal of Mathematical imaging and Vision 26(1–2):85–103

    Article  MathSciNet  Google Scholar 

  14. Criminisi A, Pérez P, Toyama K (2004) Region filling and object removal by exemplar-based image inpainting. IEEE Trans Image Process 13(9):1200–1212

    Article  Google Scholar 

  15. Drori I, Cohen-Or D, Yeshurun H (2003) Fragment-based image completion. In: ACM Transactions on graphics (TOG), vol. 22, pp. 303–312. ACM

  16. Efros AA, Leung TK (1999) Texture synthesis by non-parametric sampling. In: proceedings of the seventh IEEE international conference on Computer vision, 1999, vol 2, pp 1033–1038 . IEEE

  17. Gepshtein S, Keller Y (2013) Image completion by diffusion maps and spectral relaxation. IEEE Trans Image Process 22(8):2983–2994

    Article  MathSciNet  Google Scholar 

  18. Gerbrands JJ (1981) On the relationships between svd, klt and pca. Pattern Recog 14(1):375– 381

    Article  MathSciNet  MATH  Google Scholar 

  19. Ghorai M, Mandal S, Chanda B A two-step image inpainting algorithm using tensor svd. In: Computer Vision-ACCV 2014 Workshops, pp 63–77. Springer, p 2014

  20. Golub GH, Van Loan CF (2012) Matrix computations, vol 3, JHU Press

  21. Guleryuz OG (2005) On missing data prediction using sparse signal models: a comparison of atomic decompositions with iterated denoising. In: International society for optics and photonics. 2005, pp 59141g–59141g

  22. Heeger DJ, Bergen JR (1995) Pyramid-based texture analysis/synthesis. In: Proceedings of the 22nd annual conference on computer graphics and interactive techniques, pp 229–238. ACM

  23. Holtzman-Gazit M, Yavneh I (2006) Scale consistent image completion. In: Advances in visual computing, pp 648–659. Springer

  24. Huan X, Murali B, Ali AL (2010) Image restoration based on the fast marching method and block based sampling. Comput Vis Image Underst 114(8):847–856

    Article  Google Scholar 

  25. Huang H, Xiao N (2012) Color image inpainting based on multichannel-mca and k-svd. In: Communications and information processing, pp 666–674. Springer

  26. Jia J, Tang C-K (2003) Image repairing: Robust image synthesis by adaptive nd tensor voting. In: Proceedings of the 2003 IEEE computer society conference on Computer vision and pattern recognition, 2003, vol 1, pp i–643. IEEE

  27. Jiying W, Ruan Q (2006) Object removal by cross isophotes exemplar-based inpainting. In: 18th international conference on Pattern recognition, 2006. ICPR 2006, vol 3, pp 810–813 . IEEE

  28. Jolliffe I (2002) Principal component analysis, Wiley Online Library

  29. Kamm JL (1998) Singular value decomposition-based methods for signal and image restoration

  30. Kawai N, Sato T, Yokoya N (2009) Image inpainting considering brightness change and spatial locality of textures and its evaluation. In: Advances in image and video technology, pp 271–282. Springer

  31. Koh M-S, Rodriguez-Marek E (2009) Turbo inpainting: Iterative k-svd with a new dictionary. In: IEEE International Workshop on Multimedia Signal Processing, 2009. MMSP’09, pp 1–6. IEEE

  32. Komodakis N, Tziritas G (2007) Image completion using efficient belief propagation via priority scheduling and dynamic pruning. IEEE Trans Image Process 16(11):2649–2661

    Article  MathSciNet  Google Scholar 

  33. Lee J, Lee D-K, Park R-H (2012) Robust exemplar-based inpainting algorithm using region segmentation. IEEE Trans Consum Electron 58(2):553–561

    Article  Google Scholar 

  34. Levin A, Zomet A, Weiss Y (2003) Learning how to inpaint from global image statistics. In: Proceedings of the 9th IEEE international conference on Computer vision, 2003, pp 305–312. IEEE

  35. Liu Y, Caselles V (2013) Exemplar-based image inpainting using multiscale graph cuts. IEEE Trans Image Process 22(5):1699–1711

    Article  MathSciNet  Google Scholar 

  36. Minqin W (2011) A novel image inpainting method based on image decomposition. Procedia Engineering 15:3733–3738

    Article  Google Scholar 

  37. Moonen M, Van Dooren P, Vandewalle J (1992) A singular value decomposition updating algorithm for subspace tracking. SIAM Journal on Matrix Analysis and Applications 13(4):1015–1038

    Article  MathSciNet  MATH  Google Scholar 

  38. Padalkar MG, Zaveri MA, Joshi MV (2013) Svd based automatic detection of target regions for image inpainting. In: Computer vision-ACCV 2012 workshops, pp 61–71. Springer

  39. Papandreou G, Maragos P, Kokaram A Image inpainting with a wavelet domain hidden markov tree model. IEEE

  40. Pereira T, Leme RP, Velho L, Lewiner T (2009) Symmetry-based completion. In: GRAPP, pp 39–45. Citeseer

  41. Pritch Y, Kav-Venaki E, Peleg S (2009) Shift-map image editing. In: IEEE 12th international conference on Computer vision, 2009, pp 151–158. IEEE

  42. Saad MA, Bovik AC, Charrier C (2011) Dct statistics model-based blind image quality assessment. In: 2011 18th IEEE International Conference on Image Processing (ICIP), pp 3093–3096. IEEE

  43. Stollnitz EJ, DeRose TD (1996) Wavelets for computer graphics: theory and applications, Morgan Kaufmann

  44. Sun J, Yuan L, Jia J, Shum H-Y (2005) Image completion with structure propagation. In: ACM Transactions on graphics (tog), vol 24, pp 861–868. ACM

  45. Wall ME, Rechtsteiner A, Rocha LM (2003) Singular value decomposition and principal component analysis. In: A practical approach to microarray data analysis, pp 91–109. Springer

  46. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13 (4):600–612

    Article  Google Scholar 

  47. Wexler Y, Shechtman E, Irani M (2007) Space-time completion of video. IEEE Trans Pattern Analysis and Machine Intelligence 29(3):463–476

    Article  Google Scholar 

  48. Wong A, Orchard J (2008) A nonlocal-means approach to exemplar-based inpainting. In: 15th IEEE International Conference on Image Processing, 2008. ICIP 2008, pp 2600–2603 . IEEE

  49. Yim C, Bovik AC (2011) Quality assessment of deblocked images. IEEE Trans Image Process 20(1):88–98

    Article  MathSciNet  Google Scholar 

  50. Zheng C, Li G, Liu Y, Wang X (2012) Subspace weighted 2,1 minimization for sparse signal recovery. EURASIP Journal on Advances in Signal Processing 2012(1):1–11

    Article  Google Scholar 

  51. Zongben X, Sun J (2010) Image inpainting by patch propagation using patch sparsity. IEEE Trans Image Process 19(5):1153–1165

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Farzin Yaghmaee.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

K. Alilou, V., Yaghmaee, F. Exemplar-based image inpainting using svd-based approximation matrix and multi-scale analysis. Multimed Tools Appl 76, 7213–7234 (2017). https://doi.org/10.1007/s11042-016-3366-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-016-3366-6

Keywords

Navigation