Skip to main content
SpringerLink
Log in

Fast algorithm for color texture image inpainting using the non-local CTV model

  • Published:
Journal of Global Optimization Aims and scope Submit manuscript

Abstract

The classical non-local Total Variation model has been extensively used for gray texture image inpainting previously, but such model can not be directly applied to color texture image inpainting due to coupling of different image channels in color images. In order to solve the inpainting problem for color texture images effectively, we propose a non-local Color Total Variation model. This model is different from the recently proposed non-local Mumford–Shah model (NL-MS). Technically, the proposed model is an extension of local TV model for gray images but we take account of the relationship between different channels in color images and make use of concepts of the non-local operators. We will analyze how the coupling of different channels of color images in the proposed model makes the problem difficult for numerical implementation with the conventional split Bregman algorithm. In order to solve the proposed model efficiently, we propose a fast heuristic numerical algorithm based on the split Bregman algorithm with introduction of a threshold function. The performance of the proposed model with the proposed heuristic algorithm is compared with the NL-MS model. Extensive numerical experiments have shown that the proposed model and algorithm have superior excellent performance as well as with much faster speed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Aubert, G., Kornprobst, P.: Mathematical problems in image processing: partial differential equations and the calculus of variations, 2nd edn. Springer, Berlin, Germany (2006)

    Google Scholar 

  2. Bertalmío, M., Sapiro, G., Caselles, V., Ballester, C.: Image inpainting. In: Proceedings of SIGGRAPH, pp. 417–424 (2000)

  3. Bertalmio, M., Bertozzi, A., Sapiro, G.: Navier-Stokes, fluid-dynamics, and image and video inpainting. In Proceedings of IEEE-CVPR, pp. 355–362 (2001)

  4. Chan, T., Shen, J.: Mathematical models of local non-texture inpaintings. SIAM J. Appl. Math. 62(3), 1019–1043 (2002)

    Article  MathSciNet  Google Scholar 

  5. Bertozzi, A., Esedoglu, S., Gillette, A.: Inpainting of binary images using the cahn-hilliard equation. IEEE Trans. Image Process. 16(1), 285–291 (2007)

    Article  MathSciNet  Google Scholar 

  6. Chan, T., Shen, J.: Non-texture inpainting by curvature-driven diffusions (CDD). J. Vis. Commun. Image Represent. 12(4), 436–449 (2001)

    Article  Google Scholar 

  7. Chan, T., Kang, S., Shen, J.: Euler’s elastica and curvature-based image inpainting. SIAM J. Appl. Math. 63(2), 564–592 (2002)

    MathSciNet  Google Scholar 

  8. Esedoglu, S., Shen, J.: Digital inpainting based on the Mumford–Shah–Euler image model. Eur. J. Appl. Math. 13(4), 353–370 (2002)

    Article  MathSciNet  Google Scholar 

  9. Tai, X., Hahn, J., Chung, G.: A fast algorithm for Euler’s elastica model using augmented Lagrangian method. SIAM J. Appl. Math. 4(1), 313–344 (2011)

    MathSciNet  Google Scholar 

  10. Rudin, L., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Phys. D 60(1/4), 259–268 (1992)

    Article  Google Scholar 

  11. Nitzberg, M., Mumford, D., Shiota, T.: Filering, segmentation, and depth, LNCS, vol. 662. Springer, Berlin, Germany (1993)

    Book  Google Scholar 

  12. Efros, A., Leung, T.: Texture synthesis by nonparametric sampling. In: Proceedings of the IEEE ICCV, pp. 1033–1038 (1999)

  13. Criminisi, A., Pérez, P., Toyama, K.: Region filling and object removal by exemplar-based inpainting. IEEE Trans. Image Process. 13(9), 1200–1212 (2004)

    Article  Google Scholar 

  14. Xu, Z., Jian, S.: Image inpainting by patch propagation using patch sparsity. IEEE Trans. Image Process. 19(3), 1153–1165 (2010)

    MathSciNet  Google Scholar 

  15. Arias, P., Caselles, V., Facciolo, G., Sapiro, G.: A variational framework for exemplar-based image inpainting. Int. J. Comput. Vis. 93(3), 1–29 (2011)

    Article  MathSciNet  Google Scholar 

  16. Arias, P., Caselles, V., Facciolo, G.: Analysis of a variational framework for exemplar-based image inpainting. SIAM Multiscale Model. Simul. 10(2), 473–514 (2012)

    Article  MathSciNet  Google Scholar 

  17. Bertalmio, M., Vese, L., Sapiro, G., Osher, S.: Simultaneous structure and texture image inpainting. IEEE Trans. Image Process. 12(8), 882–889 (2003)

    Article  Google Scholar 

  18. Cai, J., Chan, R., Shen, Z.: A framelet-based image inpainting algorithm. Appl. Comput. Harmon. Anal. 24(2), 131–149 (2008)

    Article  MathSciNet  Google Scholar 

  19. Cai, J., Chan, R., Shen, Z.: Simultaneous cartoon and texture inpainting. Inverse Probl. Imaging 4(3), 379–395 (2010)

    Article  MathSciNet  Google Scholar 

  20. Buades, A., Coll, B., Morel, J.: A review of image denoising algorithms, with a new one. SIAM Multiscale Model. Simul. 4(2), 490–530 (2005)

    Article  MathSciNet  Google Scholar 

  21. Gilboa, G., Osher, S.: Nonlocal operators with applications to image processing. SIAM Multiscale Model. Simul. 7(3), 1005–1028 (2008)

    Article  MathSciNet  Google Scholar 

  22. Peyré, G., Bougleux, S., Cohen, L.: Non-local regularization of inverse problems. LNCS 5304, 57–68 (2008)

    Google Scholar 

  23. Mumford, D., Shah, J.: Optimal approximations by piecewise smooth functions and associated variational problems. CPAM XLII, 577–685 (1989)

    MathSciNet  Google Scholar 

  24. Lou, Y., Zhang, X., Osher, S., Bertozzi, A.: Image recovery via non local operators. J. Sci. Comput. 42(2), 185–197 (2010)

    Article  MathSciNet  Google Scholar 

  25. Zhang, X., Burger, M., Bresson, X., Osher, S.: Bregmanized nonlocal regularization for deconvolution and sparse reconstruction. SIAM J. Imaging Sci. 3(3), 253–276 (2010)

    Article  MathSciNet  Google Scholar 

  26. Ma, W., Osher, S.: A TV Bregman iterative model of Retinex theory, University California, Los Angeles, UCLA CAM Report, pp. 10–13 (2010)

  27. Zosso, D., Tran, G., Osher, S.: A unifying retinex model based on non-local differential operators. University California, Los Angeles, UCLA CAM Report, pp. 13–03 (2013)

  28. Jung, M., Bresson, X., Chan, T.F., Vese, L.A.: Nonlocal Mumford–Shah regularizers for color image restoration. IEEE Trans. Image Process. 20(6), 1583–1598 (2011)

    Article  MathSciNet  Google Scholar 

  29. Yang, J., Yin, W., Zhang, Y., Wang, Y.: A fast algorithm for edge-preserving variational multichannel image restoration. SIAM J. Imaging Sci. 2(2), 569–592 (2009)

    Article  MathSciNet  Google Scholar 

  30. Ambrosio, L., Tortorelli, V.M.: Approximation of functionals depending on jumps by elliptic functionals via Gamma-convergence. Commun. Pure Appl. Math. 49, 999–1036 (1990)

    Article  MathSciNet  Google Scholar 

  31. Blomgren, P., Chan, T.: Color TV: total variation methods for restoration of vector-valued images. IEEE Trans. Image Process. 7(3), 304–309 (1998)

    Article  Google Scholar 

  32. Yu, Y., Pan, Z., Wei, W., Jiang, J.: Edge preserving of some variational models for vectorial image denoising. J. Gr. Images 16(12), 2223–2230 (2011)

    Google Scholar 

  33. Duan, J., Pan, Z., Tai, X.: Some non-local TV models for restoration of color texture images. J. Gr. Images 18(7), 753–760 (2013)

    Google Scholar 

  34. Goldstein, T., O’Donoghue, B., Setzer, S., Baraniuk, R.: Fast alternating direction optimization methods. SIAM J. Image Sci. 7(3), 1588–1623 (2014)

    Article  MathSciNet  Google Scholar 

  35. Goldfarb, D., Ma, S., Scheinberg, K.: Fast alternating linearization methods for minimizing the sum of two convex functions. Math. Program. 141, 349–382 (2013)

    Article  MathSciNet  Google Scholar 

  36. Goldstein, T., Osher, S.: The split Bregman method for L1-regularized problems. SIAM J. Imaging Sci. 2, 323–343 (2009)

    Article  MathSciNet  Google Scholar 

  37. Kampa, K., Metha, S., Chou, C.A., Chaovalitwongse, W.A., Grabowski, T.J.: Sparse optimization in feature selection: application in neuroimaging. J. Glob. Optim. 59, 439–457 (2014)

    Article  Google Scholar 

  38. Xu, R., Jiao, J., Zhang, B., Ye, Q.: Pedestrian detection in images via cascaded L1-norm minimization learning method. Pattern Recognit. 45, 2573–2583 (2012)

    Article  Google Scholar 

  39. Goldstein, T., Osher, S.: The split Bregman algorithm for L1 regularized problems. SIAM J. Imaging Sci. 2(2), 323–343 (2009)

    Article  MathSciNet  Google Scholar 

  40. Duan, J., Pan, Z., Liu, W., Tai, X.: Color texture image inpainting using the non local CTV model. J. Sign. Inf. Process. 4(3B), 43–51 (2013)

    Google Scholar 

  41. Shah, J.: A common framework for curve evolution, segmentation and anisotropic diffusion. Proceedings of IEEE Conference Computer Vision and Pattern Recognition, pp. 136–142 (1996)

  42. Duan, J., Pan, Z., He, X., Wei, W., Liu, C.: Mumford–Shah model on implicit surfaces. J. Softw. 23(suppl. 2), 53–63 (2012)

    Google Scholar 

  43. Bar, L., Brook, A., Sochen, N., Kiryati, N.: Deblurring of color images corrupted by impulsive noise. IEEE Trans. Image Process. 16(4), 1101–1111 (2007)

    Article  MathSciNet  Google Scholar 

  44. Bresson, X.: A short note for nonlocal TV minimization (2009), Online

  45. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  46. Jung, M., Peyre, G., Cohen, L.: Non-local segmentation and inpainting. IEEE International Conference on Image Processing, Brusslels: Belgium (2011)

Download references

Acknowledgments

We thank the three reviewers and the associate editor for their constructive comments, which helped us improve the quality of this paper significantly. We acknowledge the support of the Natural Science Foundation of China, under Contracts 61272052 and 61363066, and of the Program for New Century Excellent Talents of the University of Ministry of Education of China.

Author information

Authors and Affiliations

  1. School of Computer Science, University of Nottingham, Nottingham, UK

    Jinming Duan

  2. College of Information Engineering, Qingdao University, Qingdao, China

    Zhenkuan Pan

  3. Department of Computing, Curtin University, Perth, Australia

    Wanquan Liu

  4. Science and Technology on Aircraft Control Laboratory, Department of ASEE, Beihang University, Beijing, China

    Baochang Zhang

  5. Department of Mathematics, University of Bergen, Bergen, Norway

    Xue-Cheng Tai

Authors
  1. Jinming Duan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhenkuan Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Baochang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wanquan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xue-Cheng Tai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Baochang Zhang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Duan, J., Pan, Z., Zhang, B. et al. Fast algorithm for color texture image inpainting using the non-local CTV model. J Glob Optim 62, 853–876 (2015). https://doi.org/10.1007/s10898-015-0290-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10898-015-0290-7

Keywords

Search

Navigation