Skip to main content
SpringerLink
Log in

A vector-valued PDE-constrained image inpainting model

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

In image inpainting, the identification and inpainting of local detail features and the preservation of global features are crucial. Models based on fractional-order partial differential equations exhibit rich evolutionary behaviors. These behaviors enable them to effectively comprehend image details. Additionally, these models possess a certain sharpening effect in image inpainting. However, they are also prone to issues such as inaccurate identification of large-scale features and over-sharpening. The optimal control model proposed in this paper uses the total variation energy of image global features as the objective function. It also employs the spatial fractional-order vector-valued Cahn–Hilliard equation as the constraint, aiming to achieve a balanced effect between local detail restoration and preservation of global features. The paper aims to optimize the objective function by designing numerical computation schemes for non-convex constraint conditions using \(L_{2}\) gradient flow, \(H^{-1}\) gradient flow, and convex splitting. The Split Bregman method is used for further optimization, and a dynamic grayscale adjustment strategy is introduced to maintain grayscale discrimination capability while enhancing computational efficiency. Numerical experiments demonstrate that the new model exhibits certain advantages over other inpainting methods in terms of PSNR values. It also shows strong competitiveness in terms of SSIM values, especially in severely damaged images where it demonstrates greater stability. Compared to traditional fractional-order equation models, the proposed model captures global features and incorporates the dynamic grayscale adjustment strategy, resulting in significantly reduced computation time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data availability

The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request

References

  1. Jam, J., Kendrick, C., Walker, K., Drouard, V., Hsu, J.G.-S., Yap, M.H.: A comprehensive review of past and present image inpainting methods. Comput. Vis. Image Underst. 203, 103147 (2021)

    Article  Google Scholar 

  2. Halim, A., Kumar, B.R.: An anisotropic pde model for image inpainting. Comput. Math. Appl. 79(9), 2701–2721 (2020)

    Article  MathSciNet  Google Scholar 

  3. Bosch, J., Stoll, M.: A fractional inpainting model based on the vector-valued Cahn–Hilliard equation. SIAM J. Imag. Sci. 8(4), 2352–2382 (2015)

    Article  MathSciNet  Google Scholar 

  4. Newson, A., Almansa, A., Gousseau, Y., Pérez, P.: Non-local patch-based image inpainting. Imag. Process. Line 7, 373–385 (2017)

    Article  MathSciNet  Google Scholar 

  5. Shen, J., Chan, T.F.: Mathematical models for local nontexture inpaintings. SIAM J. Appl. Math. 62(3), 1019–1043 (2002)

    Article  MathSciNet  Google Scholar 

  6. Chan, T.F., Shen, J.: Nontexture inpainting by curvature-driven diffusions. J. Vis. Commun. Image Represent. 12(4), 436–449 (2001)

    Article  Google Scholar 

  7. Burger, M., He, L., Schönlieb, C.-B.: Cahn–Hilliard inpainting and a generalization for gray value images. SIAM J. Imag. Sci. 2(4), 1129–1167 (2009)

    Article  Google Scholar 

  8. Bertozzi, A., Lu, S., Gillette, A.: Analysis of a two-scale Cahn–Hilliard model for binary image inpainting. Multiscale Modeling Simul. 6(3), 913–936 (2007)

    Article  MathSciNet  Google Scholar 

  9. Bertozzi, A.L., Esedoglu, S., Gillette, A.: Inpainting of binary images using the Cahn–Hilliard equation. IEEE Trans. Image Process. 16(1), 285–291 (2006)

    Article  MathSciNet  Google Scholar 

  10. Zou, Q.: An image inpainting model based on the mixture of Perona–Malik equation and Cahn–Hilliard equation. J. Appl. Math. Comput. 66(1–2), 21–38 (2021)

    Article  MathSciNet  Google Scholar 

  11. Lian, X., Fu, Q., Su, W., Zhang, X., Li, J., Yao, Z.-A.: The fractional laplacian based image inpainting. Inverse Probl. Imaging 18(2), 326–365 (2023)

    Article  MathSciNet  Google Scholar 

  12. Zhang, M., Zhang, G.-F.: Fast image inpainting strategy based on the space-fractional modified Cahn–Hilliard equations. Comput. Math. Appl. 102, 1–14 (2021)

    Article  MathSciNet  Google Scholar 

  13. Carrillo, J.A., Kalliadasis, S., Liang, F., Perez, S.P.: Enhancement of damaged-image prediction through Cahn–Hilliard image inpainting. Royal Soc. Open Sci. 8(5), 201294 (2021)

    Article  Google Scholar 

  14. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial networks. Commun. ACM 63(11), 139–144 (2020)

    Article  MathSciNet  Google Scholar 

  15. Zhao, L., Mo, Q., Lin, S., Wang, Z., Zuo, Z., Chen, H., Xing, W., Lu, D.: Uctgan: Diverse image inpainting based on unsupervised cross-space translation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 5741–5750 (2020)

  16. Elharrouss, O., Almaadeed, N., Al-Maadeed, S., Akbari, Y.: Image inpainting: A review. Neural Process. Lett. 51, 2007–2028 (2020)

    Article  Google Scholar 

  17. Dong, B., Jiang, Q., Shen, Z.: Image restoration: Wavelet frame shrinkage, nonlinear evolution pdes, and beyond. Multiscale Modeling Simul. 15(1), 606–660 (2017)

  18. Bai, X., Sun, J., Shen, J., Yao, W., Guo, Z.: A ginzburg-landau-h-1 model and its sav algorithm for image inpainting. J. Sci. Comput. 96(2), 40 (2023)

    Article  MathSciNet  Google Scholar 

  19. Ben-loghfyry, A., Hakim, A.: Caputo fractional-time of a modified Cahn–Hilliard equation for the inpainting of binary images. J. Math. Modeling 11(2), 357–373 (2023)

    MathSciNet  Google Scholar 

  20. Huang, X., Li, D., Sun, H.-W., Zhang, F.: Preconditioners with symmetrized techniques for space fractional Cahn–Hilliard equations. J. Sci. Comput. 92(2), 41 (2022)

    Article  MathSciNet  Google Scholar 

  21. Brkić, A.L., Mitrović, D., Novak, A.: On the image inpainting problem from the viewpoint of a nonlocal Cahn–Hilliard type equation. J. Adv. Res. 25, 67–76 (2020)

    Article  Google Scholar 

  22. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  23. Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60(1–4), 259–268 (1992)

    Article  MathSciNet  Google Scholar 

  24. Blank, L., Butz, M., Garcke, H.: Solving the Cahn–Hilliard variational inequality with a semi-smooth newton method. ESAIM: Control Optim. Calc. Var. 17(4), 931–954 (2011)

    MathSciNet  Google Scholar 

  25. Novick-Cohen, A., Segel, L.A.: Nonlinear aspects of the Cahn–Hilliard equation. Physica D 10(3), 277–298 (1984)

    Article  MathSciNet  Google Scholar 

  26. Elliott, C.M., Luckhaus, S.: A generalised diffusion equation for phase separation of a multi-component mixture with interfacial free energy. Prim. Care Community Psychiatr. 12(1), 23–319 (1991)

    Google Scholar 

  27. Eyre, D.J.: Systems of Cahn–Hilliard equations. SIAM J. Appl. Math. 53(6), 1686–1712 (1993)

    Article  MathSciNet  Google Scholar 

  28. Cherfils, L., Fakih, H., Miranville, A.: A Cahn–Hilliard system with a fidelity term for color image inpainting. J. Math. Imaging Vis. 54, 117–131 (2016)

    Article  MathSciNet  Google Scholar 

  29. Caselles, V., Morel, J.-M., Sbert, C.: An axiomatic approach to image interpolation. IEEE Trans. Image Process. 7(3), 376–386 (1998)

    Article  MathSciNet  Google Scholar 

Download references

Funding

This work was supported by the Science and Technology Plan of Sichuan under Grant 2021YJ0084

Author information

Author notes

  1. H. Peng and B. Zhou contributed equally to this paper.

Authors and Affiliations

  1. School of Science, Southwest Petroleum University, Chengdu, Sichuan, 610500, People’s Republic of China

    Hong Peng, Bin Zhou, Yan Sun & Linghai Zhang

  2. Institute of Articial Intelligence, Southwest Petroleum University, Chengdu, Sichuan, 610500, People’s Republic of China

    Bin Zhou

  3. School of Computer Science and Engineering, Xi’an University of Technology, Xi’an, Shaanxi, 710048, People’s Republic of China

    Wei Wei

Authors
  1. Hong Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bin Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yan Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Linghai Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wei Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

HP and BZ wrote the main manuscript and prepared figures. All authors reviewed the manuscript

Corresponding author

Correspondence to Bin Zhou.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest

Ethical approval

Not applicable

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Peng, H., Zhou, B., Sun, Y. et al. A vector-valued PDE-constrained image inpainting model. SIViP (2024). https://doi.org/10.1007/s11760-024-03124-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11760-024-03124-1

Keywords

Search

Navigation