Skip to main content
SpringerLink
Log in

Extrapolated Smoothing Descent Algorithm for Constrained Nonconvex and Nonsmooth Composite Problems

  • Published:
Chinese Annals of Mathematics, Series B Aims and scope Submit manuscript

Abstract

In this paper, the authors propose a novel smoothing descent type algorithm with extrapolation for solving a class of constrained nonsmooth and nonconvex problems, where the nonconvex term is possibly nonsmooth. Their algorithm adopts the proximal gradient algorithm with extrapolation and a safe-guarding policy to minimize the smoothed objective function for better practical and theoretical performance. Moreover, the algorithm uses a easily checking rule to update the smoothing parameter to ensure that any accumulation point of the generated sequence is an (affine-scaled) Clarke stationary point of the original nonsmooth and nonconvex problem. Their experimental results indicate the effectiveness of the proposed algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Attouch, H., Bolte, J. and Svaiter, B. F., Convergence of descent methods for semi-algebraic and tame problems: Proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel methods, Math. Program., 137, 2013, 91–129.

    Article  MATH  Google Scholar 

  2. Bao, C. L., Dong, B., Hou, L. K., et al., Image restoration by minimizing zero norm of wavelet frame coefficients, Inverse Probl., 32, 2016, 115004.

    Article  MATH  Google Scholar 

  3. Bian, W. and Chen, X. J., Linearly constrained non-Lipschitz optimization for image restoration, SIAM J. Imaging Sci., 8, 2015, 2294–2322.

    Article  MATH  Google Scholar 

  4. Bian, W. and Chen, X. J., Optimality and complexity for constrained optimization problems with non-convex regularization, Math. Oper. Res., 42, 2017, 1063–1084.

    Article  MATH  Google Scholar 

  5. Bian, W. and Chen, X. J., A smoothing proximal gradient algorithm for nonsmooth convex regression with cardinality penalty, SIAM J. Numer. Anal., 58, 2020, 858–883.

    Article  Google Scholar 

  6. Bian, W., Chen, X. J. and Ye, Y. Y., Complexity analysis of interior point algorithms for non-Lipschitz and nonconvex minimization, Math. Program., 149, 2015, 301–327.

    Article  MATH  Google Scholar 

  7. Bonettini, S., Loris, I., Porta, F., et al., On the convergence of a linesearch based proximal-gradient method for nonconvex optimization, Inverse Probl., 33, 2017, 055005.

    Article  MATH  Google Scholar 

  8. Burke, J. V., Ferris, M. C. and Qian, M. J., On the Clarke subdifferential of the distance function of a closed set, J. Math. Anal. Appl., 166, 1992, 199–213.

    Article  MATH  Google Scholar 

  9. Candes, E. J., Wakin, M. B. and Boyd, S. P., Enhancing sparsity by reweighted ℓ1 minimization, J. Fourier Anal. Appl., 14, 2008, 877–905.

    Article  MATH  Google Scholar 

  10. Chan, R. H., Tao, M. and Yuan, X. M., Constrained total variation deblurring models and fast algorithms based on alternating direction method of multipliers, SIAM J. Imaging Sci., 6, 2013, 680–697.

    Article  MATH  Google Scholar 

  11. Chen, X. J., Smoothing methods for nonsmooth, nonconvex minimization, Math. Program., 134, 2012, 71–99.

    Article  MATH  Google Scholar 

  12. Chen, X. J., Ng, M. K. and Zhang, C., Non-Lipschitz ℓp-regularization and box constrained model for image restoration, IEEE Trans. Image Process., 21, 2012, 4709–4721.

    Article  MATH  Google Scholar 

  13. Chen, X. J., Niu, L. F. and Yuan, Y. X., Optimality conditions and a smoothing trust region Newton method for nonLipschitz optimization, SIAM J. Optim., 23, 2013, 1528–1552.

    Article  MATH  Google Scholar 

  14. Chen, X. J. and Zhou, W. J., Smoothing nonlinear conjugate gradient method for image restoration using nonsmooth nonconvex minimization, SIAM J. Imaging Sci., 3, 2010, 765–790.

    Article  MATH  Google Scholar 

  15. Chen, Y. M., Liu, H. C., Ye, X. J. and Zhang, Q. C., Learnable descent algorithm for nonsmooth nonconvex image reconstruction, SIAM J. Imaging Sci., 14, 2021, 1532–1564.

    Article  MATH  Google Scholar 

  16. Clarke, F. H., Optimization and Nonsmooth Analysis, John Wiley and Sons, Philadelphia, 1990.

    Book  MATH  Google Scholar 

  17. Foucart, S. and Lai, M. J., Sparsest solutions of underdetermined linear systems via ℓq-minimization for 0 < q < 1, Appl. Comput. Harmon. Anal., 26, 2009, 395–407.

    Article  MATH  Google Scholar 

  18. Fukushima, M. and Mine, H., A generalized proximal point algorithm for certain non-convex minimization problems, International Journal of Systems Science, 12, 1981, 989–1000.

    Article  MATH  Google Scholar 

  19. Gao, Y. M. and Wu, C. L., On a general smoothly truncated regularization for variational piecewise constant image restoration: construction and convergent algorithms, Inverse Probl., 36, 2020, 045007.

    Article  MATH  Google Scholar 

  20. Ghadimi, S. and Lan, G. H., Accelerated gradient methods for nonconvex nonlinear and stochastic programming, Math. Program., 156, 2016, 59–99.

    Article  MATH  Google Scholar 

  21. Gu, B., Wang, D., Huo, Z. Y. and Huang, H., Inexact proximal gradient methods for non-convex and non-smooth optimization, in AAAI, 32, 2018.

  22. Hintermüller, M. and Wu, T., Nonconvex TVq-models in image restoration: Analysis and a trust-region regularization based superlinearly convergent solver, SIAM J. Imaging Sci., 6, 2013, 1385–1415.

  23. Kak, A. C. and Slaney, M., Principles of Computerized Tomographic Imaging, Philadelphia, PA, USA: SIAM, 2001.

    Book  MATH  Google Scholar 

  24. Kong, W. W., Melo, J. G. and Monteiro, R. D., Complexity of a quadratic penalty accelerated inexact proximal point method for solving linearly constrained nonconvex composite programs, SIAM J. Optim., 29, 2019, 2566–2593.

    Article  MATH  Google Scholar 

  25. Kong, W. W., Melo, J. G. and Monteiro, R. D., An efficient adaptive accelerated inexact proximal point method for solving linearly constrained nonconvex composite problems, Comput. Optim. Appl., 76, 2020, 305–346.

    Article  MATH  Google Scholar 

  26. Lai, M. J. and Xu, Y. Y. and Yin, W. T., Improved iteratively reweighted least squares for unconstrained smoothed ℓq minimization, SIAM J. Numer. Anal., 51, 2013, 927–957.

    Article  MATH  Google Scholar 

  27. Li, H. and Lin, Z. C., Accelerated proximal gradient methods for nonconvex programming, in NIPS, 2015, 379–387.

  28. Li, Q. W., Zhou, Y., Liang, Y. B. and Varshney, P. K., Convergence analysis of proximal gradient with momentum for nonconvex optimization, in ICML, PMLR, 2017, 2111–2119.

  29. Lions, P.-L. and Mercier, B., Splitting algorithms for the sum of two nonlinear operators, SIAM J. Numer. Anal., 16, 1979, 964–979.

    Article  MATH  Google Scholar 

  30. Liu, Z. F., Wu, C. L. and Zhao, Y, N., A new globally convergent algorithm for non-Lipschitz lp − lq minimization, Adv. Comput. Math., 45, 2019, 1369–1399.

    Article  MATH  Google Scholar 

  31. Nesterov, Y., Smooth minimization of non-smooth functions, Math. Program., 103, 2005, 127–152.

    Article  MATH  Google Scholar 

  32. Nikolova, M., Analysis of the recovery of edges in images and signals by minimizing nonconvex regularized least-squares, SIAM J. Multiscale Model. Simul., 4, 2005, 960–991.

    Article  MATH  Google Scholar 

  33. Nikolova, M., Ng, M. K. and Tam, C. P., Fast nonconvex nonsmooth minimization methods for image restoration and reconstruction, IEEE Trans. Image Process., 19, 2010, 3073–3088.

    Article  MATH  Google Scholar 

  34. Nikolova, M., Ng, M. K., Zhang, S. Q. and Ching, W. K., Efficient reconstruction of piecewise constant images using nonsmooth nonconvex minimization, SIAM J. Imaging Sci., 1, 2008, 2–25.

    Article  MATH  Google Scholar 

  35. Ochs, P., Chen, Y. J., Brox, T. and Pock, T., iPiano: Inertial proximal algorithm for nonconvex optimization, SIAM J. Imaging Sci., 7, 2014, 1388–1419.

    Article  MATH  Google Scholar 

  36. Ochs, P., Dosovitskiy, A., Brox, T. and Pock, T., On iteratively reweighted algorithms for nonsmooth nonconvex optimization in computer vision, SIAM J. Imaging Sci., 8, 2015, 331–372.

    Article  MATH  Google Scholar 

  37. Rudin, L. I., Osher, S. and Fatemi, E., Nonlinear total variation based noise removal algorithms, Phys. D, Nonlinear Phenomena, 60, 1992, 259–168.

    Article  MATH  Google Scholar 

  38. Villa, S., Salzo, S., Baldassarre, L. and Verri, A., Accelerated and inexact forward-backward algorithms, SIAM J. Optim, 23, 2013, 1607–1633.

    Article  MATH  Google Scholar 

  39. Wang, W. and Chen, Y. M., An accelerated smoothing gradient method for nonconvex nonsmooth minimization in image processing, J. Sci. Comput., 90, 2022, 1–18.

    Article  MATH  Google Scholar 

  40. Wang, W., Wu, C. L. and Gao, Y. M., A nonconvex truncated regularization and box-constrained model for CT reconstruction, Inverse Probl. Imag., 14, 2020, 867–890.

    Article  MATH  Google Scholar 

  41. Wang, W., Wu, C. L. and Tai, X. C., A globally convergent algorithm for a constrained non-Lipschitz image restoration model, J. Sci. Comput., 83, 2020, 1–19.

    Article  MATH  Google Scholar 

  42. Wen, B., Chen, X. J. and Pong, T. K., Linear convergence of proximal gradient algorithm with extrapolation for a class of nonconvex nonsmooth minimization problems, SIAM J. Optim., 27, 2017, 124–145.

    Article  MATH  Google Scholar 

  43. Wu, C. L. and Tai, X. C., Augmented Lagrangian method, dual methods, and split Bregman iteration for ROF, vectorial TV, and high order models, SIAM J. Imaging Sci., 3, 2010, 300–339.

    Article  MATH  Google Scholar 

  44. Wu, Z. and Li, M., General inertial proximal gradient method for a class of nonconvex nonsmooth optimization problems, Comput. Optim. Appl., 73, 2019, 129–158.

    Article  MATH  Google Scholar 

  45. Xu, Z. B., Chang, X. Y., Xu, F. M. and Zhang, H., ℓ½ regularization: A thresholding representation theory and a fast solver, IEEE Trans. Neural Netw. Learn. Syst., 23, 2012, 1013–1027.

    Article  Google Scholar 

  46. Yang, L., Proximal gradient method with extrapolation and line search for a class of nonconvex and nonsmooth problems, 2017, arXiv:1711.06831.

  47. Yao, Q. M., Kwok, J. T., Gao, F., et al., Efficient inexact proximal gradient algorithm for nonconvex problems, 2016, arXiv:1612.09069.

  48. Zeng, C. and Jia, R. and Wu, C. L., An iterative support shrinking algorithm for non-Lipschitz optimization in image restoration, J. Math. Imaging Vis., 61, 2019, 122–139.

    Article  MATH  Google Scholar 

  49. Zeng, C. and Wu, C. L., On the edge recovery property of noncovex nonsmooth regularization in image restoration, SIAM J. Numer. Anal., 56, 2018, 1168–1182.

    Article  MATH  Google Scholar 

  50. Zhang, H. M., Dong, B. and Liu, B. D., A reweighted joint spatial-radon domain CT image reconstruction model for metal artifact reduction, SIAM J. Imaging Sci., 11, 2018, 707–733.

    Article  MATH  Google Scholar 

  51. Zhang, X. and Zhang, X. Q., A new proximal iterative hard thresholding method with extrapolation for ℓ0 minimization, J. Sci. Comput., 79, 2019, 809–826.

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, University of Florida, Gainesville, 118105, USA

    Yunmei Chen

  2. Industrial and Systems Engineering, University of Florida, Gainesville, 118105, USA

    Hongcheng Liu

  3. Department of Mathematics, Hangzhou Dianzi University, Hangzhou, 310018, China

    Weina Wang

Authors
  1. Yunmei Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hongcheng Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Weina Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Weina Wang.

Additional information

This work was supported by the National Natural Science Foundation of China (No. 12001144), Zhejiang Provincial Natural Science Foundation of China (No. LQ20A010007) and NSF/DMS-2152961.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, Y., Liu, H. & Wang, W. Extrapolated Smoothing Descent Algorithm for Constrained Nonconvex and Nonsmooth Composite Problems. Chin. Ann. Math. Ser. B 43, 1049–1070 (2022). https://doi.org/10.1007/s11401-022-0377-7

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11401-022-0377-7

Keywords

2000 MR Subject Classification

Search

Navigation