Skip to main content
SpringerLink
Log in

A Proximal Alternating Direction Method of Multipliers for DC Programming with Structured Constraints

  • Published:
Journal of Scientific Computing Aims and scope Submit manuscript

Abstract

In this paper, we consider a class of structured DC programming, where the objective function is the difference of two (possibly nonsmooth) convex functions, and the constraint is a linear function belonging to a nonempty closed convex set. To fully exploit the favorable structure of the problem under consideration, we propose an implementable algorithm, proximal Alternating Direction Method of Multipliers (pADMM), which employs the Fenchel-Young inequality and Moreau decomposition theorem such that the potentially explicit proximal operators of the two DC parts could be efficiently explored, thereby making all subproblems quite easy in some cases. Theoretically, we prove that the sequence generated by our algorithm pADMM converges to a critical point of the problem with the help of Kurdyka–Łojasiewicz inequality. Finally, some preliminary computational results on solving \(\ell _1-\alpha \ell _2\)-norm Dantzig selector problem and automated model selection demonstrate that our proposed method runs faster than the standard ADMM solver.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Algorithm 1
Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Data Availability

Enquiries about data availability should be directed to the authors.

References

  1. Aragón Artacho, F., Fleming, R., Vuong, P.T.: Accelerating the DC algorithm for smooth functions. Math. Program. Ser. B 169(1), 95–118 (2018)

    MathSciNet  Google Scholar 

  2. Aragón Artacho, F., Vuong, P.: The boosted difference of convex functions algorithm for nonsmooth functions. SIAM J. Optim. 30, 980–1006 (2020)

    MathSciNet  Google Scholar 

  3. Attouch, H., Bolte, J., Redont, P., Soubeyran, A.: Proximal alternating minimization and projection methods for nonconvex problems: an approach based on the Kurdyka-Lojasiewicz inequality. Math. Oper. Res. 35, 438–457 (2010)

    MathSciNet  Google Scholar 

  4. Bauschke, H., Combettes, P.: Convex Analysis and Monotone Operator Theory in Hilbert Space. Springer, New York (2011)

    Google Scholar 

  5. Beck, A.: First-Order Methods in Optimization. SIAM, Philadelphia (2017)

    Google Scholar 

  6. Bolte, J., Sabach, S., Teboulle, M.: Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Math. Program. Ser. A 146, 459–494 (2014)

    MathSciNet  Google Scholar 

  7. Boţ, R.I., Nguyen, D.K.: The proximal alternating direction method of multipliers in the nonconvex setting: convergence analysis and rates. Math. Oper. Res. 45(2), 682–712 (2020)

    MathSciNet  Google Scholar 

  8. Brunton, S., Proctor, J., Kutz, J.: Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proc. Natl. Acad. Sci. 113, 3932–3937 (2016)

    MathSciNet  Google Scholar 

  9. Candés, E., Tao, T.: The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). Ann. Stat. 35, 2313–2351 (2007)

    MathSciNet  Google Scholar 

  10. Ceng, L., Ansari, Q., Yao, J.: Relaxed extragradient methods for finding minimum-norm solutions of the split feasibility problem. Nonlinear Anal. 75, 2116–2125 (2012)

    MathSciNet  Google Scholar 

  11. Censor, Y., Elfving, T.: A multiprojection algorithm using Bregman projections in product space. Numer. Algor. 8, 221–239 (1994)

    MathSciNet  Google Scholar 

  12. Chuang, C.S., He, H., Zhang, Z.: A unified Douglas-Rachford algorithm for generalized DC programming. J. Global Optim. 82, 331–349 (2022)

    MathSciNet  Google Scholar 

  13. Clarke, F.: Functional Analysis, Calculus of Variations and Optimal Control. Springer, London (2013)

    Google Scholar 

  14. Combettes, P., Wajs, V.: Signal recovery by proximal forward-backward splitting. Multiscale Model. Simul. 4, 1168–1200 (2005)

    MathSciNet  Google Scholar 

  15. Ge, H., Chen, W., Ng, M.: New restricted isometry property analysis for \(\ell _1-\ell _2\) minimization methods. SIAM J. Imaging Sci. 14, 530–557 (2021)

    MathSciNet  Google Scholar 

  16. Ge, H., Li, P.: The Dantzig selector: recovery of signal via \(\ell _1-\alpha \ell _2\) minimization. Inverse Probl. 38(1), 015006 (2021)

    Google Scholar 

  17. Gotoh, J., Takeda, A., Tono, K.: DC formulations and algorithms for sparse optimization problems. Math. Program. Ser. B 169, 141–176 (2018)

    MathSciNet  Google Scholar 

  18. Guo, K., Han, D., Wu, T.: Convergence of alternating direction method for minimizing sum of two nonconvex functions with linear constraints. Int. J. Comput. Math. 94(8), 1653–1669 (2017)

    MathSciNet  Google Scholar 

  19. He, B.S., Yang, H., Wang, S.: Alternating direction method with self-adaptive penalty parameters for monotone variational inequalities. J. Optim. Theory Appl. 106, 337–356 (2000)

    MathSciNet  Google Scholar 

  20. He, H., Ling, C., Xu, H.: An implementable splitting algorithm for the \(\ell _1\)-norm regularized split feasibility problem. J. Sci. Comput. 67, 281–298 (2016)

    MathSciNet  Google Scholar 

  21. He, H., Xu, H.K.: Splitting methods for split feasibility problems with application to Dantzig selectors. Inverse Probl. 33, 055003 (2017)

    MathSciNet  Google Scholar 

  22. He, H., Zhang, Z.: A unified Bregman alternating minimization algorithm for generalized DC programming with applications to imaging data (2022). ArXiv:2209.07323

  23. He, S., Zhu, W.: A note on approximating curve with \(\ell _1\)-norm regularization method for the split feasibility problem. J. Appl. Math. 2012, 683890 (2012)

    Google Scholar 

  24. Hong, M., Luo, Z.Q., Razaviyayn, M.: Convergence analysis of alternating direction method of multipliers for a family of nonconvex problems. SIAM J. Optim. 26(1), 337–364 (2016)

    MathSciNet  Google Scholar 

  25. Jiang, B., Lin, T., Ma, S., Zhang, S.: Structured nonconvex and nonsmooth optimization: algorithms and iteration complexity analysis. Comput. Optim. Appl. 72(1), 115–157 (2019)

    MathSciNet  Google Scholar 

  26. Le Thi, H., Pham Dinh, T.: DC programming and DCA: thirty years of developments. Math. Program. Ser. A 169, 5–68 (2018)

    MathSciNet  Google Scholar 

  27. Liu, Q., Shen, X., Gu, Y.: Linearized ADMM for nonconvex nonsmooth optimization with convergence analysis. IEEE Access 7, 76131–76144 (2019)

    Google Scholar 

  28. Lou, Y., Yan, M.: Fast \(\ell _1\)-\(\ell _2\) minimization via a proximal operator. J. Sci. Comput. 74, 767–785 (2018)

    MathSciNet  Google Scholar 

  29. Lou, Y., Yin, P., He, Q., Xin, J.: Computing sparse representation in a highly coherent dictionary based on difference of \(\ell _1\)-\(\ell _2\). J. Sci. Comput. 64, 178–196 (2015)

    MathSciNet  Google Scholar 

  30. Lou, Y., Yin, P., Xin, J.: Point source super-resolution via non-convex \(\ell _1\) based methods. J. Sci. Comput. 68, 1082–1100 (2016)

    MathSciNet  Google Scholar 

  31. Lou, Y., Zeng, T., Osher, S., Xin, J.: A weighted difference of anisotropic and isotropic total variation model for image processing. SIAM J. Imaging Sci. 8, 1798–1823 (2015)

    MathSciNet  Google Scholar 

  32. Lu, X., Zhang, L., He, H.: Structured model selection via \(\ell _1-\ell _2\) optimization. Inverse Probl. 40, 015011 (2024)

    Google Scholar 

  33. Lu, Z., Zhou, Z.: Nonmonotone enhanced proximal DC algorithms for structured nonsmooth DC programming. SIAM J. Optim. 29, 2725–2752 (2019)

    MathSciNet  Google Scholar 

  34. Lu, Z., Zhou, Z., Sun, Z.: Enhanced proximal DC algorithms with extrapolation for a class of structured nonsmooth DC minimization. Math. Progr. Ser. B 176, 369–401 (2019)

    MathSciNet  Google Scholar 

  35. Ma, T., Lou, Y., Huang, T.: Truncated \(\ell _{1-2}\) models for sparse recovery and rank minimization. SIAM J. Imaging Sci. 10, 1346–1380 (2017)

    MathSciNet  Google Scholar 

  36. Moudafi, A., Gibali, A.: \(\ell _1\)-\(\ell _2\) regularization of split feasibility problems. Numer. Algor. 78, 739–757 (2018)

    Google Scholar 

  37. Nguyen, M.: DC programming and DCA for some classes of problems in machine learning and data mining. Ph.D. thesis, Université de Lorraine (2014)

  38. Niu, Y.S.: On the convergence analysis of DCA. arXiv:2211.10942 (2022)

  39. Pham Dinh, T., Souad, E.B.: Algorithms for solving a class of nonconvex optimization problems. Methods of subgradients. In: J.B. Hiriart-Urruty (ed.) Fermat Days 85: Mathematics for Optimization, North-Holland Mathematics Studies, vol. 129, pp. 249 – 271. North-Holland (1986)

  40. Rockafellar, R.T.: Convex Analysis. Princeton University Press, Princeton (1970)

    Google Scholar 

  41. Rockafellar, R.T., Wets, R.J.B.: Variational Analysis. Springer-Verlag, Berlin (1998)

    Google Scholar 

  42. Schaeffer, H.: Learning partial differential equations via data discovery and sparse optimization. Proc. R. Soc. A. 473, 20160,446 (2017)

    MathSciNet  Google Scholar 

  43. Schaeffer, H., Tran, G., Ward, R., Zhang, L.: Extracting structured dynamical systems using sparse optimization with very few samples. Multiscale Model. Simul. 18, 1435–1461 (2020)

    MathSciNet  Google Scholar 

  44. Sun, T., Yin, P., Cheng, L., Jiang, H.: Alternating direction method of multipliers with difference of convex functions. Adv. Comput. Math. 44, 723–744 (2018)

    MathSciNet  Google Scholar 

  45. Sun, Y., Babu, P., Palomar, D.P.: Majorization-minimization algorithms in signal processing, communications, and machine learning. IEEE Trans. Signal Process. 65(3), 794–816 (2016)

    MathSciNet  Google Scholar 

  46. Takahashi, S., Fukuda, M., Tanaka, M.: New Bregman proximal type algorithms for solving DC optimization problems. Comput. Optim. Appl. 83(3), 893–931 (2022)

    MathSciNet  Google Scholar 

  47. Tu, K., Zhang, H., Gao, H., Feng, J.: A hybrid Bregman alternating direction method of multipliers for the linearly constrained difference-of-convex problems. J. Global Optim. 76(4), 665–693 (2020)

    MathSciNet  Google Scholar 

  48. Wang, Y., Yin, W., Zeng, J.: Global convergence of ADMM in nonconvex nonsmooth optimization. J. Sci. Comput. 78, 29–63 (2019)

    MathSciNet  Google Scholar 

  49. Wen, B., Chen, X., Pong, T.K.: A proximal difference-of-convex algorithm with extrapolation. Comput. Optim. Appl. 69(2), 297–324 (2018)

    MathSciNet  Google Scholar 

  50. Yin, P., Lou, Y., He, Q., Xin, J.: Minimization of \(\ell _{1-2}\) for compressed sensing. SIAM J. Sci. Comput. 37, A536–A563 (2015)

    MathSciNet  Google Scholar 

  51. Yin, P., Xin, J.: PhaseLiftOff: an accurate and stable phase retrieval method based on difference of trace and Frobenius norms. Commun. Math. Sci. 13, 1033–1049 (2015)

    MathSciNet  Google Scholar 

  52. Zhang, F., Yang, Z., Chen, Y., Yang, J., Yang, G.: Matrix completion via capped nuclear norm. IET Image Process. 12, 959–966 (2018)

    Google Scholar 

  53. de Oliveira, W.: The ABC of DC programming. Set-Valued Var. Anal. 28, 679–706 (2020)

    MathSciNet  Google Scholar 

Download references

Acknowledgements

The authors are grateful to the referees for their close reading and valuable comments, which have greatly helped us improve the quality of this paper. They also would like to thank Dr. Peng Li for kindly sharing his code of [16] with us, and Xiaofan Lu who shares code of [32].

Funding

H. He was supported in part by National Natural Science Foundation of China (NSFC) Grant #12371303, Zhejiang Provincial Natural Science Foundation of China (No. LZ24A010001), and Ningbo Natural Science Foundation (Project ID: 2023J014). L. Zhang was supported by NSFC Grant #12101342.

Author information

Authors and Affiliations

  1. School of Mathematics and Statistics, Ningbo University, Ningbo, 315211, China

    Yingxin Zhou, Hongjin He & Linan Zhang

Authors
  1. Yingxin Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hongjin He
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Linan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hongjin He.

Ethics declarations

Conflict of interest

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, Y., He, H. & Zhang, L. A Proximal Alternating Direction Method of Multipliers for DC Programming with Structured Constraints. J Sci Comput 99, 89 (2024). https://doi.org/10.1007/s10915-024-02550-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10915-024-02550-0

Keywords

Mathematics Subject Classification

Search

Navigation