Skip to main content
SpringerLink
Log in

A refined inertial DC algorithm for DC programming

  • Research Article
  • Published:
Optimization and Engineering Aims and scope Submit manuscript

Abstract

In this paper we consider the difference-of-convex (DC) programming problems, whose objective function is the difference of two convex functions. The classical DC Algorithm (DCA) is well-known for solving this kind of problems, which generally returns a critical point. Recently, an inertial DC algorithm (InDCA) equipped with heavy-ball inertial-force procedure was proposed in de Oliveira et al. (Set-Valued Variat Anal 27(4):895–919, 2019), which potentially helps to improve both the convergence speed and the solution quality. Based on InDCA, we propose a refined inertial DC algorithm (RInDCA) equipped with enlarged inertial step-size compared with InDCA. Empirically, larger step-size accelerates the convergence. We demonstrate the subsequential convergence of our refined version to a critical point. the Kurdyka-Łojasiewicz (KL) property of the objective function, we establish the sequential convergence of RInDCA. Numerical simulations on checking copositivity of matrices and image denoising problem show the benefit of larger step-size.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. Let \(\mu >0\) and \(i=0,1\), \({}^{\mu }f_i\) is the Moreau envelope of \(f_i\) with parameter \(\mu \), defined as \({}^{\mu } f_i(\mathbf{x }):= \min \{f_i(\mathbf{y })+\frac{1}{2{\mu }}\Vert \mathbf{x }-\mathbf{y } \Vert ^2:\mathbf{y }\in \mathbb {R}^n\}.\)

  2. \(\Vert \mathbf{A }\Vert \) is the spectral norm of \(\mathbf{A }\).

References

  • Adly S, Attouch H, Le HM (2021) First order inertial optimization algorithms with threshold effects associated with dry friction. hal:03284220

  • Aleksandrov AD (2012) On the surfaces representable as difference of convex functions. Sib Elektron Mat Izvest 9:360–376

    MATH  Google Scholar 

  • Artacho FJA, Campoy R, Vuong PT (2019) The boosted dc algorithm for linearly constrained dc programming. arXiv:1908.01138

  • Artacho FJA, Fleming RM, Vuong PT (2018) Accelerating the dc algorithm for smooth functions. Math Programm 169(1):95–118

    Article  MathSciNet  MATH  Google Scholar 

  • Artacho FJA, Vuong PT (2020) The boosted dc algorithm for nonsmooth functions. SIAM J Optim 30(1):980–1006

    Article  MathSciNet  MATH  Google Scholar 

  • Attouch H, Bolte J (2009) On the convergence of the proximal algorithm for nonsmooth functions involving analytic features. Math Programm 116(1):5–16

    Article  MathSciNet  MATH  Google Scholar 

  • Attouch H, Bolte J, Svaiter BF (2013) Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized gauss-seidel methods. Math Programm 137(1):91–129

    Article  MathSciNet  MATH  Google Scholar 

  • Beck A (2017) First-order methods in optimization. SIAM

  • Beck A, Teboulle M (2009) Fast gradient-based algorithms for constrained total variation image denoising and deblurring problems. IEEE Trans Image Process 18(11):2419–2434

    Article  MathSciNet  MATH  Google Scholar 

  • Bolte J, Daniilidis A, Lewis A (2007) The łojasiewicz inequality for nonsmooth subanalytic functions with applications to subgradient dynamical systems. SIAM J Optim 17(4):1205–1223

    Article  MATH  Google Scholar 

  • Bolte J, Sabach S, Teboulle M (2014) Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Math Programm 146(1):459–494

    Article  MathSciNet  MATH  Google Scholar 

  • de Oliveira W (2020) The abc of dc programming. Set-Valued Variat Anal 28(4):679–706

    Article  MathSciNet  MATH  Google Scholar 

  • de Oliveira W, Tcheou MP (2019) An inertial algorithm for dc programming. Set-Valued Variat Anal 27(4):895–919

    Article  MathSciNet  MATH  Google Scholar 

  • Dür M, Hiriart-Urruty JB (2013) Testing copositivity with the help of difference-of-convex optimization. Math Program 140(1 Ser B):31–43

    Article  MathSciNet  MATH  Google Scholar 

  • Hartman P (1959) On functions representable as a difference of convex functions. Pac J Math 9(3):707–713

    Article  MathSciNet  MATH  Google Scholar 

  • Hiriart-Urruty JB (1985) Generalized differentiability/duality and optimization for problems dealing with differences of convex functions. In: Ponstein J (ed) Convexity and duality in optimization. Springer, Berlin, pp 37–70

    Chapter  MATH  Google Scholar 

  • Horst R, Thoai NV (1999) Dc programming: overview. J Optim Theory Appl 103:1–43

    Article  MathSciNet  MATH  Google Scholar 

  • Joki K, Bagirov AM, Karmitsa N, Mäkelä MM (2017) A proximal bundle method for nonsmooth dc optimization utilizing nonconvex cutting planes. J Glob Optim 68(3):501–535

    Article  MathSciNet  MATH  Google Scholar 

  • Le Thi HA, Pham DT (2005) The dc (difference of convex functions) programming and dca revisited with dc models of real world nonconvex optimization problems. Ann Oper Res 133(1–4):23–46

    MathSciNet  MATH  Google Scholar 

  • Le Thi HA, Pham DT (2018) Dc programming and dca: thirty years of developments. Math Programm 169(1):5–68

    Article  MathSciNet  MATH  Google Scholar 

  • Le Thi HA, Pham DT, Ngai HV (2012) Exact penalty and error bounds in dc programming. J Glob Optim 52:509–535

    Article  MathSciNet  MATH  Google Scholar 

  • Liu T, Pong TK, Takeda A (2019) A refined convergence analysis of \({{\rm pDCA}}_e\) ith applications to simultaneous sparse recovery and outlier detection. Comput Optim Appl 73(1):69–100

    Article  MathSciNet  MATH  Google Scholar 

  • Lu Z, Zhou Z, Sun Z (2019) Enhanced proximal dc algorithms with extrapolation for a class of structured nonsmooth dc minimization. Math Programm 176(1):369–401

    Article  MathSciNet  MATH  Google Scholar 

  • Monga V (2017) Handbook of convex optimization methods in imaging science, vol 1. Springer, Berlin

    MATH  Google Scholar 

  • Mukkamala MC, Ochs P, Pock T, Sabach S (2020) Convex-concave backtracking for inertial Bregman proximal gradient algorithms in nonconvex optimization. SIAM J Math Data Sci 2(3):658–682

    Article  MathSciNet  MATH  Google Scholar 

  • Nesterov YE (1983) A method for solving the convex programming problem with convergence rate \(\cal{O}(1/{k^2})\). In: Dokl. akad. nauk Sssr, vol. 269, pp. 543–547

  • Nesterov YE (2013) Gradient methods for minimizing composite functions. Math Programm 140(1):125–161

    Article  MathSciNet  MATH  Google Scholar 

  • Niu YS, Wang YJ (2019) Higher-order moment portfolio optimization via the difference-of-convex programming and sums-of-squares. arXiv:1906.01509

  • Ochs P, Chen Y, Brox T, Pock T (2014) ipiano: inertial proximal algorithm for nonconvex optimization. SIAM J Imaging Sci 7(2):1388–1419

    Article  MathSciNet  MATH  Google Scholar 

  • Pang JS, Razaviyayn M, Alvarado A (2017) Computing b-stationary points of nonsmooth dc programs. Math Oper Res 42(1):95–118

    Article  MathSciNet  MATH  Google Scholar 

  • Pham DT, Huynh VN, Le Thi HA, Ho VT (2021) Alternating dc algorithm for partial dc programming problems. J Glob Optim. https://doi.org/10.1007/s10898-021-01043-w

    Article  MATH  Google Scholar 

  • Pham DT, Le Thi HA (1997) Convex analysis approach to d.c. programming: theory, algorithms and applications. Acta Math Vietnam 22(1):289–355

    MathSciNet  MATH  Google Scholar 

  • Pham DT, Le Thi HA (1998) A dc optimization algorithm for solving the trust-region subproblem. SIAM J Optim 8(2):476–505

    Article  MathSciNet  MATH  Google Scholar 

  • Pham DT, Le Thi HA (2014) Recent advances in DC programming and DCA. Transactions on Computational Intelligence XIII, pp 1–37

  • Pham DT, Souad EB (1986) Algorithms for solving a class of nonconvex optimization problems methods of subgradients. In: Hiriart-Urruty J-B (ed) Fermat days 85: mathematics for optimization, vol 129. Elsevier, New York, pp 249–271

    Chapter  Google Scholar 

  • Phan D.N, Le H.M, Le Thi H.A (2018) Accelerated difference of convex functions algorithm and its application to sparse binary logistic regression. In: IJCAI, pp. 1369–1375

  • Pock T, Sabach S (2016) Inertial proximal alternating linearized minimization (ipalm) for nonconvex and nonsmooth problems. SIAM J Imaging Sci 9(4):1756–1787

    Article  MathSciNet  MATH  Google Scholar 

  • Polyak BT (1987) Introduction to optimization, vol 1. Optimization Software Inc., New York

    MATH  Google Scholar 

  • Rockafellar RT (1970) Convex analysis, vol 36. Princeton University Press, Princeton

    Book  MATH  Google Scholar 

  • Souza JCO, Oliveira PR, Soubeyran A (2016) Global convergence of a proximal linearized algorithm for difference of convex functions. Optim Lett 10(7):1529–1539

    Article  MathSciNet  MATH  Google Scholar 

  • Wen B, Chen X, Pong TK (2018) A proximal difference-of-convex algorithm with extrapolation. Comput Optim Appl 69(2):297–324

    Article  MathSciNet  MATH  Google Scholar 

  • Zavriev S, Kostyuk F (1993) Heavy-ball method in nonconvex optimization problems. Comput Math Model 4(4):336–341

    Article  MATH  Google Scholar 

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China (Grant 11601327). We thank the anonymous referees for their valuable remarks.

Author information

Authors and Affiliations

  1. School of Mathematical Sciences, Shanghai Jiao Tong University, Shanghai, China

    Yu You

  2. Department of Applied Mathematics, The Hong Kong Polytechnic University, Kowloon, Hong Kong

    Yi-Shuai Niu

  3. School of Mathematical Sciences and SJTU-Paristech, Shanghai Jiao Tong University, Shanghai, China

    Yi-Shuai Niu

Authors
  1. Yu You
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yi-Shuai Niu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yi-Shuai Niu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work is supported by the National Natural Science Foundation of China (Grant 11601327).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

You, Y., Niu, YS. A refined inertial DC algorithm for DC programming. Optim Eng 24, 65–91 (2023). https://doi.org/10.1007/s11081-022-09716-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11081-022-09716-5

Keywords

Mathematics Subject Classification

Search

Navigation