Advertisement

Enhanced proximal DC algorithms with extrapolation for a class of structured nonsmooth DC minimization

  • Zhaosong Lu
  • Zirui Zhou
  • Zhe Sun
Full Length Paper Series B
  • 168 Downloads

Abstract

In this paper we consider a class of structured nonsmooth difference-of-convex (DC) minimization in which the first convex component is the sum of a smooth and nonsmooth functions while the second convex component is the supremum of possibly infinitely many convex smooth functions. We first propose an inexact enhanced DC algorithm for solving this problem in which the second convex component is the supremum of finitely many convex smooth functions, and show that every accumulation point of the generated sequence is an \((\alpha ,\eta )\)-D-stationary point of the problem, which is generally stronger than an ordinary D-stationary point. In addition, inspired by the recent work (Pang et al. in Math Oper Res 42(1):95–118, 2017; Wen et al. in Comput Optim Appl 69(2):297–324, 2018), we propose two proximal DC algorithms with extrapolation for solving this problem. We show that every accumulation point of the solution sequence generated by them is an \((\alpha ,\eta )\)-D-stationary point of the problem, and establish the convergence of the entire sequence under some suitable assumption. We also introduce a concept of approximate \((\alpha ,\eta )\)-D-stationary point and derive iteration complexity of the proposed algorithms for finding an approximate \((\alpha ,\eta )\)-D-stationary point. In contrast with the DC algorithm (Pang et al. 2017), our proximal DC algorithms have much simpler subproblems and also incorporate the extrapolation for possible acceleration. Moreover, one of our proximal DC algorithms is potentially applicable to the DC problem in which the second convex component is the supremum of infinitely many convex smooth functions. In addition, our algorithms have stronger convergence results than the proximal DC algorithm in Wen et al. (2018).

Keywords

Nonsmooth DC program D-stationary point Approximate D-stationary point Proximal DCA Extrapolation Iteration complexity 

Mathematics Subject Classification

90C26 90C30 65K05 

Notes

Acknowledgements

Zhaosong Lu was supported in part by NSERC Discovery Grant. Zirui Zhou was supported by NSERC Discovery Grant and the SFU Alan Mekler postdoctoral fellowship. Zhe Sun was supported by National Natural Science Foundation of China (Grant Nos. 11761037 and 11501265), Natural Science Foundation of Jiangxi Province (Grant No. 20181BAB201009), and the scholarship from China Scholarship Council. This work was conducted while the second author was a postdoctoral fellow and the third author was a visiting scholar at Simon Fraser University, Canada.

References

  1. 1.
    Alvarado, A., Scutari, G., Pang, J.-S.: A new decomposition method for multiuser DC-programming and its applications. IEEE Trans. Signal Process. 62(11), 2984–2998 (2014)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Attouch, H., Bolte, J., Svaiter, B.F.: Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel methods. Math. Program. 137(1–2), 91–129 (2013)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Bertsekas, D.P.: Nonlinear programming. Athena Scientific, Belmont (1999)zbMATHGoogle Scholar
  4. 4.
    Bolte, J., Sabach, S., Teboulle, M.: Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Math. Program. 146(1–2), 459–494 (2014)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Burer, S., Anstreicher, K.M.: Second-order-cone constraints for extended trust-region subproblems. SIAM J. Optim. 23(1), 432–451 (2013)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Combettes, P.L., Pesquet, J.-C.: Proximal splitting methods in signal processing. In: Bauschke, H., Burachik, R., Combettes, P., Elser, V., Luke, D., Wolkowicz, H. (eds.) Fixed-point algorithms for inverse problems in science and engineering, pp. 185–212. Springer, New York (2011)CrossRefGoogle Scholar
  7. 7.
    Facchinei, F., Pang, J.-S.: Finite-Dimensional Variational Inequalities and Complementarity Problems. Springer, New York (2003)zbMATHGoogle Scholar
  8. 8.
    Gong, P., Zhang, C., Lu, Z., Huang, J., Ye, J.: A general iterative shrinkage and thresholding algorithm for non-convex regularized optimization problems. In: Proceedings of the 30th International Conference on Machine Learning (ICML), pp. 37–45 (2013)Google Scholar
  9. 9.
    Gotoh, J., Takeda, A., Tono, K.: DC formulations and algorithms for sparse optimization problems. Math. Program. 169(1), 141–176 (2018)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Le Thi, H.A., Huynh, V.N., Pham, D.T.: DC programming and DCA for general DC programs. In: van Do, T., Thi, H.A.L., Nguyen, N.T. (eds.) Advanced Computational Methods for Knowledge Engineering, pp. 15–35. Springer, Berlin (2014)CrossRefGoogle Scholar
  11. 11.
    Miao, W., Pan, S., Sun, D.: A rank-corrected procedure for matrix completion with fixed basis coefficients. Math. Program. 159(1–2), 289–338 (2016)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Odonoghue, B., Candès, E.: Adaptive restart for accelerated gradient schemes. Found. Comput. Math. 15(3), 715–732 (2015)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Pang, J.-S., Razaviyayn, M., Alvarado, A.: Computing B-stationary points of nonsmooth DC programs. Math. Oper. Res. 42(1), 95–118 (2017)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Pham Dinh, T., Le Thi, H.A.: Recent advances in DC programming and DCA. In: Transactions on Computational Intelligence XIII, pp. 1–37. Springer, Berlin (2014)Google Scholar
  15. 15.
    Rockafellar, R.T.: Convex Analysis. Princeton University Press, Princeton (1970)CrossRefGoogle Scholar
  16. 16.
    Rockafellar, R.T., Wets, R.J.-B.: Variational Analysis. Springer, Berlin (1998)CrossRefGoogle Scholar
  17. 17.
    Sanjabi, M., Razaviyayn, M., Luo, Z.-Q.: Optimal joint base station assignment and beamforming for heterogeneous networks. IEEE Trans. Signal Process. 62(8), 1950–1961 (2014)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Sturm, J.F., Zhang, S.: On cones of nonnegative quadratic functions. Math. Oper. Res. 28(2), 246–267 (2003)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Wen, B., Chen, X., Pong, T.K.: A proximal difference-of-convex algorithm with extrapolation. Comput. Optim. Appl. 69(2), 297–324 (2018)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature and Mathematical Optimization Society 2018

Authors and Affiliations

  1. 1.Department of MathematicsSimon Fraser UniversityBurnabyCanada
  2. 2.Department of MathematicsHong Kong Baptist UniversityKowloon TongHong Kong
  3. 3.College of Mathematics and Information ScienceJiangxi Normal UniversityNanchangChina

Personalised recommendations