Advertisement

Computational Optimization and Applications

, Volume 68, Issue 2, pp 407–436 | Cite as

Peaceman–Rachford splitting for a class of nonconvex optimization problems

  • Guoyin Li
  • Tianxiang Liu
  • Ting Kei Pong
Article

Abstract

We study the applicability of the Peaceman–Rachford (PR) splitting method for solving nonconvex optimization problems. When applied to minimizing the sum of a strongly convex Lipschitz differentiable function and a proper closed function, we show that if the strongly convex function has a large enough strong convexity modulus and the step-size parameter is chosen below a threshold that is computable, then any cluster point of the sequence generated, if exists, will give a stationary point of the optimization problem. We also give sufficient conditions guaranteeing boundedness of the sequence generated. We then discuss one way to split the objective so that the proposed method can be suitably applied to solving optimization problems with a coercive objective that is the sum of a (not necessarily strongly) convex Lipschitz differentiable function and a proper closed function; this setting covers a large class of nonconvex feasibility problems and constrained least squares problems. Finally, we illustrate the proposed algorithm numerically.

Keywords

Peaceman–Rachford splitting Feasibility problems Nonconvex optimization problems Global convergence 

References

  1. 1.
    Attouch, H., Bolte, J., Redont, P., Soubeyran, A.: Proximal alternating minimization and projection methods for nonconvex problems. An approach based on the Kurdyka–Łojasiewicz inequality. Math. Oper. Res. 35, 438–457 (2010)MathSciNetCrossRefMATHGoogle Scholar
  2. 2.
    Attouch, H., Bolte, J., Svaiter, B.F.: Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss–Seidel methods. Math. Program. 137, 91–129 (2013)MathSciNetCrossRefMATHGoogle Scholar
  3. 3.
    Bauschke, H.H., Borwein, J.M.: On the convergence of von Neumann’s alternating projection algorithm for two sets. Set-Valued Anal. 1, 185–212 (1993)MathSciNetCrossRefMATHGoogle Scholar
  4. 4.
    Bauschke, H.H., Borwein, J.M.: On projection algorithms for solving convex feasibility problems. SIAM Rev. 38, 367–426 (1996)MathSciNetCrossRefMATHGoogle Scholar
  5. 5.
    Bauschke, H.H., Combettes, P.L.: Convex Analysis and Monotone Operator Theory in Hilbert Spaces. Springer, Berlin (2011)CrossRefMATHGoogle Scholar
  6. 6.
    Bogdan, M., van den Berg, E., Su, W., Candès, E.: Statistical estimation and testing via the sorted L1 norm. Preprint (2013). Available at arxiv:1310.1969
  7. 7.
    Bolte, J., Daniilidis, A., Lewis, A.: The Łojasiewicz inequality for nonsmooth subanalytic functions with applications to subgradient dynamical systems. SIAM J. Optim. 17, 1205–1223 (2007)CrossRefMATHGoogle Scholar
  8. 8.
    Bolte, J., Daniilidis, A., Lewis, A., Shiota, M.: Clarke subgradients of stratifiable functions. SIAM J. Optim. 18, 556–572 (2007)MathSciNetCrossRefMATHGoogle Scholar
  9. 9.
    Borwein, J.M., Li, G., Yao, L.J.: Analysis of the convergence rate for the cyclic projection algorithm applied to basic semialgebraic convex sets. SIAM J. Optim. 24, 498–527 (2014)MathSciNetCrossRefMATHGoogle Scholar
  10. 10.
    Bruckstein, A.M., Donoho, D.L., Elad, M.: From sparse solutions of systems of equations to sparse modeling of signals and images. SIAM Rev. 51, 34–81 (2009)MathSciNetCrossRefMATHGoogle Scholar
  11. 11.
    Candès, E., Tao, T.: The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). Ann. Statist. 35, 2313–2351 (2007)MathSciNetCrossRefMATHGoogle Scholar
  12. 12.
    Combettes, P.L.: Iterative construction of the resolvent of a sum of maximal monotone operators. J. Convex Anal. 16, 727–748 (2009)MathSciNetMATHGoogle Scholar
  13. 13.
    Combettes, P.L., Pesquet, J.-C.: A Douglas–Rachford splitting approach to nonsmooth convex variational signal recovery. IEEE J. Sel. Top. Signal Process. 1(4), 564–574 (2007)CrossRefGoogle Scholar
  14. 14.
    Dobra, A.: Variable selection and dependency networks for genomewide data. Biostatistics 10, 621–639 (2009)CrossRefGoogle Scholar
  15. 15.
    Douglas, J., Rachford, H.H.: On the numerical solution of heat conduction problems in two or three space variables. Trans. Am. Math. Soc. 82, 421–439 (1956)MathSciNetCrossRefMATHGoogle Scholar
  16. 16.
    Eckstein, J., Bertsekas, D.P.: On the Douglas–Rachford splitting method and the proximal point algorithm for maximal monotone operators. Math. Program. 55, 293–318 (1992)MathSciNetCrossRefMATHGoogle Scholar
  17. 17.
    Fan, J., Li, R.: Variable selection via nonconcave penalized likelihood and its oracle properties. J. Am. Stat. Assoc. 96, 1348–1360 (2001)MathSciNetCrossRefMATHGoogle Scholar
  18. 18.
    Giselsson, P., Boyd, S.: Diagonal scaling in Douglas–Rachford splitting and ADMM. In: Proceedings of the 53rd IEEE Conference on Decision and Control, pp. 5033–5039 (2014)Google Scholar
  19. 19.
    Golub, T.R., Slonim, D.K., Tamayo, P., Huard, C., Gaasenbeek, M., Mesirov, J.P., Coller, H., Loh, M.L., Downing, J.R., Caligiuri, M.A., Bloomfield, C.D., Lander, E.S.: Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. Science 286, 531–537 (1999)CrossRefGoogle Scholar
  20. 20.
    Hesse, R., Luke, D.R.: Nonconvex notions of regularity and convergence of fundamental algorithms for feasibility problems. SIAM J. Optim. 23, 2397–2419 (2013)MathSciNetCrossRefMATHGoogle Scholar
  21. 21.
    Hong, M., Luo, Z.-Q., Razaviyayn, M.: Convergence analysis of alternating direction method of multipliers for a family of nonconvex problems. SIAM J. Optim. 26, 337–364 (2016)MathSciNetCrossRefMATHGoogle Scholar
  22. 22.
    Knight, K., Fu, W.: Asymptotics for the lasso-type estimators. Ann. Stat. 28, 1356–1378 (2000)MathSciNetCrossRefMATHGoogle Scholar
  23. 23.
    Kyrillidis, A., Becker, S., Cevher, V., Koch, C.: Sparse projections onto the simplex. JMLR W&CP 28, 235–243 (2013)Google Scholar
  24. 24.
    Li, G., Pong, T.K.: Global convergence of splitting methods for nonconvex composite optimization. SIAM J. Optim. 25, 2434–2460 (2015)MathSciNetCrossRefMATHGoogle Scholar
  25. 25.
    Li, G., Pong, T.K.: Douglas–Rachford splitting for nonconvex optimization with application to nonconvex feasibility problems. Math. Program. 159, 371–401 (2016)MathSciNetCrossRefMATHGoogle Scholar
  26. 26.
    Lions, P.L., Mercier, B.: Splitting algorithms for the sum of two nonlinear operators. SIAM J. Numer. Anal. 16, 964–979 (1979)MathSciNetCrossRefMATHGoogle Scholar
  27. 27.
    Lu, Z., Pong, T.K., Zhang, Y.: An alternating direction method for finding Dantzig selectors. Comput. Stat. Data Anal. 56, 4037–4046 (2012)MathSciNetCrossRefMATHGoogle Scholar
  28. 28.
    Luke, D.R.: Finding best approximation pairs relative to a convex and a prox-regular set in a Hilbert space. SIAM J. Optim. 19, 714–739 (2008)MathSciNetCrossRefMATHGoogle Scholar
  29. 29.
    Patrinos, P., Stella, L., Bemporad, A.: Douglas–Rachford splitting: complexity estimates and accelerated variants. In: Proceedings of the 53rd IEEE Conference on Decision and Control, pp. 4234–4239 (2014)Google Scholar
  30. 30.
    Peaceman, D.W., Rachford, H.H.: The numerical solution of parabolic and elliptic differential equations. J. Soc. Ind. Appl. Math. 3, 28–41 (1955)MathSciNetCrossRefMATHGoogle Scholar
  31. 31.
    Rockafellar, R.T., Wets, R.J.-B.: Variational Analysis. Springer, Berlin (1998)CrossRefMATHGoogle Scholar
  32. 32.
    Tibshirani, R.: Regression shrinkage and selection via the lasso. J. R. Stat. Soc. Ser B 58, 267–288 (1996)MathSciNetMATHGoogle Scholar
  33. 33.
    Wang, H., Banerjee, A.: Bregman alternating direction method of multipliers. NIPS 27, 2816–2824 (2014)Google Scholar
  34. 34.
    Yeung, K.Y., Bumgarner, R.E., Raftery, A.E.: Bayesian model averaging: development of an improved multi-class, gene selection and classification tool for microarray data. Bioinformatics 21, 2394–2402 (2005)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2017

Authors and Affiliations

  1. 1.Department of Applied MathematicsUniversity of New South WalesSydneyAustralia
  2. 2.Department of Applied MathematicsThe Hong Kong Polytechnic UniversityHung HomHong Kong

Personalised recommendations