Abstract
In this paper, we study the classical nonconvex linearly constrained optimization problem. Under some mild conditions, we obtain that the penalization sequence is nonincreasing and the sequence generated by our algorithm has finite length. Based on the assumption that the objective functions have Kurdyka–Lojasiewicz property, we prove the convergence of the algorithm. We also show the numerical efficiency of our method by the concrete applications in the areas of image processing and statistics.
Similar content being viewed by others
References
Attouch, H., Bolte, J.: On the convergence of the proximal algorithm for nonsmooth functions involving analytic features. Math. Program. 116, 5–16 (2009)
Attouch, H., Bolte, J., Redont, P., Soubeyran, A.: Proximal alternating minimization and projection methods for nonconvex problems: an approach based on the Kurdyka-Łojasiewicz inequality. Math. Oper. Res. 35, 438–457 (2010)
Attouch, H., Bolte, J., Svaiter, B.F.: Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel methods. Math. Program. 137, 91–129 (2013)
Auslender, A.: Asymptotic properties of the Fenchel dual functional and applications to decomposition problems. J. Optim. Theory Appl. 73, 427–449 (1992)
Auslender, A., Teboulle, M., Ben-Tiba, S.: Coupling the logarithmic-quadratic proximal method and the block nonlinear Gauss–Seidel algorithm for linearly constrained convex minimization. In: Thera, M., Tichastschke, R. (eds.) Lecture Notes in Economics and Mathematical Systems, vol. 477, pp. 35–47 (1998)
Bolte, J., Daniilidis, A., Lewis, A.: The Łojasiewicz inequality for nonsmooth subanalytic functions with applications to subgradient dynamical systems. SIAM J. Optim. 17, 1205–1223 (2006)
Bolte, J., Daniilidis, A., Lewis, A.: A nonsmooth Morse-Sard theorem for subanalytic functions. J. Math. Anal. Appl. 321, 729–740 (2006)
Bolte, J., Daniilidis, A., Lewis, A., Shiota, M.: Clarke subgradients of stratifiable functions. SIAM J. Optim. 18, 556–572 (2007)
Bolte, J., Sabach, S., Teboulle, M.: Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Math. Program. 146, 459–494 (2014)
Fukushima, M.: Application of the alternating direction method of multipliers to separable convex programming problems. Comput. Optim. Appl. 1, 93–111 (1992)
Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variatinal problems via finite-element approximations. Comput. Math. Appl. 2, 17–40 (1976)
Glowinski, R. (ed.): Numerical Methods for Nonlinear Variational Problems. Springer, New York (1984)
Han, D.R., He, H.J., Xu, L.L.: A proximal parallel splitting method for minimizing sum of convex functions with linear constraints. J. Comput. Appl. Math. 256, 36–51 (2014)
Han, D.R., Sun, W.Y.: A new modified Goldstein-Levitin-Polyak projection method for variational inequality problems. Comput. Math. Appl. 47, 1817–1825 (2005)
Han, D.R., Yuan, X.M., Zhang, W.X.: An augmented-lagrangian-based parallel splitting method for separable convex minimization with applications to image processing. Math. Comp. 83, 2263–2291 (2014)
Han, D.R., Yuan, X.M., Zhang, W.X., Cai, X.J.: An ADM-based splitting method for separable convex programming. Comput. Optim. Appl. 54, 343–369 (2013)
He, B.S.: A Goldsteins type projection method for a class of variant variational inequalities. J. Comput. Math. 17, 425–434 (1999)
He, B.S., Liao, L.Z., Han, D.R., Yang, H.: A new inexact alternating direction method for monotone variational inequalities. Math. Program. 92, 103–118 (2002)
He, B.S., Liao, L.Z., Qian, M.J.: Alternating projection based prediction-correction method for structured variatinoal inequalities. J. Comput. Math. 24, 693–710 (2006)
He, B.S., Yang, H.: Some convergence properties of a method of multipliers for linearly constrained monotone variational inequalities. Oper. Res. Lett. 23, 151–161 (1998)
He, B.S., Yang, H., Wang, S.L.: Alternating direction method with self-adaptive penalty parameters for monotone variational inequalities. J. Optim. Theory Appl. 106, 337–356 (2000)
Łojasiewicz, S.: Une propriété topologique des sous-ensembles analytiques réels. Les Équatioons aux Dérivées Partielles, pp. 87–89. Éditions du centre National de la Recherche Scientifique, Paris (1963)
Łojasiewicz, S.: Sur la géométrie semi- et sous-analytique. Ann. Inst. Fourier 43, 1575–1595 (1993)
Martinet, B.: Regularisation d\(^{\prime }\)inequations variationelles par approximations succesives. Revue Francaise d’Informatique et de Recherche operationnelle 4, 154–159 (1970)
Nocedal, J., Wright, S.J. (eds.): Numerical Optimization, 2nd edn. Spriger, New York (2006)
Rockafellar, R.T.: Monotone operators and the proximal point algorithm. SIAM J. Control Optim. 14, 877–898 (1976)
Teboulle, M.: Convergence of proximal-like algorithms. SIAM J. Optim. 7, 1069–1083 (1997)
Tseng, P.: Convergence of a block coordinate descent method for nondifferentiable minimization. J. Optim. Theory Appl. 109, 475–494 (2001)
Wang, X.Y., Li, S.J., Kou, X.P.: An extension of subgradient method for variational inequality problems in Hilbert space. Abstr. Appl. Anal (2013). doi:10.1155/2013/531912
Wang, Y.J., Xiu, N.H., Zhang, J.Z.: Modified extragradient method for variational inequalities and verification of solution existance. J. Optim. Theory Appl. 119, 167–183 (2003)
Xue, D., Sun, W.Y., Qi, L.Q.: An alternating structured trust region algorithm for separable optimization problems with nonconvex constraints. Comput. Optim. Appl. 57, 365–386 (2014)
Yau, A.C., Tai, X.C., Ng, M.K.: \(L_o\)-norm and total variation for wavelet inpainting. In: Tai, X.C., et al. (eds.) Scale Space and Variational Method in Computer Vision, pp. 539–551. Springer, Voss (2009)
Zhang, W.X., Han, D.R., Yuan, X.M.: An efficient simultaneous method for the constrained multiple-sets split feasibility problem. Comput. Optim. Appl. 52, 825–843 (2012)
Acknowledgments
This research was supported by the National Natural Science Foundation of China (Grants number: 11171362, 11301567 and 11401058), Specialized Research Fund for the Doctoral Program of Higher Education (Grant number: 20120191110031), the Technology Project of Chongqing Education Committee (Grant number: KJ130732), and the Research Start Project of Chongqing Technology and Business University (Grant number: 2012-56-04). The authors thank the anonymous reviewers for their valuable comments and suggestions, which helped to improve the paper.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Wang, X.Y., Li, S.J., Kou, X.P. et al. A new alternating direction method for linearly constrained nonconvex optimization problems. J Glob Optim 62, 695–709 (2015). https://doi.org/10.1007/s10898-015-0268-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10898-015-0268-5