Skip to main content
SpringerLink
Log in

An alternating structured trust region algorithm for separable optimization problems with nonconvex constraints

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

In this paper, we propose a structured trust-region algorithm combining with filter technique to minimize the sum of two general functions with general constraints. Specifically, the new iterates are generated in the Gauss-Seidel type iterative procedure, whose sizes are controlled by a trust-region type parameter. The entries in the filter are a pair: one resulting from feasibility; the other resulting from optimality. The global convergence of the proposed algorithm is proved under some suitable assumptions. Some preliminary numerical results show that our algorithm is potentially efficient for solving general nonconvex optimization problems with separable structure.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Burke, J.V., Moré, J.J.: On the identification of active constraints. SIAM J. Numer. Anal. 25, 1197–1211 (1988)

    Article  MATH  MathSciNet  Google Scholar 

  2. Byrd, R.H., Schnabel, R.B., Schultz, G.A.: A trust region algorithm for nonlinearly constrained optimization. SIAM J. Numer. Anal. 24, 1152–1170 (1987)

    Article  MATH  MathSciNet  Google Scholar 

  3. Chen, Y., Gazzale, R.: When does learning in games generate convergence to Nash equilibria? The role of supermodularity in an experimental setting. Am. Econ. Rev. 9, 1505–1535 (2004)

    Article  Google Scholar 

  4. Chen, G., Teboulle, M.: A proximal-based decomposition method for convex minimization problems. Math. Program. 64, 81–101 (1994)

    Article  MATH  MathSciNet  Google Scholar 

  5. Chin, C., Fletcher, R.: On the global convergence of an SLP-filter algorithm that takes EQP steps. SIAM J. Optim. 96, 161–177 (2003)

    MATH  MathSciNet  Google Scholar 

  6. Conn, A.R., Gould, N.I.M., Toint, P.L.: LANCELOT: A Fortran Package for Large-Scale Nonlinear Optimization (Release A). Springer Series in Computational Mathematics, vol. 17. Springer, New York (1992)

    Book  MATH  Google Scholar 

  7. Conn, A.R., Gould, N.I.M., Sartenaer, A., Toint, P.L.: Convergence properties of minimization algorithms for convex constraints using a structured trust region. SIAM J. Optim. 6, 1059–1086 (1996)

    Article  MATH  MathSciNet  Google Scholar 

  8. Conn, A.R., Gould, N.I.M., Toint, P.L.: Trust Region Methods. MPS-SIAM, Philadelphia (2000)

    Book  MATH  Google Scholar 

  9. da Cruz Neto, J.X., Oliveira, P.R., Soares, P.A. Jr, Soubeyran, A.: Learning how to play Nash, potential games and alternating minimization method for structured nonconvex problems on Riemannian manifolds. http://www.optimization-online.org/DB_HTML/2012/03/3405.html. Retrieved April 2, 2012

  10. Eckstein, J., Bertsekas, D.P.: On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators. Math. Program. 55, 293–318 (1992)

    Article  MATH  MathSciNet  Google Scholar 

  11. Elalem, M.: A global convergence theory for the Dennis-Celis-Tapia trust-region algorithm for constrained optimization. SIAM J. Numer. Anal. 28, 266–290 (1991)

    Article  MathSciNet  Google Scholar 

  12. Figueiredo, M.A.T., Bioucas-Dias, J.M.: Restoration of Poissonian images using alternating direction optimization. IEEE Trans. Image Process. 19, 3133–3145 (2010)

    Article  MathSciNet  Google Scholar 

  13. Fletcher, R., Leyffer, S.: Nonlinear programming without a penalty function. Math. Program. 91, 239–269 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  14. Fletcher, R., Leyffer, S., Toint, P.L.: On the global convergence of a filter-SQP algorithm. SIAM J. Optim. 13, 44–59 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  15. Fletcher, R., Gould, N.I.M., Leyffer, S., Toint, P.L., Wächter, A.: Global convergence of a trust-region SQP-filter algorithm for general nonlinear programming. SIAM J. Optim. 13, 635–659 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  16. Forero, P.A., Cano, A., Giannakis, G.B.: Consensus-based distributed support vector machines. J. Mach. Learn. Res. 11, 1663–1707 (2010)

    MATH  MathSciNet  Google Scholar 

  17. Fortin, M., Glowinski, R.: Augmented Lagrangian Methods: Applications to the Numerical Solution of Boundary-Value Problems. North-Holland, Amsterdam (1983)

    MATH  Google Scholar 

  18. Fukushima, M.: Application of the alternating direction method of multipliers to separable convex programming problems. Comput. Optim. Appl. 1, 93–111 (1992)

    Article  MATH  MathSciNet  Google Scholar 

  19. Gabay, D.: Applications of the method of multipliers to variational inequalities. In: Fortin, M., Glowinski, R. (eds.) Augmented Lagrangian Methods: Applications to the Numerical Solution of Boundary-Value Problems, pp. 299–331. North-Holland, Amsterdam (1983)

    Chapter  Google Scholar 

  20. Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite element approximations. Comput. Math. Appl. 2, 17–40 (1976)

    Article  MATH  Google Scholar 

  21. Gao, D.Y.: Duality Principles in Nonconvex Systems: Theory, Methods and Applications. Kluwer Academic, Dordrecht (2000)

    Book  Google Scholar 

  22. Gao, D.Y., Yuan, N., Sherali, H.D.: Solutions and optimality criteria for nonconvex constrained global optimization problems with connections between canonical and Lagrangian duality. J. Glob. Optim. 45, 473–497 (2009)

    Article  MATH  Google Scholar 

  23. Glowinski, R., Marrocco, A.: Sur l’approximation, par elements finis d’ordre un, et la resolution, par penalisation-dualité, d’une classe de problems de Dirichlet non lineares. Rev. Fr. Autom. Inform. Rech. Opér. 9, 41–76 (1975)

    MATH  MathSciNet  Google Scholar 

  24. Gould, N.I.M., Sainvitu, C., Toint, Ph.L.: A filter-trust-region method for unconstrained optimization. SIAM J. Optim. 16, 341–357 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  25. Griewank, A., Toint, P.L.: On the unconstrained optimization of partially separable functions. In: Powell, M.J.D. (ed.) Nonlinear Optimization 1981, pp. 301–312. Academic Press, London (1982)

    Google Scholar 

  26. Haber, E., Ascher, U., Oldenburg, D.: On optimization techniques for solving nonlinear inverse problems. Inverse Probl. 16, 1264–1280 (2000)

    Article  MathSciNet  Google Scholar 

  27. Hager, W.W., Zhang, H.C.: Self-adaptive inexact proximal point methods. Comput. Optim. Appl. 39, 161–181 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  28. He, B.S., Liao, L.Z., Han, D.R., Yang, H.: A new inexact alternating direction method for monotone variational inequalities. Math. Program. 92, 103–118 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  29. Huang, H.Y., Aggerwal, A.K.: A class of quadratically convergent algorithms for constrained function minimization. J. Optim. Theory Appl. 16, 447–485 (1975)

    Article  MATH  Google Scholar 

  30. Kim, H., Park, H.: Nonnegative matrix factorization based on alternating nonnegativity constrained least squares and active set method. SIAM J. Matrix Anal. Appl. 30, 713–730 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  31. Kontogiorgis, S., Meyer, R.R.: A variable-penalty alternating directions method for convex optimization. Math. Program. 83, 29–53 (1998)

    MATH  MathSciNet  Google Scholar 

  32. Li, C., Sun, W.: On filter-successive linearization methods for nonlinear semidefinite programming. Sci. China Ser. A, Math. 52, 2341–2361 (2009)

    Article  MATH  Google Scholar 

  33. Li, C., Sun, W.: On filter-successive semidefinite programming methods for nonlinear semidefinite programming. Phys. Mech. Astron. 11, 23–41 (2009)

    Article  MATH  Google Scholar 

  34. Miao, W., Sun, W.: A filter trust-region method for unconstrained optimization problems. Numer. Math. J. Chin. Univ. 29, 88–96 (2007)

    MATH  MathSciNet  Google Scholar 

  35. Miele, A., Huang, H.Y., Heideman, J.C.: Sequential gradient-restoration algorithm for the minimization of constrained functions, ordinary and conjugate gradient versions. J. Optim. Theory Appl. 4, 213–243 (1969)

    Article  MATH  MathSciNet  Google Scholar 

  36. Murtagh, B.A., Saunders, M.A.: MINOS5.0 Users Guide. Systems Optimization Laboratory, Dept. of Operations Research, Stanford University, CA (1983). Appendix A: MINOS5.0, Technical Report SOL 83-20

  37. Nocedal, J., Wright, S.: Numerical Optimization. Springer, New York (1999)

    Book  MATH  Google Scholar 

  38. Powell, M.J.D., Yuan, Y.: A trust region algorithm for equality constrained optimization. Math. Program. 49, 189–213 (1990)

    Article  MATH  MathSciNet  Google Scholar 

  39. Sun, W., Yuan, Y.: Optimization Theory and Methods: Nonlinear Programming. Springer, New York (2006)

    Google Scholar 

  40. Sun, W., Hou, L., Dang, C.: A modified trust region method with Beale’s PCG technique for optimization. Comput. Optim. Appl. 40, 59–72 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  41. Toint, P.L.: On large scale nonlinear least squares calculations. SIAM J. Sci. Stat. Comput. 8, 416–435 (1987)

    Article  MATH  MathSciNet  Google Scholar 

  42. Toint, P.L.: Global convergence of a class of trust region methods for nonconvex minimization in Hilbert space. IMA J. Numer. Anal. 8, 231–252 (1988)

    Article  MATH  MathSciNet  Google Scholar 

  43. Tseng, P.: Alternating projection-proximal methods for convex programming and variational inequalities. SIAM J. Optim. 7, 951–965 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  44. Wen, Z.W., Goldfarb, D., Yin, W.T.: Alternating direction augmented Lagrangian methods for semidefinite programming. Math. Program. Comput. 2, 203–230 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  45. Xu, Y., Sun, W.: A filter successive linear programming method for nonlinear semidefinite programming problems. Numer. Algebra Control Optim. 2, 193–206 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  46. Yang, Z., Sun, W., Qi, L.: Global convergence of a filter-trust-region algorithm for solving nonsmooth equations. Int. J. Comput. Math. 87, 788–796 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  47. Yang, J.F., Zhang, Y.: Alternating direction algorithms for l 1-problems in compressive sensing. SIAM J. Sci. Comput. 332, 250–278 (2011)

    Article  Google Scholar 

  48. Zhang, Y., Sun, W., Qi, L.: A nonmonotone filter Barzilai-Borwein method for optimization. Asia-Pac. J. Oper. Res. 27, 55–69 (2010)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Mathematical Sciences, Jiangsu Key Laboratory for NSLSCS, Nanjing Normal University, Nanjing, 210046, China

    Dan Xue & Wenyu Sun

  2. Department of Applied Mathematics, The Hong Kong Polytechnic University, Kowloon, Hong Kong, China

    Liqun Qi

Authors
  1. Dan Xue
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wenyu Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Liqun Qi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wenyu Sun.

Additional information

This work is supported by the National Natural Science Foundation of China (Grants Nos. 11071122, 11171159), the Specialized Research Fund of Doctoral Program of Higher Education of China (Grant No. 20103207110002), the Research Grant Council of Hong Kong, China.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Xue, D., Sun, W. & Qi, L. An alternating structured trust region algorithm for separable optimization problems with nonconvex constraints. Comput Optim Appl 57, 365–386 (2014). https://doi.org/10.1007/s10589-013-9597-9

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-013-9597-9

Keywords

Search

Navigation