Skip to main content
SpringerLink
Log in

Partial spectral projected gradient method with active-set strategy for linearly constrained optimization

  • Original Paper
  • Published:
Numerical Algorithms Aims and scope Submit manuscript

Abstract

A method for linearly constrained optimization which modifies and generalizes recent box-constraint optimization algorithms is introduced. The new algorithm is based on a relaxed form of Spectral Projected Gradient iterations. Intercalated with these projected steps, internal iterations restricted to faces of the polytope are performed, which enhance the efficiency of the algorithm. Convergence proofs are given and numerical experiments are included and commented. Software supporting this paper is available through the Tango Project web page: http://www.ime.usp.br/∼egbirgin/tango/.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Abadie, J.: Modification of the GRG method. RAIRO Oper. Res. 3, 323–326 (1979)

    Google Scholar 

  2. Abadie, J., Guerrero, G.: The general reduced gradient method (GRG), the global Newton method and their application to mathematical programming. RAIRO Oper. Res. 18, 319–351 (1984)

    MATH  MathSciNet  Google Scholar 

  3. Andreani, R., Birgin, E.G., Martínez, J.M., Schuverdt, M.L.: On augmented Lagrangian methods with general lower-level constraints. SIAM J. Optim. 18, 1286–1309 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  4. Andreani, R., Birgin, E.G., Martínez, J.M., Schuverdt, M.L.: Augmented Lagrangian methods under the constant positive linear dependence constraint qualification. Math. Program. 111, 5–32 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  5. Andreani, R., Birgin, E.G., Martínez, J.M., Yuan, J.-Y.: Spectral projected gradient and variable metric methods for optimization with linear inequalities. IMA J. Numer. Anal. 25, 221–252 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  6. Andretta, M., Birgin, E.G., Martínez, J.M.: Practical active-set Euclidian trust-region method with spectral projected gradients for bound-constrained minimization. Optimization 54, 305–325 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  7. Andretta, M., Birgin, E.G., Martínez, J.M.: Partial spectral projected gradient method with active-set strategy for linearly constrained optimization. Technical Report MCDO090309 (see http://www.ime.usp.br/∼egbirgin/), Department of Applied Mathematics, UNICAMP, Brazil (2009)

  8. Barzilai, J., Borwein, J.M.: Two point step size gradient method. IMA J. Numer. Anal. 8, 141–148 (1988)

    Article  MATH  MathSciNet  Google Scholar 

  9. Bello, L., Raydan, M.: Convex constrained optimization for the seismic reflection tomography problem. J. Appl. Geophys. 62, 158–166 (2007)

    Article  Google Scholar 

  10. Bertsekas, D.P.: On the Goldstein-Levitin-Polyak gradient projection method. IEEE Trans. Automat. Contr. 21, 174–184 (1976)

    Article  MATH  MathSciNet  Google Scholar 

  11. Birgin, E.G., Biloti, R., Tygel, M., Santos, L.T.: Restricted optimization: a clue to a fast and accurate implementation of the common reflection surface method. J. Appl. Geophys. 42, 143–155 (1999)

    Article  Google Scholar 

  12. Birgin, E.G., Castillo, R., Martínez, J.M.: Numerical comparison of augmented Lagrangian algorithms for nonconvex problems. Comput. Optim. Appl. 31, 31–56 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  13. Birgin, E.G., Evtushenko, Y.G.: Automatic differentiation and spectral projected gradient methods for optimal control problems. Optim. Methods Softw. 10, 125–146 (1998)

    Article  MATH  MathSciNet  Google Scholar 

  14. Birgin, E.G., Martínez, J.M.: A box-constrained optimization algorithm with negative curvature directions and spectral projected gradients. Comput. [Suppl] 15, 49–60 (2001)

    Google Scholar 

  15. Birgin, E.G., Martínez, J.M.: Large-scale active-set box-constrained optimization method with spectral projected gradients. Comput. Optim. Appl. 23, 101–125 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  16. Birgin, E.G., Martínez, J.M.: Structured minimal-memory inexact quasi-Newton method and secant preconditioners for augmented Lagrangian optimization. Comput. Optim. Appl. 39, 1–16 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  17. Birgin, E.G., Martínez, J.M., Raydan, M.: Nonmonotone spectral projected gradient methods on convex sets. SIAM J. Optim. 10, 1196–1211 (2000)

    Article  MATH  MathSciNet  Google Scholar 

  18. Birgin, E.G., Martínez, J.M., Raydan, M.: Algorithm 813: SPG—software for convex-constrained optimization. ACM Trans. Math. Softw. 27, 340–349 (2001)

    Article  MATH  Google Scholar 

  19. Birgin, E.G., Martínez, J.M., Raydan, M.: Inexact spectral projected gradient methods on convex sets. IMA J. Numer. Anal. 23, 539–559 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  20. Bongartz, I., Conn, A.R., Gould, N.I.M., Toint, Ph.L.: CUTE: constrained and unconstrained testing environment. ACM Trans. Math. Softw. 21, 123–160 (1995)

    Article  MATH  Google Scholar 

  21. Conn, A.R., Gould, N.I.M., Sartenaer, A., Toint, Ph.L.: Convergence properties of an augmented Lagrangian algorithm for optimization with a combination of general equality and linear constraints. SIAM J. Optim. 6, 674–703 (1996)

    Article  MATH  MathSciNet  Google Scholar 

  22. Cores, D., Loreto, M.: A generalized two-point ellipsoidal anisotropic ray tracing for converted waves. Opt. Eng. 8, 373–396 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  23. Dai, Y.H.: Alternate step gradient method. Optimization 52, 395–415 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  24. Dai, Y.H., Fletcher, R.: Projected Barzilai-Borwein methods for large-scale box-constrained quadratic programming. Numer. Math. 100, 21–47 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  25. Dai, Y.H., Fletcher, R.: On the asymptotic behaviour of some new gradient methods. Math. Program. 103, 541–559 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  26. Dai, Y.H., Fletcher, R.: New algorithms for single linearly constrained quadratic programs subject to lower and upper bounds. Math. Program. 106, 403–421 (2005)

    Article  MathSciNet  Google Scholar 

  27. Dai, Y.H., Hager, W.W., Schittkowski, K., Zhang, H.C.: The cyclic Barzilai-Borwein method for unconstrained optimization. IMA J. Numer. Anal. 26, 604–627 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  28. Dai, Y.H., Zhang, H.C.: Adaptive two-point stepsize gradient algorithm. Numer. Algorithms 27, 377–385 (2001)

    Article  MATH  MathSciNet  Google Scholar 

  29. Deidda, G.P., Bonomi, E., Manzi, C.: Inversion of electrical conductivity data with Tikhonov regularization approach: some considerations. Ann. Geophys. 46, 549–558 (2003)

    Google Scholar 

  30. Diniz-Ehrhardt, M.A., Gomes-Ruggiero, M.A., Martínez, J.M., Santos, S.A.: Augmented Lagrangian algorithms based on the spectral projected gradient for solving nonlinear programming problems. J. Optim. Theory Appl. 123, 497–517 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  31. Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 101–213 (2002)

    Article  Google Scholar 

  32. Fletcher, R.: On the Barzilai-Borwein Method. Department of Mathematics, University of Dundee, NA/207, Dundee, Scotland (2001)

  33. Friedlander, A., Martínez, J.M., Molina, B., Raydan, M.: Gradient method with retards and generalizations. SIAM J. Numer. Anal. 36, 275–289 (1998)

    Article  MATH  Google Scholar 

  34. Gabay, D., Luenberger, D.G.: Efficiently converging minimization methods based on reduced gradient. SIAM J. Control 14, 42–61 (1976)

    Article  MATH  MathSciNet  Google Scholar 

  35. Gill, P.E., Golub, G.H., Murray, W., Saunders, M.A.: Methods for modifying matrix factorizations. Math. Comput. 28, 505–535 (1974)

    Article  MATH  MathSciNet  Google Scholar 

  36. Goldfarb, D., Idnani, A.: A numerically stable dual method for solving strictly convex quadratic programs. Math. Program. 27, 1–33 (1983)

    Article  MATH  MathSciNet  Google Scholar 

  37. Goldstein, A.A.: Convex programming in Hilbert space. Bull. Am. Math. Soc. 70, 709–710 (1964)

    Article  MATH  Google Scholar 

  38. Gomes-Ruggiero, M.A., Martínez, J.M., Santos, S.A.: Spectral projected gradient method with inexact restoration for minimization with nonconvex constraints. SIAM J. Sci. Comput. 31, 1628–1652 (2009)

    Article  MathSciNet  Google Scholar 

  39. Gonzaga, C.C., Karas, E., Vanti, M.: A globally convergent filter method for nonlinear programming. SIAM J. Optim. 14, 646–669 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  40. Grippo, L., Lampariello, F., Lucidi, S.: A nonmonotone line search technique for Newton’s method. SIAM J. Numer. Anal. 23, 707–716 (1986)

    Article  MATH  MathSciNet  Google Scholar 

  41. Grippo, L., Sciandrone, M.: Nonmonotone globalization techniques for the Barzilai-Borwein gradient method. Comput. Optim. Appl. 23, 143–169 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  42. Grippo, L., Sciandrone, M.: Nonmonotone derivative free methods for nonlinear equations. Comput. Optim. Appl. 37, 297–328 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  43. Jiang, Z.: Applications of conditional nonlinear optimal perturbation to the study of the stability and sensitivity of the Jovian atmosphere. Adv. Atmos. Sci. 23, 775–783 (2006)

    Article  Google Scholar 

  44. Júdice, J.J., Raydan, M., Rosa, S.S., Santos, S.A.: On the solution of the symmetric eigenvalue complementarity problem by the spectral projected gradient algorithm. Numer. Algorithms 47, 391–407 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  45. Levitin, E.S., Polyak, B.T.: Constrained minimization problems. USSR Comput. Math. Math. Phys. 6, 1–50 (1966)

    Article  Google Scholar 

  46. Martínez, J.M.: Inexact-restoration method with Lagrangian tangent decrease and new merit function for nonlinear programming. J. Optim. Theory Appl. 111, 39–58 (2001)

    Article  MATH  MathSciNet  Google Scholar 

  47. Martínez, J.M., Pilotta, E.A.: Inexact-restoration algorithm for constrained optimization. J. Optim. Theory Appl. 104, 135–163 (2000)

    Article  MATH  MathSciNet  Google Scholar 

  48. Martínez, J.M., Pilotta, E.A., Raydan, M.: Spectral gradient methods for linearly constrained optimization. J. Optim. Theory Appl. 125, 629–651 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  49. Murtagh, B.A., Saunders, M.A.: Large-scale linearly constrained optimization. Math. Program. 14, 41–72 (1978)

    Article  MATH  MathSciNet  Google Scholar 

  50. Powell, M.J.D.: On the quadratic programming algorithm of Goldfarb and Idnani. Math. Program. Stud. 25, 46–61 (1985)

    MATH  MathSciNet  Google Scholar 

  51. Powell, M.J.D.: A tolerant algorithm for linearly constrained optimization calculations. Math. Program. 45, 547–566 (1989)

    Article  MATH  MathSciNet  Google Scholar 

  52. Ramirez-Porras, A., Vargas-Castro, W.E.: Transmission of visible light through oxidized copper films: feasibility of using a spectral projected gradient method. Appl. Opt. 43, 1508–1514 (2004)

    Article  Google Scholar 

  53. Raydan, M.: On the Barzilai and Borwein choice of steplength for the gradient method. IMA J. Numer. Anal. 13, 321–326 (1993)

    Article  MATH  MathSciNet  Google Scholar 

  54. Raydan, M.: The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem. SIAM J. Optim. 7, 26–33 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  55. Raydan, M., Svaiter, B.F.: Relaxed steepest descent and Cauchy-Barzilai-Borwein method. Comput. Optim. Appl. 21, 155–167 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  56. Rosen, J.B.: The gradient projection method for nonlinear programming 2. Linear constraints. J. Soc. Ind. Appl. Math. 8, 181–217 (1960)

    Article  MATH  Google Scholar 

  57. Schittkowski, K.: QL: A Fortran Code for Convex Quadratic Programming. User’s Guide, Version 2.1 (2004)

  58. Serafini, T., Zanghirati, G., Zanni, L.: Gradient projection methods for quadratic programs and applications in training support vector machines. Optim. Methods Softw. 20, 347–372 (2005)

    MathSciNet  Google Scholar 

  59. Toint, Ph.L.: An assessment of non-monotone linesearch techniques for unconstrained optimization. SIAM J. Sci. Comput. 17, 725–739 (1996)

    Article  MATH  MathSciNet  Google Scholar 

  60. Vargas, W.E., Azofeifa, D.E., Clark, N.: Retrieved optical properties of thin films on absorbing substrates from transmittance measurements by application of a spectral projected gradient method. Thin Solid Films 425, 1–8 (2003)

    Article  Google Scholar 

  61. Wächter, A., Biegler, L.T.: On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Math. Program. 106, 25–57 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  62. Wang, C.Y., Liu, Q.: Convergence properties of inexact projected gradient methods. Optimization 55, 301–310 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  63. Wang, C.Y., Liu, Q., Yang, X.M.: Convergence properties of nonmonotone spectral projected gradient methods. J. Comput. Appl. Math. 182, 51–66 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  64. Yuan, Y.-X.: A new stepsize for the steepest descent method. J. Comput. Math. 24, 149–156 (2006)

    MATH  MathSciNet  Google Scholar 

  65. Zeev, N., Savasta, O., Cores, D.: Non-monotone spectral projected gradient method applied to full waveform inversion. Geophys. Prospect. 54, 525–534 (2006)

    Google Scholar 

  66. Zhang, H., Hager, W.W.: A nonmonotone line search technique and its application to unconstrained optimization. SIAM J. Optim. 14, 1043–1056 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  67. Zhang, L., Zhou, W.J.: Spectral gradient projection method for solving nonlinear monotone equations. J. Comput. Appl. Math. 196, 478–484 (2006)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Institute of Mathematics and Statistics, University of São Paulo, Rua do Matão 1010 Cidade Universitária, 05508-090, São Paulo, São Paulo, Brazil

    Marina Andretta & Ernesto G. Birgin

  2. Department of Applied Mathematics, IMECC-UNICAMP, CP 6065, 13081-970, Campinas, São Paulo, Brazil

    J. M. Martínez

Authors
  1. Marina Andretta
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ernesto G. Birgin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. J. M. Martínez
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ernesto G. Birgin.

Additional information

This work was supported by PRONEX-Optimization (PRONEX - CNPq / FAPERJ E-26 / 171.510/2006 - APQ1), FAPESP (Grant 2006/53768-0) and CNPq.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Andretta, M., Birgin, E.G. & Martínez, J.M. Partial spectral projected gradient method with active-set strategy for linearly constrained optimization . Numer Algor 53, 23–52 (2010). https://doi.org/10.1007/s11075-009-9289-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11075-009-9289-9

Keywords

Search

Navigation