Skip to main content

A trust-region approach to linearly constrained optimization

Part of the Lecture Notes in Mathematics book series (LNM,volume 1066)

Abstract

This paper suggests a class of trust-region algorithms for solving linearly constrained optimization problems. The algorithms use a “local” active-set strategy to select the steps they try. This strategy is such that degeneracy and zero Lagrange multipliers do not slow convergence (to a first-order stationary point) and that no anti-zigzagging precautions are necessary. (Unfortunately, when there are zero Lagrange multipliers, convergence to a point failing to satisfy second-order necessary conditions remains possible.) We discuss specialization of the algorithms to the case of simple bounds on the variables and report preliminary computational experience.

Keywords

  • Line Search
  • Trust Region
  • Unconstrained Optimization
  • Quadratic Programming Problem
  • Plane Rotation

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (Canada)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   34.99
Price excludes VAT (Canada)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   46.00
Price excludes VAT (Canada)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

10. References

  1. Bland, R. G. New finite pivoting rules for the simplex method. Math. Oper. Res. 2 (1977), 103–107.

    CrossRef  MathSciNet  MATH  Google Scholar 

  2. Cline, A. K., Conn, A. R. and Van Loan, C. F. Generalizing the LINPACK condition estimator. In Numerical Analysis, edited by J. P. Hennart, Lecture Notes in Mathematics 909, Springer-Verlag, Berlin, Heidelberg and New York, 1982.

    CrossRef  Google Scholar 

  3. Cline, A. K., Moler, C. B., Stewart, G. W. and Wilkinson, J. H. An estimate for the condition number of a matrix. SIAM J. Numer. Anal. 16 (1979), 368–375.

    CrossRef  MathSciNet  MATH  Google Scholar 

  4. Dennis, J. E., Gay, D. M. and Welsch, R. E. An adaptive nonlinear least-squares algorithm. ACM Trans. Math. Software 7 (1981), 348–368.

    CrossRef  MATH  Google Scholar 

  5. Dennis, J. E. and Mei, H. H-W. Two new unconstrained optimization algorithms which use function and gradient values. J. Optimization Theory Appl. 28 (1979), 453–482.

    CrossRef  MathSciNet  MATH  Google Scholar 

  6. Dennis, J. E. and Moré, J. J. Quasi-Newton methods, motivation and theory. SIAM Rev. 19 (1977), 46–89.

    CrossRef  MathSciNet  MATH  Google Scholar 

  7. Fletcher, R. Practical Methods of Optimization Vol. 2, Constrained Optimization, Wiley, Chichester and New York, 1981.

    MATH  Google Scholar 

  8. Gay, D. M. Computing optimal locally constrained steps. SIAM J. Sci. Statist. Comput. 2 (1981), 186–197.

    CrossRef  MathSciNet  MATH  Google Scholar 

  9. Gay, D. M. On Convergence Testing in Model/Trust-Region Algorithms for Unconstrained Optimization. Computing Science Technical Report No. 104, Bell Laboratories.

    Google Scholar 

  10. Gay, D. M. Subroutines for unconstrained minimization using a model/trust-region approach. To appear in ACM Trans. Math. Software.

    Google Scholar 

  11. Gill, P. E., Golub, G. H., Murray, W. and Saunders, M. A. Methods for modifying matrix factorizations. Math. Comput. 28 (1974), 505–535.

    CrossRef  MathSciNet  MATH  Google Scholar 

  12. Gill, P. E. and Murray, W. Newton-type methods for linearly constrained optimization. In Numerical Methods for Constrained Optimization, edited by P. E. Gill and W. Murray, Academic Press, London and New York, 1974.

    Google Scholar 

  13. Gill, P. E. and Murray, W. Minimization subject to bounds on the variables. NPL Report NAC 72, National Physical Laboratory, England, 1976.

    Google Scholar 

  14. Gill, P. E. and Murray, W. The computation of Lagrange-multiplier estimates for constrained optimization. NPL Report NAC 77, National Physical Laboratory, England, 1977.

    MATH  Google Scholar 

  15. Gill, P. E., Murray, W. and Wright, M. H. Practical Optimization. Academic Press, London and New York, 1981.

    MATH  Google Scholar 

  16. Goldfarb, D. Factorized variable metric methods for unconstrained optimization. Math. Comput. 30 (1976), 796–811.

    CrossRef  MathSciNet  MATH  Google Scholar 

  17. Kuhn, H. W. Nonlinear programming: a historical view. In Nonlinear Programming (SIAM-AMS Proceedings, vol. IX), edited by R. W. Cottle and C. E. Lemke, American Mathematical Society, Providence, R.I., 1976.

    Google Scholar 

  18. Lawson, C. L. and Hanson, R. J., Solving Least Squares Problems. Prentice-Hall, Englewood Cliffs, N.J., 1974.

    MATH  Google Scholar 

  19. McCormick, G. P. Optimality criteria in nonlinear programming. In Nonlinear Programming (SIAM-AMS Proceedings, vol. IX), edited by R. W. Cottle and C. E. Lemke, American Mathematical Society, Providence, R.I., 1976.

    Google Scholar 

  20. Moré, J. J. The Levenberg-Marquardt algorithm: implementation and theory. In Numerical Analysis, edited by G. A. Watson, Lecture Notes in Mathematics 630, Springer-Verlag, Berlin, Heidelberg and New York, 1978.

    CrossRef  Google Scholar 

  21. Moré, J. J., Garbow, B. S., and Hillstrom, K. E. Testing unconstrained optimization software. ACM Trans. Math. Software 7 (1981), 17–41.

    CrossRef  MathSciNet  MATH  Google Scholar 

  22. Moré, J. J., and Sorensen, D. C. Computing a Trust Region Step. Technical Report ANL-81-83, Applied Mathematics Division, Argonne National Lab.

    Google Scholar 

  23. NAG FORTRAN Library Manual Mark 9, vol. 3, Numerical Algorithms Group, Oxford, 1982.

    Google Scholar 

  24. Powell, M. J. D. A new algorithm for unconstrained optimization. In Nonlinear Programming, edited by J. B. Rosen, O. L. Mangasarian, and K. Ritter; Academic Press, New York, 1970.

    Google Scholar 

  25. Powell, M. J. D. A FORTRAN subroutine for unconstrained minimization, requiring first derivatives of the objective function. Report AERE-R.6469, A.E.R.E. Harwell, Oxon., England, 1970.

    Google Scholar 

  26. Powell, M. J. D. Quadratic termination properties of minimization algorithms I. Statement and discussion of results. J. Inst. Math. Applic. 10 (1972), 333–342.

    CrossRef  MathSciNet  MATH  Google Scholar 

  27. Rosen, J. B. The gradient projection method for non-linear programming, Part I: linear constraints. J. SIAM 8 (1960), 181–217.

    MATH  Google Scholar 

  28. Schnabel, R. B., Koontz, J. E., and Weiss, B. E. A modular system of algorithms for unconstrained minimization. Technical report CU-CS-240-82, Department of Computer Science, University of Colorado at Boulder.

    Google Scholar 

  29. Wolfe, P. A technique for resolving degeneracy in linear programming. J. SIAM 11 (1963), 205–211.

    MathSciNet  MATH  Google Scholar 

Download references

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 1984 Springer-Verlag

About this paper

Cite this paper

Gay, D.M. (1984). A trust-region approach to linearly constrained optimization. In: Griffiths, D.F. (eds) Numerical Analysis. Lecture Notes in Mathematics, vol 1066. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0099519

Download citation

  • DOI: https://doi.org/10.1007/BFb0099519

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-13344-5

  • Online ISBN: 978-3-540-38881-4

  • eBook Packages: Springer Book Archive