Advertisement

Dual techniques for constrained optimization

  • W. W. Hager
Contributed Papers

Abstract

Algorithms to solve constrained optimization problems are derived. These schemes combine an unconstrained minimization scheme like the conjugate gradient method, an augmented Lagrangian, and multiplier updates to obtain global quadratic convergence. Since an augmented Lagrangian can be ill conditioned, a preconditioning strategy is developed to eliminate the instabilities associated with the penalty term. A criterion for deciding when to increase the penalty is presented.

Key Words

Constrained optimization duality augmented Lagrangians multiplier methods preconditioning null space methods 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Rosen, J. B., andKreuser, J. L.,A Gradient Projection Algorithm for Nonlinear Constraints, Numerical Methods for Nonlinear Optimization, Edited by F. A. Lootsma, Academic Press, London, England, pp. 297–300, 1972.Google Scholar
  2. 2.
    Rosen, J. B., andKreuser, J. L.,A Quadratically Convergent Lagrangian Algorithm for Nonlinear Constraints, University of Wisconsin, Computer Sciences Department, Report No. 166, Madison, Wisconsin, 1972.Google Scholar
  3. 3.
    Tapia, R. A.,Diagonalized Multiplier Methods and Quasi-Newton Methods for Constrained Optimization, Journal of Optimization Theory and Applications, Vol. 22, pp. 135–194, 1977.Google Scholar
  4. 4.
    Robinson, S. M.,A Quadratically Convergent Algorithm for General Nonlinear Programming Problems, Mathematical Programming, Vol. 3, pp. 145–156, 1972.Google Scholar
  5. 5.
    Miele, A., Pritchard, R. E., andDamoulakis, J. N.,Sequential Gradient-Restoration Algorithm for Optimal Control Problems, Journal of Optimization Theory and Applications, Vol. 5, pp. 235–282, 1970.Google Scholar
  6. 6.
    Miele, A., Damoulakis, J. N., Cloutier, J. R., andTietze, J. L.,Sequential Gradient-Restoration Algorithm for Optimal Control Problems with Nondifferential Constraints, Journal of Optimization Theory and Applications, Vol. 13, pp. 218–225, 1974.Google Scholar
  7. 7.
    Gonzalez, S., andMiele, A.,Sequential Gradient-Restoration Algorithm for Optimal Control Problems with General Boundary Conditions, Journal of Optimization Theory and Applications, Vol. 26, pp. 395–425, 1978.Google Scholar
  8. 8.
    Best, M. J., Bräuninger, J., Ritter, K., andRobinson, S. M.,A Globally and Quadratically Convergent Algorithm for General Nonlinear Programming Problems, University of Wisconsin, Mathematics Research Center, Report No. 1973, 1979.Google Scholar
  9. 9.
    Rosen, J. B.,Two-Phase Algorithm for Nonlinear Constraint Problems, University of Minnesota, Computer Science Department, Report No. 77–8, 1977.Google Scholar
  10. 10.
    Han, S. P.,Superlinearly Convergent Variable-Metric Algorithms for General Nonlinear Programming Problems, Mathematical Programming, Vol. 11, pp. 263–282, 1976.Google Scholar
  11. 11.
    Nocedal, J., andOverton, M.,Projected Hessian Updating Algorithms for Nonlinearly Constrained Optimization, SIAM Journal on Numerical Analysis, Vol. 22, pp. 821–850, 1985.Google Scholar
  12. 12.
    Powell, M. J. D.,Algorithms for Nonlinear Constraints That Use Lagrangian Functions, Mathematical Programming, Vol. 14, pp. 224–248, 1978.Google Scholar
  13. 13.
    Yamashita, H.,A Globally Convergent Constrained Quasi-Newton Method with an Augmented Lagrangian-Type Penalty Function, Mathematical Programming, Vol. 23, pp. 75–86, 1982.Google Scholar
  14. 14.
    Hager, W. W.,Dual Techniques for Constrained Optimization, Proceedings of the 24th Conference on Decision and Control, Fort Lauderdale, Florida, pp. 616–621, 1985.Google Scholar
  15. 15.
    Arrow, K. J., andSolow, R. M.,Gradient Methods for Constrained Maxima, with Weakened Assumptions, Studies in Linear and Nonlinear Programming, Edited by K. Arrow, L. Hurwicz, and H. Uzawa, Stanford University Press, Stanford, California, 1958.Google Scholar
  16. 16.
    Hestenes, M. R.,Multiplier and Gradient Methods, Journal of Optimization Theory and Applications, Vol. 4, pp. 303–320, 1969.Google Scholar
  17. 17.
    Powell, M. J. D.,A Method for Nonlinear Constraints in Minimization Problems, Optimization, Edited by R. Fletcher, Academic Press, New York, New York, 1972.Google Scholar
  18. 18.
    Hager, W. W.,Approximations to the Multiplier Method, SIAM Journal on Numerical Analysis, Vol. 22, pp. 16–46, 1985.Google Scholar
  19. 19.
    Polyak, B. T., andTret'yakov, N. V.,The Method of Penalty Estimates for Conditional Extremum Problems, Zhurnal Vychislitel'noi Matematiki i Matematicheskoi Fiziki, Vol. 13, pp. 34–46, 1973 (Translated into USSR Computational Mathematics and Mathematical Physics, Vol. 13, pp. 42–58, 1973).Google Scholar
  20. 20.
    Bertsekas, D. P.,Constrained Optimization and Lagrange Multiplier Methods, Academic Press, New York, New York, 1982.Google Scholar
  21. 21.
    Rockafellar, R. T.,Augmented Lagrange Multiplier Functions and Duality in Nonconvex Programming, SIAM Journal on Control, Vol. 12, pp. 268–285, 1974.Google Scholar
  22. 22.
    Rockafellar, R. T.,The Multiplier Method of Hestenes and Powell Applied to Convex Programming, Journal of Optimization Theory and Applications, Vol. 12, pp. 555–562, 1973.Google Scholar
  23. 23.
    Valentine, F. A.,The Problem of Lagrange with Differential Inequalities as Added Side Conditions, Contributions to the Calculus of Variations, University of Chicago Press, Chicago, Illinois, pp. 407–448, 1937.Google Scholar
  24. 24.
    Golub, G. H., andVan Loan, C. F.,Matrix Computations, Johns Hopkins University Press, Baltimore, Maryland, 1983.Google Scholar
  25. 25.
    Dongarra, J. J., Moler, C. B., Bunch, J. R., andStewart, G. W.,LINPACK User's Guide, Society for Industrial and Applied Mathematics, Philadelphia, Pennsylvania, 1979.Google Scholar
  26. 26.
    Hager, W. W.,Lipschitz Continuity for Constrained Processes, SIAM Journal on Control and Optimization, Vol. 17, pp. 321–338, 1979.Google Scholar
  27. 27.
    Luenberger, D. G.,Introduction to Linear and Nonlinear Programming, Addison-Wesley, Reading, Massachusetts, 1973.Google Scholar
  28. 28.
    Ben-Israel, A.,A Newton-Raphson Method for the Solution of Systems of Equations, Journal of Mathematical Analysis and Applications, Vol. 15, pp. 243–252, 1966.Google Scholar
  29. 29.
    Ortega, J. M., andRheinboldt, W. C.,Iterative Solution of Nonlinear Equations in Several Variables, Academic Press, New York, New York, 1970.Google Scholar
  30. 30.
    Cohen, A.,Rate of Convergence for Root Finding and Optimization Algorithms, University of California at Berkeley, PhD Thesis, 1970.Google Scholar
  31. 31.
    Akaike, H.,On a Successive Transformation of Probability Distribution and Its Application to the Analysis of the Optimum Gradient Method, Annals of the Institute of Statistical Mathematics, Vol. 11, pp. 1–16, 1959.Google Scholar
  32. 32.
    Loewner, C.,Uber Monotone Matrixfunktionen, Mathematische Zeitschrift, Vol. 38, pp. 177–216, 1934.Google Scholar
  33. 33.
    Mitchell, A. R., andGriffiths, D. F.,The Finite Difference Method in Partial Differential Equations, John Wiley, Chichester, England, 1980.Google Scholar
  34. 34.
    Ghosh, N., andHager, W. W.,A Derivative-Free Bracketing Scheme for Univariate Minimization (to appear).Google Scholar
  35. 35.
    Hager, W. W.,A Derivative-Based Bracketing Scheme for Univariate Minimization (to appear).Google Scholar
  36. 36.
    Polak, E.,Computational Methods in Optimization: A Unified Approach, Academic Press, New York, New York, 1971.Google Scholar
  37. 37.
    Davidon, W. C.,Variable-Metric Method for Minimization, Report No. 5990, Argonne National Laboratory, Lemont, Illinois, 1959.Google Scholar
  38. 38.
    Fletcher, R., andPowell, M. J. D.,A Rapidly Convergent Descent Method for Minimization, Computer Journal, Vol. 6, pp. 163–168, 1963.Google Scholar
  39. 39.
    Brezzi, F.,On the Existence, Uniqueness, and Approximation of Saddle-Point Problems Arising from Lagrange Multipliers, RAIRO Analyse Numérique, Vol. 8, pp. 129–151, 1974.Google Scholar
  40. 40.
    Hager, W. W., andRostamian, R.,Optimal Coatings, Bang-Bang Controls, and Gradient Techniques, Optimal Control: Applications and Methods, Vol. 8, pp. 1–20, 1987.Google Scholar
  41. 41.
    Hager, W. W., andIanculescu, G. D.,Dual Approximations in Optimal Control, SIAM Journal on Control and Optimization, Vol. 22, pp. 423–465, 1984.Google Scholar

Copyright information

© Plenum Publishing Corporation 1987

Authors and Affiliations

  • W. W. Hager
    • 1
  1. 1.Department of MathematicsPennsylvania State UniversityUniversity Park

Personalised recommendations