A projected Lagrangian algorithm and its implementation for sparse nonlinear constraints

  • Bruce A. Murtagh
  • Michael A. Saunders
Chapter
First Online:
Received:
Revised:
Part of the Mathematical Programming Studies book series (MATHPROGRAMM, volume 16)

Abstract

An algorithm is described for solving large-scale nonlinear programs whose objective and constraint functions are smooth and continuously differentiable. The algorithm is of the projected Lagrangian type, involving a sequence of sparse, linearly constrained subproblems whose objective functions include a modified Lagrangian term and a modified quadratic penalty function.

The algorithm has been implemented in a general purpose FORTRAN program called MINOS/AUGMENTED. Some aspects of the implementation are described, and computational results are given for some nontrivial test problems.

The system is intended for use on problems whose Jacobian matrix is sparse. (Such problems usually include a large set of purely linear constraints.) The bulk of the data for a problem may be assembled using a standard linear-programming matrix generator. Function and gradient values for nonlinear terms are supplied by two user-written subroutines.

Future applications could include, some of the problems that are currently being solved in industry by the method of successive linear programming (SLP). We would expect the rate of convergence and the attainable accuracy to be better than that achieved by SLP, but comparisons are not yet available on problems of sinificant size.

One of the largest nonlinear programs solved by MINOS/AUGENTED involved about 850 constraints and 4000 variables, with a nonlinear objective function and 32 nonlinear constraints. From a cold start, about 6000 iterations and 1 hour of computer time were required on a DEC VAX 11/780.

Key words

Large-scale Optimization Optimization Software Nonlinear Programming Projected Lagrangian 
This is a preview of subscription content, log in to check access.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    J. Abadie and J. Carpentier, “Generalization of the Wolfe reduced gradient method to the case of nonlinear constraints”, in: R. Fletcher, ed., Optimization (Academic Press, London, 1969) pp. 37–49.Google Scholar
  2. [2]
    J. Abadie and J. Guigou, “Numerical experiments with the GRG method”, in: J. Abadie, ed., Integer and nonlinear programming (North-Holland, Amsterdam, 1970) pp. 529–536.Google Scholar
  3. [3]
    K. J. Arrow and R. M. Solow, “Gradient methods for constrained maxima, with weakened assumptions”, in: K. J. Arrow, L. Hurwicz, and H. Uzawa, eds., Studies in linear and nonlinear programming (Stanford University Press, Stanford, CA, 1958) pp. 166–176.Google Scholar
  4. [4]
    T. E. Baker and R. Ventker, “Successive linear programming in refinery logistic models”, presented at ORSA/TIMS joint national meeting, Colorado Springs, CO (November 1980).Google Scholar
  5. [5]
    A.S.J. Batchelor and E.M.L. Beale, “A revised method of conjugate gradient approximation programming”, presented to the Ninth International Symposium on Mathematical Programming (Budapest, 1976).Google Scholar
  6. [6]
    E.M.L. Beale, “A conjugate gradient-method of approximation programming”, in: R.W. Cottle and J. Krarup, eds., Optimization methods for resource allocation, (English Universities Press, 1974) pp. 261–277.Google Scholar
  7. [7]
    E. M. L. Beale, “Nonlinear programming using a general Mathematical Programming System”, in: H. J. Greenberg, ed., Design and implementation of optimization software (Sijthoff and Noordhoff, The Netherlands, 1978) pp. 259–279.Google Scholar
  8. [8]
    J. F. Benders, “Partitioning procedures for solving mixed variables programming problems”, Numerische Mathematik 4 (1962) 238–252.MATHCrossRefMathSciNetGoogle Scholar
  9. [9]
    M. J. Best, J. Bräuninger, K. Ritter and S. M. Robinson, “A globally and quadratically convergent algorithm for general nonlinear programming problems”, Computing 26 (1981) 141–153.MATHCrossRefMathSciNetGoogle Scholar
  10. [10]
    M. C. Biggs and M. A. Laughton, “Optimal electric power scheduling: a large nonlinear test problem solved by recursive quadratic programming”, Mathematical Programming 13 (1977) 167–182.MATHCrossRefMathSciNetGoogle Scholar
  11. [11]
    J. Brancken and G. P. McCormick, Selected applications for unconstrained and linearly constrained optimization (Wiley, New York, 1968) pp. 58–82.Google Scholar
  12. [12]
    A. R. Colville, “A comparative study on nonlinear programming codes”, Report 320-2949 (1968), IBM New York Scientific Center, New York.Google Scholar
  13. [13]
    I. D. Coope and R. Fletcher, “Some numerical experience with a globally convergent algorithm for nonlinearly constrained optimization”, Journal of Optimization Theory and Applications 32(1) (1980) 1–16.MATHCrossRefMathSciNetGoogle Scholar
  14. [14]
    G. B. Dantzig, Linear programming and extensions (Princeton University Press, New Jersey, 1963).MATHGoogle Scholar
  15. [15]
    R. S. Dembo, “A set of geometric programming test problems and their solutions”, Mathematical Programming 10 (1976) 192–213.MATHCrossRefMathSciNetGoogle Scholar
  16. [16]
    L. F. Escudero, “A projected Lagrangian method for nonlinear, programming”, Report G320-3401 (1980), IBM Palo Alto Scientific Center, CA.Google Scholar
  17. [17]
    U. M. Garcia-Palomares and O. L. Mangasarian, “Superlinearly convergent quasi-Newton algorithms for nonlinearly constrained optimization problems”, Mathematical Programming 11 (1976) 1–13.MATHCrossRefMathSciNetGoogle Scholar
  18. [18]
    P. E. Gill and W. Murray, “Newton-type methods for unconstrained and linearly constrained optimization”, Mathematical Programming 7 (1974) 311–350.MATHCrossRefMathSciNetGoogle Scholar
  19. [19]
    P. E. Gill and W. Murray, “Safeguarded steplength algorithms for optimization using descent methods”, Report NAC 37 (1974). National Physical Laboratory, Teddington, England.Google Scholar
  20. [20]
    P. E Gill and W. Murray, Private communication (1975), National Physical Laboratory, Teddington, England.Google Scholar
  21. [21]
    P. E. Gill, W. Murray, M. A. Saunders and M. H. Wright, “Two steplength algorithms for numerical optimization”, Report SOL 79-25 (1979), Department of Operations Research, Stanford University, CA (Revised 1981.)Google Scholar
  22. [22]
    H.J. Greenberg, Private communication, (1979), Department of Energy, Washington, DC.Google Scholar
  23. [23]
    R. E. Griffith and R. A. Stewart, “A nonlinear programming technique for the optimization of continuous processing systems” Management Science 7 (1961) 379–392.MATHCrossRefMathSciNetGoogle Scholar
  24. [24]
    E. Hellerman and D. C. Rarick, “The partitioned preassigned pivot procedure”, in: D. J. Rose and R. A. Willoughby, eds., Sparse matrices and their applications (Plenum Press, New York, 1972) pp. 67–76.Google Scholar
  25. [25]
    M. R. Hestenes, “Multiplier and gradient methods”, Journal of Optimization Theory and Applications 4 (1969) 303–320.MATHCrossRefMathSciNetGoogle Scholar
  26. [26]
    R. J. Hillestad and S. E. Jacobsen, “Reverse convex programming”, Research paper (1979), RAND Corporation, Santa Monica, CA, and Department of System Science, University of California, Los Angeles, CA.Google Scholar
  27. [27]
    A. Jain, L.S. Lasdon and M.A. Saunders, “An in-core nonlinear mathematical programming system for large sparse nonlinear programs”, presented at ORSA/TIMS joint national meeting, Miami, FL (November 1976).Google Scholar
  28. [28]
    C.D. Kolstad., Private communication (1980), Los Alamos Scientific Laboratory, New Mexico.Google Scholar
  29. [29]
    L. S. Lasdon, Optimization theory for large systems (Macmillan, New York, 1970).MATHGoogle Scholar
  30. [30]
    L. S. Lasdon and A. D. Waren, “Generalized reduced gradient sorftware for linearly and nonlinearly constrained problems”, in: H. J. Greenberg, ed., Design and implementation of optimization software (Sijthoff and Noordhoff, The Netherlands, 1978) pp. 363–396.Google Scholar
  31. [31]
    A.S. Manne, Private communication (1979), Stanford University, CA.Google Scholar
  32. [32]
    P.H. Merz, Private communication (1980), Chevron Research Company, Richmond, CA.Google Scholar
  33. [33]
    R. R. Meyer, “The validity of a family of optimization methods”, SIAM Journal on Control 8 (1970) 41–54.MATHCrossRefGoogle Scholar
  34. [34]
    W. Murray and M.H. Wright, “Projected Lagrangian methods based on trajectories of penalty and barrier functions”, Report SOL 78-23 (1978), Department of Operations Research Stanford University, CA.Google Scholar
  35. [35]
    B. A. Murtagh and M. A. Saunders, “Large-scale linearly constrained optimization”, Mathematical Programming 14 (1978) 41–72.MATHCrossRefMathSciNetGoogle Scholar
  36. [36]
    B. A. Murtagh and M. A. Saunders, “MINOS user’s guide”, Report SOL 77-9 (1977), Department of Operations Research Stanford University, CA.Google Scholar
  37. [37]
    B. A. Murtagh and M. A. Saunders, “MINOS/AUGMENTED user’s manual”, Report SOL 80-14 (1980), Department of Operations Research, Stanford University, CA.Google Scholar
  38. [38]
    D. A. Pierre and M. J. Lowe, Mathematical programming via augmented Lagrangians (Addison-Wesley, Reading, MA, 1975) pp. 239–241.MATHGoogle Scholar
  39. [39]
    M. J. D. Powell, “A method for nonlinear constraints in optimization problems”, in: R. Fletcher, ed., Optimization (Academic Press, New York, 1969) pp. 283–297.Google Scholar
  40. [40]
    M. J. D. Powell, “Algorithms for nonlinear constraints that use Lagrangian functions,” Mathematical Programming 14 (1978) 224–248.MATHCrossRefMathSciNetGoogle Scholar
  41. [41]
    M.J.D. Powell, “A fast, algorithm for nonlinearly constrained optimization calculations”, presented at the 1977 Dundee Conference on Numerical Analysis, Dundee, Scotland.Google Scholar
  42. [42]
    M. J. D. Powell, “Constrained optimization by a variable metric method”, Report 77/NA6 (1977), Department of Applied Mathematics and Theoretical Physics, Cambridge University, England.Google Scholar
  43. [43]
    P. V. Preckel, “Modules for use with MINOS/AUGMENTED in solving sequences of mathematical programs”, Report SOL 80-15 (1980), Department of Operations Research, Stanford University, CA.Google Scholar
  44. [44]
    P. S. Ritch, “Discrete optimal control with multiple constraints I: constraint separation and transformation technique”, Automatica 9 (1973) 415–429.CrossRefMathSciNetGoogle Scholar
  45. [45]
    S. M. Robinson, “A quadratically convergent algorithm for general nonlinear programming problems”, Mathematical Programming 3 (1972) 145–156.MATHCrossRefMathSciNetGoogle Scholar
  46. [46]
    J. B. Rosen, “Convex partition programming”, in: R. L. Graves and P. Wolfe, eds., Recent advances in mathematical programming (McGraw-Hill, New York, 1963) pp. 159–176.Google Scholar
  47. [47]
    J. B. Rosen, “Iterative solution of nonlinear optimal control problems”, SIAM Journal on Control 4 (1966) 223–244.MATHCrossRefGoogle Scholar
  48. [48]
    J. B. Rosen, “Two-phase algorithm for nonlinear constraint problems”, in: O. L. Mangasarian, R. R. Meyer and S. M. Robinson, eds., Nonlinear programming 3, (Academic Press, London, 1978) pp. 97–124.Google Scholar
  49. [49]
    B. C. Rush, J. Bracken and G. P. McCormick, “A nonlinear programming model for launch vehicle design and costing”, Operations Research 15 (1967) 185–210.CrossRefGoogle Scholar
  50. [50]
    R. W. H. Sargent and B. A. Murtagh, “Projection methods for nonlinear programming”, Mathematical Programming 4 (1973) 245–268.MATHCrossRefMathSciNetGoogle Scholar
  51. [51]
    M. A. Saunders, “A fast, stable implementation of the simplex method using Bartels-Golub updating”, in: J. R. Bunch and D. J. Rose, eds., Sparse matrix computations (Academic Press, New York, 1976) pp. 213–226.Google Scholar
  52. [52]
    M. A. Saunders, “MINOS system manual” Report SOL 77-31 (1977), Department of Operations Research, Stanford University, CA.Google Scholar
  53. [53]
    C. M. Shen and M. A. Laughton, “Determination of optimum power system operating conditions under constraints”, Proceedings of the Institute of Electrical Engineers 116 (1969) 225–239.CrossRefGoogle Scholar
  54. [54]
    P. Wolfe, “Methods of nonlinear programming”, in: J. Abadie, ed., Nonlinear programming (North-Holland, Amsterdam, 1967) pp. 97–131.Google Scholar
  55. [55]
    M. H. Wright, “Numerical methods for nonlinearly constrained optimization”, SLAC Report. No. 193 (1976), Stanford University, CA (Ph. D. Dissertation).Google Scholar
  56. [56]
    R. C. Burchett, Private communication (1981), General Electric Company, Schenectady, NY.Google Scholar

Copyright information

© The Mathematical Programming Society, Inc. 1982

Authors and Affiliations

  • Bruce A. Murtagh
    • 1
  • Michael A. Saunders
    • 2
  1. 1.The University of New South WalesSydneyAustralia
  2. 2.Stanford UniversityStanfordUSA

Personalised recommendations