Skip to main content
SpringerLink
Log in
Book cover

Encyclopedia of Optimization pp 2230–2239Cite as

Rosen’s Method, Global Convergence, and Powell’s Conjecture

FSP

  • Reference work entry

  • 172 Accesses

The gradient projection method proposed by J.B. Rosen in 1960 [82], [83] is one of earliest methods in the history of mathematical programming for solving constrained optimization problems. The importance of this method in the literature also stems from the fact that many more efficient algorithms, in linear and nonlinear programming, developed later (e.g. by D. Goldfarb [42], by B.H. Murtagh and R.W.H. Sargent [63] and by N.K. Karmarkar [53]) incorporated the basic ideas propounded by Rosen.

The global convergence of Rosen’s method was a long-standing open problem. Since Rosen’s method is included in many textbooks, the convergence problem became quite well-known. In fact, almost all books (such as [3], [4], [57], [67]) that have a chapter or a section to introduce Rosen’s method recognize the problem on the global convergence of Rosen’s method. Through efforts of 26 years, the proof was finally found [35], [36], [47].

The study on the global convergence of Rosen’s method had a great...

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   1,699.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Allen, E., Helgason, R., and Kennington, J.: ‘A generalization of Polyak’s convergence result for subgradient optimization’, Math. Program.37 (1987), 309–318.

    MathSciNet  MATH  Google Scholar 

  2. Armijo, L.: ‘Minimization of functions having Lipschitz continuous first partial derivatives’, Pacific J. Math.16 (1966), 1–3.

    MathSciNet  MATH  Google Scholar 

  3. Avriel, M.:Nonlinear programming: Analysis and methods, Prentice-Hall, 1976.

    Google Scholar 

  4. Bazaraa, M.S., and Shetty, C.M.:Nonlinear programming: Theory and algorithms, Wiley, 1979.

    Google Scholar 

  5. Bland, R.G.: ‘New finite pivoting rules for the simplex method’, Math. Oper. Res.2, no. 2 (1977), 103–107.

    Article  MathSciNet  MATH  Google Scholar 

  6. Broyden, C.G.: ‘A class of methods for solving nonlinear simultaneous equations’, Math. Comput.19 (1965), 577–593.

    MathSciNet  MATH  Google Scholar 

  7. Broyden, C.G.: ‘Quasi-Newton methods and their application to function minimization’, Math. Comput.21 (1967), 368–381.

    MathSciNet  MATH  Google Scholar 

  8. Broyden, C.G.: ‘The convergence of a class of double-rank minimization algorithms 1: general consideration’, J. Inst. Math. Appl.6 (1970), 76–90.

    MathSciNet  MATH  Google Scholar 

  9. Broyden, C.G.: ‘The convergence of a class of double-rank minimization algorithms 2: the new algorithms’, J. Inst. Math. Appl.6 (1970), 222–231.

    MathSciNet  MATH  Google Scholar 

  10. Buckley, A.: ‘An alternate implementation of Goldfarb’s minimization algorithm’, Math. Program.8 (1975), 207–231.

    MathSciNet  MATH  Google Scholar 

  11. Byrd, R.H.: ‘An example of irregular convergence in some constrained optimization methods that use the projected Hessian’, Math. Program.32 (1985), 232–237.

    MathSciNet  MATH  Google Scholar 

  12. Byrd, R.H., Nocedal, J., and Yuan, Y.: ‘Global convergence of a class of quasi-Newton method on convex problems’, SIAM J. Numer. Anal.24 (1987), 1171–1190.

    MathSciNet  MATH  Google Scholar 

  13. Byrd, R.H., and Shultz, G.A.: ‘A practical class of globally convergent active set strategies for linearly constrained optimization’, Manuscript (unpublished).

    Google Scholar 

  14. Calamai, P.H., and More, J.J.: ‘Projected gradient methods for linearly constrained problems’, Math. Program.39 (1987), 93–116.

    MathSciNet  MATH  Google Scholar 

  15. Coleman, T.F., and Conn, A.R.: ‘On the local convergence of a quasi-Newton method for the nonlinear programming problem’, SIAM J. Numer. Anal.1 (1984), 755–769.

    MathSciNet  Google Scholar 

  16. Davidon, W.C.: ‘Variable metric method for minimization’, AEC Res. Developm. ReportANL-5990, no. nov (1959).

    Google Scholar 

  17. Davidon, W.C.: ‘Variable metric method for minimization’, Computer J.10 (1968), 406–410.

    MathSciNet  MATH  Google Scholar 

  18. Dax, A.: ‘The gradient projection method for quadratic programming’, Report Inst. Math. Hebrew Univ. (Jerusalem) (1978).

    Google Scholar 

  19. Dembo, R.S., and Klincewicz, J.G.: ‘Dealing with degeneracy in reduced gradient algorithms’, Math. Program.31 (1985), 257–363.

    MathSciNet  Google Scholar 

  20. Dixon, L.C.W.: ‘Quasi-Newton algorithms generate identical points’, Math. Program.2 (1972), 383–387.

    MathSciNet  MATH  Google Scholar 

  21. Dixon, L.C.W.: ‘Quasi-Newton algorithms generate identical points II, the proof of four new theorems’, Math. Program.3 (1972), 345–358.

    MathSciNet  MATH  Google Scholar 

  22. Du, D.-Z.: ‘A modification of Rosen-Polak’s algorithm’, Kexue Tongbao28 (1983), 301–305.

    MATH  Google Scholar 

  23. Du, D.-Z.: ‘Changing of point-to-set maps and point-to-set map families for continuities’, Acta Math. Applic. Sinica8, no. 2 (1985), 142–150(In Chinese.)

    MATH  Google Scholar 

  24. Du, D.-Z.: ‘A family of gradient projection algorithms’, Acta Math. Applic. Sinica, English Ser.2 (1985), 1–13.

    MATH  Google Scholar 

  25. Du, D.-Z.: ‘A gradient projection algorithm for convex programming with nonlinear constraints’, Acta Math. Applic. Sinica8 (1985), 7–16(In Chinese.)

    MATH  Google Scholar 

  26. Du, D.-Z: ‘Remarks on the convergence of Rosen’s gradient projection method’, Acta Math. Applic. Sinica, English Ser.3 (1987), 270–279.

    MATH  Google Scholar 

  27. Du, D.-Z:Gradient projection methods in linear and nonlinear programming, Hadronic Press, 1988.

    Google Scholar 

  28. Du, D.-Z:Convergence theory of feasible direction methods, Sci. Press, 1991.

    Google Scholar 

  29. Du, D.-Z: ‘Rosen’s method and slope lemmas’, in P.M. Pardalos (ed.): Advances in Optimization and Parallel Computing, Elsevier, 1992, pp. 68–84.

    Google Scholar 

  30. Du, D.-Z., Sun, J., and Song, T.-T: ‘A counterexample for Rosen’s gradient projection method’, Math. Assortment1 (1980), 4–6(In Chinese.)

    Google Scholar 

  31. Du, D.-Z, and Du, X.-F: ‘A convergent reduced gradient algorithm without using special pivot’, Math. Numer. Sinica (to appear).

    Google Scholar 

  32. Du, D.-Z, and Sun, J.: ‘A new gradient projection method’, Math. Numer. Sinica4 (1983), 378–386(In Chinese.)

    MathSciNet  Google Scholar 

  33. Du, D.-Z, Sun, J., and Song, T.-T: ‘Simplified finite pivoting processes in the reduced gradient algorithms’, Acta Math. Applic. Sinica7 (1984), 142–146(In Chinese.)

    MathSciNet  MATH  Google Scholar 

  34. Du, D.-Z, Wu, F., and Zhang, X.-S: ‘On Rosen’s gradient projection methods’, Ann. Oper. Res. (to appear).

    Google Scholar 

  35. Du, D.-Z, and Zhang, X.-S: ‘A convergence theorem of Rosen’s gradient projection method’, Math. Program.36 (1986), 135–144.

    MathSciNet  MATH  Google Scholar 

  36. Du, D.-Z, and Zhang, X.-S: ‘Global convergence of Rosen’s gradient projection methods’, Math. Program.44 (1989).

    Google Scholar 

  37. Du, D.-Z, and Zhang, X.-S: ‘Notes on a new gradient projection method’, System Sci. and Math. Sci.2 (1989).

    Google Scholar 

  38. Dunn, J.C.: ‘Global and asymptotic convergence rate estimates for a class of projected gradient processes’, SIAM J. Control Optim.19 (1981), 368–400.

    MathSciNet  MATH  Google Scholar 

  39. Dunn, J.C.: ‘On the convergence of projected gradient processes to singular critical point’, J. Optim. Th. Appl.55 (1987), 203–216.

    MathSciNet  MATH  Google Scholar 

  40. Fletcher, R.:Practical methods of optimization, unconstrained optimization, Wiley, 1987.

    Google Scholar 

  41. Fletcher, R., and Powell, M.J.D.: ‘A rapidly convergent descent method for minimization’, Computer J.6 (1963), 163–168.

    MathSciNet  MATH  Google Scholar 

  42. Goldfarb, D.: ‘Extension of Davidon’s variable metric method to maximization under linear inequality and equality constraints’, SIAM J. Appl. Math.17 (1969), 739–764.

    MathSciNet  MATH  Google Scholar 

  43. Goldfarb, D.: ‘A family of variable metric methods derived by variational means’, Math. Comput.24 (1970), 23–26.

    MathSciNet  MATH  Google Scholar 

  44. Goldstein, J.: ‘Variations on variable-metric methods’, Math. Comput.24 (1970), 1–22.

    Google Scholar 

  45. Gui, X.-Y, and Du, D.-Z: ‘A superlinearly convergent method to linearly constrained optimization problems under degeneracy’, Acta Math. Applic. Sinica, English Ser.1 (1984), 76–84.

    MATH  Google Scholar 

  46. Haraux, A.: ‘How to differentiate the projection on a convex set in Hilbert space: Some applications to variational inequalities’, J. Math. Soc. Japan20 (1977), 615–631.

    Article  MathSciNet  Google Scholar 

  47. He, G.-Z: ‘Proof of convergence of Rosen’s gradient projection method’, Acta Math. Applic. Sinica1 (1990).

    Google Scholar 

  48. Hu, X.-D.: ‘Nonlinear programming: A unified approach and the upper bound of partial concentrator’, PhD Thesis Inst. Appl. Math. Chinese Acad. Sci. (1989).

    Google Scholar 

  49. Huang, H.Y.: ‘Unified approach to quadratically convergent algorithms for function minimization’, J. Optim. Th. Appl.5 (1970), 405–423.

    MATH  Google Scholar 

  50. Huard, P.: ‘Optimization algorithms and point-to-set maps’, Math. Program.8 (1975), 308–331.

    MathSciNet  MATH  Google Scholar 

  51. Huard, P.: ‘Extensions of Zangwill’s theorem’, Math. Program. Stud.10 (1979), 98–103.

    MathSciNet  MATH  Google Scholar 

  52. Kalfon, P., Ribiere, G., and Sogno, J.C.: ‘A method of feasible directions using projection operators’, Proc. IFIP congr.68: Information Processing, 1 North-Holland (1969).

    Google Scholar 

  53. Karmarkar, N.: ‘A new polynomial-time algorithm for linear programming’, Combinatorica4 (1984), 373–395.

    MathSciNet  MATH  Google Scholar 

  54. Klee, V., and Minty, G.J.: ‘How good is the simplex algorithm?’, in O. Shisha (ed.): Inequalities, Vol. III Acad. Press, 1972.

    Google Scholar 

  55. Kwei, H.-Y, Gui, X.-Y, Wu, F., and Lai, Y.-L: ‘Extension of a variable metric algorithm to a linearly constrained optimization problem: A variation of Goldfarb’s algorithm’, in K.B. Haley (ed.): O.R. 1978, North-Holland, 1979.

    Google Scholar 

  56. Levitin, E.S., and Polyak, B.T.: ‘Constrained minimization methods’, USSR Comput. Math. Math. Phys.6, no. 5 (1966), 1–50.

    Google Scholar 

  57. Luenberger, D.G.:Introduction to linear and nonlinear programming, Addison-Wesley, 1973.

    Google Scholar 

  58. McCormick, G.P.: ‘Anti-zigzagging by bending’, Managem. Sci.15 (1969), 315–320.

    Google Scholar 

  59. McCormick, G.P.: ‘A second order method for the linearly constrained nonlinear programming problem’, in J.B. Rosen, O.L. Mangasarian, and K. Ritter, (eds.): Nonlinear Programming, Acad. Press, 1990.

    Google Scholar 

  60. McCormick, G.P., and Tapia, R.A.: ‘The gradient projection method under mild differentiability conditions’, SIAM J. Control Optim.10 (1972), 93–98.

    MathSciNet  MATH  Google Scholar 

  61. Meyer, R.R.: ‘The validity of a family of optimization methods’, SIAM J. Control Optim.8 (1970), 41–54.

    MATH  Google Scholar 

  62. Meyer, R.R.: ‘Sufficient conditions for the convergence of monotonic mathematical programming algorithms’, J. Comput. Syst. Sci.12 (1976), 108–121.

    MATH  Google Scholar 

  63. Murtagh, B.A., and Sargent, R.W.H.: ‘A constrained minimization method with quadratic convergence’, in R. Fletcher, (ed.), Optimization, Acad. Press, 1969.

    Google Scholar 

  64. Murtagh, B.A., and Saunders, M.A.: ‘Large-scale linearly constrained optimization’, Math. Program. 14 (1978), 14–72.

    MathSciNet  Google Scholar 

  65. Nocedal, J., and Overton, M.L.: ‘Projected Hessian updating algorithms for nonlinearly constrained optimization’, SIAM J. Numer. Anal.1 (1985), 821–850.

    MathSciNet  Google Scholar 

  66. Polak, E.: ‘On the convergence of optimization algorithms’, Revue Franc. Inform. Rech. Oper.3 (1969), 17–34.

    MathSciNet  MATH  Google Scholar 

  67. Polak, E.:Computational methods in optimization, Acad. Press, 1971.

    Google Scholar 

  68. Pollak, H.O.: ‘Some remarks on the Steiner problem’, J. Combin. Th. A24 (1978), 278–295.

    MathSciNet  MATH  Google Scholar 

  69. Powell, M.J.D.: ‘On the convergence of the variable metric algorithm’, J. Inst. Math. Appl.7 (1971), 21–36.

    MathSciNet  MATH  Google Scholar 

  70. Powell, M.J.D.: ‘Some properties of the variable metric algorithm’, in F.A. Lootsma, (ed.): Numerical Methods for Nonlinear Optimization, Acad. Press, 1972.

    Google Scholar 

  71. Powell, M.J.D.: ‘Unconstrained minimization and extensions for constraints’, in P.L. Hammer, and G. Zoutendijk, (eds.): Mathematical Programming Theory and Practice, North-Holland, 1974, 31–79.

    Google Scholar 

  72. Powell, M.J.D.: ‘Some global convergence properties of a variable metric algorithm for minimization without exact line searches’, in R.W. Cottle, and C.E. Lemke, (eds.): Nonlinear Programming, Amer. Math. Soc., 1976.

    Google Scholar 

  73. Powell, M.J.D.: ‘The convergence of variable metric methods for nonlinearly constrained optimization calculations’, in O. Mangasarian, R. Meyer, and S. Robinson, (eds.): Nonlinear Programming, Vol 3 Acad. Press, 1978, 27–63.

    Google Scholar 

  74. Powell, M.J.D., and Yuan, Y.: ‘A trust region algorithm for equality unconstrained optimization’, Math. Program.49 (1991), 189–211.

    MathSciNet  Google Scholar 

  75. Pu, D.: ‘A class of DFP algorithm without exact linear search’, Asia-Pacific J. Oper. Res.9 (1992), 207–220.

    MathSciNet  MATH  Google Scholar 

  76. Pu, D.: ‘The convergence of Broyden algorithms without convexity assumption’, System Sci. and Math. Sci.10 (1997), 289–298.

    MathSciNet  MATH  Google Scholar 

  77. Pu, D., and Yu, W.: ‘On convergence property of DFP algorithm’, J. Qüfu Normal Univ. 14, no. 3 (1988), 63–69.

    MathSciNet  Google Scholar 

  78. Ritter, K.: ‘A superlinearly convergent method for minimization problems with linear inequality constraints’, Math. Program.4 (1973), 44–71.

    MathSciNet  MATH  Google Scholar 

  79. Ritter, K.: ‘A method of conjugate directions for linearly constrained nonlinear programming problems’, SIAM J. Numer. Anal.12 (1975), 272–303.

    MathSciNet  Google Scholar 

  80. Ritter, K.: ‘Convergence and superlinear convergence of algorithms for linearly constrained minimization problems’, in L.C.W. Dixon, E. Spedicado, and G.P. Szego (eds.): Nonlinear Optimization: Theory and Algorithms, II Birkhäuser, 1980, 221–251.

    Google Scholar 

  81. Ritter, K.: ‘Global and superlinear convergence of a class of variable metric methods’, Math. Program. Stud.14 (1981), 178–205.

    MathSciNet  MATH  Google Scholar 

  82. Rosen, J.B.: ‘The gradient projection method for nonlinear programming, Part I: linear constraints’, SIAM J. Appl. Math.8 (1960), 181–217.

    MATH  Google Scholar 

  83. Rosen, J.B.: ‘The gradient projection method for nonlinear programming, Part II: nonlinear constraints’, SIAM J. Appl. Math.9 (1961), 514–553.

    Google Scholar 

  84. Shanno, D.F.: ‘Conditioning of quasi-Newton methods for function minimization’, Math. Comput.24 (1970), 647–656.

    MathSciNet  Google Scholar 

  85. Wang, C.Y.: ‘Simplifications of a new pivoting rule and Levitin-Polyak gradient projection method and their convergent property’, Acta Math. Applic. Sinica4 (1981), 37–52(In Chinese.)

    MATH  Google Scholar 

  86. Wang, C.Y.: ‘On convergence property of an improved reduced gradient method’, Kexue Tongbao28 (1983), 577–582.

    MATH  Google Scholar 

  87. Wolfe, P.: ‘Methods of nonlinear programming’, in R.L. Graves, and P. Wolfe, (eds.): Recent Advances in Mathematical Programming, 1963.

    Google Scholar 

  88. Wolfe, P.: ‘Convergence conditions for ascent methods’, SIAM Rev.11 (1969), 226–235.

    MathSciNet  MATH  Google Scholar 

  89. Wolfe, P.: ‘Convergence theory in nonlinear programming’, in J. Abadie, (ed.): Integer and Nonlinear Programming, North-Holland, 1970.

    Google Scholar 

  90. Wolfe, P.: ‘On the convergence of gradient methods under constraints’, IBM J. Res. Developm.16 (1972), 407–411.

    Article  MathSciNet  MATH  Google Scholar 

  91. Wu, F., and Gui, X.: ‘A class of variable metric methods with \BMI n+ 1’, parameters, Acta Math.24 (1981), 921–930.

    MathSciNet  MATH  Google Scholar 

  92. Yue, M., and Han, J.: ‘A new reduced gradient method’, Sci. Sinica22 (1979), 1099–1113.

    MathSciNet  MATH  Google Scholar 

  93. Yue, M., and Han, J.: ‘A unified approach to the feasible direction methods for nonlinear programming with linear constraints’, Acta Math. Applic. Sinica, English Ser.1 (1984), 63–73.

    MATH  Google Scholar 

  94. Zangwill, W.I.: ‘Convergence conditions for nonlinear programming algorithms’, Managem. Sci.16 (1969), 1–13.

    MathSciNet  MATH  Google Scholar 

  95. Zangwill, W.I.:Nonlinear programming: A unified approach, Prentice-Hall, 1969.

    Google Scholar 

  96. Zhang, X.-S: ‘An improved Rosen-Polak method’, Acta Math. Applic. Sinica2 (1979), 257–267 (In Chinese).

    Google Scholar 

  97. Zhang, X.-S: ‘On the convergence of Rosen’s gradient projection method’, Acta Math. Applic. Sinica8 (1985), 125–128(In Chinese.) Also in: English Ser. 3 (1987), 280–288.

    MATH  Google Scholar 

  98. Zoutendijk, G.:Methods of feasible direction methods, Elsevier, 1960.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Dept. Computer Sci. and Engin. Univ. Minnesota, Minneapolis, MN, 55455, USA

    Ding-Zhu Du

  2. Center for Applied Optim. Dept. Industrial and Systems Engin. Univ. Florida, Gainesville, FL, 32611, USA

    Panos M. Pardalos

  3. Dept. Computer Sci. and Engin. Univ. Minnesota, Minneapolis, MN, 55455, USA

    Weili Wu

Authors
  1. Ding-Zhu Du
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Panos M. Pardalos
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Weili Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Dept. Chemical Engin., Princeton Univ., Princeton, NJ, 08544-5263, USA

    Christodoulos A. Floudas

  2. Center for Applied Optim. Dept. Industrial and Systems Engin., Univ. Florida, Gainesville, FL, 32611, USA

    Panos M. Pardalos

Rights and permissions

Reprints and permissions

Copyright information

© 2001 Kluwer Academic Publishers

About this entry

Cite this entry

Du, DZ., Pardalos, P.M., Wu, W. (2001). Rosen’s Method, Global Convergence, and Powell’s Conjecture . In: Floudas, C.A., Pardalos, P.M. (eds) Encyclopedia of Optimization. Springer, Boston, MA. https://doi.org/10.1007/0-306-48332-7_436

Download citation

  • DOI: https://doi.org/10.1007/0-306-48332-7_436

  • Publisher Name: Springer, Boston, MA

  • Print ISBN: 978-0-7923-6932-5

  • Online ISBN: 978-0-306-48332-5

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation