Lagrangian Relaxation

  • Claude Lemaréchal
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2241)


Lagrangian relaxation is a tool to find upper bounds on a given (arbitrary) maximization problem. Sometimes, the bound is exact and an optimal solution is found. Our aim in this paper is to review this technique, the theory behind it, its numerical aspects, its relation with other techniques such as column generation.


Dual Problem Dual Function Column Generation Lagrangian Relaxation Nonsmooth Optimization 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    F. Alizadeh. Interior point methods in semidefinite programming with applications to combinatorial optimization. SIAM Journal on Optimization, 5(1):13–51, 1995.MathSciNetzbMATHGoogle Scholar
  2. 2.
    K. Anstreicher and L.A. Wolsey. On dual solutions in subgradient opimization. Unpublished manuscript, CORE, Louvain-la-Neuve, Belgium, 1993.Google Scholar
  3. 3.
    D.P. Bertsekas. Projected Newton methods for optimization problems with simple constraints. SIAM Journal on Control and Optimization, 20:221–246, 1982.MathSciNetzbMATHGoogle Scholar
  4. 4.
    M.J. Best. Equivalence of some quadratic programming algorithms. Mathematical Programming, 30:71–87, 1984.MathSciNetzbMATHGoogle Scholar
  5. 5.
    U. Brännlund. On relaxation methods for nonsmooth convex optimization. PhD thesis, Royal Institute of Technology-Stockholm, 1993.Google Scholar
  6. 6.
    P.M. Camerini, L. Fratta, and F. Maffioli. On improving relaxation methods by modified gradient techniques. Mathematical Programming Study, 3:26–34, 1975.MathSciNetzbMATHGoogle Scholar
  7. 7.
    E. Cheney and A. Goldstein. Newton’s method for convex programming and Tchebyche. approximations. Numerische Mathematik, 1:253–268, 1959.MathSciNetzbMATHGoogle Scholar
  8. 8.
    R. Correa and C. Lemaréchal. Convergence of some algorithms for convex minimization. Mathematical Programming, 62(2):261–275, 1993.MathSciNetzbMATHGoogle Scholar
  9. 9.
    A. Decarreau, D. Hilhorst, C. Lemarechal, and J. Navaza. Dual methods in entropy maximization. application to some problems in crystallography. SIAM Journal on Optimization, 2(2):173–197, 1992.MathSciNetzbMATHGoogle Scholar
  10. 10.
    I. Ekeland and R. Temam. Convex Analysis and Variational Problems. North Holland, 1976; reprinted by SIAM, 1999.Google Scholar
  11. 11.
    Y.M. Ermol'ev. Methods of solution of nonlinear extremal problems. Kibernetica, 2(4):1–17, 1966.Google Scholar
  12. 12.
    H. Everett III. Generalized lagrange multiplier method for solving problems of optimum allocation of resources. Operations Research, 11:399–417, 1963.MathSciNetzbMATHGoogle Scholar
  13. 13.
    J.E. Falk. Lagrange multipliers and nonconvex programs. SIAM Journal on Control, 7(4):534–545, 1969.MathSciNetzbMATHGoogle Scholar
  14. 14.
    S. Feltenmark and K. C. Kiwiel. Dual applications of proximal bundle methods, including lagrangian relaxation of nonconvex problems. SIAM Journal on Optimization, 10(3):697–721, 2000.MathSciNetzbMATHGoogle Scholar
  15. 15.
    R. Fletcher. Practical Methods of Optimization. John Wiley & Sons, Chichester (second edition), 1987.zbMATHGoogle Scholar
  16. 16.
    A. Frangioni. Solving semidefinite quadratic problems within nonsmooth optimization algorithms. Computational Operational Research, 23(11):1099–1118, 1996.MathSciNetzbMATHGoogle Scholar
  17. 17.
    M. Fukushima and L. Qi. A globally and superlinearly convergent algorithm for nonsmooth convex minimization. SIAM Journal on Optimization, 6:1106–1120, 1996.MathSciNetzbMATHGoogle Scholar
  18. 18.
    C. Garrod and J.K. Percus. Reduction of the N-particle variational problem. Journal of Mathematical Physics, 5(12), 1964.Google Scholar
  19. 19.
    A.M. Geoffrion. Lagrangean relaxation for integer programming. Mathematical Programming Study, 2:82–114, 1974.MathSciNetzbMATHGoogle Scholar
  20. 20.
    M. X. Goemans and D. P. Williamson. Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming. Journal of the ACM, 6:1115–1145, 1995.MathSciNetzbMATHGoogle Scholar
  21. 21.
    J.L. Goffin, A. Haurie, and J.Ph. Vial. Decomposition and nondifferentiable optimization with the projective algorithm. Management Science, 38(2):284–302, 1992.zbMATHGoogle Scholar
  22. 22.
    J.L. Goffin and J.Ph. Vial. Convex nondifferentiable optimization: a survey focussed on the analytic center cutting plane method. to appear in Optimization Methods and Software; also as HEC/Logilab Technical Report 99.02, Univ. of Geneva, Switzerland.Google Scholar
  23. 23.
    M. Held and R. Karp. The travelling salesman problem and minimum spanning trees: Part II. Mathematical Programming, 1:6–25, 1971.MathSciNetzbMATHGoogle Scholar
  24. 24.
    C. Helmberg and F. Rendl. A spectral bundle method for semidefinite programming. SIAM Journal on Optimization, 10(3):673–696, 2000.MathSciNetzbMATHGoogle Scholar
  25. 25.
    J.-B. Hiriart-Urruty and C. Lemaréchal. Convex Analysis and Minimization Algorithms. Springer Verlag, Heidelberg, 1993.zbMATHGoogle Scholar
  26. 26.
    J.-B. Hiriart-Urruty and C. Lemaréchal. Fundamentals of Convex Analysis. Springer Verlag, Heidelberg, 2001. to appear.zbMATHGoogle Scholar
  27. 27.
    R.A. Horn and Ch.R. Johnson. Matrix Analysis. Cambridge University Press, 1989. (New edition, 1999).Google Scholar
  28. 28.
    J. E. Kelley. The cutting plane method for solving convex programs. J. Soc. Indust. Appl. Math., 8:703–712, 1960.MathSciNetzbMATHGoogle Scholar
  29. 29.
    K.C. Kiwiel. Methods of Descent for Nondifferentiable Optimization. Lecture Notes in Mathematics 1133. Springer Verlag, Heidelberg, 1985.zbMATHGoogle Scholar
  30. 30.
    K.C. Kiwiel. A method for solving certain quadratic programming problems arising in nonsmooth optimization. IMA Journal of Numerical Analysis, 6:137–152, 1986.MathSciNetzbMATHGoogle Scholar
  31. 31.
    K.C. Kiwiel. A dual method for certain positive semidefinite quadratic programming problems. SIAM Journal on Scientific and Statistical Computing, 10:175–186, 1989.MathSciNetzbMATHGoogle Scholar
  32. 32.
    K.C. Kiwiel. Proximity control in bundle methods for convex nondifferentiable minimization. Mathematical Programming, 46(1):105–122, 1990.MathSciNetzbMATHGoogle Scholar
  33. 33.
    K.C. Kiwiel. Efficiency of proximal bundle methods. Journal of Optimization Theory and Applications, 104(3):589–603, 2000.MathSciNetzbMATHGoogle Scholar
  34. 34.
    K.C. Kiwiel, T. Larsson, and P.O. Lindberg. The efficiency of ballstep subgradient level methods for convex optimization. Mathematics of Operations Research, 24(1):237–254, 1999.MathSciNetzbMATHGoogle Scholar
  35. 35.
    T. Larsson, M. Patriksson, and A.B. Strömberg. Ergodic, primal convergence in dual subgradient schemes for convex programming. Mathematical Programming, 86(2):283–312, 1999.MathSciNetzbMATHGoogle Scholar
  36. 36.
    C. Lemaréchal, A.S. Nemirovskii, and Yu.E. Nesterov. New variants of bundle methods. Mathematical Programming, 69:111–148, 1995.MathSciNetzbMATHGoogle Scholar
  37. 37.
    C. Lemaréchal, Yu. Nesterov, and F. Oustry. Duality gap analysis for problems with quadratic constraints, 2001. In preparation.Google Scholar
  38. 38.
    C. Lemaréchal and F. Oustry. Semi-definite relaxations and lagrangian duality with application to combinatorial optimization. Rapport de Recherche 3710, Inria, 1999.
  39. 39.
    C. Lemaréchal and F. Oustry. Nonsmooth algorithms to solve semidefinite programs. In L. El Ghaoui and S-I. Niculescu, editors, Advances in Linear Matrix Inequality Methods in Control, Advances in Design and Control, 2, pages 57–77. SIAM, 2000.Google Scholar
  40. 40.
    C. Lemaréchal, F. Oustry, and C. Sagastizábal. The U-lagrangian of a convex function. Transactions of the AMS, 352(2):711–729, 2000.MathSciNetzbMATHGoogle Scholar
  41. 41.
    C. Lemaréchal, F. Pellegrino, A. Renaud, and C. Sagastizábal. Bundle methods applied to the unit-commitment problem. In J. Doležal and J. Fidler, editors, System Modelling and Optimization, pages 395–402, 1996.Google Scholar
  42. 42.
    C. Lemaréchal and A. Renaud. A geometric study of duality gaps, with applications. Mathematical Programming, 90(3):399–427, 2001.MathSciNetzbMATHGoogle Scholar
  43. 43.
    C. Lemaréchal and C. Sagastizábal. Variable metric bundle methods: from conceptual to implementable forms. Mathematical Programming, 76(3):393–410, 1997.MathSciNetzbMATHGoogle Scholar
  44. 44.
    C. Lemaréchal and J. Zowe. Some remarks on the construction of higher order algorithms in convex optimization. Applied Mathematics and Optimization, 10(1):51–68, 1983.MathSciNetzbMATHGoogle Scholar
  45. 45.
    C. Lemaréchal and J. Zowe. The eclipsing concept to approximate a multi-valued mapping. Optimization, 22(1):3–37, 1991.MathSciNetzbMATHGoogle Scholar
  46. 46.
    L. Lovász. On the Shannon capacity of a graph. IEEE Transactions on Information Theory, IT 25:1–7, 1979.MathSciNetzbMATHGoogle Scholar
  47. 47.
    D.G. Luenberger. Optimization by Vector Space Methods. Wiley, New York, 1969.zbMATHGoogle Scholar
  48. 48.
    L. Luksan and J. Vlcek. A bundle-newton method for nonsmooth unconstrained minimization. Mathematical Programming, 83A(3):373–391, 1998.MathSciNetzbMATHGoogle Scholar
  49. 49.
    T.L. Magnanti, J.F. Shapiro, and M.H. Wagner. Generalized linear programming solves the dual. Management Science, 22(11):1195–1203, 1976.zbMATHGoogle Scholar
  50. 50.
    R.E. Marsten, W.W. Hogan, and J.W. Blankenship. The boxstep method for largescale optimization. Operations Research, 23(3):389–405, 1975.MathSciNetzbMATHGoogle Scholar
  51. 51.
    R. Mifflin. A quasi-second-order proximal bundle algorithm. Mathematical Programming, 73(1):51–72, 1996.MathSciNetzbMATHGoogle Scholar
  52. 52.
    R. Mifflin and C. Sagastizábal. On VU-theory for functions with primal-dual gradient structure. SIAM Journal on Optimization, 11(2):547–571, 2000.MathSciNetzbMATHGoogle Scholar
  53. 53.
    R. Mifflin, D.F. Sun, and L.Q. Qi. Quasi-Newton bundle-type methods for nondifferentiable convex optimization. SIAM Journal on Optimization, 8(2):583–603, 1998.MathSciNetzbMATHGoogle Scholar
  54. 54.
    A.S. Nemirovskii and D. Yudin. Informational complexity and efficient methods for the solution of convex extremal problems. Ékonomika i Mathematicheskie Metody, 12:357–369, 1976. (in Russian. English translation: Matekon, 13, 3-25).zbMATHGoogle Scholar
  55. 55.
    Yu.E. Nesterov. Complexity estimates of some cutting plane methods based on the analytic barrier. Mathematical Programming, 69(1):149–176, 1995.MathSciNetzbMATHGoogle Scholar
  56. 56.
    Yu.E. Nesterov and A.S. Nemirovskii. Interior-Point Polynomial Algorithms in Convex Programming. Number 13 in SIAM Studies in Applied Mathematics. SIAM, Philadelphia, 1994.zbMATHGoogle Scholar
  57. 57.
    Yu.E. Nesterov and A.S. Nemirovskii. Interior-Point Polynomial Algorithms in Convex Programming. SIAM Studies in Applied Mathematics 13. SIAM, Philadelphia, 1994.zbMATHGoogle Scholar
  58. 58.
    J. Nocedal and S.J. Wright. Numerical Optimization. Springer Verlag, New York, 1999.zbMATHGoogle Scholar
  59. 59.
    F. Oustry. A second-order bundle method to minimize the maximum eigenvalue function. Mathematical Programming, 89(1):1–33, 2000.MathSciNetzbMATHGoogle Scholar
  60. 60.
    S. Poljak, F. Rendl, and H. Wolkowicz. A recipe for semidefinite relaxation for (0,1)-quadratic programming. Journal of Global Optimization, 7:51–73, 1995.MathSciNetzbMATHGoogle Scholar
  61. 61.
    B.T. Polyak. A general method for solving extremum problems. Soviet Mathematics Doklady, 8:593–597, 1967.zbMATHGoogle Scholar
  62. 62.
    B.N. Pshenichnyi. The Linearization Method for Constrained Optimization. Springer Verlag, 1994.Google Scholar
  63. 63.
    C.R. Reeves. Modern Heuristic Techniques for Combinatorial Problems. Blackwell Scientific Publications, New York, 1993.zbMATHGoogle Scholar
  64. 64.
    N. Shor. Utilization of the operation of space dilatation in the minimization of convex functions. Cybernetics, 6(1):7–15, 1970.Google Scholar
  65. 65.
    N.Z. Shor. Minimization methods for non-differentiable functions. Springer Verlag, Berlin, 1985.zbMATHGoogle Scholar
  66. 66.
    N.Z. Shor and A.S. Davydov. Method of opbtaining estimates in quadratic extremal probems with boolean variables. Cybernetics, 21(2):207–211, 1985.zbMATHGoogle Scholar
  67. 67.
    N.Z. Shor and N.G. Zhurbenko. A method for minimization, using the spacedilation operation in the direction of difference between two gradient sequences. Cybernetics, 7(3):450–459, 1971.Google Scholar
  68. 68.
    V.N. Solov'ev. The subdifferential and the directional derivatives of the maximum of a family of convex functions. Izvestiya: Mathematics, 62(4):807–832, 1998.MathSciNetzbMATHGoogle Scholar
  69. 69.
    G. Sonnevend. An “analytical centre” for polyhedra and new classes of global algorithms for linear (smooth, convex) programming. In A. Prékopa, J. Szelezsan, and B. Strazicky, editors, Proc. 12th IFIP Conf. System Modelling and Optimization, L.N. in Control and Information Sciences, pages 866–875. Springer Verlag, Berlin, 1986.Google Scholar
  70. 70.
    D.M. Topkis. A note on cutting-plane methods without nested constraint sets. Operations Research, 18:1216–1224, 1970.MathSciNetzbMATHGoogle Scholar
  71. 71.
    T. Terlaky. On lp programming. European Journal of Operational Research, 22:70–100, 1985.MathSciNetGoogle Scholar
  72. 72.
    H. Uzawa. Iterative methods for concave programming. In K. Arrow, L. Hurwicz, and H. Uzawa, editors, Studies in Linear and Nonlinear Programming, pages 154–165. Stanford University Press, 1959.Google Scholar
  73. 73.
    L. Vandenberghe and S. Boyd. Semidefinite programming. SIAM Review, 38(1):49–95, 1996.MathSciNetzbMATHGoogle Scholar
  74. 74.
    P. Wolfe. A method of conjugate subgradients for minimizing nondifferentiable functions. Mathematical Programming Study, 3:145–173, 1975.MathSciNetzbMATHGoogle Scholar
  75. 75.
    L.A. Wolsey. Integer Programming. Wiley-Interscience, 1998.Google Scholar
  76. 76.
    S.J. Wright. Primal-Dual Interior-Point Methods. SIAM Publication, Philadelphia, 1997.zbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2001

Authors and Affiliations

  • Claude Lemaréchal
    • 1
  1. 1.InriaMontbonnotFrance

Personalised recommendations