Mathematical Programming

, Volume 113, Issue 2, pp 299–344 | Cite as

Comparison of bundle and classical column generation

  • O. Briant
  • C. Lemaréchal
  • Ph. Meurdesoif
  • S. Michel
  • N. Perrot
  • F. Vanderbeck


When a column generation approach is applied to decomposable mixed integer programming problems, it is standard to formulate and solve the master problem as a linear program. Seen in the dual space, this results in the algorithm known in the nonlinear programming community as the cutting-plane algorithm of Kelley and Cheney-Goldstein. However, more stable methods with better theoretical convergence rates are known and have been used as alternatives to this standard. One of them is the bundle method; our aim is to illustrate its differences with Kelley’s method. In the process we review alternative stabilization techniques used in column generation, comparing them from both primal and dual points of view. Numerical comparisons are presented for five applications: cutting stock (which includes bin packing), vertex coloring, capacitated vehicle routing, multi-item lot sizing, and traveling salesman. We also give a sketchy comparison with the volume algorithm.


Lagrangian duality Dantzig–Wolfe decomposition Stabilized column generation Cutting-plane algorithms Bundle algorithm Volume algorithm Nonsmooth convex optimization 

Mathematics Subject Classification (2000)

66K05 90C25 90C27 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Anstreicher, K., Wolsey, L.A.: On Dual Solutions in Subgradient Optimization. Unpublished manuscript, CORE, Louvain-la-Neuve (1993)Google Scholar
  2. 2.
    Augerat, P.: VRP problem instances (1995) Scholar
  3. 3.
    Barahona F. and Anbil R. (2000). The volume algorithm: Producing primal solutions with a subgradient method. Math. Program. 87(3): 385–399 MATHCrossRefMathSciNetGoogle Scholar
  4. 4.
    Beasley, J.E.: Or-library: distributing test problems by electronic mail. J. Oper. Res. Soc. 41(11), 1069–1072 (1990) Scholar
  5. 5.
    Ben-Amor, H., Desrosiers, J.: A proximal-like algorithm for column generation stabilization. Technical report G-2003-43, Les Cahiers du Gerad, Montréal (2003)Google Scholar
  6. 6.
    Benameur, W., Neto, J.: Acceleration of cutting plane and column generation algorithms: application to network design (to appear, 2006)Google Scholar
  7. 7.
    Cheney E. and Goldstein A. (1959). Newton’s method for convex programming and Tchebycheff approximations. Numer. Math. 1: 253–268 MATHCrossRefMathSciNetGoogle Scholar
  8. 8.
    COmputational INfrastructure for Operations Research: http://www.coin-or.orgGoogle Scholar
  9. 9.
    Dash Optimization: Xpress-MP: User guide and reference manual, release 12 (2001) http://www.dashoptimization.comGoogle Scholar
  10. 10.
    Degraeve Z. and Peeters M. (2003). Optimal integer solutions to industrial cutting stock problems: part 2: benchmark results. INFORMS J. Comput. 15(1–3): 58–81 CrossRefMathSciNetGoogle Scholar
  11. 11.
    du Merle O., Villeneuve D., Desrosiers J. and Hansen P. (1999). Stabilized column generation. Disc. Math. 194(1–3): 229–237 MATHCrossRefMathSciNetGoogle Scholar
  12. 12.
    Ermol’ev Y.M. (1966). Methods of solution of nonlinear extremal problems. Kibernetica 2(4): 1–17 MathSciNetGoogle Scholar
  13. 13.
    Feillet D., Dejax P., Gendreau M. and Gueguen C. (2004). An exact algorithm for the elementary shortest path problem with resource constraints: application to some vehicle routing problems. Networks 44(3): 216–229 MATHCrossRefMathSciNetGoogle Scholar
  14. 14.
    Frangioni A. (2003). Generalized bundle methods. SIAM J. Optim. 13(1): 117–156 CrossRefMathSciNetGoogle Scholar
  15. 15.
    Gau T. and Wäscher G. (1995). CUTGEN1: a problem generator for the standard one-dimensional cutting stock problem. Eur. J. Oper. Res. 84: 572–579 MATHCrossRefGoogle Scholar
  16. 16.
    Geoffrion A.M. (1974). Lagrangean relaxation for integer programming. Math. Program. Study 2: 82–114 MathSciNetGoogle Scholar
  17. 17.
    Goffin J.-L., Haurie A. and Vial J.-Ph. (1992). Decomposition and nondifferentiable optimization with the projective algorithm. Manage. Sci. 38(2): 284–302 MATHGoogle Scholar
  18. 18.
    Goffin J.-L., Luo Z.-Q. and Ye Y. (1996). Complexity analysis of an interior cutting plane for convex feasibility problems. SIAM J. Optim. 6(3): 638–652 MATHCrossRefMathSciNetGoogle Scholar
  19. 19.
    Goffin J.-L. and Vial J.-Ph. (2002). Convex nondifferentiable optimization: a survey focused on the analytic center cutting plane method. Optim. Methods Softw. 17(5): 805–867 MATHCrossRefMathSciNetGoogle Scholar
  20. 20.
    Held M. and Karp R. (1970). The traveling salesman problem and minimum spanning trees. Oper. Res. 18: 1138–1162 MATHMathSciNetGoogle Scholar
  21. 21.
    Held M. and Karp R. (1971). The traveling salesman problem and minimum spanning trees: part II. Math. Program. 1(1): 6–25 MATHCrossRefMathSciNetGoogle Scholar
  22. 22.
    Hiriart-Urruty, J.-B., Lemaréchal, C.: Convex Analysis and Minimization Algorithms. Springer, Berlin Heidelberg New York (1993, Two volumes)Google Scholar
  23. 23.
    Hiriart-Urruty J.-B. and Lemaréchal C. (2001). Fundamentals of Convex Analysis. Springer, Berlin Heidelberg New York MATHGoogle Scholar
  24. 24.
    Kelley J.E. (1960). The cutting plane method for solving convex programs. J. Soc. Indust. Appl. Math. 8: 703–712 CrossRefMathSciNetGoogle Scholar
  25. 25.
    Kim S., Chang K.N. and Lee J.Y. (1995). A descent method with linear programming subproblems for nondifferentiable convex optimization. Math. Program. 71(1): 17–28 CrossRefMathSciNetGoogle Scholar
  26. 26.
    Kiwiel K.C. (1989). A dual method for certain positive semidefinite quadratic programming problems. SIAM J. Sci. Stat. Comput. 10(1): 175–186 MATHCrossRefMathSciNetGoogle Scholar
  27. 27.
    Kiwiel K.C. (1983). An aggregate subgradient method for nonsmooth convex minimization. Math. Program. 27: 320–341 MATHCrossRefMathSciNetGoogle Scholar
  28. 28.
    Kiwiel K.C. (1985). Methods of Descent for Nondifferentiable Optimization. Lecture Notes in Mathematics 1133. Springer, Berlin Heidelberg New York Google Scholar
  29. 29.
    Kiwiel K.C. (1994). A Cholesky dual method for proximal piecewise linear programming. Numer. Math. 68: 325–340 MATHCrossRefMathSciNetGoogle Scholar
  30. 30.
    Kiwiel, K.C.: An inexact bundle approach to cutting stock problems. Technical report, Systems Research Institute, Warsaw. Submitted to INFORMS J. Comput. (2004)Google Scholar
  31. 31.
    Kiwiel K.C. (2006). A proximal bundle method with approximate subgradient linearizations. SIAM J. Optim. 16(4): 1007–1023 MATHCrossRefMathSciNetGoogle Scholar
  32. 32.
    Kiwiel, K.C., Lemaréchal, C.: An inexact conic bundle variant suited to column generation (in preparation)Google Scholar
  33. 33.
    Sagastizábal C., Bahiense L. and Maculan N. (2002). The volume algorithm revisited: relation with bundle methods. Math. Program. 94(1): 41–69 MATHCrossRefMathSciNetGoogle Scholar
  34. 34.
    Larsson T., Patriksson M. and Strömberg A.B. (1999). Ergodic, primal convergence in dual subgradient schemes for convex programming. Math. Program. 86(2): 283–312 MATHCrossRefMathSciNetGoogle Scholar
  35. 35.
    Lemaréchal, C.: An algorithm for minimizing convex functions. In: Rosenfeld, J.L. (ed.) Information Processing ’74, pp. 552–556. North Holland, Amsterdam (1974)Google Scholar
  36. 36.
    Lemaréchal, C.: Nonsmooth optimization and descent methods. Research Report 78-4, IIASA (1978)Google Scholar
  37. 37.
    Lemaréchal, C.: Lagrangian relaxation. In: Jünger, M., Naddef, D. (eds.) Computational Combinatorial Optimization, pp. 112–156. Springer, Berlin Heidelberg New York (2001)Google Scholar
  38. 38.
    Lemaréchal, C.: The omnipresence of Lagrange. 4OR 1(1), 7,25 (2003)Google Scholar
  39. 39.
    Lemaréchal C., Nemirovskii A.S. and Nesterov Yu.E. (1995). New variants of bundle methods. Math. Program. 69: 111–148 CrossRefGoogle Scholar
  40. 40.
    Lemaréchal, C., Pellegrino, F., Renaud, A., Sagastizábal, C.: Bundle methods applied to the unit-commitment problem. In: Dolež al, J., Fidler, J. (eds.) System Modelling and Optimization, pp. 395–402. Chapman and Hall, London (1996)Google Scholar
  41. 41.
    Lemaréchal C. and Sagastizábal C. (1997). Variable metric bundle methods: from conceptual to implementable forms. Math. Program. 76(3): 393–410 CrossRefGoogle Scholar
  42. 42.
    Marsten R.E., Hogan W.W. and Blankenship J.W. (1975). The boxstep method for large-scale optimization. Oper. Res. 23(3): 389–405 MATHMathSciNetGoogle Scholar
  43. 43.
    Mehrotra A. and Trick M.A. (1996). A column generation approach to graph coloring. INFORMS J. Comput. 8(4): 344–354 MATHCrossRefGoogle Scholar
  44. 44.
    du Merle O., Villeneuve D., Desrosiers J. and Hansen P. (1999). Stabilized column generation. Discrete Math. 194: 229–237 MATHCrossRefMathSciNetGoogle Scholar
  45. 45.
    Nemirovskii, A.S., Yudin, D.: Informational complexity and efficient methods for the solution of convex extremal problems. Ékonomika i Mathematicheskie Metody 12, 357–369 (1976) (in Russian. English translation: Matekon 13, 3–25)Google Scholar
  46. 46.
    Nemirovskii, A.S., Yudin, D.: Problem Complexity and Method Efficiency in Optimization. Wiley-Interscience Series in Discrete Mathematics (1983). (Original Russian: Nauka, 1979)Google Scholar
  47. 47.
    Nesterov Yu.E. (1995). Complexity estimates of some cutting plane methods based on the analytic barrier. Math. Program. 69(1): 149–176 CrossRefMathSciNetGoogle Scholar
  48. 48.
    Nesterov Yu.E. and Vial J.-Ph. (1999). Homogeneous analytic center cutting plane methods for convex problems and variational inequalities. SIAM J. Optim. 9(3): 707–728 MATHCrossRefMathSciNetGoogle Scholar
  49. 49.
    Polyak B.T. (1967). A general method for solving extremum problems. Soviet Math. Doklady 8: 593–597 MATHGoogle Scholar
  50. 50.
    Prim R.C. (1957). Shortest connection networks and some generalizations. Bell Syst. Technol. J. 36: 1389–1401 Google Scholar
  51. 51.
    Rockafellar R.T. (1970). Convex Analysis. Princeton University Press, Princeton MATHGoogle Scholar
  52. 52.
    Rockafellar R.T. (1973). A dual approach to solving nonlinear programming problems by constrained optimization. Math. Program. 5: 354–373 MATHCrossRefMathSciNetGoogle Scholar
  53. 53.
    Rousseau, L.-M., Gendreau, M., Feillet, D.: Interior point stabilization for column generation. Working paper 39, Centre de Recherche sur les Transports, Univ. Montréal (2003)Google Scholar
  54. 54.
    Shor N. (1970). Utilization of the operation of space dilatation in the minimization of convex functions. Cybernetics 6(1): 7–15 CrossRefMathSciNetGoogle Scholar
  55. 55.
    Shor N.Z. (1985). Minimization methods for non-differentiable functions. Springer, Berlin Heidelberg New York MATHGoogle Scholar
  56. 56.
    Thiongane B., Nagih A. and Plateau G. (2004). Adapted step size in a 0-1 biknapsack lagrangean dual solving algoritm. Ann. Oper. Res. 139(1): 353–373 CrossRefMathSciNetGoogle Scholar
  57. 57.
    Uzawa, H.: Iterative methods for concave programming. In: Arrow, K., Hurwicz, L., Uzawa, H. (eds.) Studies in Linear and Nonlinear Programming, pp. 154–165. Stanford University Press, Stanford (1959)Google Scholar
  58. 58.
    Vanderbeck F. (2002). Extending Dantzig’s bound to the bounded multi-class binary knapsack problem. Math. Program. 94(1): 125–16 MATHCrossRefMathSciNetGoogle Scholar
  59. 59.
    Vanderbeck, F.: Dantzig-Wolfe re-formulation or how to exploit simultaneaously original formulation and column generation re-formulation. Working paper U-03.24, University Bordeaux 1, Talence (2003)Google Scholar

Copyright information

© Springer-Verlag 2006

Authors and Affiliations

  • O. Briant
    • 1
  • C. Lemaréchal
    • 2
  • Ph. Meurdesoif
    • 3
  • S. Michel
    • 3
  • N. Perrot
    • 3
  • F. Vanderbeck
    • 3
  1. 1.GilcoGrenobleFrance
  2. 2.INRIASaint IsmierFrance
  3. 3.University of Bordeaux 1; MABTalenceFrance

Personalised recommendations