Skip to main content
SpringerLink
Log in

On learning and branching: a survey

  • Invited Paper
  • Published:
TOP Aims and scope Submit manuscript

Abstract

This paper surveys learning techniques to deal with the two most crucial decisions in the branch-and-bound algorithm for Mixed-Integer Linear Programming, namely variable and node selections. Because of the lack of deep mathematical understanding on those decisions, the classical and vast literature in the field is inherently based on computational studies and heuristic, often problem-specific, strategies. We will both interpret some of those early contributions in the light of modern (machine) learning techniques, and give the details of the recent algorithms that instead explicitly incorporate machine learning paradigms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. Note that SB + PC and RB differ on the switch from SB to PC: at a certain fixed depth of the tree for the former method, whereas depending on each variable’s reliability (i.e., past usage) for the latter.

  2. Note that Marcos Alvarez et al. (2016) comes later than Marcos Alvarez et al. (2017), where 2017 is the year of journal publication of Marcos Alvarez et al. (2014).

References

  • Abdi H, Williams LJ (2010) Principal component analysis. Wiley Interdiscip Rev 2(4):433–459

    Article  Google Scholar 

  • Achterberg T, Berthold T (2009) Hybrid branching. Springer, Berlin, pp 309–311. doi:10.1007/978-3-642-01929-6_23

  • Achterberg T, Koch T, Martin A (2005) Branching rules revisited. Oper Res Lett 33(1):42–54. doi:10.1016/j.orl.2004.04.002

    Article  Google Scholar 

  • Achterberg T, Koch T, Martin A (2006) MIPLIB 2003. Oper Res Lett 34(4):361–372

    Article  Google Scholar 

  • Ansótegui C, Sellmann M, Tierney K (2009) A gender-based genetic algorithm for the automatic configuration of algorithms. Springer, Berlin, pp 142–157. doi:10.1007/978-3-642-04244-7_14

  • Applegate D, Bixby R, Chvátal V, Cook W (2007) The traveling salesman problem. A computational study. Princeton University Press, Princeton

    Google Scholar 

  • Auer P, Cesa-Bianchi N, Fischer P (2002) Finite-time analysis of the multiarmed bandit problem. Mach Learn 47(2):235–256. doi:10.1023/A:1013689704352

    Article  Google Scholar 

  • Bellman R (1961) Adaptive control processes. Princeton University Press, Princeton

    Book  Google Scholar 

  • Benichou M, Gauthier J, Girodet P, Hentges G (1971) Experiments in mixed-integer programming. Math Program 1:76–94

    Article  Google Scholar 

  • Bertsekas DP, Tsitsiklis JN (1996) Neuro-dynamic programming, 1st edn. Anthropological field studies. Athena Scientific, Belmont

    Google Scholar 

  • Bischl B, Lang M, Kotthoff L, Schiffner J, Richter J, Studerus E, Casalicchio G, Jones ZM (2016) mlr: machine learning in R. J Mach Learn Res 17(170):1–5

    Google Scholar 

  • Bishop CM (2006) Pattern recognition and machine learning. Information science and statistics. Springer, New York

    Google Scholar 

  • Bixby RE, Ceria S, McZeal CM, Savelsbergh MWP (1998) An updated mixed integer programming library: MIPLIB 3.0

  • COR@L (2017) Computational Optimization Research at Lehigh. https://coral.ise.lehigh.edu

  • Cornuéjols G, Karamanov M, Li Y (2006) Early estimates of the size of branch-and-bound trees. INFORMS J Comput 18(1):86–96. doi:10.1287/ijoc.1040.0107

    Article  Google Scholar 

  • CPLEX (2017) http://www-01.ibm.com/software/commerce/optimization/cplex-optimizer/index.html

  • Di Liberto G, Kadioglu S, Leo K, Malitsky Y (2016) DASH: dynamic approach for switching heuristics. Eur J Oper Res 248(3):943–953. doi:10.1016/j.ejor.2015.08.018

    Article  Google Scholar 

  • Domingos P (2012) A few useful things to know about machine learning. Commun ACM 55(10):78–87. doi:10.1145/2347736.2347755

    Article  Google Scholar 

  • Fischetti M, Monaci M (2012a) Backdoor branching. INFORMS J Comput 25(4):693–700. doi:10.1287/ijoc.1120.0531

    Article  Google Scholar 

  • Fischetti M, Monaci M (2012b) Branching on nonchimerical fractionalities. Oper Res Lett 40(3):159–164

    Article  Google Scholar 

  • Fischetti M, Monaci M (2014) Exploiting erraticism in search. Oper Res 62(1):114–122. doi:10.1287/opre.2013.1231

    Article  Google Scholar 

  • Fischetti M, Lodi A, Monaci M, Salvagnin D, Tramontani A (2016) Improving branch-and-cut performance by random sampling. Math Program Comput 8(1):113–132. doi:10.1007/s12532-015-0096-0

    Article  Google Scholar 

  • Geurts P, Ernst D, Wehenkel L (2006) Extremely randomized trees. Mach Learn 63(1):3–42. doi:10.1007/s10994-006-6226-1

    Article  Google Scholar 

  • Gilpin A, Sandholm T (2011) Information-theoretic approaches to branching in search. Discret Optim 8(2):147–159. doi:10.1016/j.disopt.2010.07.001

    Article  Google Scholar 

  • Glankwamdee W, Linderoth J (2011) Lookahead branching for mixed integer programming. In: Twelfth INFORMS computing society meeting, INFORMS, pp 130–150

  • Gomory R (1960) An algorithm for the mixed integer problem. Tech. Rep. RM-2597, The Rand Corporation

  • Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT Press. http://www.deeplearningbook.org

  • Gurobi (2017) http://www.gurobi.com

  • Hamerly G, Elkan C (2003) Learning the k in k-means. In: NIPS, vol. 3, MIT Press, pp 281–288

  • Hastie T, Tibshirani R, Friedman J (2009) The elements of statistical learning: data mining, inference and prediction, 2nd edn. Springer series in statistics, Springer, New York. doi:10.1007/978-0-387-84858-7

  • He H, Daume III H, Eisner JM (2014) Learning to search in branch and bound algorithms. In: Ghahramani Z, Welling M, Cortes C, Lawrence ND, Weinberger KQ (eds) Advances in neural information processing systems, vol. 27. Curran Associates, Inc., pp 3293–3301

  • Hutter F, Xu L, Hoos HH, Leyton-Brown K (2014) Algorithm runtime prediction: methods & evaluation. Artif Intell 206:79–111. doi:10.1016/j.artint.2013.10.003

    Article  Google Scholar 

  • Joachims T (2006) Training linear SVMs in linear time. In: Proceedings of the 12th ACM SIGKDD international conference on knowledge discovery and data mining, ACM, pp 217–226

  • JuliaComputing (2017) https://juliacomputing.com/domains/machine-learning.html

  • Kadioglu S, Malitsky Y, Sellmann M (2012) Non-model-based search guidance for set partitioning problems. In: AAAI

  • Karzan FK, Nemhauser GL, Savelsbergh MWP (2009) Information-based branching schemes for binary linear mixed integer problems. Math Program Comput 1(4):249–293. doi:10.1007/s12532-009-0009-1

    Article  Google Scholar 

  • Khalil E (2016) Machine learning for integer programming. In: Proceedings of the doctoral consortium at the twenty-fifth international joint conference on artificial intelligence (IJCAI)

  • Khalil E, Le Bodic P, Song L, Nemhauser G, Dilkina B (2016) Learning to branch in mixed integer programming. In: Proceedings of the 30th AAAI conference on artificial intelligence

  • Knuth DE (1975) Estimating the efficiency of backtrack programs. Math Comput 29(129):122–136

    Article  Google Scholar 

  • Koch T, Achterberg T, Andersen E, Bastert O, Berthold T, Bixby R, Danna E, Gamrath G, Gleixner A, Heinz S, Lodi A, Mittelmann H, Ralphs T, Salvagnin D, Steffy D, Wolter K (2011) MIPLIB 2010. Math Program Comput 3:103–163

  • Kocsis L, Szepesvári C (2006) Bandit based Monte-Carlo planning. Springer, Berlin, pp 282–293. doi:10.1007/11871842_29

  • Land A, Doig A (1960) An automatic method of solving discrete programming problems. Econometrica 28:497–520

    Article  Google Scholar 

  • Le Bodic P, Nemhauser G (2017) An abstract model for branching and its application to mixed integer programming. Math Program. doi:10.1007/s10107-016-1101-8

  • Linderoth JT, Lodi A (2011) MILP software. In: Cochran J (ed) Wiley encyclopedia of operations research and management science, vol 5. Wiley, pp 3239–3248

  • Linderoth JT, Savelsbergh MWP (1999) A computational study of search strategies for mixed integer programming. INFORMS J Comput 11(2):173–187. doi:10.1287/ijoc.11.2.173

    Article  Google Scholar 

  • Lodi A (2009) Mixed integer programming computation. In: Jünger M, Liebling T, Naddef D, Nemhauser G, Pulleyblank W, Reinelt G, Rinaldi G, Wolsey L (eds) 50 Years of Integer Programming 1958–2008. Springer, Berlin Heidelberg, pp 619–645

    Google Scholar 

  • Lodi A (2013) The heuristic (dark) side of MIP solvers. In: Talbi EG (ed) Hybrid metaheuristics, vol 434. Studies in computational intelligence. Springer, Berlin, pp 273–284

    Chapter  Google Scholar 

  • Lodi A, Tramontani A (2013) Performance variability in mixed-integer programming. INFORMS, chap 1, pp 1–12. doi:10.1287/educ.2013.0112

  • MacQueen J (1967) Some methods for classification and analysis of multivariate observations. In: Proceedings of the fifth Berkeley symposium on mathematical statistics and probability, vol. 1, University of California Press, pp 281–297

  • Marcos Alvarez A (2016) Computational and theoretical synergies between linear optimization and supervised machine learning. PhD thesis, Université de Liège, Liège, Belgique

  • Marcos Alvarez A, Louveaux Q, Wehenkel L (2014) A supervised machine learning approach to variable branching in branch-and-bound. Tech. rep., Université de Liège. http://hdl.handle.net/2268/167559

  • Marcos Alvarez A, Wehenkel L, Louveaux Q (2015) Machine learning to balance the load in parallel branch-and-bound. Tech. rep., Université de Liège. http://hdl.handle.net/2268/181086

  • Marcos Alvarez A, Wehenkel L, Louveaux Q (2016) Online learning for strong branching approximation in branch-and-bound. Tech. rep., Université de Liège. http://hdl.handle.net/2268/192361

  • Marcos Alvarez A, Louveaux Q, Wehenkel L (2017) A machine learning-based approximation of strong branching. INFORMS J Comput 29(1):185–195. doi:10.1287/ijoc.2016.0723

    Article  Google Scholar 

  • Nocedal J, Wright S (2006) Numerical optimization, 2nd edn. Springer, New York

    Google Scholar 

  • Padberg M, Rinaldi G (1991) A branch and cut algorithm for the resolution of large-scale symmetric traveling salesmen problems. SIAM Rev 33(1):60–100. doi:10.1137/1033004

    Article  Google Scholar 

  • Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E (2011) Scikit-learn: machine learning in Python. J Mach Learn Res 12:2825–2830

    Google Scholar 

  • Robbins H (1952) Some aspects of the sequential design of experiments. Bull Am Math Soc 58(5):527–535

    Article  Google Scholar 

  • Sabharwal A, Samulowitz H, Reddy C (2012) Guiding combinatorial optimization with UCT. In: Beldiceanu N, Jussien N, Pinson É (eds) Integration of AI and OR techniques in constraint programming for combinatorial optimization problems: 9th international conference, CPAIOR 2012, Nantes, France, May 28–June 1, 2012. Proceedings. Lecture notes in computer science. Springer, Berlin, pp 356–361. doi:10.1007/978-3-642-29828-8_23

  • Sammut C (2010) Behavioral cloning. Springer, Boston, pp 93–97. doi:10.1007/978-0-387-30164-8_69

  • SCIP (2017) http://scip.zib.de/

  • Shannon CE (1948) A mathematical theory of communication. Bell Syst Tech J 27:379–423 and 623–656

    Article  Google Scholar 

  • Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT Press, Cambridge

  • Syed U, Schapire RE (2010) A reduction from apprenticeship learning to classification. In: Lafferty JD, Williams CKI, Shawe-Taylor J, Zemel RS, Culotta A (eds) Advances in neural information processing systems, vol. 23. Curran Associates, Inc., pp 2253–2261

  • Szepesvári C (2010) Algorithms for reinforcement learning, vol 4. Morgan & Claypool Publishers, San Rafael

    Google Scholar 

Download references

Acknowledgements

We like to thank Yoshua Bengio for his support in our learning curve. Additional thanks go to Laurent Charlin, Mathieu Tanneau, Claudio Sole and François Laviolette for interesting discussions on the topic. A final round of discussion happened at the Bellairs Workshop on “Data, Learning and Optimization”, so we are indebted to all participants for exchanging on the topic and especially to Bruce Shepherd to have made the workshop happening.

Author information

Authors and Affiliations

  1. Canada Excellence Research Chair, École Polytechnique de Montréal, C.P. 6079, Succ. Centre-ville, Montréal, QC, H3C 3A7, Canada

    Andrea Lodi & Giulia Zarpellon

Authors
  1. Andrea Lodi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Giulia Zarpellon
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Andrea Lodi.

Additional information

This invited paper is discussed in the comments available at doi:10.1007/s11750-017-0449-0, doi:10.1007/s11750-017-0453-4, and doi:10.1007/s11750-017-0454-3.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lodi, A., Zarpellon, G. On learning and branching: a survey. TOP 25, 207–236 (2017). https://doi.org/10.1007/s11750-017-0451-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11750-017-0451-6

Keywords

Mathematics Subject Classification

Search

Navigation