Skip to main content
SpringerLink
Log in

A General Model for Automated Algorithm Design

  • Chapter
  • First Online:
Automated Design of Machine Learning and Search Algorithms

Part of the book series: Natural Computing Series ((NCS))

Abstract

This chapter presents a newly defined novel combinatorial optimisation problem, namely, the General Combinatorial Optimisation Problem (GCOP), whose decision variables are a set of elementary algorithm components. The combinations of these algorithm components, i.e. solutions of GCOP, thus represent different search algorithms. The objective of GCOP is to find the optimal combinations of algorithm components for solving optimisation problems. Solving the GCOP is thus equivalent to automatically designing the best search algorithms for optimisation problems. The definition of the GCOP is presented with a new taxonomy which categorises relevant literature on automated algorithm design into three lines of research, namely, automated algorithm configuration, selection and composition. Based on the decision space under consideration, the algorithm design itself is defined as an optimisation problem. Relevant literature is briefly reviewed, motivating a new line of exciting and challenging directions on the emerging research of automated algorithm design.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://people.brunel.ac.uk/~mastjjb/jeb/info.html.

  2. 2.

    https://sites.google.com/view/general-cop.

References

  1. T. Adamo, G. Ghiani, A. Grieco, E. Guerriero, E. Manni, MIP neighborhood synthesis through semantic feature extraction and automatic algorithm configuration. Comput. Oper. Res. 83, 106–119 (2017)

    Article  MathSciNet  Google Scholar 

  2. T. Adamo, G. Ghiani, E. Guerriero, E. Manni, Automatic instantiation of a variable neighborhood descent from a mixed integer programming model. Oper. Res. Perspect. 4, 123–135 (2017)

    MathSciNet  Google Scholar 

  3. B. Adenso-Díaz, M. Laguna, Fine-tuning of algorithms using fractional experimental designs and local search. Oper. Res. 54(1), 99–114 (2006)

    Google Scholar 

  4. T. Agasiev, A. Karpenko, The program system for automated parameter tuning of optimization algorithms. Procedia Comput. Sci. 103, 347–354 (2017)

    Article  Google Scholar 

  5. R. Akay, A. Basturk, A. Kalinli, X. Yao, Parallel population-based algorithm portfolios: an empirical study. Neurocomputing 247, 115–125 (2017)

    Article  Google Scholar 

  6. C. Ansótegui, M. Sellmann, K. Tierney, Gga: a gender-based genetic algorithm for the automatic configuration of algorithms, in Proceedings of 2009 15th International Conference on Principles and Practice of Constraint Programming (Lisbon, Portugal, 2009), pp. 142–157

    Google Scholar 

  7. P. Balaprakash, M. Birattari, T. Stützle, Improvement strategies for the f-race algorithm: sampling design and iterative refinement, in HM 2007: Hybrid Metaheuristics (Dortmund, Germany, October 8-9, 2007), pp. 108–122

    Google Scholar 

  8. J. Beasley, OR-library: distributing test problems by electronic mail. J. Oper. Res. Soc. 41(11), 1069–1072 (1990)

    Article  Google Scholar 

  9. L. Bezerra, M. Lòpez-Ibáñez, T. Stützle, Automatic design of evolutionary algorithms for multi-objective combinatorial optimization, in Proceedings of Parallel Problem Solving from Nature (Ljubljana, September 13–17, 2014), pp. 508–517

    Google Scholar 

  10. M. Birattari, The Problem of Tuning Metaheuristics As Seen From a Machine Learning Perspective (IOS Press, US, 2005)

    MATH  Google Scholar 

  11. M. Birattari, T. Stützle, L. Paquete, K. Varrentrapp, A racing algorithm for configuring metaheuristics, in Proceedings of the 4th Annual Conference on Genetic and Evolutionary Computation (GECCO’02) (2002), pp. 11–18

    Google Scholar 

  12. M. Birattari, Z. Yuan, P. Balaprakash, T. Stützle, F-race and iterated F-race: an overview, in Experimental Methods for the Analysis of Optimization Algorithms (2010), pp. 311–336

    Google Scholar 

  13. B. Bischl, O. Mersmann, H. Trautmann, M. Preuss, M. Preuß, Algorithm selection based on exploratory landscape analysis and cost-sensitive learning, in GECCO ’12: Proceedings of the 14th Annual Conference on Genetic and Evolutionary Computation (Philly, 2012), pp. 313–320

    Google Scholar 

  14. E.K. Burke, M. Gendreau, M. Hyde, G. Kendall, B. McCollum, G. Ochoa, A.J. Parkes, S. Petrovic, The cross-domain heuristic search challenge - an international research competition, in Proceedings of Intelligent Conference Learning and Intelligent Optimization (Rome, January 17-21, 2011), pp. 631–634

    Google Scholar 

  15. E.K. Burke, M. Gendreau, M. Hyde, G. Kendall, G. Ochoa, E. Özcan, Hyper-heuristics: a survey of the state of the art. J. Oper. Res. Soc. 64(12), 1695–1724 (2013)

    Article  Google Scholar 

  16. T. Carchrae, J.C. Beck, Applying machine learning to low-knowledge ccontrol of optimisztion algorithms. Comput. Intell. 4(21), 372–387 (2005)

    Article  Google Scholar 

  17. P. Cowling, G. Kendall, E. Soubeiga, A hyperheuristic approach to scheduling a sales summit, in Proceedings of Practice and Theory of Automated Timetabling (Konstanz, August 16–18, 2000), pp. 176–190

    Google Scholar 

  18. S.P. Coy, B.L. Golden, G.C. Runger, E.A. Wasil, Using experimental design to find effective parameter settings for heuristics. J. Heuristics 1(7), 77–97 (2001)

    Article  Google Scholar 

  19. N.T.T. Dang, P. De Causmaecker, Characterization of neighborhood behaviours in a multi-neighborhood local search algorithm, in LION 2016: Learning and Intelligent Optimization. Lecture Notes in Computer Science 10079 (2016), pp. 234–239

    Google Scholar 

  20. O. François, C. Lavergne, Design of evolutionary algorithms - a statistical perspective. IEEE Trans. Evol. Comput. 5(2), 129–148 (2001)

    Article  Google Scholar 

  21. M.R. Garey, D.S. Johnson, Computers and Intractability: A Guide to the Theory of NP-Completeness (W.H. Freeman, New York, 1979)

    MATH  Google Scholar 

  22. D.B. Gümüs, E. Özcan, J. Atkin, An analysis of the taguchi method for tuning a memetic algorithm with reduced computational time budget, in ISCIS 2016: Computer and Information Sciences (Poland, 2016), pp. 12–20

    Google Scholar 

  23. Y. He, S.Y. Yuen, Y. Lou, X. Zhang, A sequential algorithm portfolio approach for black box optimization. Swarm Evol. Comput. 44, 559–570 (2019)

    Article  Google Scholar 

  24. B.A. Huberman, R.M. Lukose, T. Hogg, An economics approach to hard computational problems. Science 275(5296), 51–54 (1997)

    Article  Google Scholar 

  25. F. Hutter, H.H. Hoos, K. Leyton-Brown, Sequential model-based optimization for general algorithm configuration, in LION 2011: Learning and Intelligent Optimization (Rome, Italy, 2011), pp. 507–523

    Google Scholar 

  26. F. Hutter, H.H. Hoos, K. Leyton-Brown, T. Stützle, ParamILS: An automatic algorithm configuration framework. Journal of Artificial Intelligence Research 36, 267–306 (2009)

    Article  Google Scholar 

  27. S. Kadioglu, Y. Malitsky, M. Sellmann, K. Tierney, and K. Tierney. Isac - instance-specific algorithm configuration. In Proceedings of the 2010 conference on ECAI 2010: 19th European Conference on Artificial Intelligence, pages 751–756, Lisbon, Portugal, Aug, 2010

    Google Scholar 

  28. G. Kendall, R. Bai, J. Blazewicz, P. De Causmaecker, M. Gendreau, R. John, J. Li, B. McCollum, E. Pesch, R. Qu, N. Sabar, G. Vanden Berghe, and A. Yee. Good laboratory practice for optimization research. Journal of Operational Research Society, 67(4):676–689, Apr. 2016

    Google Scholar 

  29. P. Kerschke, H.H. Hoos, F. Neumann, H. Trautmann, Is evolutionary computation evolving fast enough? Evolutionary Computation 27(1), 3–45 (2019)

    Article  Google Scholar 

  30. T. Liao, D. Molina, T. Stützle, Performance evaluation of automatically tuned continuous optimizers on different benchmark sets. Applied Soft Computing 27, 490–503 (2015)

    Article  Google Scholar 

  31. A. Liefooghe, B. Derbel, S. Verel, H. Aguirre, K. Tanaka, Towards landscape-aware automatic algorithm configuration: preliminary experiments on neutral and rugged landscapes, in EvoCOP 2017: Evolutionary Computation in Combinatorial Optimization (2017), pp. 215–232

    Google Scholar 

  32. S. Liu, K. Tang, X. Yao, Automatic construction of parallel portfolios via explicit instance grouping, in Proceedings of AAAI Conference on Artificial Intelligence (New Orleans, 2018), pp. 2–7

    Google Scholar 

  33. M. López-Ibáñez, J. Dubois-Lacoste, L. P. Cáceres, T. Stützle, M. Birattari, The irace package: Iterated racing for automatic algorithm configuration. Oper. Res. Perspect. 3, 43–58 (2016)

    Google Scholar 

  34. M. López-Ibáñez, T. Stützle, The automatic design of multi-objective ant colony optimization algorithms. IEEE Trans. Evol. Comput. 16(6), 861–875 (2012)

    Google Scholar 

  35. J. MacLachlan, Y. Mei, J. Branke, M. Zhang, Genetic programming hyper-heuristics with vehicle collaboration for uncertain capacitated arc routing problems, in Evolutionary Computation 28(4), 563–593 (2020)

    Google Scholar 

  36. Y. Malitsky, M. Sellmann, Instance-specific algorithm configuration as a method for non-model-based portfolio generation (2012), pp. 244–259

    Google Scholar 

  37. T. Messelis, P. De Causmaecker, An automatic algorithm selection approach for the multi-mode resource-constrained project scheduling problem. Eur. J. Oper. Res. 233(3), 511–528 (2014)

    Article  MathSciNet  Google Scholar 

  38. S. Minton, Automatically configuring constraint satisfaction programs: a case study. Constraints 1–2(1), 7–43 (1996)

    Article  MathSciNet  Google Scholar 

  39. M. Misir, K. Verbeeck, P. De Causmaecker, G.V. Berghe, An investigation on the generality level of selection hyper-heuristics under different empirical conditions. Appl. Soft Comput. 13(7), 3335–3353 (2013)

    Article  Google Scholar 

  40. G. Kendall, R. Qu, N.R. Sabar, M. Ayob, A dynamic multiarmed bandit-gene expression programming hyper-heuristic for combinatorial optimization problems. IEEE Trans. Cybern. 45(2), 217–228 (2015)

    Article  Google Scholar 

  41. M. Oltean, Evolving evolutionary algorithms using linear genetic programming. Evol. Comput. 13(3), 387–410 (2005)

    Article  Google Scholar 

  42. C. Papadimitriou, K. Steiglitz, Combinatorial Optimization: Algorithms and Complexity (Dover Publications Inc., 1982)

    Google Scholar 

  43. M.-W. Park, Y.-D. Kim, A systematic procedure for setting parameters in simulated annealing algorithms. Comput. Oper. Res. 25(3), 207–217 (1998)

    Article  Google Scholar 

  44. J. Pérez, R.A. Pazos, J. Frausto, G. Rodríguez, D. Romero, L. Cruz, A statistical approach for algorithm selection, in WEA 2004: Experimental and Efficient Algorithms (Angra dos Reis, Brazil, May, 2004), pp. 417–431

    Google Scholar 

  45. J. Pihera, N. Musliu, Application of machine learning to algorithm selection for tsp, in 2014 IEEE 26th International Conference on Tools with Artificial Intelligence (Limassol, Cyprus, 2014), pp. 47–54

    Google Scholar 

  46. N. Pillay, D. Beckedahl, EvoHyp - a Java toolkit for evolutionary algorithm hyper-heuristics, in Proceedings of IEEE Congress on Evolutionary Computation (San Sebastian, June 5-8, 2017), pp. 2707–2713

    Google Scholar 

  47. N. Pillay, R. Qu, Hyper-Heuristics: Theory and Applications (Springer Nature, 2019)

    Google Scholar 

  48. N. Pillay, R. Qu, Assessing hyper-heuristic performance. J. Oper. Res. Soc. accepted (2020)

    Google Scholar 

  49. M. Preuss, T. Bartz-Beielstein, Sequential parameter optimization applied to self-adaptation for binary-coded evolutionary algorithms, in Parameter Setting in Evolutionary Algorithms (2007), pp. 91–120

    Google Scholar 

  50. R. Qu, E.K. Burke, Hybridisations withing a graph based hyper-heuristic framework for university timetabling problems. J. Oper. Res. Soc. 60, 1273–1285 (2009)

    Article  Google Scholar 

  51. R. Qu, G. Kendall, N. Pillay, The general combinatorial optimisation problem - towards automated algorithm design. IEEE Comput. Intell. Mag. 15(2), 14–23 (2020)

    Article  Google Scholar 

  52. I.C.O. Ramos, M.C. Goldbarg, E.G. Goldbarg, A.D.D. Neto, Logistic regression for parameter tuning on an evolutionary algorithm, in 2005 IEEE Congress on Evolutionary Computation (Edinburgh, Scotland, 2-5 Sept. 2005)

    Google Scholar 

  53. M.-C. Riff, E. Montero, A new algorithm for reducing metaheuristic design effort, in 2013 IEEE Congress on Evolutionary Computation (Mexico, 2013)

    Google Scholar 

  54. S.K. Smit, A.E. Eiben, Comparing parameter tuning methods for evolutionary algorithms, in 2009 IEEE Congress on Evolutionary Computation (Trondheim, Norway, 2009)

    Google Scholar 

  55. K. Tang, F. Peng, G. Chen, X. Yao, Population-based algorithm portfolios with automated constituent algorithms selection. Inf. Sci. 279, 94–104 (2014)

    Article  Google Scholar 

  56. H. Terashima-Marín, P. Ross, M. Valenzuela-Rendón, Evolution of constraint satisfaction strategies in examination timetabling, in GECCO’99: Proceedings of the 1st Annual Conference on Genetic and Evolutionary Computation (1999), pp. 635–642

    Google Scholar 

  57. A.A. Visheratin, M. Melnik, D. Nasonov, Automatic workflow scheduling tuning for distributed processing systems. Procedia Comput. Sci. 101, 388–397 (2016)

    Article  Google Scholar 

  58. D.H. Wolpert, W.G. McReady, No free lunch theorems for optimisation. IEEE Trans. Evol. Comput. 1(1), 67–82 (1997)

    Article  Google Scholar 

  59. L. Xu, H. Hoos, K. Leyton-Brown, Hydra: automatically configuring algorithms for portfolio-based selection, in Proceedings of AAAI Conference on Artificial Intelligence (Atlanta, July 11–15, 2010)

    Google Scholar 

  60. L. Xu, F. Hutter, H.H. Hoos, K. Leyton-Brown, Satzilla: portfolio-based algorithm selection for sat. J. Artif. Intell. Res. 32, 565–606 (2008)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Nottingham, Nottingham, United Kingdom

    Rong Qu

Authors
  1. Rong Qu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rong Qu .

Editor information

Editors and Affiliations

  1. Dept. of Computer Science, University of Pretoria, Pretoria, South Africa

    Nelishia Pillay

  2. School of Computer Science, University of Nottingham, Nottingham, UK

    Rong Qu

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Qu, R. (2021). A General Model for Automated Algorithm Design. In: Pillay, N., Qu, R. (eds) Automated Design of Machine Learning and Search Algorithms. Natural Computing Series. Springer, Cham. https://doi.org/10.1007/978-3-030-72069-8_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-72069-8_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-72068-1

  • Online ISBN: 978-3-030-72069-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation