Contrasting meta-learning and hyper-heuristic research: the role of evolutionary algorithms

  • Gisele L. Pappa
  • Gabriela Ochoa
  • Matthew R. Hyde
  • Alex A. Freitas
  • John Woodward
  • Jerry Swan


The fields of machine meta-learning and hyper-heuristic optimisation have developed mostly independently of each other, although evolutionary algorithms (particularly genetic programming) have recently played an important role in the development of both fields. Recent work in both fields shares a common goal, that of automating as much of the algorithm design process as possible. In this paper we first provide a historical perspective on automated algorithm design, and then we discuss similarities and differences between meta-learning in the field of supervised machine learning (classification) and hyper-heuristics in the field of optimisation. This discussion focuses on the dimensions of the problem space, the algorithm space and the performance measure, as well as clarifying important issues related to different levels of automation and generality in both fields. We also discuss important research directions, challenges and foundational issues in meta-learning and hyper-heuristic research. It is important to emphasize that this paper is not a survey, as several surveys on the areas of meta-learning and hyper-heuristics (separately) have been previously published. The main contribution of the paper is to contrast meta-learning and hyper-heuristics methods and concepts, in order to promote awareness and cross-fertilisation of ideas across the (by and large, non-overlapping) different communities of meta-learning and hyper-heuristic researchers. We hope that this cross-fertilisation of ideas can inspire interesting new research in both fields and in the new emerging research area which consists of integrating those fields.


Hyper-heuristics Meta-learning Genetic programming Automated algorithm design 



G. L. Pappa, A. A. Freitas, J. Woodward and J. Swan would like to thank the 2011 GECCO’s workshop organizers for putting them in contact. G. L. Pappa was partially supported by CNPq, CAPES and FAPEMIG, all Brazilian Research Agencies. G. Ochoa and J. Swan were supported by UK Research Council EPSRC, grants EP/F033214/1 and EP/D061571/1. J. Wooward was supported by UK Research Council EPSRC, grant EP/J017515/1. A.A. Freitas was partially supported by UK Research Council EPSRC, grant EP/H020217/1.


  1. 1.
    C. Adami, T.C. Brown, Evolutionary learning in the 2d artificial life system avida. in Artificial Life IV, ed. by R.A. Brooks, P. Maes (MIT Press, Cambridge, 1994), pp. 377–381Google Scholar
  2. 2.
    D.W. Aha, Generalizing from case studies: A case study. in Proceedings of the Ninth International Conference on Machine Learning. (Morgan Kaufmann, Burlington, 1992), pp. 1–10Google Scholar
  3. 3.
    R. Aler, D. Borrajo, P. Isasi, Evolving heuristics for planning. in Lecture Notes in Computer Science. (1998)Google Scholar
  4. 4.
    R. Aler, D. Borrajo, P. Isasi, Learning to solve planning problems efficiently by means of genetic programming. Evol. Comput. 9(4), 387–420 (2001)CrossRefGoogle Scholar
  5. 5.
    R. Aler, D. Borrajo, P. Isasi, Using genetic programming to learn and improve control knowledge. Artif. Intell. 141(1-2), 2956 (2002)CrossRefGoogle Scholar
  6. 6.
    P.J. Angeline, Adaptive and self-adaptive evolutionary computations. in Computational Intelligence: A Dynamic Systems Perspective. (IEEE Press, New York, 1995), pp. 152–163Google Scholar
  7. 7.
    T. Bäck, An overview of parameter control methods by self-adaption in evolutionary algorithms. Fundam. Inf. 35(1-4), 51–66 (1998)zbMATHGoogle Scholar
  8. 8.
    W. Banzhaf, P. Nordin, R.E. Keller, F.D. Francone, Genetic Programming: An Introduction; On the Automatic Evolution of Computer Programs and its Applications. (Morgan Kaufmann, San Francisco, 1998)CrossRefzbMATHGoogle Scholar
  9. 9.
    R.C. Barros, M.P. Basgalupp, A.C. de Carvalho, A.A. Freitas, A hyper-heuristic evolutionary algorithm for automatically designing decision-tree algorithms. in Proceedings of the fourteenth international conference on Genetic and evolutionary computation conference, GECCO ’12. (2012), pp. 1237–1244Google Scholar
  10. 10.
    D. Borrajo, M. Veloso, Lazy incremental learning of control knowledge for efficiently obtaining quality plans. AI Rev. J. Spec. Issue Lazy Learn. 11, 371–405 (1996)Google Scholar
  11. 11.
    P. Brazdil, C. Giraud-Carrier, C. Soares, R. Vilalta, Metalearning: Applications to Data Mining. (Springer, Berlin, 2008)Google Scholar
  12. 12.
    P.B. Brazdil, C. Soares, J.P. Da Costa, Ranking learning algorithms: using ibl and meta-learning on accuracy and time results. Mach. Learn. 50(3), 251–277 (2003)CrossRefzbMATHGoogle Scholar
  13. 13.
    L. Breiman, Bagging predictors. Mach. Learn. 24, 123–140 (1996)zbMATHMathSciNetGoogle Scholar
  14. 14.
    E.K. Burke, E. Hart, G. Kendall, J. Newall, P. Ross, S. Schulenburg, Hyper-heuristics: an emerging direction in modern search technology. in Handbook of Metaheuristics, ed. by F. Glover, G. Kochenberger (Kluwer, Dordrecht, 2003), pp. 457–474Google Scholar
  15. 15.
    E.K. Burke, M. Hyde, G. Kendall, G. Ochoa, E. Ozcan, J. Woodward, Exploring hyper-heuristic methodologies with genetic programming. in Computational Intelligence: Collaboration, Fusion and Emergence, Intelligent Systems Reference Library. ed. by C. Mumford, L. Jain (Springer, Berlin, 2009), pp. 177–201CrossRefGoogle Scholar
  16. 16.
    E.K. Burke, M. Hyde, G. Kendall, G. Ochoa, E. Ozcan, J. Woodward, Handbook of Metaheuristics, International Series in Operations Research & Management Science, vol. 146, chap. A Classification of Hyper-heuristic Approaches. (Springer 2010), Chapter 15, pp. 449–468Google Scholar
  17. 17.
    E.K. Burke, M. Hyde, G. Kendall, J. Woodward, Automatic heuristic generation with genetic programming: evolving a jack-of-all-trades or a master of one. in Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2007). (London, UK 2007), pp. 1559–1565Google Scholar
  18. 18.
    E.K. Burke, M.R. Hyde, G. Kendall, Grammatical evolution of local search heuristics. IEEE Transactions on Evolutionary Computation 16(3), 406–417 (2012)CrossRefGoogle Scholar
  19. 19.
    E.K. Burke, M.R. Hyde, G. Kendall, J. Woodward, A genetic programming hyper-heuristic approach for evolving two dimensional strip packing heuristics. IEEE Transactions on Evolutionary Computation 14(6), 942–958 (2010)CrossRefGoogle Scholar
  20. 20.
    E.K. Burke, M.R. Hyde, G. Kendall, J. Woodward, Automating the packing heuristic design process with genetic programming. Evol. Comput. 20(1), 63–89 (2012)CrossRefGoogle Scholar
  21. 21.
    E.K. Burke, G. Kendall, J.D. Landa-Silva, R. O’Brien, E. Soubeiga, An ant algorithm hyperheuristic for the project presentation scheduling problem. in Proceedings of the 2005 IEEE Congress on Evolutionary Computation, vol. 3. (2005), pp. 2263–2270Google Scholar
  22. 22.
    E.K. Burke, G. Kendall, E. Soubeiga, A tabu-search hyperheuristic for timetabling and rostering. J. Heuristics 9(6), 451–470 (2003)CrossRefGoogle Scholar
  23. 23.
    E.K. Burke, B. McCollum, A. Meisels, S. Petrovic, R. Qu, A graph-based hyper-heuristic for educational timetabling problems. Eur. J. Oper. Res. 176, 177–192 (2007)CrossRefzbMATHMathSciNetGoogle Scholar
  24. 24.
    E.K. Burke, S. Petrovic, R. Qu, Case based heuristic selection for timetabling problems. J. Sched. 9(2), 115–132 (2006)CrossRefzbMATHGoogle Scholar
  25. 25.
    J. Cano-Belmán, R. Ríos-Mercado, J. Bautista, A scatter search based hyper-heuristic for sequencing a mixed-model assembly line. J. Heuristics 16, 749–770 (2010)CrossRefzbMATHGoogle Scholar
  26. 26.
    E. Cantu-Paz, C. Kamath, An empirical comparison of combinations of evolutionary algorithms and neural networks for classification problems. IEEE Trans. Syst. Man Cybern. Part B Cybern. 35(5), 915–927 (2005)CrossRefGoogle Scholar
  27. 27.
    K. Chakhlevitch, P.I. Cowling, Hyperheuristics: Recent developments. in Adaptive and Multilevel Metaheuristics Studies in Computational Intelligence, vol. 136, ed. by C. Cotta, M. Sevaux, K. Sörensen (Springer, Berlin, 2008), pp. 3–29CrossRefGoogle Scholar
  28. 28.
    A. Chandra, X. Yao, Ensemble learning using multi-objective evolutionary algorithms. J Math. Model. Algorithms 5, 417–445 (2006)CrossRefzbMATHMathSciNetGoogle Scholar
  29. 29.
    P.C. Chen, G. Kendall, G. Vanden Berghe, An ant based hyper-heuristic for the travelling tournament problem. in Proceedings of IEEE Symposium of Computational Intelligence in Scheduling (CISched 2007), (2007), pp. 19–26Google Scholar
  30. 30.
    P. Cowling, G. Kendall, E. Soubeiga, A hyperheuristic approach for scheduling a sales summit. in Selected Papers of the Third International Conference on the Practice And Theory of Automated Timetabling, PATAT 2000, LNCS (Springer, Konstanz, Germany, 2000), pp. 176–190Google Scholar
  31. 31.
    P. Cowling, G. Kendall, E. Soubeiga, A hyperheuristic approach for scheduling a sales summit. in Selected Papers of the Third International Conference on the Practice And Theory of Automated Timetabling, PATAT 2000 (Springer, Berlin, 2001), pp. 176–190Google Scholar
  32. 32.
    L. Cruz-Reyes, C. Gómez-Santillán, J. Pérez-Ortega, V. Landero, M. Quiroz, A. Ochoa, Intelligent Systems, chap. Algorithm Selection: From Meta-Learning to Hyper-Heuristics. (InTech, 2012), pp. 77–102Google Scholar
  33. 33.
    A. Cuesta-Cañada, L. Garrido, H. Terashima-Marin, Building hyper-heuristics through ant colony optimization for the 2d bin packing problem. in Knowledge-Based Intelligent Information and Engineering Systems. ed. by R. Khosla, R. Howlett, L. Jain (Springer, Berlin, 2005), p. 907Google Scholar
  34. 34.
    R. Curry, P. Lichodzijewski, M. Heywood, Scaling genetic programming to large datasets using hierarchical dynamic subset selection. IEEE Trans. Syst. Man Cybern. Part B Cybern. 37(4), 1065–1073 (2007)CrossRefGoogle Scholar
  35. 35.
    C. Dimopoulos, A.M.S. Zalzala, Investigating the use of genetic programming for a classic one-machine scheduling problem. Adv. Eng. Softw. 32(6), 489–498 (2001)CrossRefzbMATHGoogle Scholar
  36. 36.
    L.S. Diosan, M. Oltean, Evolving evolutionary algorithms using evolutionary algorithms. in Proceedings of the 2007 GECCO conference companion on Genetic and evolutionary computation, GECCO ’07. (New York, NY, USA, 2007), pp. 2442–2449Google Scholar
  37. 37.
    K.A. Dowsland, E. Soubeiga, E.K. Burke, A simulated annealing hyper-heuristic for determining shipper sizes. Eur. J. Oper. Res. 179(3), 759–774 (2007)CrossRefzbMATHGoogle Scholar
  38. 38.
    B. Edmonds, Meta-genetic programming: Co-evolving the operators of variation. Tech. rep., Centre for Policy Modelling, Manchester Metropolitan University (1998)Google Scholar
  39. 39.
    B. Edmonds, Meta-genetic programming: Co-evolving the operators of variation. Turk. J. Elec. Engin. 9(1), 13–29 (2001)Google Scholar
  40. 40.
    A.E. Eiben, Z. Michalewicz, M. Schoenauer, J.E. Smith, Parameter control in evolutionary algorithms. IEEE Trans. Evol. Comput. 2(3), 124–141 (1999)CrossRefGoogle Scholar
  41. 41.
    A. Elyasaf, A. Hauptman, M. Sipper, Ga-freecell: evolving solvers for the game of freecell. in Proceedings of the 13th annual conference on Genetic and evolutionary computation, GECCO ’11. (ACM, New York, NY, USA, 2011), pp. 1931–1938Google Scholar
  42. 42.
    A. Elyasaf, A. Hauptman, M. Sipper, Evolutionary design of freecell solvers. IEEE Trans. Comput. Intell. AI Games 4(4), 270–281 (2012)CrossRefGoogle Scholar
  43. 43.
    H.L. Fang, P. Ross, D. Corne, A promising genetic algorithm approach to job shop scheduling, rescheduling, and open-shop scheduling problems. in 5th International Conference on Genetic Algorithms ed. by S. Forrest (Morgan Kaufmann, San Mateo, 1993), pp. 375–382Google Scholar
  44. 44.
    R. Fikes, N.J. Nilsson, Strips: a new approach to the application of theorem proving to problem solving. in IJCAI. (1971), pp. 608–620Google Scholar
  45. 45.
    H. Fisher, G.L. Thompson, Probabilistic learning combinations of local job-shop scheduling rules. in Industrial Scheduling, ed. by J.F. Muth, G.L. Thompson (Prentice-Hall, Inc, New Jersey, 1963), pp. 225–251Google Scholar
  46. 46.
    D. Floreano, P. Durr, C. Mattiussi, Neuroevolution: from architectures to learning. Evol. Intel. 1, 47–62 (2008)CrossRefGoogle Scholar
  47. 47.
    A. Frank, A. Asuncion, UCI machine learning repository (2010).
  48. 48.
    P.W. Frey, D.J. Slate, Letter recognition using holland-style adaptive classifiers. Mach. Learn. 6, 161–182 (1991)Google Scholar
  49. 49.
    A.S. Fukunaga, Automated discovery of local search heuristics for satisfiability testing. Evol. Comput. 16(1), 31–61 (2008)CrossRefGoogle Scholar
  50. 50.
    M.R. Garey, D.S. Johnson, Computers and Intractability: A Guide to the theory of NP-Completeness. (W.H. Freeman and Company, San Fransisco, 1979)zbMATHGoogle Scholar
  51. 51.
    P. Garrido, M. Riff, Dvrp: A hard dynamic combinatorial optimisation problem tackled by an evolutionary hyper-heuristic. J. Heuristics 16, 795–834 (2010)CrossRefzbMATHGoogle Scholar
  52. 52.
    C.D. Geiger, R. Uzsoy, H. Aytug, Rapid modeling and discovery of priority dispatching rules: An autonomous learning approach. J. Sched. 9(1), 7–34 (2006)CrossRefzbMATHGoogle Scholar
  53. 53.
    L. Georgiou, W.J. Teahan, jGE: a java implementation of grammatical evolution. in Proceedings of the 10th WSEAS International Conference on Systems. (World Scientific and Engineering Academy and Society (WSEAS), 2006), pp. 410–415Google Scholar
  54. 54.
    M. Ghallab, C.K. Isi, S. Penberthy, D.E. Smith, Y. Sun, D. Weld, PDDL - The Planning Domain Definition Language. Tech. Rep. CVC TR-98-003/DCS TR-1165, Yale Center for Computational Vision and Control (1998)Google Scholar
  55. 55.
    F. Glover, Future paths for integer programming and links to artificial intelligence. Comput. Opert. Res. 13(5), 533–549 (1986)CrossRefzbMATHMathSciNetGoogle Scholar
  56. 56.
    H.J. Goldsby, B.H. Cheng, Avida-mde: a digital evolution approach to generating models of adaptive software behavior. in Proceedings of the 10th Annual Conference on Genetic and Evolutionary Computation, GECCO ’08 (2008), pp. 1751–1758Google Scholar
  57. 57.
    J. Gratch, S. Chien, Adaptive problem-solving for large-scale scheduling problems: a case study. J. Artif. Intel. Res. 4, 365–396 (1996)Google Scholar
  58. 58.
    J. Grefenstette, Optimization of control parameters for genetic algorithms. IEEE Trans. Syst. Man Cybern. 16(1), 122–128 (1986)CrossRefGoogle Scholar
  59. 59.
    J.J. Grefenstette, Optimization of control parameters for genetic algorithms. IEEE Trans. Syst. Man Cybern. SMC-16(1), 122–128 (1986)CrossRefGoogle Scholar
  60. 60.
    A. Hauptman, A. Elyasaf, M. Sipper, A. Karmon, Gp-rush: using genetic programming to evolve solversforthe rushhour puzzle. in Genetic and evolutionary computation (GECCO 2009). (ACM, 2009), pp. 955–962Google Scholar
  61. 61.
    M.R. Hyde, E.K. Burke, G. Kendall, Automated code generation by local search. J. Oper. Res. Soc. (2012). doi: 10.1057/jors.2012.149
  62. 62.
    A. Keleş, A. Yayimli, A.C. Uyar, Ant based hyper heuristic for physical impairment aware routing and wavelength assignment. in Proceedings of the 33rd IEEE conference on Sarnoff. (Piscataway, NJ, USA, 2010), pp. 90–94Google Scholar
  63. 63.
    Y. Kodratoff, D. Sleeman, M. Uszynski, K. Causse, S. Craw, Building a machine learning toolbox. in Enhancing the Knowledge Engineering Process, ed. by Steels L., Lepape (1992), pp. 81–108Google Scholar
  64. 64.
    J.R. Koza, Genetic Programming: On the Programming of Computers by the Means of Natural Selection. (The MIT Press, Massachusetts, 1992)zbMATHGoogle Scholar
  65. 65.
    O. Kramer, Evolutionary self-adaptation: a survey of operators and strategy parameters. Evol. Intel. 3, 51–65 (2010)CrossRefzbMATHGoogle Scholar
  66. 66.
    N. Krasnogor, S. Gustafson, A study on the use of “self-generation” in memetic algorithms. Nat. Comput. 3(1), 53–76 (2004)CrossRefzbMATHMathSciNetGoogle Scholar
  67. 67.
    C.W.G. Lasarczyk, P. Dittrich, J.C.F. Bioinformatics, W. Banzhaf, Dynamic subset selection based on a fitness case topology. Evol. Comput. 12, 223–242 (2004)CrossRefGoogle Scholar
  68. 68.
    J. Levine, D. Humphreys (2003) Learning action strategies for planning domains using genetic programming. in EvoWorkshops. (2003), pp. 684–695Google Scholar
  69. 69.
    J. Levine, H. Westerberg, M. Galea, D. Humphreys, Evolutionary-based learning of generalised policies for ai planning domains. in Proceedings of the 11th Annual Conference on Genetic and Evolutionary Computation, GECCO ’09. (ACM, New York, 2009), pp. 1195–1202Google Scholar
  70. 70.
    K. Leyton-Brown, E. Nudelman, Y. Shoham, Learning the empirical hardness of optimization problems: The case of combinatorial auctions. in Principles and Practice of Constraint Programming - CP 2002, Lecture Notes in Computer Science, vol. 2470, ed. by P. Van Hentenryck (Springer, Berlin, 2002), pp. 91–100Google Scholar
  71. 71.
    J. Maturana, F. Lardeux, F. Saubion, Autonomous operator management for evolutionary algorithms. J. Heuristics 16, 881–909 (2010)CrossRefzbMATHGoogle Scholar
  72. 72.
    D.V. McDermott, Pddl2.1 - the art of the possible? commentary on fox and long. J. Artif. Intell. Res. (JAIR) 20, 145–148 (2003)zbMATHGoogle Scholar
  73. 73.
    D. Michie, D. Spiegelhalter, C. Taylor (eds), Machine Learning, Neural and Statistical Classification. (Ellis Horwood, Chichester, 1994)zbMATHGoogle Scholar
  74. 74.
    S. Minton, Automatically configuring constraint satisfaction problems: a case study. Constraints 1(1), 7–43 (1996)CrossRefMathSciNetGoogle Scholar
  75. 75.
    T. Mitchell, Machine Learning (Mcgraw-Hill International Edit), 1st edn. (McGraw-Hill Education, New York, (ISE Editions), 1997)Google Scholar
  76. 76.
    A.Y. Ng, Preventing overfitting of cross-validation data. in Proceedings of the Fourteenth International Conference on Machine Learning, ICML ’97. (Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1997), pp. 245–253Google Scholar
  77. 77.
    M. Nicolau, libGE: Grammatical evolution library for c++. Available from: (2006)
  78. 78.
    G. Ochoa, M. Hyde, T. Curtois, J. Vazquez-Rodriguez, J. Walker, M. Gendreau, G. Kendall, B. McCollum, A. Parkes, S. Petrovic, E. Burke, HyFlex: A Benchmark Framework for Cross-domain Heuristic Search 7245, 136–147 (2012)Google Scholar
  79. 79.
    G. Ochoa, R. Qu, E.K. Burke, Analyzing the landscape of a graph based hyper-heuristic for timetabling problems. in Proceedings of Genetic and Evolutionary Computation Conference (GECCO 2009). (Montreal, Canada, 2009)Google Scholar
  80. 80.
    G. Ochoa, J.A. Váquez-Rodríguez, S. Petrovic, E.K. Burke, Dispatching rules for production scheduling: a hyper-heuristic landscape analysis. in Proceedings of the IEEE Congress on Evolutionary Computation (CEC 2009). (Montreal, Norway, 2009)Google Scholar
  81. 81.
    G. Ochoa, J. Walker, M. Hyde, T. Curtois, Adaptive evolutionary algorithms and extensions to the hyflex hyper-heuristic framework. in Parallel Problem Solving from Nature - PPSN 2012, vol. 7492. (Springer, Berlin, 2012), pp. 418–427Google Scholar
  82. 82.
    C. Ofria, C.O. Wilke, Avida: A software platform for research in computational evolutionary biology. Artif. Life 10(2), 191–229 (2004)CrossRefGoogle Scholar
  83. 83.
    M. Oltean, Evolving evolutionary algorithms using linear genetic programming. Evol. Comput. 13, 387–410 (2005)CrossRefGoogle Scholar
  84. 84.
    M. O’Neill, E. Hemberg, C. Gilligan, E. Bartley, J. McDermott, A. Brabazon, GEVA: Grammatical evolution in Java. SIGEVOlution 3(2), (2008)Google Scholar
  85. 85.
    E. Özcan, B. Bilgin, E.E. Korkmaz, A comprehensive analysis of hyper-heuristics. Intell. Data Anal. 12(1), 3–23 (2008)Google Scholar
  86. 86.
    G.L. Pappa, A.A. Freitas, Automatically evolving rule induction algorithms tailored to the prediction of postsynaptic activity in proteins. Intell. Data Anal. 13(2), 243–259 (2009)Google Scholar
  87. 87.
    G.L. Pappa, A.A. Freitas, Automating the Design of Data Mining Algorithms: An Evolutionary Computation Approach. (Springer, Berlin, 2009)Google Scholar
  88. 88.
    D. Pisinger, S. Ropke, A general heuristic for vehicle routing problems. Comput. Oper. Res. 34, 2403–2435 (2007)CrossRefzbMATHMathSciNetGoogle Scholar
  89. 89.
    R. Poli, L. Vanneschi, W.B. Langdon, N.F. McPhee, Theoretical results in genetic programming: the next ten years? Genet. Program. Evolvable Mach. 11(3-4), 285–320 (2010)CrossRefGoogle Scholar
  90. 90.
    D. Posada, K.A. Crandall, Modeltest: testing the model of dna substitution. Bioinformatics 14(9), 817–818 (1998)CrossRefGoogle Scholar
  91. 91.
    R. Qu, E.K. Burke, Hybridisations within a graph based hyper-heuristic framework for university timetabling problems. J. Oper. Res. Soc. 60, 1273–1285 (2009)CrossRefzbMATHGoogle Scholar
  92. 92.
    R.B. Rao, D. Gordon, W. Spears, For every generalization action, is there really an equal and opposite reaction? Analysis of the conservation law for generalization performance. in Proc. of the 12th International Conference on Machine Learning. (Morgan Kaufmann, 1995), pp. 471–479Google Scholar
  93. 93.
    I. Rechenberg, Evolutionsstrategie: Optimierung technischer Systeme nach Prinzipien der biologischen Evolution. (Frommann-Holzboog, Stuttgart, 1973)Google Scholar
  94. 94.
    J.R. Rice, The algorithm selection problem. Adv. Comput. 15, 65–118 (1976)CrossRefGoogle Scholar
  95. 95.
    L. Rokach, Taxonomy for characterizing ensemble methods in classification tasks: A review and annotated bibliography. Comput. Stat. Data Anal. 53(12), 4046–4072 (2009)CrossRefzbMATHMathSciNetGoogle Scholar
  96. 96.
    P. Ross, Hyper-heuristics. in Search Methodologies: Introductory Tutorials in Optimization and Decision Support Techniques, chap. 17, ed. by E.K. Burke, G. Kendall (Springer, Berlin, 2005), pp. 529–556CrossRefGoogle Scholar
  97. 97.
    P. Ross, J.G. Marín-Blázquez, Constructive hyper-heuristics in class timetabling. in IEEE Congress on Evolutionary Computation. (2005), pp. 1493–1500Google Scholar
  98. 98.
    P. Ross, S. Schulenburg, J.G. Marin-Blazquez, E. Hart, Hyper-heuristics: learning to combine simple heuristics in bin-packing problem. in Proceedings of the Genetic and Evolutionary Computation Conference, GECCO’02 (2002)Google Scholar
  99. 99.
    O. Roux, C. Fonlupt, Ant programming: or how to use ants for automatic programming. in Proceedings of ANTS’2000, ed. by M. Dorigo, E. Al (Brussels, Belgium, 2000), pp. 121–129Google Scholar
  100. 100.
    A. Salehi-Abari, T. White, Enhanced generalized ant programming. in Proceedings of the 2008 Genetic and Evolutionary Computation Conference GECCO. (ACM Press, 2008), pp. 111–118Google Scholar
  101. 101.
    C. Schaffer, A conservation law for generalization performance. in Proc. of the 11th International Conference on Machine Learning. (Morgan Kaufmann, 1994), pp. 259–265Google Scholar
  102. 102.
    R. Schapire, The strength of weak learnability. Mach. Learn. 5, 197–227 (1990)Google Scholar
  103. 103.
    H.P. Schwefel, Numerische Optimierung von Computer-Modellen Mittels der Evolutionstrategie, ISR, vol. 26. (Birkhaeuser, Basel/Stuttgart, 1977)CrossRefGoogle Scholar
  104. 104.
    Y. Shan, R. McKay, D. Essam, H. Abbass, A survey of probabilistic model building genetic programming. in Scalable Optimization via Probabilistic Modeling: From Algorithms to Applications, ed. by M. Pelikan, K. Sastry, E. Cantu-Paz (Springer, Berlin, UK, 2006), pp. 121–160CrossRefGoogle Scholar
  105. 105.
    K. Smith-Miles, Towards insightful algorithm selection for optimisation using meta-learning concepts. in Proc. of IEEE International Joint Conference on Neural Networks IJCNN 2008. (2008), pp. 4118–4124Google Scholar
  106. 106.
    K.A. Smith-Miles, Cross-disciplinary perspectives on meta-learning for algorithm selection. ACM Comput. Surv. 41, 6:1–6:25 (2008)CrossRefGoogle Scholar
  107. 107.
    L. Spector, Towards Practical Autoconstructive Evolution: Self-Evolution of Problem-Solving Genetic Programming Systems, vol. 8 (Springer, Berlin, 2010), pp. 17–33Google Scholar
  108. 108.
    K.O. Stanley, R. Miikkulainen, Evolving neural networks through augmenting topologies. Evol. Comput. 10, 99–127 (2002)CrossRefGoogle Scholar
  109. 109.
    T. Stutzle, S. Fernandes, New Benchmark Instances for the QAP and the Experimental Analysis of Algorithms, Lecture Notes in Computer Science, vol. 3004. (Springer, Berlin/Heidelberg, 2004), pp. 199–209Google Scholar
  110. 110.
    E.G. Talbi, Metaheuristics: From Design to Implementation. (Wiley, London, 2009)CrossRefGoogle Scholar
  111. 111.
    J.C. Tay, N.B. Ho, Evolving dispatching rules using genetic programming for solving multi-objective flexible job-shop problems. Comput. Ind. Eng. 54, 453–473 (2008)CrossRefGoogle Scholar
  112. 112.
    H. Terashima-Marin, E.J. Flores-Alvarez, P. Ross, Hyper-heuristics and classifier systems for solving 2D-regular cutting stock problems. in Proceedings of the Genetic and Evolutionary Computation Conference GECCO 2005. (2005), pp. 637–643Google Scholar
  113. 113.
    H. Terashima-Marin, P. Ross, C.J. Farias Zarate, E. Lopez-Camacho, M. Valenzuela-Rendon, Generalized hyper-heuristics for solving 2D regular and irregular packing problems. Ann. Oper. Res. 179(1), 369–392 (2010)CrossRefzbMATHMathSciNetGoogle Scholar
  114. 114.
    A. Vella, D. Corne, C. Murphy, Hyper-heuristic decision tree induction. in Nature Biologically Inspired Computing, 2009. NaBIC 2009. (World Congress on, 2009), pp. 409 – 414Google Scholar
  115. 115.
    R. Vilalta, Y. Drissi, A perspective view and survey of meta-learning. Artif. Intell. Rev. 18, 77–95 (2002)CrossRefGoogle Scholar
  116. 116.
    G. Wäscher, H. Haußner, H. Schumann, An improved typology of cutting and packing problems. European Journal of Operational Research 183(3), 1109–1130 (2007)CrossRefzbMATHGoogle Scholar
  117. 117.
    D.H. Wolpert, Stacked generalization. Neural Netw. 5, 241–259 (1992)CrossRefGoogle Scholar
  118. 118.
    D.H. Wolpert, W.G. Macready, No free lunch theorems for optimization. IEEE Trans. Evol. Comput. 1(1), 67–82 (1997)CrossRefGoogle Scholar
  119. 119.
    X. Yao, Evolving artificial neural networks. Proc. IEEE 87(9), 1423–1447 (1999)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Gisele L. Pappa
    • 1
  • Gabriela Ochoa
    • 2
  • Matthew R. Hyde
    • 3
  • Alex A. Freitas
    • 4
  • John Woodward
    • 2
  • Jerry Swan
    • 2
  1. 1.Computer Science DepartmentUniversidade Federal de Minas GeraisBelo HorizonteBrazil
  2. 2.Computing Science and MathematicsUniversity of StirlingStirlingScotland, UK
  3. 3.School of Environmental SciencesUniversity of East AngliaNorwichUK
  4. 4.School of ComputingUniversity of KentKentUK

Personalised recommendations