Artificial Intelligence Review

, Volume 46, Issue 3, pp 327–349 | Cite as

Combine and conquer: an evolutionary hyper-heuristic approach for solving constraint satisfaction problems

  • José Carlos Ortiz-BaylissEmail author
  • Hugo Terashima-Marín
  • Santiago Enrique Conant-Pablos


Selection hyper-heuristics are a technology for optimization in which a high-level mechanism controls low-level heuristics, so as to be capable of solving a wide range of problem instances efficiently. Hyper-heuristics are used to generate a solution process rather than producing an immediate solution to a given problem. This process is a re-usable mechanism that can be applied both to seen and unseen problem instances. In this paper, we propose a selection hyper-heuristic process with the intention to rise the level of generality and solve consistently well a wide range of constraint satisfaction problems. The hyper-heuristic technique is based on a messy genetic algorithm that generates high-level heuristics formed by rules (condition \(\rightarrow \) heuristic). The high-level heuristics produced are seen to be good at solving instances from certain parts of the parameterized space of problems, producing results using effort comparable to the best single heuristic per instance. This is beneficial, as the choice of best heuristic varies from instance to instance, so the high-level heuristics are definitely preferable to selecting any one low-level heuristic for all instances. The results confirm the robustness of the proposed approach and how high-level heuristics trained for some specific classes of instances can also be applied to unseen classes without significant lost of efficiency. This paper contributes to the understanding of heuristics and the way they can be used in a collaborative way to benefit from their combined strengths.


Heuristics Hyper-heuristics Constraint satisfaction Genetic algorithms 



This research was supported in part by ITESM Strategic Project PRY075, CONACyT Basic Science Projects Under Grants 99695 and 241461, and ITESM Research Group with Strategic Focus in Intelligent Systems.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.


  1. Achlioptas D, Molloy MSO, Kirousis LM, Stamatiou YC, Kranakis E, Krizanc D (2001) Random constraint satisfaction: a more accurate picture. Constraints 6(4):329–344MathSciNetzbMATHCrossRefGoogle Scholar
  2. Berlier J, McCollum J (2010) A constraint satisfaction algorithm for microcontroller selection and pin assignment. In: Proceedings of the IEEE SoutheastCon 2010 (SoutheastCon), pp 348–351Google Scholar
  3. Bitner JR, Reingold EM (1975) Backtrack programming techniques. Commun ACM 18:651–656zbMATHCrossRefGoogle Scholar
  4. Bittle SA, Fox MS (2009) Learning and using hyper-heuristics for variable and value ordering in constraint satisfaction problems. GECCO ’09: proceedings of the 11th annual conference companion on genetic and evolutionary computation conference. ACM, New York, pp 2209–2212CrossRefGoogle Scholar
  5. Brailsford SC, Potts CN, Smith BM (1999) Constraint satisfaction problems: algorithms and applications. Eur J Oper Res 119(3):557–581zbMATHCrossRefGoogle Scholar
  6. Burke E, Hart E, Kendall G, Newall J, Ross P, Schulenburg S (2003) Hyper-heuristics: an emerging direction in modern research technolology. In: Glover FW, Kochenberger GA (eds) Handbook of metaheuristics. Kluwer, Boston, MA, pp 457–474Google Scholar
  7. Burke EK, Gendrau M, Hyde M, Kendall G, Ochoa G, Özcan E, Qu R (2012) Hyper-heuristics: a survey of the state of the art. J Oper Res Soc 64:1695–1724CrossRefGoogle Scholar
  8. Cheeseman P, Kanefsky B, Taylor WM (1991) Where the really hard problems are. In: Proceedings of IJCAI, pp 331–337Google Scholar
  9. Craenen BGW, Eiben AE, van Hemert JI (2003) Comparing evolutionary algorithms on binary constraint satisfaction problems. IEEE Trans Evol Comput 7:424–444CrossRefGoogle Scholar
  10. Crawford B, Soto R, Castro C, Monfroy E (2011) A hyperheuristic approach for dynamic enumeration strategy selection in constraint satisfaction. In: Proceedings of the 4th international conference on interplay between natural and artificial computation: new challenges on bioinspired applications—volume part II, IWINAC’11. Springer, Berlin, pp 295–304Google Scholar
  11. Crawford B, Soto R, Monfroy E, Palma W, Castro C, Paredes F (2013) Parameter tuning of a choice-function based hyperheuristic using particle swarm optimization. Expert Syst Appl 40(5):1690–1695CrossRefGoogle Scholar
  12. Diao R, Shen Q (2015) Nature inspired feature selection meta-heuristics. Artif Intell Rev 44(3):311–340. doi: 10.1007/s10462-015-9428-8 CrossRefGoogle Scholar
  13. Dunkin N, Allen S (1997) Frequency assignment problems: representations and solutions. Tech Rep CSD-TR-97-14, University of LondonGoogle Scholar
  14. Epstein SL, Freuder EC, Wallace R, Morozov A, Samuels B (2002) The adaptive constraint engine. In: Proceedings of the 8th international conference on principles and practice of constraint programming. Springer, London, CP ’02, pp 525–542Google Scholar
  15. Freuder EC (1978) Synthesizing constraint expressions. Commun ACM 21:958–966MathSciNetzbMATHCrossRefGoogle Scholar
  16. Gent I, MacIntyre E, Prosser P, Smith B, Walsh T (1996) An empirical study of dynamic variable ordering heuristics for the constraint satisfaction problem. In: Proceedings of CP-96, pp 179–193Google Scholar
  17. Ginsberg M (1993) Dynamic backtracking. Artif Intell 1:25–46zbMATHGoogle Scholar
  18. Goldberg D, Korb B, Deb K (1989) Messy genetic algorithms: motivation, analysis and first results. Complex Syst 3:493–530MathSciNetzbMATHGoogle Scholar
  19. Haralick RM, Elliott GL (1980) Increasing tree search efficiency for constraint satisfaction problems. Artif Intell 14:263–313CrossRefGoogle Scholar
  20. Hell P, Nesetril J (2008) Colouring, constraint satisfaction, and complexity. Comput Sci Rev 2(3):143–163zbMATHCrossRefGoogle Scholar
  21. Jussien N, Lhomme O (2000) Local search with constraint propagation and conflict-based heuristics. In: Proceedings of the seventeenth national conference on artificial intelligence and twelfth conference on innovative applications of artificial intelligence. AAAI Press/The MIT Press, pp 169–174Google Scholar
  22. Marchiori E, Steenbeek A (2000) Genetic local search algorithm for random binary constraint satisfaction problems. In: Proceedings of the ACM symposium on applied computing, pp 458–462Google Scholar
  23. Minton S, Phillips A, Laird P (1990) Solving large-scale CSP and scheduling problems using a heuristic repair method. In: Proceedings of the 8th AAAI conference, pp 17–24Google Scholar
  24. Minton S, Johnston MD, Phillips A, Laird P (1992) Minimizing conflicts: a heuristic repair method for constraint satisfaction and scheduling problems. Artif Intell 58:161–205MathSciNetzbMATHCrossRefGoogle Scholar
  25. Nguyen S, Zhang M, Johnston M (2011) A genetic programming based hyper-heuristic approach for combinatorial optimisation. In: Proceedings of the 13th annual conference on genetic and evolutionary computation. GECCO ’11. ACM, New York, pp 1299–1306. doi: 10.1145/2001576.2001752
  26. Nonobe K, Ibaraki T (1998) A tabu search approach to the constraint satisfaction problem as a general problem solver. Eur J Oper Res 106(2–3):599–623zbMATHCrossRefGoogle Scholar
  27. O’Mahony E, Hebrard E, Holland A, Nugent C, O’Sullivan B (2008) Using case-based reasoning in an algorithm portfolio for constraint solving. In: Proceedings of the 19th Irish conference on artificial intelligence and cognitive scienceGoogle Scholar
  28. Ortiz-Bayliss JC, Özcan E, Parkes AJ, Terashima-Marin H (2010) Mapping the performance of heuristics for constraint satisfaction. In: 2010 IEEE congress on evolutionary computation (CEC), pp 1–8Google Scholar
  29. Ortiz-Bayliss JC, Terashima-Marín H, Conant-Pablos SE (2013) Learning vector quantization for variable ordering in constraint satisfaction problems. Pattern Recogn Lett 34(4):423–432CrossRefGoogle Scholar
  30. Petrovic S, Qu R (2002) Case-based reasoning as a heuristic selector in a hyper-heuristic for course timetabling problems. In: Proceedings of the 6th international conference on knowledge-based intelligent information engineering systems and applied technologies (KES’02), vol 82, pp 336–340Google Scholar
  31. Prosser P (1994) Binary constraint satisfaction problems: some are harder than others. Proceedings of the European conference in artificial intelligence. Holland, Amsterdam, pp 95–99Google Scholar
  32. Rice JR (1976) The algorithm selection problem. Adv Comput 15:65–118CrossRefGoogle Scholar
  33. Ross P (2012) Hyper-heuristics. In: Burke E, Kendall G (eds) Search methodologies: introductory tutorials in optimization and decision support techniques. Springer, Berlin, pp 611–638Google Scholar
  34. Ross P, Hart E (2002) Using evolutionary algorithms to solve problems by combining choices of heuristics. In: Sarker R, Mohammadian M, Yao X (eds) Evolutionary optimization. Kluwer, DordrechtGoogle Scholar
  35. Sabar NR, Kendall G (2015) Population based monte carlo tree search hyper-heuristic for combinatorial optimization problems. Inf Sci 314:225–239. doi: 10.1016/j.ins.2014.10.045 CrossRefGoogle Scholar
  36. Sim K, Hart E (2014) An improved immune inspired hyper-heuristic for combinatorial optimisation problems. In: Proceedings of the 2014 conference on genetic and evolutionary computation, GECCO ’14. ACM, New York, pp 121–128. doi: 10.1145/2576768.2598241
  37. Sim K, Hart E, Paechter B (2013) Learning to solve bin packing problems with an immune inspired hyper-heuristic. In: Liò P, Miglino O, Nicosia G, Nolfi S, Pavone M (eds) Advances in artificial life, ECAL 2013. MIT Press, pp 856–863Google Scholar
  38. Smith BM (2001) Constructing an asymptotic phase transition in binary constraint satisfaction problems. J Theor Comput Sci 265(1):265–283MathSciNetzbMATHCrossRefGoogle Scholar
  39. Soto R, Crawford B, Monfroy E, Bustos V (2012) Using autonomous search for generating good enumeration strategy blends in constraint programming. In: ICCSA (3)’12, pp 607–617Google Scholar
  40. Terashima-Marín H, Ortiz-Bayliss JC, Ross P, Valenzuela-Rendón M (2008) Hyper-heuristics for the dynamic variable ordering in constraint satisfaction problems. In: GECCO’08: proceedings of the 10th annual conference on genetic and evolutionary computation. ACM, pp 571–578Google Scholar
  41. Wallace R (2006) Analysis of heuristic synergies. In: Hnich B, Carlsson M, Fages F, Rossi F (eds) Recent advances in constraints, lecture notes in computer science, vol 3978. Springer, Berlin, pp 73–87CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2016

Authors and Affiliations

  • José Carlos Ortiz-Bayliss
    • 1
    Email author
  • Hugo Terashima-Marín
    • 1
  • Santiago Enrique Conant-Pablos
    • 1
  1. 1.Tecnológico de MonterreyNational School of Engineering and SciencesMonterreyMexico

Personalised recommendations