Genetic Programming and Evolvable Machines

, Volume 17, Issue 2, pp 119–144 | Cite as

Grammar-based generation of variable-selection heuristics for constraint satisfaction problems

  • Alejandro Sosa-Ascencio
  • Gabriela Ochoa
  • Hugo Terashima-Marin
  • Santiago Enrique Conant-Pablos


We propose a grammar-based genetic programming framework that generates variable-selection heuristics for solving constraint satisfaction problems. This approach can be considered as a generation hyper-heuristic. A grammar to express heuristics is extracted from successful human-designed variable-selection heuristics. The search is performed on the derivation sequences of this grammar using a strongly typed genetic programming framework. The approach brings two innovations to grammar-based hyper-heuristics in this domain: the incorporation of if-then-else rules to the function set, and the implementation of overloaded functions capable of handling different input dimensionality. Moreover, the heuristic search space is explored using not only evolutionary search, but also two alternative simpler strategies, namely, iterated local search and parallel hill climbing. We tested our approach on synthetic and real-world instances. The newly generated heuristics have an improved performance when compared against human-designed heuristics. Our results suggest that the constrained search space imposed by the proposed grammar is the main factor in the generation of good heuristics. However, to generate more general heuristics, the composition of the training set and the search methodology played an important role. We found that increasing the variability of the training set improved the generality of the evolved heuristics, and the evolutionary search strategy produced slightly better results.


Constraint satisfaction problems Hyper-heuristics  Genetic programming Variable ordering heuristics Grammar-based framework 



This research was supported in part by Tecnológico de Monterrey under the strategic project PRY075 and the Research Group with Strategic Focus in Intelligent Systems, and the CONACyT Projects (Basic Science) under Grants 99695 and 241461. G. Ochoa acknowledges funding from the Engineering and Physical Sciences Research Council, UK (EPSRC) Grant Number EP/J017515.


  1. 1.
    D. Achlioptas, M.S.O. Molloy, L.M. Kirousis, Y.C. Stamatiou, E. Kranakis, D. Krizanc, Random Constraint Satisfaction: A More Accurate Picture. Constraints. 6(4), 329–344 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    M. Bader-El-Den, R. Poli, A GP-based hyper-heuristic framework for evolving 3-SAT heuristics, in Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation, GECCO ’7 (2007), p. 1749Google Scholar
  3. 3.
    M. Bader-El-Den, R. Poli, S. Fatima, Evolving timetabling heuristics using a grammar-based genetic programming hyper-heuristic framework. Memet. Comput. 1(3), 205–219 (2009)CrossRefGoogle Scholar
  4. 4.
    S. Bain, J. Thornton, A. Sattar, Evolving algorithms for constraint satisfaction, in Congress on Evolutionary Computation (2004), pp. 265–272Google Scholar
  5. 5.
    S. Bain, J. Thornton, A. Sattar, Methods of automatic algorithm generation, in PRICAI 2004: Trends in Artificial Intelligence (2004), pp. 1–10Google Scholar
  6. 6.
    E.B. Baum, Iterated descent: a better algorithm for local search in combinatorial optimization problems. Technical report (Caltech, Pasadena, CA, 1986)Google Scholar
  7. 7.
    J. Berlier, J. McCollum, A constraint satisfaction algorithm for microcontroller selection and pin assignment, in Proceedings of the IEEE SoutheastCon 2010 (SoutheastCon) (2010), pp. 348–351Google Scholar
  8. 8.
    S.C. Brailsford, C.N. Potts, B.M. Smith, Constraint satisfaction problems: algorithms and applications. Eur. J. Oper. Res. 119(3), 557–581 (1999)CrossRefzbMATHGoogle Scholar
  9. 9.
    D. Brelaz, New methods to colour the vertices of a graph. Commun. ACM 22, 251–256 (1979)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    E.K. Burke, T. Curtois, M. Hyde, G. Kendall, G.a. Ochoa, S. Petrovic, J.A. Vázquez-Rodríguez, M. Gendreau, Iterated local search vs. hyper-heuristics: Towards general-purpose search algorithms. IEEE Congress on Evolutionary Computation. Barcelona (2010)Google Scholar
  11. 11.
    E.K. Burke, M. Gendreau, M. Hyde, G. Kendall, G. Ochoa, E. Ozcan, R. Qu, Hyper-heuristics: a survey of the state of the art. J. Oper. Res. Soc 64(12), 1695–1724 (2013). doi: 10.1057/jors.2013.71 CrossRefGoogle Scholar
  12. 12.
    E.K. Burke, M. Hyde, G. Kendall, G. Ochoa, E. Özcan, J. Woodward, Exploring hyper-heuristic methodologies with genetic programming, in Computational Intelligence: Collaboration, Fusion and Emergence, Intelligent Systems Reference Library, ed. by C. Mumford, L. Jain (Springer, Berlin, 2009), pp. 177–201CrossRefGoogle Scholar
  13. 13.
    E.K. Burke, M.R. Hyde, G. Kendall, S. Member, Grammatical evolution of local search heuristics. Trans. Evol. Comput. 16(3), 406–417 (2012)CrossRefGoogle Scholar
  14. 14.
    E.K. Burke, M.R. Hyde, G. Kendall, G. Ochoa, E. Özcan, J. Woodward, A Classification of Hyper-heuristic Approaches. International Series in Operations Research & Management Science 146, pp. 449–468 . Springer US (2010). doi: 10.1007/978-1-4419-1665-5_15
  15. 15.
    R. Dechter, I. Meiri, Experimental evaluation of preprocessing algorithms for constraint satisfaction problems. Artif. Intell. 38(2), 211–242 (1994)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    N. Dunkin, S. Allen, Frequency assignment problems: representations and solutions. Technical report CSD-TR-97-14, (University of London, 1997)Google Scholar
  17. 17.
    J. Gaschnig, Experimental case studies of backtrack versus waltz-type versus new algorithms for satisfying assignment problems, in Second Biennial Conference of the Canadian Society for Computational Studies of Intelligence, ed. by C.I.P. Society, Toronto (1978)Google Scholar
  18. 18.
    I. Gent, E. MacIntyre, P. Prosser, B. Smith, T. Walsh, An empirical study of dynamic variable ordering heuristics for the constraint satisfaction problem, in Proceedings of the International Conference on Principles and Practice of Constraint Programming (CP’96) (1996), pp. 179–193Google Scholar
  19. 19.
    R.M. Haralick, G.L. Elliott, Increasing tree search efficiency for constraint satisfaction problems. Artif. Intell. 14(3), 263–313 (1980)CrossRefGoogle Scholar
  20. 20.
    P. Hell, J. Nesetril, Colouring, constraint satisfaction, and complexity. Comput. Sci. Rev. 2(3), 143–163 (2008)CrossRefzbMATHGoogle Scholar
  21. 21.
    R. Keller, R. Poli, Linear genetic programming of parsimonious metaheuristics, in IEEE Congress on Evolutionary Computation 2007. CEC 2007 (2007), pp. 4508–4515Google Scholar
  22. 22.
    R.E. Keller, R. Poli, Self-adaptive hyperheuristic and greedy search, in 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence) (2008), pp. 3801–3808Google Scholar
  23. 23.
    V. Kumar, Algorithms for constraint satisfaction problems: a survey. AI Mag. 13(1), 32–44 (1992)Google Scholar
  24. 24.
    H.R. Lourenco, O. Martin, T. Stutzle, Iterated local search. Technical report (Kluwer Academic, Norwell, 2002)Google Scholar
  25. 25.
    A. Mackworth, Consistency in networks of relations. Artif. Intell. 8(1), 99–118 (1977)MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    O. Martin, S.W. Otto, E.W. Felten, Large-step Markov chains for the traveling salesman problem. Complex Syst. 5(3), 299–326 (1991)MathSciNetzbMATHGoogle Scholar
  27. 27.
    R.I. McKay, N.X. Hoai, P.A. Whigham, Y. Shan, M. O’Neill, Grammar-based genetic programming: a survey. Genet. Program. Evol. Mach. 11(3–4), 365–396 (2010)CrossRefGoogle Scholar
  28. 28.
    S. Minton, An analytic learning system for specializing heuristics, in IJCAI (1993), pp. 922–928Google Scholar
  29. 29.
    S. Minton, M.D. Johnston, A.B. Philips, P. Laird, Minimizing conflicts: a heuristic repair method for constraint satisfaction and scheduling problems. Artif. Intell. 58, 161–205 (1992)MathSciNetCrossRefzbMATHGoogle Scholar
  30. 30.
    M. Mouhoub, B. Jafari, Heuristic techniques for variable and value ordering in csps, in Proceedings of the 13th annual conference on Genetic and evolutionary computation-GECCO ’11 (2011), p. 457Google Scholar
  31. 31.
    J.C. Ortiz Bayliss, Exploring hyper-heuristic approaches for solving constraint satisfaction problems. Ph.D. thesis, Tecnológico de Monterrey (2011)Google Scholar
  32. 32.
    J.C. Ortiz-Bayliss, J.H. Moreno-Scott, H. Terashima-Marín, Automatic generation of heuristics for constraint satisfaction problems, in International Workshop on Nature Inspired Cooperative Strategies for Optimization (NICSO 2014) (2014), pp. 1–14Google Scholar
  33. 33.
    J.C. Ortiz-Bayliss, E. Özcan, A.J. Parkes, H. Terashima-Marin, Mapping the performance of heuristics for constraint satisfaction, in CEC’10: Proceedings of the Congress on Evolutionary Computation IEEE (2010)Google Scholar
  34. 34.
    J.C. Ortiz-Bayliss, A.J. Parkes, H. Terashima-Marín, A genetic programming hyper-heuristic: turning features into heuristics for constraint satisfaction, in Workshop on Computational Intelligence (2013)Google Scholar
  35. 35.
    J.C. Ortiz-Bayliss, H. Terashima-Marín, S.E. Conant-Pablos, Learning vector quantization for variable ordering in constraint satisfaction problems. Pattern Recognit. Lett. 34(4), 423–432 (2013)CrossRefGoogle Scholar
  36. 36.
    F.J. Ovalle-Martínez, J. Solano-González, I. Stojmenovic, A parallel hill climbing algorithm for pushing dependent data in clients–providers–servers systems. Mob. Netw. Appl. 9, 257–264 (2004)CrossRefGoogle Scholar
  37. 37.
    G. Pappa, G. Ochoa, M. Hyde, A. Freitas, J. Woodward, J. Swan, Contrasting meta-learning and hyper-heuristic research: the role of evolutionary algorithms. Genet. Program. Evol. Mach. 15(1), 3–35 (2014). doi: 10.1007/s10710-013-9186-9 CrossRefGoogle Scholar
  38. 38.
    P.W. Purdom, Search rearrangement backtracking and polynomial average time. Artif. Intell. 21, 117–133 (1983)CrossRefGoogle Scholar
  39. 39.
    N. Sabar, M. Ayob, G. Kendall, R. Qu, Grammatical evolution hyper-heuristic for combinatorial optimization problems. IEEE Trans. Evol. Comput. 17(6), 840–861 (2013). doi: 10.1109/TEVC.2013.2281527 CrossRefGoogle Scholar
  40. 40.
    J.A. Soria-Alcaraz, G. Ochoa, J. Swan, Effective learning hyper-heuristics for the course timetabling problem. Eur. J. Oper. Res. 238(1), 77–86 (2014)MathSciNetCrossRefGoogle Scholar
  41. 41.
    A. Sosa-Ascencio, H. Terashima-Marín, M. Valenzuela-Rendón, Grammar-based genetic programming for evolving variable ordering heuristics, in IEEE Congress on Evolutionary Computation (2013), pp. 1154–1161Google Scholar
  42. 42.
    L. Spector, Towards Practical Autoconstructive Evolution: Self-Evolution of Problem-Solving Genetic Programming Systems, vol. 8 (Springer, Berlin, 2010)Google Scholar
  43. 43.
    H. Terashima-Marín, J.C. Ortiz-Bayliss, P. Ross, M. Valenzuela-Rendón, Using hyper-heuristics for the dynamic variable ordering in binary constraint satisfaction problems, in 7th Mexican International Conference on Artificial Intelligence, Lecture Notes in Computer Science, vol. 5317, ed. by A. Gelbukh, E. Morales (Springer, Berlin, 2008), pp. 407–417Google Scholar
  44. 44.
    J.D. Walker, G. Ochoa, M. Gendreau, E.K. Burke, Vehicle routing and adaptive iterated local search within the hyflex hyper-heuristic framework, in Learning and Intelligent Optimization, Lecture Notes in Computer Science ed. by Y. Hamadi, M. Schoenauer (Springer, Berlin, 2012), pp. 265–276Google Scholar
  45. 45.
    R. Wallace, Analysis of heuristic synergies, in Recent Advances in Constraints, Lecture Notes in Computer Science, vol. 3978, ed. by B. Hnich, M. Carlsson, F. Fages, F. Rossi (Springer, Berlin, 2006), pp. 73–87CrossRefGoogle Scholar
  46. 46.
    P.A. Whigham, Grammatically-based genetic programming, in Proceedings of the Workshop on Genetic Programming: From Theory to Real-World Applications, ed. by J.P. Rosca (Tahoe City, CA, 1995), pp. 33–41Google Scholar
  47. 47.
    C.P. Williams, T. Gogg, Using deep structure to locate hard problems, in Proceedings of AAAI (1992), pp. 472–477Google Scholar
  48. 48.
    K. Xu, W. Li, Exact phase transitions in random constraint satisfaction problems. J. Artif. Intell. Res. 12, 93–103 (2000)MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  • Alejandro Sosa-Ascencio
    • 1
  • Gabriela Ochoa
    • 2
  • Hugo Terashima-Marin
    • 1
  • Santiago Enrique Conant-Pablos
    • 1
  1. 1.Instituto Tecnológico y de Estudios Superiores de MonterreyMonterreyMexico
  2. 2.Computing Science and MathematicsUniversity of StirlingStirlingUK

Personalised recommendations