Abstract
We propose a grammar-based genetic programming framework that generates variable-selection heuristics for solving constraint satisfaction problems. This approach can be considered as a generation hyper-heuristic. A grammar to express heuristics is extracted from successful human-designed variable-selection heuristics. The search is performed on the derivation sequences of this grammar using a strongly typed genetic programming framework. The approach brings two innovations to grammar-based hyper-heuristics in this domain: the incorporation of if-then-else rules to the function set, and the implementation of overloaded functions capable of handling different input dimensionality. Moreover, the heuristic search space is explored using not only evolutionary search, but also two alternative simpler strategies, namely, iterated local search and parallel hill climbing. We tested our approach on synthetic and real-world instances. The newly generated heuristics have an improved performance when compared against human-designed heuristics. Our results suggest that the constrained search space imposed by the proposed grammar is the main factor in the generation of good heuristics. However, to generate more general heuristics, the composition of the training set and the search methodology played an important role. We found that increasing the variability of the training set improved the generality of the evolved heuristics, and the evolutionary search strategy produced slightly better results.
Similar content being viewed by others
Notes
The real-world benchmark problems used can be found at http://www.cril.univ-artois.fr/lecoutre/benchmarks.html, under the names ‘RLFAP-graphs’ and ‘jobShop-e0ddr1’
References
D. Achlioptas, M.S.O. Molloy, L.M. Kirousis, Y.C. Stamatiou, E. Kranakis, D. Krizanc, Random Constraint Satisfaction: A More Accurate Picture. Constraints. 6(4), 329–344 (2001)
M. Bader-El-Den, R. Poli, A GP-based hyper-heuristic framework for evolving 3-SAT heuristics, in Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation, GECCO ’7 (2007), p. 1749
M. Bader-El-Den, R. Poli, S. Fatima, Evolving timetabling heuristics using a grammar-based genetic programming hyper-heuristic framework. Memet. Comput. 1(3), 205–219 (2009)
S. Bain, J. Thornton, A. Sattar, Evolving algorithms for constraint satisfaction, in Congress on Evolutionary Computation (2004), pp. 265–272
S. Bain, J. Thornton, A. Sattar, Methods of automatic algorithm generation, in PRICAI 2004: Trends in Artificial Intelligence (2004), pp. 1–10
E.B. Baum, Iterated descent: a better algorithm for local search in combinatorial optimization problems. Technical report (Caltech, Pasadena, CA, 1986)
J. Berlier, J. McCollum, A constraint satisfaction algorithm for microcontroller selection and pin assignment, in Proceedings of the IEEE SoutheastCon 2010 (SoutheastCon) (2010), pp. 348–351
S.C. Brailsford, C.N. Potts, B.M. Smith, Constraint satisfaction problems: algorithms and applications. Eur. J. Oper. Res. 119(3), 557–581 (1999)
D. Brelaz, New methods to colour the vertices of a graph. Commun. ACM 22, 251–256 (1979)
E.K. Burke, T. Curtois, M. Hyde, G. Kendall, G.a. Ochoa, S. Petrovic, J.A. Vázquez-Rodríguez, M. Gendreau, Iterated local search vs. hyper-heuristics: Towards general-purpose search algorithms. IEEE Congress on Evolutionary Computation. Barcelona (2010)
E.K. Burke, M. Gendreau, M. Hyde, G. Kendall, G. Ochoa, E. Ozcan, R. Qu, Hyper-heuristics: a survey of the state of the art. J. Oper. Res. Soc 64(12), 1695–1724 (2013). doi:10.1057/jors.2013.71
E.K. Burke, M. Hyde, G. Kendall, G. Ochoa, E. Özcan, J. Woodward, Exploring hyper-heuristic methodologies with genetic programming, in Computational Intelligence: Collaboration, Fusion and Emergence, Intelligent Systems Reference Library, ed. by C. Mumford, L. Jain (Springer, Berlin, 2009), pp. 177–201
E.K. Burke, M.R. Hyde, G. Kendall, S. Member, Grammatical evolution of local search heuristics. Trans. Evol. Comput. 16(3), 406–417 (2012)
E.K. Burke, M.R. Hyde, G. Kendall, G. Ochoa, E. Özcan, J. Woodward, A Classification of Hyper-heuristic Approaches. International Series in Operations Research & Management Science 146, pp. 449–468 . Springer US (2010). doi:10.1007/978-1-4419-1665-5_15
R. Dechter, I. Meiri, Experimental evaluation of preprocessing algorithms for constraint satisfaction problems. Artif. Intell. 38(2), 211–242 (1994)
N. Dunkin, S. Allen, Frequency assignment problems: representations and solutions. Technical report CSD-TR-97-14, (University of London, 1997)
J. Gaschnig, Experimental case studies of backtrack versus waltz-type versus new algorithms for satisfying assignment problems, in Second Biennial Conference of the Canadian Society for Computational Studies of Intelligence, ed. by C.I.P. Society, Toronto (1978)
I. Gent, E. MacIntyre, P. Prosser, B. Smith, T. Walsh, An empirical study of dynamic variable ordering heuristics for the constraint satisfaction problem, in Proceedings of the International Conference on Principles and Practice of Constraint Programming (CP’96) (1996), pp. 179–193
R.M. Haralick, G.L. Elliott, Increasing tree search efficiency for constraint satisfaction problems. Artif. Intell. 14(3), 263–313 (1980)
P. Hell, J. Nesetril, Colouring, constraint satisfaction, and complexity. Comput. Sci. Rev. 2(3), 143–163 (2008)
R. Keller, R. Poli, Linear genetic programming of parsimonious metaheuristics, in IEEE Congress on Evolutionary Computation 2007. CEC 2007 (2007), pp. 4508–4515
R.E. Keller, R. Poli, Self-adaptive hyperheuristic and greedy search, in 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence) (2008), pp. 3801–3808
V. Kumar, Algorithms for constraint satisfaction problems: a survey. AI Mag. 13(1), 32–44 (1992)
H.R. Lourenco, O. Martin, T. Stutzle, Iterated local search. Technical report (Kluwer Academic, Norwell, 2002)
A. Mackworth, Consistency in networks of relations. Artif. Intell. 8(1), 99–118 (1977)
O. Martin, S.W. Otto, E.W. Felten, Large-step Markov chains for the traveling salesman problem. Complex Syst. 5(3), 299–326 (1991)
R.I. McKay, N.X. Hoai, P.A. Whigham, Y. Shan, M. O’Neill, Grammar-based genetic programming: a survey. Genet. Program. Evol. Mach. 11(3–4), 365–396 (2010)
S. Minton, An analytic learning system for specializing heuristics, in IJCAI (1993), pp. 922–928
S. Minton, M.D. Johnston, A.B. Philips, P. Laird, Minimizing conflicts: a heuristic repair method for constraint satisfaction and scheduling problems. Artif. Intell. 58, 161–205 (1992)
M. Mouhoub, B. Jafari, Heuristic techniques for variable and value ordering in csps, in Proceedings of the 13th annual conference on Genetic and evolutionary computation-GECCO ’11 (2011), p. 457
J.C. Ortiz Bayliss, Exploring hyper-heuristic approaches for solving constraint satisfaction problems. Ph.D. thesis, Tecnológico de Monterrey (2011)
J.C. Ortiz-Bayliss, J.H. Moreno-Scott, H. Terashima-Marín, Automatic generation of heuristics for constraint satisfaction problems, in International Workshop on Nature Inspired Cooperative Strategies for Optimization (NICSO 2014) (2014), pp. 1–14
J.C. Ortiz-Bayliss, E. Özcan, A.J. Parkes, H. Terashima-Marin, Mapping the performance of heuristics for constraint satisfaction, in CEC’10: Proceedings of the Congress on Evolutionary Computation IEEE (2010)
J.C. Ortiz-Bayliss, A.J. Parkes, H. Terashima-Marín, A genetic programming hyper-heuristic: turning features into heuristics for constraint satisfaction, in Workshop on Computational Intelligence (2013)
J.C. Ortiz-Bayliss, H. Terashima-Marín, S.E. Conant-Pablos, Learning vector quantization for variable ordering in constraint satisfaction problems. Pattern Recognit. Lett. 34(4), 423–432 (2013)
F.J. Ovalle-Martínez, J. Solano-González, I. Stojmenovic, A parallel hill climbing algorithm for pushing dependent data in clients–providers–servers systems. Mob. Netw. Appl. 9, 257–264 (2004)
G. Pappa, G. Ochoa, M. Hyde, A. Freitas, J. Woodward, J. Swan, Contrasting meta-learning and hyper-heuristic research: the role of evolutionary algorithms. Genet. Program. Evol. Mach. 15(1), 3–35 (2014). doi:10.1007/s10710-013-9186-9
P.W. Purdom, Search rearrangement backtracking and polynomial average time. Artif. Intell. 21, 117–133 (1983)
N. Sabar, M. Ayob, G. Kendall, R. Qu, Grammatical evolution hyper-heuristic for combinatorial optimization problems. IEEE Trans. Evol. Comput. 17(6), 840–861 (2013). doi:10.1109/TEVC.2013.2281527
J.A. Soria-Alcaraz, G. Ochoa, J. Swan, Effective learning hyper-heuristics for the course timetabling problem. Eur. J. Oper. Res. 238(1), 77–86 (2014)
A. Sosa-Ascencio, H. Terashima-Marín, M. Valenzuela-Rendón, Grammar-based genetic programming for evolving variable ordering heuristics, in IEEE Congress on Evolutionary Computation (2013), pp. 1154–1161
L. Spector, Towards Practical Autoconstructive Evolution: Self-Evolution of Problem-Solving Genetic Programming Systems, vol. 8 (Springer, Berlin, 2010)
H. Terashima-Marín, J.C. Ortiz-Bayliss, P. Ross, M. Valenzuela-Rendón, Using hyper-heuristics for the dynamic variable ordering in binary constraint satisfaction problems, in 7th Mexican International Conference on Artificial Intelligence, Lecture Notes in Computer Science, vol. 5317, ed. by A. Gelbukh, E. Morales (Springer, Berlin, 2008), pp. 407–417
J.D. Walker, G. Ochoa, M. Gendreau, E.K. Burke, Vehicle routing and adaptive iterated local search within the hyflex hyper-heuristic framework, in Learning and Intelligent Optimization, Lecture Notes in Computer Science ed. by Y. Hamadi, M. Schoenauer (Springer, Berlin, 2012), pp. 265–276
R. Wallace, Analysis of heuristic synergies, in Recent Advances in Constraints, Lecture Notes in Computer Science, vol. 3978, ed. by B. Hnich, M. Carlsson, F. Fages, F. Rossi (Springer, Berlin, 2006), pp. 73–87
P.A. Whigham, Grammatically-based genetic programming, in Proceedings of the Workshop on Genetic Programming: From Theory to Real-World Applications, ed. by J.P. Rosca (Tahoe City, CA, 1995), pp. 33–41
C.P. Williams, T. Gogg, Using deep structure to locate hard problems, in Proceedings of AAAI (1992), pp. 472–477
K. Xu, W. Li, Exact phase transitions in random constraint satisfaction problems. J. Artif. Intell. Res. 12, 93–103 (2000)
Acknowledgments
This research was supported in part by Tecnológico de Monterrey under the strategic project PRY075 and the Research Group with Strategic Focus in Intelligent Systems, and the CONACyT Projects (Basic Science) under Grants 99695 and 241461. G. Ochoa acknowledges funding from the Engineering and Physical Sciences Research Council, UK (EPSRC) Grant Number EP/J017515.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Sosa-Ascencio, A., Ochoa, G., Terashima-Marin, H. et al. Grammar-based generation of variable-selection heuristics for constraint satisfaction problems. Genet Program Evolvable Mach 17, 119–144 (2016). https://doi.org/10.1007/s10710-015-9249-1
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10710-015-9249-1