# Monte Carlo hyper-heuristics for examination timetabling

- 560 Downloads
- 23 Citations

## Abstract

Automating the neighbourhood selection process in an iterative approach that uses multiple heuristics is not a trivial task. Hyper-heuristics are search methodologies that not only aim to provide a general framework for solving problem instances at different difficulty levels in a given domain, but a key goal is also to extend the level of generality so that different problems from different domains can also be solved. Indeed, a major challenge is to explore how the heuristic design process might be automated. Almost all existing iterative selection hyper-heuristics performing single point search contain two successive stages; heuristic selection and move acceptance. Different operators can be used in either of the stages. Recent studies explore ways of introducing learning mechanisms into the search process for improving the performance of hyper-heuristics. In this study, a broad empirical analysis is performed comparing Monte Carlo based hyper-heuristics for solving capacitated examination timetabling problems. One of these hyper-heuristics is an approach that overlaps two stages and presents them in a single algorithmic body. A learning heuristic selection method (L) operates in harmony with a simulated annealing move acceptance method using reheating (SA) based on some shared variables. Yet, the heuristic selection and move acceptance methods can be separated as the proposed approach respects the common selection hyper-heuristic framework. The experimental results show that simulated annealing with reheating as a hyper-heuristic move acceptance method has significant potential. On the other hand, the learning hyper-heuristic using simulated annealing with reheating move acceptance (L–SA) performs poorly due to certain weaknesses, such as the choice of rewarding mechanism and the evaluation of utility values for heuristic selection as compared to some other hyper-heuristics in examination timetabling. Trials with other heuristic selection methods confirm that the best alternative for the simulated annealing with reheating move acceptance for examination timetabling is a previously proposed strategy known as the choice function.

## Keywords

Hyper-heuristics Simulated annealing Meta-heuristics Examination timetabling Reinforcement learning## Preview

Unable to display preview. Download preview PDF.

## References

- Abdullah, S., Ahmadi, S., Burke, E. K., Dror, M., & McCollum, B. (2007). A tabu-based large neighbourhood search methodology for the capacitated examination timetabling problem.
*Journal of the Operational Research Society*,*58*, 1494–1502. CrossRefGoogle Scholar - Alkan, A., & Özcan, E. (2003). Memetic algorithms for timetabling. In
*Proc. of the congress on evolutionary computation*(Vol. 3, pp. 1796–1802). Google Scholar - Asmuni, H., Burke, E. K., & Garibaldi, J. M. (2005). Fuzzy multiple ordering criteria for examination timetabling. In
*Lecture notes in computer science**: Vol.*3616.*Selected papers from the 5th international conference on the practice and theory of automated timetabling*(pp. 334–353). Berlin: Springer. CrossRefGoogle Scholar - Ayob, M., & Kendall, G. (2003). A Monte Carlo hyper-heuristic to optimise component placement sequencing for multi head placement machine. In
*Proceedings of the international conference on intelligent technologies (InTech’03)*, Chiang Mai, Thailand (pp. 132–141). Google Scholar - Azimi, Z. N. (2005). Hybrid heuristics for examination timetabling problem.
*Applied Mathematics and Computation*,*163*(2), 705–733. CrossRefGoogle Scholar - Bai, R., & Kendall, G. (2005). An investigation of automated planograms using a simulated annealing based hyper-heuristics. In T. Ibaraki, K. Nonobe, & M. Yagiura (Eds.),
*Operations research/computer science interface series**: Vol.*32.*Metaheuristics: progress as real problem solver*(pp. 87–108). Berlin: Springer. CrossRefGoogle Scholar - Bai, R., Blazewicz, J., Burke, E. K., Kendall, G., & McCollum, B. (2007).
*A simulated annealing hyper-heuristic methodology for flexible decision support*(Tech. Rep. NOTTCS-TR-2007-8). School of CSiT, University of Nottingham. Google Scholar - Bilgin, B., Özcan, E., & Korkmaz, E. E. (2007). An experimental study on hyper-heuristics and exam timetabling. In
*Lecture notes in computer science**: Vol.*3867.*Practice and theory of automated timetabling VI (PATAT 2006)*(pp. 394–412). Berlin: Springer. CrossRefGoogle Scholar - Broder, S. (1964). Final examination scheduling.
*Communications of the ACM*,*7*, 494–498. CrossRefGoogle Scholar - Burke, E. K., & Newall, J. P. (1999). A multistage evolutionary algorithm for the timetable problem.
*IEEE Trans Evolutionary Computation*,*3*(1), 63–74. CrossRefGoogle Scholar - Burke, E. K., & Newall, J. P. (2004). Solving examination timetabling problems through adaption of heuristic orderings.
*Annals of Operations Research*,*129*, 107–134. CrossRefGoogle Scholar - Burke, E. K., & Petrovic, S. (2002). Recent research directions in automated timetabling.
*European Journal of Operational Research*,*140*(2), 266–280. CrossRefGoogle Scholar - Burke, E. K., Newall, J. P., & Weare, R. F. (1996a). A memetic algorithm for university exam timetabling. In
*Lecture notes in computer science**: Vol.*1153.*Selected papers from the first international conference on practice and theory of automated timetabling*(pp. 241–250). Berlin: Springer. Google Scholar - Burke, E. K., Elliman, D. G., Ford, P. H., & Weare, R. F. (1996b). Examination timetabling in British universities: a survey. In
*Lecture notes in computer science**: Vol.*1153.*Selected papers from the first international conference on practice and theory of automated timetabling*(pp. 76–90). Berlin: Springer. Google Scholar - Burke, E. K., Hart, E., Kendall, G., Newall, J., Ross, P., & Schulenburg, S. (2003). Hyper-heuristics: An emerging direction in modern search technology. In F. Glover & G. Kochenberger (Eds.),
*Handbook of metaheuristics*(pp. 457–474). Norwell: Kluwer Academic. Google Scholar - Burke, E. K., Hyde, M. R., & Kendall, G. (2006). Evolving bin packing heuristics with genetic programming. In
*Lecture notes in computer science**: Vol.*4193.*Proceedings of the 9th international conference on parallel problem solving from nature (PPSN 2006)*, Reykjavik, Iceland (pp. 860–869). Berlin: Springer. CrossRefGoogle Scholar - Burke, E. K., Hyde, M. R., Kendall, G., & Woodward, J. (2007a). Automatic heuristic generation with genetic programming: evolving a jack-of-all-trades or a master of one. In
*GECCO ’07: proceedings of the 9th annual conference on genetic and evolutionary computation*(pp. 1559–1565). New York: ACM. doi: 10.1145/1276958.1277273. CrossRefGoogle Scholar - Burke, E. K., McCollum, B., Meisels, A., Petrovic, S., & Qu, R. (2007b). A graph-based hyper-heuristic for educational timetabling problems.
*European Journal of Operational Research*,*176*(1), 177–192. CrossRefGoogle Scholar - Caramia, M., Dell’Olmo, P., & Italiano, G. F. (2001). New algorithms for examination timetabling. In
*Lecture notes in computer science**: Vol.*1982.*WAE ’00: the 4th international workshop on algorithm engineering*(pp. 230–242). London: Springer. Google Scholar - Caramia, M., Dellolmo, P., & Italiano, G. F. (2008). Novel local search-based approaches to university examination timetabling.
*INFORMS Journal on Computing*,*20*(1), 86–99. CrossRefGoogle Scholar - Carter, M. W. (1986). A survey of practical applications of examination timetabling algorithms.
*Operations Research Society of America*,*34*(2), 193–202. Google Scholar - Carter, M. W., & Laporte, G. (1996). Recent developments in practical examination timetabling. In
*Lecture notes in computer science**: Vol.*1153.*Selected papers from the first international conference on practice and theory of automated timetabling*(pp. 373–383). Berlin: Springer. Google Scholar - Carter, M. W., Laporte, G., & Lee, S. (1996). Examination timetabling: Algorithmic strategies and applications.
*Journal of the Operational Research Society*,*47*(3), 373–383. Google Scholar - Cerny, V. (1985). Thermodynamical approach to the traveling salesman problem: An efficient simulation algorithm.
*Journal of Optimization Theory and Applications*,*45*(1), 41–51. CrossRefGoogle Scholar - Cole, A. J. (1964). The preparation of examination timetables using a small-store computer.
*The Computer Journal*,*7*, 117–121. CrossRefGoogle Scholar - Corne, D., Ross, P., & Fang, H. L. (1994). Fast practical evolutionary timetabling. In
*Selected papers from AISB workshop on evolutionary computing*(pp. 250–263). Google Scholar - Corr, P. H., McCollum, B., McGreevy, M.A.J., & McMullan, P. (2006). A new neural network based construction heuristic for the examination timetabling problem. In
*Parallel problem solving from nature—PPSN IX*(pp. 392–401). Google Scholar - Cowling, P., Kendall, G., & Soubeiga, E. (2001a). A hyperheuristic approach to scheduling a sales summit. In
*PATAT ’00: selected papers from the third international conference on practice and theory of automated timetabling III*(pp. 176–190). London: Springer. Google Scholar - Cowling, P., Kendall, G., & Soubeiga, E. (2001b). A parameter-free hyperheuristic for scheduling a sales summit. In
*Proceedings of the 4th metaheuristic international conference*(pp. 127–131). Google Scholar - Cowling, P., Kendall, G., & Soubeiga, E. (2002). Hyperheuristics: A tool for rapid prototyping in scheduling and optimisation. In
*Lecture notes in computer science**: Vol.*4193.*EvoWorkShops*(pp. 1–10). Berlin: Springer. Google Scholar - Dowsland, K., & Thompson, J. (2005). Ant colony optimization for the examination scheduling problem.
*Journal of the Operational Research Society*,*56*(4), 426–438. CrossRefGoogle Scholar - Dowsland, K. A., Soubeiga, E., & Burke, E. (2007). A simulated annealing based hyperheuristic for determining shipper sizes for storage and transportation.
*European Journal of Operational Research*,*179*(3), 759–774. CrossRefGoogle Scholar - Eley, M. (2006). Ant algorithms for the exam timetabling problem. In
*Proc. of the 5th international conference on the practice and theory of automated timetabling*(pp. 364–382). Google Scholar - Ergul, A. (1996). Ga-based examination scheduling experience at middle east technical university. In
*Lecture notes in computer science**: Vol.*1153.*Practice and theory of automated timetabling*(pp. 212–226). Berlin: Springer. CrossRefGoogle Scholar - Ersoy, E., Özcan, E., & Uyar, S. (2007). Memetic algorithms and hyperhill-climbers. In
*Proc. of the 3rd multidisciplinary int. conf. on scheduling: theory and applications*(*MISTA’07*) (pp. 159–166). Google Scholar - Even, S., Itai, A., & Shamir, A. (1976). On the complexity of timetable and multicommodity flow problems.
*SIAM Journal on Computing*,*5*(4), 691–703. CrossRefGoogle Scholar - Gaspero, L. D., & Schaerf, A. (2001). Tabu search techniques for examination timetabling. In E. K. Burke & W. Erben (Eds.),
*Lecture notes in computer science**: Vol.*2079.*Third international conference on practice and theory of automated timetabling, PATAT2000*(pp. 104–117). Berlin: Springer. CrossRefGoogle Scholar - Gogos, C., Alefragis, P., & Housos, E. (2010). An improved multi-staged algorithmic process for the solution of the examination timetabling problem.
*Annals of Operations Research*. doi: 10.1007/s10479-010-0712-3. Google Scholar - Goldberg, DE (1989).
*Genetic algorithms in search, optimization and machine learning*. Boston: Addison-Wesley. Google Scholar - Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996). Reinforcement learning: a survey.
*Journal of Artificial Intelligence Research*,*4*, 237–285. Google Scholar - Kendall, G., & Hussin, N. M. (2005). A tabu search hyper-heuristic approach to the examination timetabling problem at the Mara University of Technology. In
*Lecture notes in computer science**: Vol.*3616.*Practice and theory of automated timetabling V*(pp. 270–293). Berlin: Springer. CrossRefGoogle Scholar - Kirkpatrick, S., Gelatt, C. D., & Vecchi, M. P. (1983). Optimization by simulated annealing.
*Science*,*220*, 671–680. CrossRefGoogle Scholar - Krasnogor, N., & Gustafson, S. (2004). A study on the use of ‘self-generation’ in memetic algorithms.
*Natural Computing*,*3*(1), 53–76. CrossRefGoogle Scholar - Leighton, F. T. (1979). A graph coloring algorithm for large scheduling problems.
*Journal of Research of the National Bureau of Standards*,*84*, 489–506. Google Scholar - Lundy, M., & Mees, A. (1986). Convergence of an annealing algorithm.
*Mathematical Programming*,*34*, 111–124. CrossRefGoogle Scholar - Marin, H. T. (1998).
*Combinations of ga and csp strategies for solving the examination timetabling problem*(PhD thesis). Instituto Technologico y de Estudios Superiores de Monterrey. Google Scholar - McCollum, B. (2006). University timetabling: Bridging the gap between research and practice. In
*Proc. of the 5th international conference on the practice and theory of automated timetabling*(pp. 15–35). Berlin: Springer. Google Scholar - McCollum, B., Schaerf, A., Paechter, B., McMullan, P., Lewis, R., Parkes, A. J., Gaspero, L., Qu, R., & Burke, E. K. (2010). Setting the research agenda in automated timetabling: the second international timetabling competition.
*INFORMS Journal on Computing*,*22*, 120–130. CrossRefGoogle Scholar - Merlot, L. T., Boland, N., Hughes, B. D., & Stuckey, P. J. (2003). A hybrid algorithm for the examination timetabling problem. In
*Lecture notes in computer science**: Vol.*1153.*Practice and theory of automated timetabling IV, PATAT 2002*(pp. 207–231). Berlin: Springer. CrossRefGoogle Scholar - Müller, T. (2009). Itc2007 solver description: A hybrid approach.
*Annals of Operations Research*,*172*(1), 429–446. CrossRefGoogle Scholar - Nareyek, A. (2003). Choosing search heuristics by non-stationary reinforcement learning. In M. G. C. Resende & J. P. de Sousa (Eds.),
*Metaheuristics: computer decision-making*(pp. 523–544). Norwell: Kluwer Academic, Chap. 9. Google Scholar - Ong, Y. S., Lim, M. H., Zhu, N., & Wong, K. W. (2006). Classification of adaptive memetic algorithms: a comparative study.
*IEEE Transactions on Systems, Man, and Cybernetics, Part B*,*36*(1), 141–152. CrossRefGoogle Scholar - Özcan, E., & Alkan, A. (2007). A memetic algorithm for solving a timetabling problem: An incremental strategy. In: P. Baptiste, G. Kendall, A.M. Kordon & F. Sourd (Eds.),
*Proc. of the 3rd multidisciplinary int. conf. on scheduling: theory and applications*(pp. 394–401). Google Scholar - Özcan, E., & Ersoy, E. (2005). Final exam scheduler—fes. In
*Proc. of the congress on evolutionary computation*(pp. 1356–1363). New York: IEEE Press. CrossRefGoogle Scholar - Özcan, E., Bilgin, B., & Korkmaz, E. E. (2006). Hill climbers and mutational heuristics in hyperheuristics. In
*Lecture notes in computer science**: Vol.*4193.*Proceedings of the 9th international conference on parallel problem solving from nature (PPSN 2006)*, Reykjavik, Iceland (pp. 202–211). Berlin: Springer. CrossRefGoogle Scholar - Özcan, E., Bilgin, B., & Korkmaz, E. E. (2008). A comprehensive survey of hyperheuristics.
*Intelligent Data Analysis*,*12*(1), 3–23. Google Scholar - Paquete, L. F., & Fonseca, C. M. (2001). A study of examination timetabling with multiobjective evolutionary algorithms. In
*Proc. of the 4th metaheuristics international conference (MIC 2001)*(pp. 149–154). Google Scholar - Petrovic, S., Patel, V., & Yang, Y. (2005). Examination timetabling with fuzzy constraints. In
*Lecture notes in computer science**: Vol.*3616.*The 5th int. conf. on the practice and theory of automated timetabling*(pp. 313–333). Berlin: Springer. CrossRefGoogle Scholar - Petrovic, S., Yang, Y., & Dror, M. (2007). Case-based selection of initialisation heuristics for metaheuristic examination timetabling.
*Expert Systems with Applications*,*33*(3), 772–785. CrossRefGoogle Scholar - Qu, R., Burke, E. K., McCollum, B., Merlot, L., & Lee, S. (2009). A survey of search methodologies and automated system development for examination timetabling.
*Journal of Scheduling*,*12*(1), 55–89. CrossRefGoogle Scholar - Ross, P., Corne, D., & Fang, H. L. (1994). Improving evolutionary timetabling with delta evaluation and directed mutation. In
*PPSN III: proceedings of the international conference on evolutionary computation. The third conference on parallel problem solving from nature*(pp. 556–565). London: Springer. Google Scholar - Schaerf, A. (1999). A survey of automated timetabling.
*Artificial Intelligence Review*,*13*(2), 87–127. CrossRefGoogle Scholar - Schaerf, A. & Gaspero, L.D. (2006). Measurability and reproducibility in timetabling research: State-of the-art and discussion (invited paper). In
*Proc. of the 6th int. conf. on the practice and theory of automated timetabling*(pp. 53–62). Google Scholar - Wong, T., Cote, P., & Gely, P. (2002). Final exam timetabling: a practical approach. In
*Proc. of the IEEE canadian conference on electrical and computer engineering*(Vol. 2, pp. 726–731). Google Scholar