# Ensemble Move Acceptance in Selection Hyper-heuristics

## Abstract

Selection hyper-heuristics are high level search methodologies which control a set of low level heuristics while solving a given problem. Move acceptance is a crucial component of selection hyper-heuristics, deciding whether to accept or reject a new solution at each step during the search process. This study investigates group decision making strategies as ensemble methods exploiting the strengths of multiple move acceptance methods for improved performance. The empirical results indicate the success of the proposed methods across six combinatorial optimisation problems from a benchmark as well as an examination timetabling problem.

## Keywords

Metaheuristic Optimisation Parameter control Timetabling Group decision making## 1 Introduction

A selection hyper-heuristic is an iterative improvement oriented search method which embeds two key components; *heuristic selection* and *move acceptance* [3]. The heuristic selection method chooses and applies a heuristic from a set of low level heuristics to the solution in hand, producing a new one. Then the move acceptance method decides whether to accept or reject this solution. The modularity, use of machine learning techniques and utilisation of the *domain barrier* make hyper-heuristics more general search methodologies than the current techniques tailored for a particular domain are. A selection hyper-heuristic or its components can be reused on another problem domain without requiring any change. There is a growing number of studies on selection hyper-heuristics combining a range of simple heuristic selection and move acceptance methods [6, 13]. More on any type of hyper-heuristic, such as their components and application areas can be found in [3].

*Hyper-heuristics Flexible Framework* (HyFlex) [11] was proposed as a software platform for rapid development and testing of hyper-heuristics. HyFlex is implemented in Java along with six different problem domains: boolean satisfiability, bin-packing, permutation flow-shop, personnel scheduling, travelling salesman problem and vehicle routing problem. HyFlex was used in the first Cross-Domain Heuristic Search Challenge, CHeSC 2011 (http://www.asap.cs.nott.ac.uk/chesc2011/) to detect the best selection hyper-heuristic. Following the competition, the results from twenty competing selection hyper-heuristics across thirty problem instances (containing five instances from each HyFlex domain) and the description of their algorithms were provided at the competition web-page.

A recent theoretical study on selection hyper-heuristics in [10] showed that the mixing of simple move acceptance criteria could lead to an improved running-time complexity than using each move acceptance method standalone on some simple benchmark functions. In [1, 8] different move acceptance criteria were used under an iterative two-stage framework which switches from one move acceptance to another at each stage. The previous work [2, 13] indicates that the overall performance of a hyper-heuristic depends on the choice of selection hyper-heuristic components. This study extends the initial work in Özcan et al. [12] by applying and evaluating four group decision making strategies as ensemble methods using three different move acceptance methods in combination with seven heuristic selection methods on an examination timetabling problem [2]. The same selection hyper-heuristics are then tested on thirty problem instances from six different domains from the HyFlex benchmark.

## 2 Group Decision Making Selection Hyper-heuristics

An overview of heuristic selection and move acceptance methods as a part of the selection hyper-heuristics as well as the group decision making methods forming an ensemble of move acceptance used in this study is described in this section.

A range of simple heuristic selection methods were studied in [6]. *Simple Random* (SR) selects a heuristic at random at each decision point. *Random Descent* (RD) also selects a heuristic at random, and then applies it to the candidate solution as long as the solution is improved. *Random Permutation* (RP) generates a random permutation of heuristics and applies one heuristic at a time in that order. *Random Permutation Descent* (RPD) is based on the same RP strategy, however similar to RD, applies the same heuristic repeatedly until there is no more improvement. *Greedy* (GR) applies all low level heuristics to the current solution and selects the heuristic which generates the best improvement. *Choice Function* (CF) is an online learning heuristic selection method that scores each low level heuristic based on their utility value and selects the one with the highest score. A *Tabu Search* based hyper-heuristic (TABU) that maintains a tabu list of badly performing low level heuristics to disallow the selection of these heuristics was tested in [5].

*k*methods. In this model, each move acceptance carries a certain

*strength*(\(s_i\)) which adjusts its contribution towards a final acceptance decision.

*D*(

*m*) returns 1, if a solution is accepted by the move acceptance method

*m*, and 0, if rejected.

In this study, we use group decision making strategies which make an accept/reject decision based on *authority*, *minority* and *majority* rules, namely G-OR (the move acceptance method which accepts the solution has the authority), G-AND (minority decides rejection), G-VOT and G-PVO (considers majority of the votes for the accept/reject decision). G-PVO probabilistically makes the accept/reject decisions. The probability that a new solution is accepted changes dynamically in proportional to the number of members that voted to the acceptance of the new solution. For instance, assuming 6 members in the group out of 10 move acceptance methods accepts a solution at a given step, then G-PVO accepts the solution with a probability of 60 %. It is preferable in G-VOT to have an odd number of members for the group decision making move acceptance criteria, where none of the other strategies requires this. More formally, using Eq. 1, assuming *k* move acceptance methods, then for G-AND, G-OR and G-VOT, \(\alpha \) is *k*, 0.5 and *k*/2, respectively, where all \(s_i\) values are set to 1. For G-PVO, \(\alpha \) equals \(k*r\), where *r* is uniform random number in [0, 1], and \(s_i\) values equal 1/*k*.

In this study, the heuristic selection methods in {SR, RD, RP, RPD, CF, GR, TABU} are paired with four group decision making move acceptance mechanisms {G-AND, G-OR, G-VOT, G-PVO}, generating twenty eight group decision making selection hyper-heuristics. From this point forward, a selection hyper-heuristic will be denoted as *“heuristic selection method”*_*“move acceptance method”*. For example, SR_G-AND denotes the selection hyper-heuristic using SR as the heuristic selection method and G-AND as the move acceptance method.

*t*,

*T*is the time limit or the maximum number of steps and \(\varDelta F\) is an expected range for the maximum fitness change. GD acceptance criterion accepts all the improving moves but the non-improving moves are accepted if the objective value of the current solution is not worse than an expected value, named as level [7]. Equation 3 is used to update the threshold level \(\tau _t\) at time or step

*t*.

*T*is the time limit or the maximum number of steps, \(\varDelta f\) is an expected range for the maximum fitness change and

*F*is the final objective value.

## 3 Computational Experiments

Two sets of benchmarks are used for examination timetabling: Yeditepe [14, 15] and Toronto benchmarks [4] consisting of eight and fourteen instances, respectively. The mean performance of each group decision making move acceptance method in a selection hyper-heuristic regardless of the heuristic selection method is compared to each other based on their ranks. The group decision making move acceptance methods are ranked from 1 to 4 for each problem instance and heuristic selection method from best to worst based on the mean cost over fifty runs. The approaches are assigned to different ranks if their performances vary in a statistically significant manner for a given instance. Otherwise, their performances are considered to be similar and an average rank is assigned to them all. A similar outcome is observed for the online performances of the group decision making strategies as in the benchmark functions reported in [12]. G-VOT is the best acceptance mechanism based on the average rank over all the problems, while G-PVO, G-AND and G-OR follows it in that order, respectively as illustrated in Fig. 1.

\(\% imp\). denotes the percentage improvement over the average best cost across fifty runs that the ‘current’ best hyper-heuristic(s) (investigated in this work) produces over the ‘previous’ best hyper-heuristic (reported in [2]) for each problem instance. If a hyper-heuristic delivers a statistically significant performance, it appears in the ‘current’ column. Bold entries highlight the best performing method. The hyper-heuristics that have a similar performance to the bold entry are displayed in parentheses. “+” indicates that all hyper-heuristics in {GR_G-VOT, TABU_G-VOT, RP_G-VOT, GR_G-PVO, SR_G-VOT, CF_G-VOT} has similar performance. “/” excludes the hyper-heuristic from this set that is displayed afterwards

instance | current | previous | \(\%imp.\) |
---|---|---|---|

yue20011 | | SR_GD | 20.84 |

yue20012 | | SR_GD | 24.93 |

yue20013 | + | | 0 |

yue20021 | | SR_GD | 17.97 |

yue20022 | | CF_MC | 3.97 |

yue20023 | | CF_MC | 1.97 |

yue20031 | | CF_MC | 4.4 |

yue20032 | n/a | | n/a |

car91 I | | TABU_IE | 81.37 |

car92 I | | TABU_IE | 196.89 |

ear83 I | | CF_MC | 1.1 |

hecs92 I | | CF_MC | 21.46 |

kfu93 | | SR_GD | 30.88 |

lse91 | | CF_MC | 13.38 |

pur93 I | | SR_IE | 15.6 |

rye92 | | CF_MC | 41.67 |

sta83 I | + | | 0 |

tre92 | | SR_GD | 92.93 |

uta92 I | | TABU_IE | 36.36 |

ute92 | | | 0 |

yor83 I | | CF_MC | 9.01 |

Ranking of selected group decision making hyper-heuristics to the CHeSC 2011 competitors based on Formula 1

Rank | HH | Total | SAT | BP | PS | PFS | TSP | VRP |
---|---|---|---|---|---|---|---|---|

1 | AdapHH | 170.00 | 33.75 | 43.00 | 6.00 | 37.00 | 40.25 | 10.00 |

7 | HAHA | 65.75 | 31.75 | 0.00 | 19.50 | 3.50 | 0.00 | 11.00 |

11 | CF_G-AND | 39.00 | 0.00 | 25.00 | 10.00 | 0.00 | 0.00 | 4.00 |

14 | CF_G-OR | 27.50 | 9.50 | 0.00 | 2.00 | 0.00 | 8.00 | 8.00 |

15 | CF_G-VOT | 23.50 | 0.00 | 0.00 | 8.50 | 0.00 | 4.00 | 11.00 |

20 | CF_G-PVO | 16.14 | 0.14 | 0.00 | 1.00 | 0.00 | 6.00 | 9.00 |

22 | TABU_G-VOT | 11.50 | 0.00 | 0.00 | 11.50 | 0.00 | 0.00 | 0.00 |

23 | RP_G-PVO | 7.00 | 6.50 | 0.00 | 0.50 | 0.00 | 0.00 | 0.00 |

Firstly, performance of all group decision making hyper-heuristics are compared to each other. Figure 3 summarises the results including top twelve out of twenty eight approaches. In the overall, CF_G-OR, CF_G-VOT and TABU_G-VOT are the top three group decision making methods, while GR_G-AND and GR_G-OR are the worst. RP_G-PVO, CF_G-AND, CF_G-OR, TABU_G-VOT, CF_G-PVO and CF_G-OR perform the best on boolean satisfiability (SAT), bin-packing (BP), personnel scheduling (PS), permutation flow-shop (PFS), travelling salesman (TSP) and vehicle routing problems (VRP), respectively. Table 2 summarises the ranking of those six group decision making hyper-heuristics and all competing hyper-heuristics at CHeSC 2011, including the top ranking method, denoted as AdapHH. The top ten ranking hyper-heuristics from the competition remains in their positions and group decision making methods perform relatively poor. CF_G-AND is the third best approach for BP. TABU_G-VOT comes sixth for PS. TABU_G-VOT, CF_G-AND and CF_G-VOT score better than the CHeSC 2011 winner for the same problem. CF_G-OR is the best among the group decision making methods for SAT, ranking the eighth. The best group decision making hyper-heuristic for TSP, i.e. CF_G-OR, takes the ninth place. For VRP, CF_G-VOT as the best hyper-heuristic with group decision making is the sixth best approach among the CHeSC 2011 competitors. However, its performance on VRP is still better than the winning approach. The performance of all group decision making methods is poor on the PFS problem. CF_G-AND is the group decision making hyper-heuristic winner and it ranks the eleventh when compared to the CHeSC 2011 hyper-heuristics with a total score of 39.00.

## 4 Conclusion

The experimental results show that the ensemble move acceptance methods based on group decision making models can exploit the strength of constituent move acceptance methods yielding an improved performance. In general, learning heuristic selection performs well within group decision making hyper-heuristics. Considering their performance over the examination timetabling benchmark problems, Greedy performs the best as a heuristic selection method. Combining multiple move acceptance methods using a majority rule improves the performance of Greedy as compared to using a single move acceptance method. On the other side, CF outperforms other standard heuristic selection schemes on the CHeSC 2011 benchmark, performing reasonably well in combination with AND-operator group decision making move acceptance. The proposed ensemble move acceptance methods enable the use of the existing move acceptance methods and do not introduce any extra parameters other than the constituent methods have. Discovering the best choice of move acceptance methods in the ensemble as well as their weights is left as a future work. More interestingly, new adaptive ensemble move acceptance methods, which are capable of adjusting the weight/strength of each constituent move acceptance during the search process, can be designed for improved cross domain performance.

## References

- 1.Asta, S., Özcan, E.: A tensor-based selection hyper-heuristic for cross-domain heuristic search. Inf. Sci.
**299**, 412–432 (2015)CrossRefGoogle Scholar - 2.Bilgin, B., Özcan, E., Korkmaz, E.E.: An experimental study on hyper-heuristics and exam timetabling. In: Burke, E.K., Rudová, H. (eds.) PATAT 2006. LNCS, vol. 3867, pp. 394–412. Springer, Heidelberg (2007). doi: 10.1007/978-3-540-77345-0_25 CrossRefGoogle Scholar
- 3.Burke, E.K., Gendreau, M., Hyde, M., Kendall, G., Ochoa, G., Özcan, E., Qu, R.: Hyper-heuristics: A survey of the state of the art. J. Oper. Res. Soc.
**64**(12), 1695–1724 (2013)CrossRefGoogle Scholar - 4.Carter, M.W., Laporte, G., Lee, S.Y.: Examination timetabling: algorithmic strategies and applications. J. Oper. Res. Soc.
**47**(3), 373–383 (1996)CrossRefGoogle Scholar - 5.Cowling, P., Chakhlevitch, K.: Hyperheuristics for managing a large collection of low level heuristics to schedule personnel. In: IEEE Congress on Evolutionary Computation, pp. 1214–1221 (2003)Google Scholar
- 6.Cowling, P.I., Kendall, G., Soubeiga, E.: A hyperheuristic approach to scheduling a sales summit. In: Burke, E., Erben, W. (eds.) PATAT 2000. LNCS, vol. 2079, p. 176. Springer, Heidelberg (2001)CrossRefGoogle Scholar
- 7.Kendall, G., Mohamad, M.: Channel assignment optimisation using a hyper-heuristic. In: IEEE Conference on Cybernetic and Intelligent Systems, pp. 790–795, 1–3 December 2004Google Scholar
- 8.Kheiri, A., Özcan, E.: An iterated multi-stage selection hyper-heuristic. Eur. J. Oper. Res.
**250**(1), 77–90 (2016)MathSciNetCrossRefzbMATHGoogle Scholar - 9.Kirkpatrick, S., Gelatt, C.D., Vecchi, M.P.: Optimization by simulated annealing. Science
**220**, 671–680 (1983)MathSciNetCrossRefzbMATHGoogle Scholar - 10.Lehre, P.K., Özcan, E.: A runtime analysis of simple hyper-heuristics: to mix or not to mix operators. In: Workshop on Foundations of Genetic Algorithms XII, pp. 97–104 (2013)Google Scholar
- 11.Ochoa, G., et al.: HyFlex: a benchmark framework for cross-domain heuristic search. In: Hao, J.-K., Middendorf, M. (eds.) EvoCOP 2012. LNCS, vol. 7245, pp. 136–147. Springer, Heidelberg (2012)CrossRefGoogle Scholar
- 12.Özcan, E., Misir, M., Kheiri, A.: Group decision making hyper-heuristics for function optimisation. In: The 13th UK Workshop on Computational Intelligence, pp. 327–333, September 2013Google Scholar
- 13.Özcan, E., Bilgin, B., Korkmaz, E.E.: A comprehensive analysis of hyper-heuristics. Intell. Data Anal.
**12**(1), 3–23 (2008)Google Scholar - 14.Özcan, E., Ersoy, E.: Final exam scheduler - fes. In: Corne, D., Michalewicz, Z., McKay, B., Eiben, G., Fogel, D., Fonseca, C., Greenwood, G., Raidl, G., Tan, K.C., Zalzala, A. (eds.) IEEE Congress on Evolutionary Computation, pp. 1356–1363 (2005)Google Scholar
- 15.Parkes, A.J., Özcan, E.: Properties of yeditepe examination timetabling benchmark instances. In: PATAT VIII, pp. 531–534 (2010)Google Scholar

## Copyright information

**Open Access** This chapter is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, duplication, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, a link is provided to the Creative Commons license and any changes made are indicated.

The images or other third party material in this chapter are included in the work’s Creative Commons license, unless indicated otherwise in the credit line; if such material is not included in the work’s Creative Commons license and the respective action is not permitted by statutory regulation, users will need to obtain permission from the license holder to duplicate, adapt or reproduce the material.