Abstract
Machine-learning approaches to algorithm-selection typically take data describing an instance as input. Input data can take the form of features derived from the instance description or fitness landscape, or can be a direct representation of the instance itself, i.e. an image or textual description. Regardless of the choice of input, there is an implicit assumption that instances that are similar will elicit similar performance from algorithm, and that a model is capable of learning this relationship. We argue that viewing algorithm-selection purely from an instance perspective can be misleading as it fails to account for how an algorithm ‘views’ similarity between instances. We propose a novel ‘algorithm-centric’ method for describing instances that can be used to train models for algorithm-selection: specifically, we use short probing trajectories calculated by applying a solver to an instance for a very short period of time. The approach is demonstrated to be promising, providing comparable or better results to computationally expensive landscape-based feature-based approaches. Furthermore, projecting the trajectories into a 2-dimensional space illustrates that functions that are similar from an algorithm-perspective do not necessarily correspond to the accepted categorisation of these functions from a human perspective.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
We evaluate the effect of this choice later in Sect. 4.3.
- 2.
A classifier trained only on e.g. CMA-ES trajectories can predict any of the three solvers, etc.
References
Alissa, M., Sim, K., Hart, E.: Automated algorithm selection: from feature-based to feature-free approaches. J. Heuristics 29(1), 1–38 (2023). https://doi.org/10.1007/s10732-022-09505-4
Alissa, M., Sim, K., Hart, E.: Algorithm selection using deep learning without feature extraction. In: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 198–206 (2019)
Ardeh, M., Mei, Y., Zhang, M.: Genetic programming hyper-heuristics with probabilistic prototype tree knowledge transfer for uncertain capacitated arc routing problems. In: 2020 IEEE Congress on Evolutionary Computation (CEC), pp. 1–8 (2020). https://doi.org/10.1109/CEC48606.2020.9185714
Belkhir, N.: Per Instance Algorithm Configuration for Continuous Black Box Optimization. phdthesis, Université Paris-Saclay (2017). https://hal.inria.fr/tel-01669527/document
Belkhir, N., Dréo, J., Savéant, P., Schoenauer, M.: Per instance algorithm configuration of CMA-ES with limited budget. In: Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2017, pp. 681–688. ACM (2017). https://doi.org/10.1145/3071178.3071343
Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001). https://doi.org/10.1023/A:1010933404324
Cenikj, G., Petelin, G., Doerr, C., Korosec, P., Eftimov, T.: Dynamorep: trajectory-based population dynamics for classification of black-box optimization problems. In: Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2023, Lisbon, Portugal, July 15–19, 2023, pp. 813–821. ACM (2023). https://doi.org/10.1145/3583131.3590401
Derbel, B., Liefooghe, A., Vérel, S., Aguirre, H., Tanaka, K.: New features for continuous exploratory landscape analysis based on the SOO tree. In: Proceedings of Foundations of Genetic Algorithms (FOGA) 2019, pp. 72–86. ACM (2019). https://doi.org/10.1145/3299904.3340308
Finck, S., Hansen, N., Ros, R., Auger, A.: Real-parameter black-box optimization benchmarking 2010: presentation of the noiseless functions (2010). http://coco.gforge.inria.fr/downloads/download16.00/bbobdocfunctions.pdf
Hansen, N., Auger, A., Ros, R., Mersmann, O., Tusar, T., Brockhoff, D.: COCO: a platform for comparing continuous optimizers in a black-box setting. Opt. Meth. Software 36(1), 114–144 (2021). https://doi.org/10.1080/10556788.2020.1808977
Hansen, N., Ostermeier, A.: Completely derandomized self-adaptation in evolution strategies. Evol. Comput. 9(2), 159–195 (2001). https://doi.org/10.1162/106365601750190398
Heins, J., Bossek, J., Pohl, J., Seiler, M., Trautmann, H., Kerschke, P.: A study on the effects of normalized TSP features for automated algorithm selection. Theor. Comput. Sci. 940(Part), 123–145 (2023). https://doi.org/10.1016/j.tcs.2022.10.019
Jankovic, A., Eftimov, T., Doerr, C.: Towards feature-based performance regression using trajectory data. In: Castillo, P.A., Jiménez Laredo, J.L. (eds.) EvoApplications 2021. LNCS, vol. 12694, pp. 601–617. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-72699-7_38
Jankovic, A., Vermetten, D., Kostovska, A., de Nobel, J., Eftimov, T., Doerr, C.: Trajectory-based algorithm selection with warm-starting. In: IEEE Congress on Evolutionary Computation, CEC 2022, Padua, Italy, July 18–23, 2022, pp. 1–8. IEEE (2022). https://doi.org/10.1109/CEC55065.2022.9870222
Kennedy, J., Eberhart, R.: Particle swarm optimization. In: Proceedings of ICNN 1995 - International Conference on Neural Networks, vol. 4, pp. 1942–1948 (1995). https://doi.org/10.1109/ICNN.1995.488968
Kerschke, P., Hoos, H., Neumann, F., Trautmann, H.: Automated algorithm selection: survey and perspectives. Evol. Comput. 27(1), 3–45 (2019)
Kerschke, P., Preuss, M., Wessing, S., Trautmann, H.: Detecting funnel structures by means of exploratory landscape analysis. In: Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2015, pp. 265–272. ACM (2015). https://doi.org/10.1145/2739480.2754642,http://dl.acm.org/citation.cfm?doid=2739480.2754642
Kerschke, P., Preuss, M., Wessing, S., Trautmann, H.: Low-budget exploratory landscape analysis on multiple peaks models. In: Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2016, pp. 229–236. ACM (2016). https://doi.org/10.1145/2908812.2908845
Kostovska, A., et al.: Per-run algorithm selection with warm-starting using trajectory-based features. In: Rudolph, G., Kononova, A.V., Aguirre, H., Kerschke, P., Ochoa, G., Tusar, T. (eds.) PPSN 2022, Part I. LNCS, vol. 13398, pp. 46–60. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-14714-2_4
Kostovska, A., et al.: Per-run algorithm selection with warm-starting using trajectory-based features. In: Rudolph, G., Kononova, A.V., Aguirre, H., Kerschke, P., Ochoa, G., Tusar, T. (eds.) PPSN 2022, pp. 46–60. Springer, Cham (2022)
Kotthoff, L., Kerschke, P., Hoos, H., Trautmann, H.: Improving the state of the art in inexact TSP solving using per-instance algorithm selection. In: Rudolph, G., et al. (eds.) PPSN 2022. LNCS, vol. 8994, pp. 202–217. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-19084-6_18
Mersmann, O., Bischl, B., Trautmann, H., Preuss, M., Weihs, C., Rudolph, G.: Exploratory landscape analysis. In: Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2011, pp. 829–836. ACM (2011). https://doi.org/10.1145/2001576.2001690
Muñoz, M., Kirley, M., Halgamuge, S.: Exploratory landscape analysis of continuous space optimization problems using information content. IEEE Trans. Evol. Comput. 19(1), 74–87 (2015). https://doi.org/10.1109/TEVC.2014.2302006
de Nobel, J., Wang, H., Bäck, T.: Explorative data analysis of time series based algorithm features of CMA-ES variants. In: GECCO 2021: Genetic and Evolutionary Computation Conference, Lille, France, July 10–14, 2021, pp. 510–518. ACM (2021). https://doi.org/10.1145/3449639.3459399
Pedregosa, F., et al.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)
Pitra, Z., Repický, J., Holena, M.: Landscape analysis of Gaussian process surrogates for the covariance matrix adaptation evolution strategy. In: Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2019, pp. 691–699 (2019). https://doi.org/10.1145/3321707.3321861
Renau, Q.: Landscape-Aware Selection of Metaheuristics for the Optimization of Radar Networks. Ph.D. thesis, Polytechnic Institute of Paris, Palaiseau, France (2022). https://tel.archives-ouvertes.fr/tel-03593606
Renau, Q., Dreo, J., Doerr, C., Doerr, B.: Towards explainable exploratory landscape analysis: extreme feature selection for classifying BBOB functions. In: Castillo, P.A., Jiménez Laredo, J.L. (eds.) EvoApplications 2021. LNCS, vol. 12694, pp. 17–33. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-72699-7_2
Renau, Q., Dréo, J., Peres, A., Semet, Y., Doerr, C., Doerr, B.: Automated algorithm selection for radar network configuration. In: Fieldsend, J.E., Wagner, M. (eds.) Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2022, pp. 1263–1271. ACM (2022). https://doi.org/10.1145/3512290.3528825
Renau, Q., Dreo, J., Doerr, C., Doerr, B.: Exploratory Landscape Analysis Feature Values for the 24 Noiseless BBOB Functions (2021). https://doi.org/10.5281/zenodo.4449934
Rodríguez, J., Kuncheva, L., Alonso, C.: Rotation forest: a new classifier ensemble method. IEEE Trans. Pattern Anal. Mach. Intell. 28(10), 1619–1630 (2006). https://doi.org/10.1109/TPAMI.2006.211
Seiler, M.V., Prager, R.P., Kerschke, P., Trautmann, H.: A collection of deep learning-based feature-free approaches for characterizing single-objective continuous fitness landscapes. In: GECCO 2022: Genetic and Evolutionary Computation Conference, Boston, Massachusetts, USA, July 9–13, 2022, pp. 657–665. ACM (2022). https://doi.org/10.1145/3512290.3528834
Sim, K., Hart, E.: Evolutionary approaches to improving the layouts of instance-spaces. In: Rudolph, G., et al. (eds.) PPSN 2022, Part I. LNCS, vol. 13398, pp. 207–219. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-14714-2_15
Skvorc, U., Eftimov, T., Korosec, P.: A comprehensive analysis of the invariance of exploratory landscape analysis features to function transformations. In: IEEE Congress on Evolutionary Computation, CEC 2022, Padua, Italy, July 18–23, 2022, pp. 1–8. IEEE (2022). https://doi.org/10.1109/CEC55065.2022.9870313
Song, Y., Bliek, L., Zhang, Y.: Revisit the algorithm selection problem for tsp with spatial information enhanced graph neural networks (2023)
Storn, R., Price, K.: Differential evolution - a simple and efficient heuristic for global optimization over continuous spaces. J. Global Optim. 11(4), 341–359 (1997). https://doi.org/10.1023/A:1008202821328
Vermetten, D., Hao, W., Sim, K., Hart, E.: To Switch or not to Switch: Predicting the Benefit of Switching between Algorithms based on Trajectory Features - Dataset (2022). https://doi.org/10.5281/zenodo.7249389
Vermetten, D., Wang, H., Sim, K., Hart, E.: To switch or not to switch: predicting the benefit of switching between algorithms based on trajectory features. In: Correia, J., Smith, S., Qaddoura, R. (eds.) Applications of Evolutionary Computation. LNCS, vol. 13989, pp. 335–350. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-30229-9_22
Zhuang, F., et al.: A comprehensive survey on transfer learning. Proc. IEEE 109(1), 43–76 (2021). https://doi.org/10.1109/JPROC.2020.3004555
Acknowledgements
The authors are funded by the EPSRC ‘Keep-Learning’ project: EP/V026534/1 and EP/V027182/1
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Renau, Q., Hart, E. (2024). On the Utility of Probing Trajectories for Algorithm-Selection. In: Smith, S., Correia, J., Cintrano, C. (eds) Applications of Evolutionary Computation. EvoApplications 2024. Lecture Notes in Computer Science, vol 14634. Springer, Cham. https://doi.org/10.1007/978-3-031-56852-7_7
Download citation
DOI: https://doi.org/10.1007/978-3-031-56852-7_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-56851-0
Online ISBN: 978-3-031-56852-7
eBook Packages: Computer ScienceComputer Science (R0)