Abstract
Empirical data plays an important role in evolutionary computation research. To make better use of the available data, ontologies have been proposed in the literature to organize their storage in a structured way. However, the full potential of these formal methods to capture our domain knowledge has yet to be demonstrated. In this work, we evaluate a performance prediction model built on top of the extension of the recently proposed OPTION ontology. More specifically, we first extend the OPTION ontology with the vocabulary needed to represent modular black-box optimization algorithms. Then, we use the extended OPTION ontology, to create knowledge graphs with fixed-budget performance data for two modular algorithm frameworks, modCMA, and modDE, for the 24 noiseless BBOB benchmark functions. We build the performance prediction model using a knowledge graph embedding-based methodology. Using a number of different evaluation scenarios, we show that a triple classification approach, a fairly standard predictive modeling task in the context of knowledge graphs, can correctly predict whether a given algorithm instance will be able to achieve a certain target precision for a given problem instance. This approach requires feature representation of algorithms and problems. While the latter is already well developed, we hope that our work will motivate the community to collaborate on appropriate algorithm representations.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Data and Code Availability.
Our source code, data, the OPTION ontology extension, the generated KGs, and figures are available at: https://github.com/KostovskaAna/KG4AlgorithmPerformancePrediction.git.
Notes
- 1.
In the rest of this paper, we will refer to the ontology classes in italic, while the relations between the classes will be written in typewriter.
References
Basto-Fernandes, V., Yevseyeva, I., Deutz, A., Emmerich, M.: A survey of diversity oriented optimization: problems, indicators, and algorithms. In: Emmerich, M., Deutz, A., Schütze, O., Legrand, P., Tantar, E., Tantar, A.-A. (eds.) EVOLVE – A Bridge between Probability, Set Oriented Numerics and Evolutionary Computation VII. SCI, vol. 662, pp. 3–23. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-49325-1_1
Chen, X., Jia, S., Xiang, Y.: A review: Knowledge reasoning over knowledge graph. Expert Syst. Appl. 141, 112948 (2020)
Costabello, L., Pai, S., Le Van, C., McGrath, R., McCarthy, N., Tabacof, P.: Ampligraph: a library for representation learning on knowledge graphs (2019). Accessed 10 Oct 2019
Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the AISTATS. JMLR Workshop and Conference Proceedings, pp. 249–256 (2010)
Gruber, T.: Toward principles for the design of ontologies used for knowledge sharing? Int. J. Hum Comput Stud. 43(5–6), 907–928 (1995)
Hansen, N., Finck, S., Ros, R., Auger, A.: Real-Parameter Black-Box Optimization Benchmarking 2009: Noiseless Functions Definitions. Research Report RR-6829, INRIA (2009). https://hal.inria.fr/inria-00362633
Hansen, N., Ostermeier, A.: Adapting arbitrary normal mutation distributions in evolution strategies: the covariance matrix adaptation. In: Proceedings of the CEC, pp. 312–317. IEEE (1996)
Kerschke, P., Trautmann, H.: The R-Package FLACCO for exploratory landscape analysis with applications to multi-objective optimization problems. In: CEC, pp. 5262–5269, July 2016. https://doi.org/10.1109/CEC.2016.7748359
Kostovska, A., Vermetten, D., Doerr, C., Dzeroski, S., Panov, P., Eftimov, T.: OPTION: OPTImization Algorithm Benchmarking ONtology. In: Proceedings of the GECCO, Companion Material (2021)
Li, L., Yevseyeva, I., Basto-Fernandes, V., Trautmann, H., Jing, N., Emmerich, M.: Building and using an ontology of preference-based multiobjective evolutionary algorithms. In: Trautmann, H., et al. (eds.) EMO 2017. LNCS, vol. 10173, pp. 406–421. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-54157-0_28
López-Ibáñez, M., Branke, J., Paquete, L.: Reproducibility in evolutionary computation. ACM Trans. Evol. Learn. Optim. 1(4), 1–21 (2021)
Mahmoudi, G., Muller-Schloer, C.: Semantic multi-criteria decision making semcdm. In: 2009 IEEE Symposium on Computational Intelligence in Multi-Criteria Decision-Making (MCDM), pp. 149–156. IEEE (2009)
de Nobel, J., Vermetten, D., Wang, H., Doerr, C., Bäck, T.: Tuning as a means of assessing the benefits of new ideas in interplay with existing algorithmic modules. In: Proceedings of the GECCO, Companion Material, pp. 1375–1384. ACM (2021)
Pedregosa, F., et al.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)
Renau, Q., Doerr, C., Dreo, J., Doerr, B.: Experimental data set for the study “Exploratory Landscape Analysis is Strongly Sensitive to the Sampling Strategy”, June 2020. https://doi.org/10.5281/zenodo.3886816
Smith, B., et al.: Relations in biomedical ontologies. Genome Biol. 6(5), R46 (2005)
Storn, R., Price, K.: Differential evolution-a simple and efficient heuristic for global optimization over continuous spaces. J. Global Optim. 11(4), 341–359 (1997)
Trouillon, T., Welbl, J., Riedel, S., Gaussier, É., Bouchard, G.: Complex embeddings for simple link prediction. In: International Conference on Machine Learning, pp. 2071–2080. PMLR (2016)
Wang, Q., Mao, Z., Wang, B., Guo, L.: Knowledge graph embedding: a survey of approaches and applications. IEEE Trans. Knowl. Data Eng. 29(12), 2724–2743 (2017)
Yaman, A., Hallawa, A., Coler, M., Iacca, G.: Presenting the ECO: evolutionary computation ontology. In: Squillero, G., Sim, K. (eds.) EvoApplications 2017. LNCS, vol. 10199, pp. 603–619. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-55849-3_39
Acknowledgments
The authors acknowledge the support of the Slovenian Research Agency through program grant No. P2-0103 and P2-0098, project grants N2-0239 and J2-4460, a young researcher grant to AK, and a bilateral project between Slovenia and France grant No. BI-FR/23-24-PROTEUS-001 (PR-12040), as well as the EC through grant No. 952215 (TAILOR). Our work is also supported by ANR-22-ERCS-0003-01 project VARIATION, and via a SPECIES scholarship for Ana Kostovska.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Kostovska, A., Vermetten, D., Džeroski, S., Panov, P., Eftimov, T., Doerr, C. (2023). Using Knowledge Graphs for Performance Prediction of Modular Optimization Algorithms. In: Correia, J., Smith, S., Qaddoura, R. (eds) Applications of Evolutionary Computation. EvoApplications 2023. Lecture Notes in Computer Science, vol 13989. Springer, Cham. https://doi.org/10.1007/978-3-031-30229-9_17
Download citation
DOI: https://doi.org/10.1007/978-3-031-30229-9_17
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-30228-2
Online ISBN: 978-3-031-30229-9
eBook Packages: Computer ScienceComputer Science (R0)