Abstract
Combinatorial optimization problems are very common in the real world but difficult to solve. Among the promising algorithms that have been successful in solving these problems are metaheuristics. The two basic search behaviors used in metaheuristics are exploration and exploitation, and the success of metaheuristic search largely depends on the balance of these two behaviors. Machine learning techniques have provided considerable support to improve data-driven optimization algorithms. One of the techniques that stands out is Q-Learning, which is a reinforcement learning technique that penalizes or rewards actions according to the consequence it entails. In this work, a general discretization framework is proposed where Q-Learning can adapt a continuous metaheuristic to work in discrete domains. In particular, we use Q-learning so that the algorithm learns an optimal binarization schemEqe selection policy. The policy is dynamically updated based on the performance of the binarization schemes in each iteration. Preliminary experiments using our framework with sine cosine algorithm show that the proposal presents promising results compared to other algorithms.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Song, H., Triguero, I., Özcan, E.: A review on the self and dual interactions between machine learning and optimisations. Progr. Artifi. Intell. 8(2019), 143–165 (2019)
Talbi, E.G.: Metaheuristics: From Design to Implementation, vol. 74. Wiley, Hoboke (2009)
El-Ghazali, T.: Machine learning into metaheuristics: a survey and taxonomy of data-driven metaheuristics, hal-02745295 (2020)
Watkins, C., Dayan, P.: Technical note: q-learning. Mach. Learn. 8(1992), 279–292 (1992)
Dorigo, M., Gambardella, L.M.: A study of some properties of ant-q. In: Proceeding of PPSN IV-Fourth International Conference on Parallel Problem Solving From Nature, pp. 656–665. Springer-Verlag, Berlin (1996)
Sadeg, S., Hamdad, L., Remache, A.R., Karech, M.N., Benatchba, K., Habbas, Z.: QBSO-FS: a reinforcement learning based bee swarm optimization metaheuristic for feature selection. LNCS 11507, pp. 99–110 (2019)
Sadhu, A.K., Konar, A., Bhattacharjee, T., Das, S.: Synergism of firefly algorithm and q-learning for robot arm path planning. Swarm Evol. Comput. 43, 50–68 (2018)
Lanza-Gutierrez, J.M., Caballe, N.C., Crawford, B., Soto, R., Gómez-Pulido, J., Paredes, F.: Exploring further advantages in an alternative formulation for the set covering problem. Math. Prob. Eng. 2020, Article ID 5473501 (2020)
Mirjalili, S.: SCA: a sine cosine algorithm for solving optimization problems. Knowl. Based Syst. 96, 120–133 (2016)
Crawford, B., Soto, R., Astorga, G., Garcia, J., Castro, C., Paredes, F.: Putting continuous metaheuristics to work in binary search spaces. Complexity 2017, Article ID 8404231 (2017)
García, J., Moraga, P., Valenzuela, M., Crawford, B., Soto, R., Pinto, H., Peña, A., Altimiras, F., Astorga, G.: A Db-scan binarization algorithm applied to matrix covering problems. Comput. Intell. Neurosci. 2019, Article ID 3238574 (2019)
Mirjalili, S., Lewis, A.: S-shaped versus V-shaped transfer functions for binary particle swarm optimization. Swarm Evol. Comput. 9, 1–14 (2013)
Lanza-Gutierrez, J.M., Crawford, B., Soto, R., Berrios, N., Gomez-Pulido, J.A., Paredes, F.: Analyzing the effects of binarization techniques when solving the set covering problem through swarm optimization. Expert Syst. Appl. 70(15), 67–82 (2017)
Zamli, K.Z., Din, F., Ahmed, B.S., Bures, M.: A hybrid Q-learning sine-cosine-based strategy for addressing the combinatorial test suite minimization problem. PLoS One 13(5) (2018)
Mullen, R.L., Monekosso, D., Barman, S., Remagnino, P.: A review of ant algorithms. Expert Syst. Appl. 36(2009), 9608–9617 (2009)
Soto, R., Crawford, B., Muñoz, A., Johnson, F., Paredes, F.: Pre-processing, repairing and transfer functions can help binary electromagnetism-like algorithms. In: Silhavy, R., et al. (eds.) Artificial Intelligence Perspectives and Applications. Advances in Intelligent Systems and Computing, vol. 347, pp. 89–97 (2015)
Crawford, B., Soto, R., Berríos, N., Johnson, F., Paredes, F., Castro, C., Norero, E.: A binary cat swarm optimization algorithm for the non-unicost set covering problem. Math. Prob. Eng. 2015, Article ID 578541 (2015)
Crawford, B., Soto, R., Cuesta, R., Paredes, F.: Application of the artificial bee colony algorithm for solving the set covering problem. Sci. World J. 2014, Article ID 189164 (2014)
Crawford, B., Soto, R., Peña, C., Palma, W., Johnson, F., Paredes, F.: Solving the set covering problem with a shuffled frog leaping algorithm. In: Silhavy, R., et al. (eds.) Software Engineering in Intelligent Systems. Advances in Intelligent Systems and Computing, vol. 349, pp. 317–326 (2015)
Crawford, B., Soto, R., Olivares-Suárez, M., Paredes, F., Johnson, F.: Binary firefly algorithm for the set covering problem. In: proceeding 9th Iberian Conference on Information Systems and Technologies (CISTI), pp. 1–5 (2014)
Acknowledgements
Felipe Cisternas-Caneo and Marcelo Becerra-Rozas are supported by Grant DI Investigación Interdisciplinaria del Pregrado/VRIEA/PUCV/039.324/2020. Broderick Crawford is supported by Grant CONICYT/FONDECYT/REGULAR/1171243. Ricardo Soto is supported by Grant CONICYT/FONDECYT/REGULAR/1190129. Broderick Crawford, Ricardo Soto and Hanns de la Fuente-Mella are supported by Grant Núcleo de Investigación en Data Analytics/VRIEA/PUCV/039.432/2020. José Lemus-Romani is supported by National Agency for Research and Development (ANID)/Scholarship Program/DOCTORADO NACIONAL/2019-21191692.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Cisternas-Caneo, F. et al. (2021). A Data-Driven Dynamic Discretization Framework to Solve Combinatorial Problems Using Continuous Metaheuristics. In: Abraham, A., Sasaki, H., Rios, R., Gandhi, N., Singh, U., Ma, K. (eds) Innovations in Bio-Inspired Computing and Applications. IBICA 2020. Advances in Intelligent Systems and Computing, vol 1372. Springer, Cham. https://doi.org/10.1007/978-3-030-73603-3_7
Download citation
DOI: https://doi.org/10.1007/978-3-030-73603-3_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-73602-6
Online ISBN: 978-3-030-73603-3
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)