Abstract
Adaptive Operator Selection (AOS) is a method used to dynamically determine which operator should be applied in an optimization algorithm based on its performance history. Recently, Upper Confidence Bound (UCB) algorithms have been successfully applied for this task. UCB algorithms have special features to tackle the Exploration versus Exploitation (EvE) dilemma presented on the AOS problem. However, it is important to note that the use of UCB algorithms for AOS is still incipient on Multiobjective Evolutionary Algorithms (MOEAs) and many contributions can be made. The aim of this paper is to extend the study of UCB based AOS methods. Two methods are proposed: MOEA/D-UCB-Tuned and MOEA/D-UCB-V, both use the variance of the operators’ rewards in order to obtain a better EvE tradeoff. In these proposals the UCB-Tuned and UCB-V algorithms from the multiarmed bandit (MAB) literature are combined with MOEA/D (MOEA based on decomposition), one of the most successful MOEAs. Experimental results demonstrate that MOEA/D-UCB-Tuned can be favorably compared with state-of-the-art adaptive operator selection MOEA/D variants based on probability (ENS-MOEA/D and ADEMO/D) and multi-armed bandits (MOEA/D-FRRMAB) methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Auer, P.: Using confidence bounds for exploitation-exploration trade-offs. J. Mach. Learn. Res. 3, 397–422 (2003)
Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Mach. Learn. 47(2–3), 235–256 (2002)
Conover, W.J.: Practical Nonparametric Statistics, 3rd edn. Wiley (1999)
Ehrgott, M.: A discussion of scalarization techniques for multiple objective integer programming. Ann. Oper. Res. 147, 343–360 (2006)
Fialho, A.: Adaptive operator selection for optimization. Ph.D. thesis, Comput. Sci. Dept. - Univ. Paris-Sud XI (2010)
Fialho, A., Schoenauer, M., Sebag, M.: Analysis of adaptive operator selection techniques on the royal road and long k-path problems. In: Conference on Genetic and Evolutionary Computation, pp. 779–786 (2009)
Goldberg, D.E.: Probability matching, the magnitude of reinforcement, and classifier system bidding. Mach. Learn. 5, 407–425 (1990)
Gong, W., Fialho, l, Cai, Z., Li, H.: Adaptive strategy selection in differential evolution for numerical optimization: An empirical study. Inform. Sciences 181(24), 5364–5386 (2011)
Mashwani, Khan: W., Salhi, A.: A decomposition-based hybrid multiobjective evolutionary algorithm with dynamic resource allocation. Appl. Soft Comput. 12(9), 2765–2780 (2012)
Knowles, J., Thiele, L., Zitzler, E.: A Tutorial on the Performance Assessment of Stochastic Multiobjective Optimizers. TIK Report 214, Computer Engineering and Networks Laboratory (TIK), ETH Zurich (February 2006)
Li, K., Fialho, A., Kwong, S., Zhang, Q.: Adaptive operator selection with bandits for a multiobjective evolutionary algorithm based on decomposition. IEEE Transactions on Evolutionary Computation 18(1), 114–130 (2014)
Sato, H.: Inverted PBI in MOEA/D and its impact on the search performance on multi and many-objective optimization. In: Proceedings of the 2014 Conference on Genetic and Evolutionary Computation, GECCO 2014, pp. 645–652. ACM, New York (2014). http://doi.acm.org/10.1145/2576768.2598297
Thierens, D.: An adaptive pursuit strategy for allocating operator probabilities. In: Conference on Genetic and Evolutionary Computation, pp. 1539–1546 (2005)
Venske, S.M., Gonalves, R.A., Delgado, M.R.: ADEMO/D: Multiobjective optimization by an adaptive differential evolution algorithm. Neurocomputing 127, 65–77 (2014), advances in Intelligent Systems Selected papers from the 2012 Brazilian Symposium on Neural Networks
Zhang, Q., Zhou, A., Zhao, S., Suganthan, P.N., Liu, W., Tiwari, S.: Multiobjective optimization test instances for the CEC 2009 special session and competition. Tech. rep., University of Essex and Nanyang Technological University, CES-487 (2008)
Zhang, Q., Liu, W., Li, H.: The performance of a new version of MOEA/D on CEC09 unconstrained mop test instances. In: IEEE Congress on Evolutionary Computation, CEC 2009, pp. 203–208 (May 2009)
Zhao, S.Z., Suganthan, P.N., Zhang, Q.: Decomposition-based multiobjective evolutionary algorithm with an ensemble of neighborhood sizes. IEEE Trans. Evol. Comput. 16(3), 442–446 (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Gonçalves, R.A., Almeida, C.P., Pozo, A. (2015). Upper Confidence Bound (UCB) Algorithms for Adaptive Operator Selection in MOEA/D. In: Gaspar-Cunha, A., Henggeler Antunes, C., Coello, C. (eds) Evolutionary Multi-Criterion Optimization. EMO 2015. Lecture Notes in Computer Science(), vol 9018. Springer, Cham. https://doi.org/10.1007/978-3-319-15934-8_28
Download citation
DOI: https://doi.org/10.1007/978-3-319-15934-8_28
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-15933-1
Online ISBN: 978-3-319-15934-8
eBook Packages: Computer ScienceComputer Science (R0)