Abstract
This paper proposes an algorithm for combinatorial optimizations that uses reinforcement learning and estimation of joint probability distribution of promising solutions to generate a new population of solutions. We call it Reinforcement Learning Estimation of Distribution Algorithm (RELEDA). For the estimation of the joint probability distribution we consider each variable as univariate. Then we update the probability of each variable by applying reinforcement learning method. Though we consider variables independent of one another, the proposed method can solve problems of highly correlated variables. To compare the efficiency of our proposed algorithm with other Estimation of Distribution Algorithms (EDAs) we provide the experimental results of the two problems: four peaks problem and bipolar function.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Baluja, S.: Population based incremental learning: A method for integrating genetic search based function optimization and competitive learning. Technical Report No. CMU-CS-94-163, Carnegie Mellon University, Pittsburgh, Pennsylvania, USA (1994).
Baluja, S. and Caruana, R.: Removing the genetics from standard genetic algorithm. In A. Prieditis and S. Russell, editors, Proceedings of the International Conference on Machine Learning, Morgan Kaufmann, (1995) 38–46.
Baluja, S. and Davies, S.: Using optimal dependency trees for combinatorial optimization: Learning the structure of search space. Technical Report No. CMU-CS-97-107, Carnegie Mellon University, Pittsburgh, Pennsylvania, USA (1997).
Berny, A.: Statistical Machine Learning and Combinatorial Optimization. In Kallel, L., Naudts, B. and Rogers, A., editors, Theoretical Aspects of Evolutionary Computing, Springer (2001).
De Bonet, J.S., Isbell, C.L. and Viola, P.: MIMIC: Finding Optima by estimating probability densities. Advances in Neural Information Processing Systems, 9 (1997).
Deb, K. and Goldberg, D.E.: Sufficient conditions for deceptive and easy binary functions. Annals of Mathematics and Artificial Intelligence, 10 (1994), 385–408.
Goldberg, D.E., Korb, B. and Deb, K.: Messy genetic algorithms: Motivation, analysis and first results. Complex Systems 3(5) (1989) 493–530.
González, C., Lozano, J.A. and Larrañaga, P.: Mathematical modeling of discrete estimation of distribution algorithms. In P. Larrañaga and J.A. Lozano, editors, Estimation of Distribution Algorithms: A New Tool for Evolutionary Optimization. Kluwer Academic Publishers, Boston (2001).
Harik, G.: Learning gene linkage to efficiently solve problems of bounded difficulty using genetic algorithms. IlliGAL Report No. 97005, Illinois Genetic Algorithms Laboratory, University of Illinois, Urbana, Illinois, USA (1997).
Harik, G.: Linkage learning via probabilistic modeling in the ECGA. Illigal Report No. 99010, Illinois Genetic Algorithm Laboratory, University of Illinois, Urbana, Illinois, USA (1999).
Harik, G.R., Lobo, F.G. and Goldberg, D.E.: The compact genetic algorithm. In Proceedings of the IEEE Conference on Evolutionary Computation, (1998) 523–528
Holland, J.H.: Adaptation in Natural and Artificial Systems. The University of Michigan Press (1975).
Kargupta, H.: Revisiting the GEMGA: Scalable evolutionary optimization through linkage learning. In Proceedings of 1998 IEEE International Conference on Evolutionary Computation, IEEE Press, Piscataway, New Jersey, USA (1998) 603–608.
Larrañaga, P. and Lozano, J.A.: Estimation of Distribution Algorithms: A New Tool for Evolutionary Optimization. Kluwer Academic Publishers, Boston, (2001).
Mühlenbein, H.: The equation for response to selection and its use for prediction. Evolutionary Computation, 5(3) (1998) 303–346.
Mühlenbein, H. and Mahnig, T.: The Factorized Distribution Algorithm for additively decomposed functions. Proceedings of the 1999 Congress on Evolutionary Computation, IEEE press (1999) 752–759.
Mühlenbein, H. and Paaß, G.: From recombination of genes to the estimation of distributions I. Binary parameters. In Lecture Notes in Computer Science 1411: Parallel Problem Solving from Nature-PPSN IV, (1996) 178–187.
Paul, T.K. and Iba, H.: Linear and Combinatorial Optimizations by Estimation of Distribution Algorithms. 9th MPS Symposium on Evolutionary Computation, IPSJ Symposium 2003, Japan (2002), 99–106.
Pelikan, M., Goldberg, D.E. and Cantú-Paz, E.: Linkage Problem, Distribution Estimation and Bayesian Networks. Evolutionary Computation, 8(3) (2000) 311–340.
Pelikan, M., Goldberg, D.E. and Lobo, F.G.: A survey of optimization by building and using probabilistic models. Technical Report, Illigal Report No. 99018, University of Illinois at Urbana-Champaign, USA (1999).
Pelikan, M. and Mühlenbein, H.: The bivariate marginal distribution algorithm. Advances in Soft Computing-Engineering Design and Manufacturing, (1999) 521–535.
Williams, R.J.: Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning 8 (1992) 229–256.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Paul, T.K., Iba, H. (2003). Reinforcement Learning Estimation of Distribution Algorithm. In: Cantú-Paz, E., et al. Genetic and Evolutionary Computation — GECCO 2003. GECCO 2003. Lecture Notes in Computer Science, vol 2724. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45110-2_2
Download citation
DOI: https://doi.org/10.1007/3-540-45110-2_2
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-40603-7
Online ISBN: 978-3-540-45110-5
eBook Packages: Springer Book Archive