Skip to main content
SpringerLink
Log in

A latent space-based estimation of distribution algorithm for large-scale global optimization

  • Foundations
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Large-scale global optimization problems (LSGOs) have received considerable attention in the field of meta-heuristic algorithms. Estimation of distribution algorithms (EDAs) are a major branch of meta-heuristic algorithms. However, how to effectively build the probabilistic model for EDA in high dimensions is confronted with obstacle, making them less attractive due to the large computational requirements. To overcome the shortcomings of EDAs, this paper proposed a latent space-based EDA (LS-EDA), which transforms the multivariate probabilistic model of Gaussian-based EDA into its principal component latent subspace with lower dimensionality. LS-EDA can efficiently reduce the complexity of EDA while maintaining its probability model without losing key information to scale up its performance for LSGOs. When the original dimensions are projected to the latent subspace, those dimensions with larger projected value make more contribution to the optimization process. LS-EDA can also help recognize and understand the problem structure, especially for black-box optimization problems. Due to dimensionality reduction, its computational budget and population size can be effectively reduced while its performance is highly competitive in comparison with the state-of-the-art meta-heuristic algorithms for LSGOs. In order to understand the strengths and weaknesses of LS-EDA, we have carried out extensive computational studies. Our results revealed LS-EDA outperforms the others on the benchmark functions with overlap and nonseparate variables.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  • Armananzas R, Saeys Y, Inza I, Garcia-Torres M (2011) Peakbin selection in mass spectrometry data using a consensus approach with estimation of distribution algorithms. IEEE ACM Trans Comput Biol Bioinform 8(3):760–774

    Article  Google Scholar 

  • Bielza C, Robles V, Larranaga P (2009) Estimation of distribution algorithms as logistic regression regularizers of microarray classifiers. Methods Inf Med 48(3):236–241

    Article  Google Scholar 

  • Bishop CM (2006) Pattern recognition and machine learning. Springer, Berlin

    MATH  Google Scholar 

  • Bosman PA, Thierens D (2000) Continuous iterated density estimation evolutionary algorithms within the idea framework. In: Genetic and Evolutionary Computation Conference, page 197200. Optimization Building Using Probabilistic Models OBUPM Workshop GECCO

  • Bosman PAN, Thierens D (2001) Expanding from discrete to continuous estimation of distribution algorithms: the idea. In: Parallel problem solving from nature—PPSN VI, pp 767–776

  • Che A, Wu P, Chu F, Zhou M (2015) Improved quantum-inspired evolutionary algorithm for large-size lane reservation. IEEE Trans Syst Man Cybern Syst 45(12):1535–1548

    Article  Google Scholar 

  • Dong W, Yao X (2008) Unified eigen analysis on multivariate gaussian based estimation of distribution algorithms. Inf Sci 178(15):3000–3023

    Article  MathSciNet  MATH  Google Scholar 

  • Dong W, Zhou M (2014) Gaussian classifier-based evolutionary strategy for multimodal optimization. IEEE Trans Neural Netw Learn Syst 25(6):1200–1216

    Article  Google Scholar 

  • Dong WS, Chen TS, Tino P, Yao X (2013) Scaling up estimation of distribution algorithms for continuous optimization. IEEE Trans Evolut Comput 17(6):797–822. 265RU Times Cited:4 Cited References Count:58

  • Etxeberria R, Lozano JA (2000) Optimization in continuous domains by learning and simulation of gaussian networks. In: Genetic and Evolutionary Computation Conference

  • Fogel DB (1996) Evolutionary algorithms in theory and practice, evolutionary programming, genetic algorithms. Oxford University Press, Oxford

    Google Scholar 

  • Gomes J, Mariano P, Christensen AL (2017) Novelty-driven cooperative coevolution. Evol Comput 25(2):275–307

    Article  Google Scholar 

  • Guo Z, Zhou MC, Jiang G (2008) Adaptive sensor placement and boundary estimation for monitoring mass objects. IEEE Trans Syst Man Cybern Part B 38(1):222–232

    Article  Google Scholar 

  • Hansen N, Ostermeier A (2001) Completely derandomized self-adaptation in evolution strategies. Evol Comput 9(2):159–195

    Article  Google Scholar 

  • Hauschild MW, Pelikan M, Sastry K, Goldberg DE (2008) Using previous models to bias structural learning in the hierarchical boa. In: Genetic and Evolutionary Computation Conference, GECCO-2008, pp 415–422

  • Kabn A, Bootkrajang J, Durrant RJ (2015) Toward large-scale continuous eda: a random matrix theory perspective. Evol Comput 24:255–291

    Article  Google Scholar 

  • Larranaga P, Lozano JA (2002) Estimation of distribution algorithms: a new tool for evolutionary computation, vol 2. Springer, Berlin

    Book  MATH  Google Scholar 

  • Lin T, Zhang H, Zhang K, Tu Z, Cui N (2016) An adaptive multiobjective estimation of distribution algorithm with a novel Gaussian sampling strategy. Soft Comput 21:6043–6061

    Article  Google Scholar 

  • Lohn JD, Colombano SP (1999) A circuit representation technique for automated circuit design. IEEE Trans Evol Comput 3(3):205–219

    Article  Google Scholar 

  • Lu Q, Yao X (2005) Clustering and learning gaussian distribution for continuous optimization. IEEE Trans Syst Man Cybern Part C Appl Rev 35(2):195–204

    Article  Google Scholar 

  • Mahdavi S, Shiri ME, Rahnamayan S (2015) Metaheuristics in large-scale global continues optimization: a survey. Inf Sci 295:407–428

    Article  MathSciNet  Google Scholar 

  • Mahdavi S, Rahnamayan S, Shiri ME (2016a) Incremental cooperative coevolution for large-scale global optimization. Soft Comput 22:2045–2064

    Article  MATH  Google Scholar 

  • Mahdavi S, Rahnamayan S, Shiri ME (2016b) Multilevel framework for large-scale global optimization. Soft Comput 21:4111–4140

    Article  Google Scholar 

  • Mishra KM, Gallagher M (2014) A modified screening estimation of distribution algorithm for large-scale continuous optimization. In: Asia-Pacific Conference on Simulated Evolution and Learning. Springer, pp 119–130

  • Mühlenbein H, Paass G (1996) From recombination of genes to the estimation of distributions I. Binary parameters. In: Parallel Problem Solving from Nature—PPSN IV. Springer, pp 178–187

  • Omidvar MN, Li X, Yao X (2011) Smart use of computational resources based on contribution for cooperative co-evolutionary algorithms. In: Proceedings of the 13th annual conference on Genetic and evolutionary computation. ACM, pp 1115–1122

  • Omidvar MN, Li X, Mei Y, Yao X (2014) Cooperative co-evolution with differential grouping for large scale optimization. IEEE Trans Evol Comput 18(3):378–393

    Article  Google Scholar 

  • Omidvar MN, Yang M, Mei Y, Li X, Yao X (2017) Dg2: a faster and more accurate differential grouping for large-scale black-box optimization. IEEE Trans Evol Comput 21(6):929–942

    Article  Google Scholar 

  • Pelikan M, Goldberg DE, Lobo FG (2002) A survey of optimization by building and using probabilistic models. Comput Optim Appl 21(1):5–20

    Article  MathSciNet  MATH  Google Scholar 

  • Pok P (2005) On the utility of linear transformations for population-based optimization algorithms. In: Preprints of the 16th World Congress of the International Federation of Automatic Control, pp 281–286

  • Potter MA, De Jong KA (1994) A cooperative coevolutionary approach to function optimization. In: Parallel problem solving from nature—PPSN III. Springer, pp 249–257

  • Ray T, Liew KM (2003) Society and civilization: an optimization algorithm based on the simulation of social behavior. IEEE Trans Evol Comput 7(4):386–396

    Article  Google Scholar 

  • Santana R, Mendiburu A, Lozano JA (2012) Structural transfer using EDAs: an application to multi-marker tagging SNP selection. In Evolutionary Computation, pp 1–8

  • Santana R, Armaanzas R, Bielza C, Larraaga P (2013) Network measures for information extraction in evolutionary algorithms. Int J Comput Intell Syst 6(6):1163–1188

    Article  Google Scholar 

  • Sanyang ML, Kabn A (2015) Heavy tails with parameter adaptation in random projection based continuous EDA. In: 2015 IEEE Congress on Evolutionary Computation (CEC). IEEE, pp 2074–2081

  • Sebag M, Ducoulombier A (1998) Extending population-based incremental learning to continuous search spaces. In: Parallel Problem Solving from NaturePPSN V. Springer, pp 418–427

  • Shang YW, Qiu YH (2006) A note on the extended Rosenbrock function. Evol Comput 14(1):119–126

    Article  Google Scholar 

  • Suganthan PN, Hansen N, Liang JJ, Deb K, Chen YP, Auger A, Tiwari S (2005) Problem definitions and evaluation criteria for the CEC 2005 special session on real-parameter optimization. Nanyang Technological University

  • Sun J, Zhang Q, Tsang E, Ford J (2003) Hybrid estimation of distribution algorithm for global optimization. Eng Comput 21(1):91–107

    MATH  Google Scholar 

  • Tang K, Li X, Suganthan PN, Yang Z, Weise T (2013) Benchmark functions for the CEC’2013 special session and competition on large-scale global optimization. Nature Inspired Computation & Applications

  • Wagner M, Auger A, Schoenauer M (2004) EEDA: a new robust estimation of distribution algorithms. Rapport de Recherche (Res. Rep.)RR-5190

  • Wu FX, Liu LZ, Zhang WJ (2012) Inference of biological S-system using the separable estimation method and the genetic algorithm. IEEE ACM Trans Comput Biol Bioinform 9(4):955–965

    Article  Google Scholar 

  • Yang XS (2014) Swarm intelligence based algorithms: a critical analysis. Evol Intell 7(1):17–28

    Article  Google Scholar 

  • Yang Z, Tang K, Yao X (2007) Differential evolution for high-dimensional function optimization. In IEEE Congress on Evolutionary Computation, 2007. CEC 2007, pp 3523–3530

  • Yang Z, Ke T, Xin Y (2008a) Large scale evolutionary optimization using cooperative coevolution. Inf Sci 178(15):2985–2999

    Article  MathSciNet  MATH  Google Scholar 

  • Yang Z, Tang K, Yao X (2008b) Multilevel cooperative coevolution for large scale optimization. In: IEEE Congress on Evolutionary Computation, 2008. CEC 2008. (IEEE World Congress on Computational Intelligence), pp 1663–1670

  • Yang M, Omidvar MN, Li C, Li X, Cai Z, Kazimipour B, Yao X (2017) Efficient resource allocation in cooperative co-evolution for large-scale global optimization. IEEE Trans Evol Comput 21(4):493–505

    Article  Google Scholar 

  • Yao X, Liu Y, Lin G (1999) Evolutionary programming made faster. IEEE Trans Evol Comput 3(2):82–102

    Article  Google Scholar 

  • Zhou A, Sun J, Zhang Q (2015) An estimation of distribution algorithm with cheap and expensive local search. IEEE Trans Evol Comput 19(6):807–822

    Article  Google Scholar 

Download references

Acknowledgements

This study was funded by the NSF of China (Grant Numbers 61170305, 61672024), the NSF of USA (Grant Number CMMI-1162482) and the Key Research Program in Higher Education of Henan (Grant Number 17A520046).

Author information

Authors and Affiliations

  1. Computer School, Wuhan University, Wuhan, 430072, China

    Wenyong Dong & Yufeng Wang

  2. Software School, Nanyang Institute of Technology, Nanyang, 473000, Henan, China

    Wenyong Dong & Yufeng Wang

  3. Department of Electrical and Computer Engineering, New Jersey Institute of Technology, Newark, NJ, 07102, USA

    Mengchu Zhou

Authors
  1. Wenyong Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yufeng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mengchu Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yufeng Wang.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants performed by any of the authors.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Additional information

Communicated by A. Di Nola.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dong, W., Wang, Y. & Zhou, M. A latent space-based estimation of distribution algorithm for large-scale global optimization. Soft Comput 23, 4593–4615 (2019). https://doi.org/10.1007/s00500-018-3390-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-018-3390-8

Keywords

Search

Navigation