Skip to main content
SpringerLink
Log in

MONEDA: scalable multi-objective optimization with a neural network-based estimation of distribution algorithm

  • Published:
Journal of Global Optimization Aims and scope Submit manuscript

Abstract

The extension of estimation of distribution algorithms (EDAs) to the multi-objective domain has led to multi-objective optimization EDAs (MOEDAs). Most MOEDAs have limited themselves to porting single-objective EDAs to the multi-objective domain. Although MOEDAs have proved to be a valid approach, the last point is an obstacle to the achievement of a significant improvement regarding “standard” multi-objective optimization evolutionary algorithms. Adapting the model-building algorithm is one way to achieve a substantial advance. Most model-building schemes used so far by EDAs employ off-the-shelf machine learning methods. However, the model-building problem has particular requirements that those methods do not meet and even evade. The focus of this paper is on the model-building issue and how it has not been properly understood and addressed by most MOEDAs. We delve down into the roots of this matter and hypothesize about its causes. To gain a deeper understanding of the subject we propose a novel algorithm intended to overcome the drawbacks of current MOEDAs. This new algorithm is the multi-objective neural estimation of distribution algorithm (MONEDA). MONEDA uses a modified growing neural gas network for model-building (MB-GNG). MB-GNG is a custom-made clustering algorithm that meets the above demands. Thanks to its custom-made model-building algorithm, the preservation of elite individuals and its individual replacement scheme, MONEDA is capable of scalably solving continuous multi-objective optimization problems. It performs better than similar algorithms in terms of a set of quality indicators and computational resource requirements.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  1. Ahn, C.W.: Advances in Evolutionary Algorithms. Theory, Design and Practice. Springer, ISBN 3-540-31758-9 (2006)

  2. Ahn, C.W., Ramakrishna, R.S.: Multiobjective real-coded Bayesian optimization algorithm revisited: Diversity preservation. In: GECCO’07. Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation, ACM Press, New York, NY, USA, pp. 593–600, (2007). doi:10.1145/1276958.1277079

  3. Ahn, C.W., Goldberg, D.E., Ramakrishna, R.S.: Real-coded Bayesian optimization algorithm: bringing the strength of BOA into the continuous world. In: 2004 Genetic and Evolutionary Computation (GECCO 2004), Lecture Notes in Computer Science, vol. 3102, Springer, pp. 840–851 (2004)

  4. Bäck, T.: Evolutionary Algorithms in Theory and Practice: Evolution Strategies, Evolutionary Programming, Genetic Algorithms. Oxford University Press, New York (1996)

    MATH  Google Scholar 

  5. Bäck, T., Fogel, D.B., Michalewicz, Z. (eds.): Handbook of Evolutionary Computation. Institute of Physics Publishing and Oxford University Press (1997)

  6. Bader, J.: Hypervolume-Based Search for Multiobjective Optimization: Theory and Methods. PhD thesis, ETH Zurich, Switzerland (2010)

  7. Bader, J., Zitzler, E.: HypE: An Algorithm for Fast Hypervolume-Based Many-Objective Optimization. TIK Report 286, Computer Engineering and Networks Laboratory (TIK), ETH Zurich (2008)

  8. Bader, J., Zitzler, E.: HypE: an algorithm for fast hypervolume-based many-objective optimization. Evolut. Comput. 19(1), 45–76 (2011). doi:10.1162/EVCO_a_00009

    Article  Google Scholar 

  9. Bader, J., Deb, K., Zitzler, E.: Faster hypervolume-based search using Monte Carlo sampling. In: Beckmann, M., Künzi, H.P., Fandel, G., Trockel, W., Basile, A., Drexl, A., Dawid, H., Inderfurth, K., Kürsten, W., Schittko, U., Ehrgott, M., Naujoks, B., Stewart, T.J., Wallenius, J. (eds.) Multiple Criteria Decision Making for Sustainable Energy and Transportation Systems, Springer, Berlin/Heidelberg, Lecture Notes in Economics and Mathematical Systems, vol. 634, pp. 313–326, (2010). doi:10.1007/978-3-642-04045-0_27

  10. Bandyopadhyay, S., Pal, S.K., Aruna, B.: Multiobjective GAs, quantitative indices, and pattern classification. IEEE Trans. Syst. Man Cybern. Part B Cybern. 34(5), 2088–2099 (2004)

    Article  Google Scholar 

  11. Basseur, M., Zitzler, E.: Handling uncertainty in indicator-based multiobjective optimization. Int. J. Comput. Intell. Res. 2(3), 255–272 (2006)

    Article  MathSciNet  Google Scholar 

  12. Basseur, M., Zitzler, E.: A preliminary study on handling uncertainty in indicator-based multiobjective optimization. In: Rothlauf, F., Branke, J., Cagnoni, S., Costa, E., Cotta, C., Drechsler, R., Lutton, E., Machado, P., Moore, J.H., Romero, J., Smith, G.D., Squillero, G., Takagi, H. (eds.) Applications of Evolutionary Computing. EvoWorkshops 2006: EvoBIO, EvoCOMNET, EvoHOT, EvoIASP, EvoINTERACTION, EvoMUSART, and EvoSTOC, Springer, Budapest, Hungary, Lecture Notes in Computer Science, vol. 3907, pp. 727–739 (2006)

  13. Bellman, R.E.: Adaptive Control Processes. Princeton University Press, Princeton (1961)

    Book  MATH  Google Scholar 

  14. Benjamini, Y.: Opening the box of a boxplot. Am. Stat. 42(4), 257–262 (1988). doi:10.2307/2685133

    Google Scholar 

  15. Berkhin, P.: Survey of Clustering Data Mining Techniques. Tech. rep., Accrue Software, San Jose, CA. http://citeseer.ist.psu.edu/berkhin02survey.html. (2002)

  16. Beume, N.: S-metric calculation by considering dominated hypervolume as Klee’s measure problem. Evolut. Comput. 17(4), 477–492 (2009). doi:10.1162/evco.2009.17.4.17402

    Article  Google Scholar 

  17. Beume, N., Rudolph, G.: Faster S-metric calculation by considering dominated hypervolume as Klee’s measure problem. In: Kovalerchuk, B. (ed.) Proceedings of the Second IASTED International Conference on Computational Intelligence. IASTED/ACTA Press, pp. 233–238 (2006)

  18. Bleuler, S., Laumanns, M., Thiele, L., Zitzler, E.: PISA—a platform and programming language independent interface for search algorithms. In: Fonseca, C.M., Fleming, P.J., Zitzler, E., Deb, K., Thiele, L. (eds.) Evolutionary Multi-Criterion Optimization. Second International Conference, EMO 2003, Springer. Lecture Notes in Computer Science, vol. 2632, Faro, Portugal, pp. 494–508 (2003)

  19. Bosman, P.A., Thierens, D.: Adaptive variance scaling in continuous multi-objective estimation-of-distribution algorithms. In: Proceedings of the 9th annual conference on Genetic and evolutionary computation—GECCO’07. ACM Press, New York, New York, USA, p 500 (2007). doi:10.1145/1276958.1277067. http://portal.acm.org/citation.cfm?doid=1276958.1277067

  20. Bosman, P.A.N.: Design and Application of Iterated Density-Estimation Evolutionary Algorithms. PhD thesis, Institute of Information and Computing Sciences, Universiteit Utrecht, Utrecht, The Netherlands (2003)

  21. Bosman, P.A.N.: The anticipated mean shift and cluster registration in mixture-based EDAs for multi-objective optimization. In: Proceedings of the 12th Annual Conference on Genetic and Evolutionary Computation—GECCO’10. ACM Press, New York, New York, USA, p. 351 (2010). doi:10.1145/1830483.1830549. http://portal.acm.org/citation.cfm?doid=1830483.1830549

  22. Bosman, P.A.N., Thierens, D.: Multi-objective optimization with diversity preserving mixture-based iterated density estimation evolutionary algorithms. Int. J. Approx. Reason. 31(3), 259–289 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  23. Bosman, P.A.N., Thierens, D.: The balance between proximity and diversity in multiobjective evolutionary algorithms. IEEE Trans. Evolut. Comput. 7(2), 174–188 (2003)

    Article  Google Scholar 

  24. Bosman, P.A.N, Thierens, D.: The naïve MIDEA: a baseline multi-objective EA. In: Coello, C.A.C., Hernández A.A., Zitzler, E, (eds.) Evolutionary Multi-Criterion Optimization. Third International Conference, EMO 2005, Springer. Lecture Notes in Computer Science, vol. 3410, Guanajuato, México, pp. 428–442 (2005)

  25. Bosman, P.A.N., Grahl, J., Thierens, D.: Benchmarking parameter-free amalgam on functions with and without noise. Evolut. Comput. 21(3), 445–469 (2013). doi:10.1162/EVCO_a_00094

    Article  Google Scholar 

  26. Box, G.E.P., Muller, M.E.: A note on the generation of random normal deviates. Ann. Math. Stat. 29, 610–611 (1958)

    Article  MATH  Google Scholar 

  27. Branke, J., Miettinen, K., Deb, K., Słowiǹski, R. (eds.) Multiobjective Optimization, Lecture Notes in Computer Science, vol. 5252. Springer, Berlin/Heidelberg (2008)

  28. Brockhoff, D., Zitzler, E.: Dimensionality reduction in multiobjective optimization: the minimum objective subset problem. In: Waldmann, K.H., Stocker, U.M. (eds.) Operations Research Proceedings 2006. Springer, pp. 423–429 (2007)

  29. Brockhoff, D., Zitzler, E.: Improving hypervolume-based multiobjective evolutionary algorithms by using objective reduction methods. In: IEEE Congress on Evolutionary Computation (CEC 2007). IEEE Press, pp. 2086–2093 (2007)

  30. Brockhoff, D., Saxena, D.K., Deb, K., Zitzler, E.: On handling a large number of objectives a posteriori and during optimization. In: Knowles, J., Corne, D., Deb, K. (eds.) Multi-Objective Problem Solving from Nature: From Concepts to Applications, Natural Computing Series. Springer, pp. 377–403 (2008). doi:10.1007/978-3-540-72964-8

  31. Chambers, J., Cleveland, W., Kleiner, B., Tukey, P.: Graphical Methods for Data Analysis. Wadsworth, Belmont (1983)

  32. Coello C.A.C.: 20 years of evolutionary multi-objective optimization: what has been done and what remains to be done. In: Yen, G.Y., Fogel, D.B. (eds.) Computational Intelligence: Principles and Practice, IEEE Computational Intelligence Society, Vancouver, Canada, chap 4, pp. 73–88 (2006)

  33. Coello, C.A.C.: Evolutionary multiobjective optimization: a historical view of the field. IEEE Comput. Intell. Mag. 1(1), 28–36 (2006)

  34. Coello C.A.C., Lamont, G.B., Van Veldhuizen D.A.: Evolutionary Algorithms for Solving Multi-Objective Problems, 2nd edn. Genetic and Evolutionary Computation, Springer, New York. http://www.springer.com/west/home/computer/foundations?SGWID=4-156-22-173660344-0 (2007)

  35. Conover, W.J.: Practical Nonparametric Statistics, 3rd edn. Wiley, New York (1999)

    Google Scholar 

  36. Cooper, G., Herskovits, E.: A Bayesian method for the induction of probabilistic networks from data. Mach. Learn. 9(4), 309–347 (1992)

    MATH  Google Scholar 

  37. Corne, D.W., Knowles, J.D.: No free lunch and free leftovers theorems for multiobjective optimisation problems. In: Fonseca, C.M., Fleming, P.J., Zitzler, E., Deb, K., Thiele, L. (eds.) Evolutionary Multi-Criterion Optimization. Second International Conference, EMO 2003, Springer, Faro, Portugal, Lecture Notes in Computer Science, vol. 2632, pp. 327–341 (2003)

  38. Corne, D.W., Knowles, J.D., Oates. M.J.: The Pareto envelope-based selection algorithm for multiobjective optimization. In: Schoenauer, M., Deb, K., Rudolph, G., Yao, X., Lutton, E., Merelo, J.J., Schwefel, H.P. (eds.) Proceedings of the Parallel Problem Solving from Nature VI Conference, Springer. Lecture Notes in Computer Science No. 1917, Paris, France, pp. 839–848 (2000)

  39. Corne, D.W., Jerram, N.R., Knowles, J.D., Oates, M.J.: PESA-II: region-based selection in evolutionary multiobjective optimization. In: Spector, L., Goodman, E.D., Wu, A., Langdon, W., Voigt, H.M., Gen. M., Sen, S., Dorigo, M., Pezeshk, S., Garzon, M.H., Burke, E. (eds.) Proceedings of the Genetic and Evolutionary Computation Conference (GECCO’2001), Morgan Kaufmann Publishers, San Francisco, California, pp. 283–290 (2001)

  40. Costa, M., Minisci, E.: MOPED: a multi-objective Parzen-based estimation of distribution algorithm for continuous problems. In: Fonseca, C.M., Fleming, P.J., Zitzler, E., Deb, K., Thiele, L. (eds.) Evolutionary Multi-Criterion Optimization. Second International Conference, EMO 2003, Springer. Lecture Notes in Computer Science, vol. 2632, Faro, Portugal, pp. 282–294 (2003)

  41. Costa, M., Minisci, E., Pasero, E.: An hybrid neural/genetic approach to continuous multi-objective optimization problems. In: Apolloni, B., Marinaro, M., Tagliaferri, R. (eds.) Italian Workshop on Neural Neural Nets (WIRN), Springer, Lecture Notes in Computer Science, vol. 2859, pp. 61–69 (2003)

  42. Darwin, C.: On the Origin of Species by Means of Natural Selection, or The Preservation of Favoured Races in the Struggle for Life. John Murray, London (1859)

  43. De Jong, K.A.: Evolutionary Computation: A Unified Approach. MIT Press, Cambridge (2006)

    MATH  Google Scholar 

  44. Deb, K.: Multi-Objective Optimization using Evolutionary Algorithms. Wiley, Chichester, UK. ISBN 0-471-87339-X (2001)

  45. Deb, K., Saxena, D.K.: On finding Pareto-optimal solutions through dimensionality reduction for certain large-dimensional multi-objective optimization problems. Tech. Rep. 2005011, KanGAL (2005)

  46. Deb, K., Saxena, D.K.: Searching for Pareto-optimal solutions through dimensionality reduction for certain large-dimensional multi-objective optimization problems. In: 2006 IEEE Conference on Evolutionary Computation (CEC’2006), IEEE Press, Piscataway, New Jersey, pp. 3352–3360 (2006)

  47. Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. Evolut. Comput. 6(2), 182–197 (2002)

    Article  Google Scholar 

  48. Deb, K., Thiele, L., Laumanns, M., Zitzler, E.: Scalable test problems for evolutionary multiobjective optimization. In: Abraham,A., Jain, L., Goldberg, R. (eds.) Evolutionary Multiobjective Optimization: Theoretical Advances and Applications, Advanced Information and Knowledge Processing, Springer Verlag, pp. 105–145 (2004)

  49. Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. B 39, 1–38 (1977)

    MathSciNet  MATH  Google Scholar 

  50. Ehrgott, M.: Multicriteria Optimization, Lecture Notes in Economics and Mathematical Systems, vol. 491. Springer (2005)

  51. Etxeberria, R., Larrañaga, P.: Global optimization using Bayesian networks. In: Ochoa, A., Soto, M.R., Santana, R. (eds.) Proceedings of the Second Symposium on Artificial Intelligence (CIMAF-99), Habana, Cuba, pp. 151–173 (1999)

  52. Fleischer, M.: The measure of Pareto optima. Applications to multi-objective metaheuristics. In: Fonseca, C.M., Fleming, P.J., Zitzler, E., Deb, K., Thiele, L. (eds.) Evolutionary Multi-Criterion Optimization. Second International Conference, EMO 2003, Springer. Lecture Notes in Computer Science, vol. 2632, Faro, Portugal, pp. 519–533 (2003)

  53. Flentge, F.: Locally weighted interpolating growing neural gas. IEEE Trans. Neural Netw. 17(6), 1382–1393 (2006)

    Article  Google Scholar 

  54. Fonseca, C., Fleming, P.: Multiobjective Genetic Algorithms. In: IEE Colloquium on Genetic Algorithms for Control Systems Engineering, IEE, pp. 6/1-6/5 (1993)

  55. Fonseca, C.M., Fleming, P.J.: Genetic algorithms for multiobjective optimization: formulation, discussion and generalization. In: Forrest, S. (ed.) Proceedings of the FifthInternational Conference on Genetic Algorithms, University of Illinois at Urbana-Champaign, Morgan Kauffman Publishers, San Mateo, California, pp. 416–423 (1993)

  56. Fonseca, C.M., Paquete, L., López-Ibánez, M.: An improved dimension-sweep algorithm for the hypervolume indicator. In: 2006 IEEE Congress on Evolutionary Computation (CEC’2006), pp. 1157–1163 (2006)

  57. Fritzke, B.: Fast learning with incremental RBF networks. Neural Process. Lett. 1, 2–5 (1994)

    Article  Google Scholar 

  58. Fritzke, B.: A growing neural gas network learns topologies. In: Tesauro, G., Touretzky, D.S., Leen, T.K. (eds.) Advances in Neural Information Processing Systems, vol. 7, pp. 625–632. MIT Press, Cambridge (1995)

    Google Scholar 

  59. Fritzke, B.: Incremental neuro-fuzzy systems. In: Application of Soft Computing, SPIE International Symposium on Optical Science, Engineering and Instrumentation, San Diego, CA (1997)

  60. Grünwald, P.D.: The Minimum Description Length Principle (Adaptive Computation and Machine Learning). The MIT Press (2007)

  61. Hartigan, J.A.: Clustering Algorithms. Wiley Series in Probability and Mathematical Statistics. Wiley, New York (1975)

  62. Horn, J., Nafpliotis, N., Goldberg, D.E.: A Niched Pareto genetic algorithm for multiobjective optimization. In: Proceedings of the First IEEE Conference on Evolutionary Computation, IEEE World Congress on Computational Intelligence, IEEE Service Center, Piscataway, New Jersey, vol. 1, pp. 82–87 (1994)

  63. Huband, S., Barone, L., While, L., Hingston, P.: A scalable multi-objective test problem toolkit. In: Coello C.A.C., Hernández Aguirre, A., Zitzler, E. (eds.) Evolutionary Multi-Criterion Optimization. Third International Conference, EMO 2005, Springer. Lecture Notes in Computer Science, vol. 3410, Guanajuato, México, pp. 280–295 (2005)

  64. Huband, S., Hingston, P., Barone, L., While, L.: A review of multiobjective test problems and a scalable test problem toolkit. IEEE Trans. Evolut. Comput. 10(5), 477–506 (2006)

    Article  MATH  Google Scholar 

  65. Ishibuchi, H., Murata, T.: Multi-objective genetic local search algorithm and its application to flowshop scheduling. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 28(3), 392–403 (1998)

    Article  Google Scholar 

  66. Jain, A.K., Murty, M.N., Flynn, P.J.: Data clustering: a review. ACM Comput. Surv. 31(3), 264–323 (1999). doi:10.1145/331499.331504

    Article  Google Scholar 

  67. Kambhatla, N., Leen, T.K.: Dimension reduction by local principal component analysis. Neural Comput. 9(7), 1493–1516 (1997)

    Article  Google Scholar 

  68. Karshenas, H., Santana, R., Bielza, C., Larranaga, P.: Multiobjective estimation of distribution algorithm based on joint modeling of objectives and variables. IEEE Trans. Evolut. Comput. 18(4), 519–542 (2014). doi:10.1109/TEVC.2013.2281524

    Article  Google Scholar 

  69. Khan, N.: Bayesian Optimization Algorithms for Multiobjective and Hierarchically Difficult Problems. Master’s thesis, Graduate College of the University of Illinois at Urbana-Champaign, Urbana, Illinois, USA (2003)

  70. Khan, N., Goldberg, D.E., Pelikan, M.: Multi-objective Bayesian optimization algorithm. In: Langdon, W., Cantú-Paz, E., Mathias, K., Roy, R., Davis, D., Poli, R., Balakrishnan, K., Honavar, V., Rudolph, G., Wegener, J., Bull, L., Potter, M., Schultz, A., Miller, J., Burke, E., Jonoska, N. (eds.) Proceedings of the Genetic and Evolutionary Computation Conference (GECCO’2002), Morgan Kaufmann Publishers, San Francisco, California, p. 684 (2002)

  71. Khare, V., Yao, X., Deb, K.: Performance scaling of multi-objective evolutionary algorithms. In: Fonseca, C.M., Fleming, P.J., Zitzler, E., Deb, K., Thiele, L. (eds.) Evolutionary Multi-Criterion Optimization. Second International Conference, EMO 2003, Springer. Lecture Notes in Computer Science, vol. 2632, Faro, Portugal, pp. 376–390 (2003)

  72. Knowles, J., Corne, D.: Quantifying the effects of objective space dimension in evolutionary multiobjective optimization. In: Obayashi, S., Deb, K., Poloni, C., Hiroyasu, T., Murata, T. (eds.) Proceedings 4th International Conference Evolutionary Multi-Criterion Optimization (EMO 2007), Springer, Berlin/Heidelberg, pp. 757–771 (2007). doi:10.1007/978-3-540-70928-2_57

  73. Knowles, J., Thiele, L., Zitzler, E.: A Tutorial on the Performance Assessment of Stochastic Multiobjective Optimizers. 214, Computer Engineering and Networks Laboratory (TIK), ETH Zurich, Switzerland, revised version (2006)

  74. Knowles, J., Thiele, L., Zitzler, E.: A Tutorial on the Performance Assessment of Stochastic Multiobjective Optimizers. TIK Report 214, Computer Engineering and Networks Laboratory (TIK), ETH Zurich (2006)

  75. Knowles, J.D.: Local-Search and Hybrid Evolutionary Algorithms for Pareto Optimization. PhD thesis, The University of Reading, Department of Computer Science, Reading, UK (2002)

  76. Knowles, J.D., Corne, D.W.: The Pareto archived evolution strategy: a new baseline algorithm for multiobjective optimisation. In: 1999 Congress on Evolutionary Computation, IEEE Service Center, Washington, D.C., pp. 98–105 (1999)

  77. Knowles, J.D., Corne, D.W.: Approximating the nondominated front using the Pareto archived evolution strategy. Evolut. Comput. 8(2), 149–172 (2000)

    Article  Google Scholar 

  78. Kruskal, W.H., Wallis, W.A.: Use of ranks in one-criterion analysis of variance. J. Am. Stat. Assoc. 47, 583–621 (1952)

    Article  MATH  Google Scholar 

  79. Larrañaga, P.: A review on estimation of distribution algorithms. In: Larrañaga, P., Lozano, J.A. (eds.) Estimation of Distribution Algorithms. A New Tool forEvolutionary Computation, Kluwer Academic Publishers, Boston/Dordrecht/London, pp. 55–98 (2002)

  80. Larrañaga, P., Lozano, J.A. (eds.) Estimation of Distribution Algorithms. A New Tool for Evolutionary Computation. Genetic Algorithms and Evolutionary Computation, Kluwer Academic Publishers, Boston/Dordrecht/London (2002)

  81. Laumanns, M., Ocenasek, J.: Bayesian optimization algorithms for multi-objective optimization. In: Merelo Guervós JJ, Adamidis P, Beyer HG, nas JLFV, Schwefel HP (eds.) Parallel Problem Solving from Nature–PPSN VII, Springer-Verlag. Lecture Notes in Computer Science No. 2439, Granada, Spain, pp. 298–307 (2002)

  82. Levon, J.: OProfile manual. Victoria University of Manchester. http://oprofile.sourceforge.net/ (2004)

  83. Lozano, J.A., Larrañaga, P., Inza, I., Bengoetxea, E.: (eds) Towards a New Evolutionary Computation: Advances on Estimation of Distribution Algorithms. Springer, Berlin (2006)

  84. MacQueen, J.: Some methods for classification and analysis ofmultivariate observations. In: Proceedings of the Fifth Berkeley Symposium on Mathematical, vol 1, pp. 281–297 (1967)

  85. Mann, H.B., Whitney, D.R.: On a test of whether one of two random variables is stochastically larger than the other. Ann. Math. Stat. 18, 50–60 (1947)

    Article  MathSciNet  MATH  Google Scholar 

  86. Martí, L., García, J., Berlanga, A., Molina, J.M.: A cumulative evidential stopping criterion for multiobjective optimization evolutionary algorithms. In: Thierens, D., Deb, K., Pelikan, M., Beyer, H.G., Doerr, B., Poli, R., Bittari, M. (eds.) GECCO’09: Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation, ACM Press, New York, p. 911, (2007). doi:10.1145/1276958.1277141. http://portal.acm.org/citation.cfm?doid=1276958.1277141

  87. Martí, L., García, J., Berlanga, A., Molina, J.M.: Introducing MONEDA: scalable multiobjective optimization with a neural estimation of distribution algorithm. In: GECCO’08: 10th Annual Conference on Genetic and Evolutionary Computation, ACM Press, New York, NY, USA, pp. 689–696 (2008). doi:10.1145/1389095.1389230

  88. Martí, L., García, J., Berlanga, A., Molina, J.M.: Model-building algorithms for multiobjective EDAs: directions for improvement. In: 2008 IEEE Conference on Evolutionary Computation (CEC), Part of 2008 IEEE World Congress on Computational Intelligence (WCCI 2008), IEEE Press, Piscataway, New Jersey, pp. 2848–2855 (2008). doi:10.1109/CEC.2008.4631179. http://ieeexplore.ieee.org/iel5/4625778/4630767/04631179.pdf?tp=&arnumber=4631179&isnumber=4630767

  89. Martí, L., García, J., Berlanga, A., Molina, J.M.: An approach to stopping criteria for multi-objective optimization evolutionary algorithms: The MGBM criterion. In: 2009 IEEE Conference on Evolutionary Computation (CEC 2009), IEEE Press, Piscataway, New Jersey, pp. 1263–1270 (2009). doi:10.1109/CEC.2009.4983090

  90. Martí, L., García, J., Berlanga, A., Molina, J.M.: Solving complex high-dimensional problems with the multi-objective neural estimation of distribution algorithm. In: Raidl, G., Alba, E., Bacardit, J., Bates Congdon, C., Beyer, H.G., Birattari, M., Blum, C., Bosman, P.A.N., Corne, D., Cotta, C., Di Penta, M., Doerr, B., Drechsler, R., Ebner, M., Grahl, J., Jansen, T., Knowles, J., Lenaerts, T., Middendorf, M., Miller, J.F., O’Neill, M., Poli, R., Squillero, G., Stanley, K., Stützle, T., van Hemert, J. (eds.) GECCO 2009: 11th Annual Conference on Genetic and Evolutionary Computation, ACM Press, New York, NY, USA, pp. 619–626 (2009). doi:10.1145/1569901.1569987. http://portal.acm.org/citation.cfm?id=1569901.1569987

  91. Martí, L., García, J., Berlanga, A., Coello, C.A.C., Molina, J.M.: On Current Model-building Methods for Multi-objective Estimation of Distribution Algorithms: Shortcommings and Directions for Improvement. Tech. Rep. GIAA2010E001, Grupo de Inteligencia Artificial Aplicada, Universidad Carlos III de Madrid, Colmenarejo, Spain. http://www.giaa.inf.uc3m.es/miembros/lmarti/model-building (2010)

  92. Martinetz, T.M.: Competitive Hebbian learning rule forms perfectly topology preserving maps. In: International Conference on Artificial Neural Networks (ICANN’93), Springer-Verlag, Amsterdam, pp. 427–434 (1993)

  93. Martinetz, T.M., Berkovich, S.G., Shulten, K.J.: Neural-gas network for vector quantization and its application to time-series prediction. IEEE Trans. Neural Netw. 4, 558–560 (1993)

    Article  Google Scholar 

  94. Massey, F.J.: The Kolmogorov–Smirnov test for goodness of fit. J. Am. Stat. Assoc. 46(253), 68–78 (1951)

    Article  MATH  Google Scholar 

  95. Miettinen, K.: Nonlinear Multiobjective Optimization, International Series in Operations Research & Management Science, vol. 12. Kluwer, Norwell, MA (1999)

  96. Mühlenbein, H., Mahnig, T.: FDA—a scalable evolutionary algorithm for the optimization of additively decomposed functions. Evolut. Comput. 7(4), 353–376 (1999)

    Article  Google Scholar 

  97. Ocenasek, J.: Parallel Estimation of Distribution Algorithms. PhD thesis, Faculty of Information Technology, Brno University of Technology, Brno, Czech Republic (2002)

  98. Ocenasek, J., Schwarz, J.: Estimation of distribution algorithm for mixed continuous-discrete optimization problems. In: 2nd Euro-International Symposium on Computational Intelligence, pp. 227–232 (2002)

  99. Pareto, V.: Cours D’Économie Politique. F. Rouge, Lausanne(1896)

  100. Parzen, E.: On estimation of a probability density function and mode. Ann. Math. Stat. 33, 1065–1076 (1962)

    Article  MathSciNet  MATH  Google Scholar 

  101. Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, San Francisco (1988)

    MATH  Google Scholar 

  102. Pelikan, M.: Hierarchical Bayesian Optimization Algorithm. Toward a New Generation of Evolutionary Algorithms. Studies in Fuzziness and Soft Computing, Springer (2005)

  103. Pelikan, M., Goldberg, D.E.: Hierarchical bayesian optimizationalgorithm. In: Pelikan, M., Sastry, K., Cantú-Paz, E. (eds.) Scalable Optimization via Probabilistic Modeling: From Algorithms to Applications, Studies in Computational Intelligence, Springer–Verlag, pp. 63–90 (2006)

  104. Pelikan, M., Goldberg, D.E., Cantú-Paz, E.: BOA: The Bayesian optimization algorithm. In: Banzhaf, W., Daida, J., Eiben, A.E., Garzon, M.H., Honavar, V., Jakiela, M., Smith, R.E. (eds.) Proceedings of the Genetic and Evolutionary Computation Conference GECCO-1999, Morgan Kaufmann Publishers, San Francisco, CA, Orlando, FL, vol. I, pp. 525–532 (1999)

  105. Pelikan, M., Goldberg, D.E., Lobo, F. A survey of optimization by building and using probabilistic models. IlliGAL Report No. 99018, University of Illinois at Urbana-Champaign, Illinois Genetic Algorithms Laboratory, Urbana, IL (1999)

  106. Pelikan, M., Goldberg, D.E., Cantú-Paz, E.: Hierarchical problem solving by the Bayesian optimization algorithm. IlliGAL Report No. 2000002, University of Illinois at Urbana-Champaign, Illinois Genetic Algorithms Laboratory, Urbana, IL (2000)

  107. Pelikan, M., Sastry, K., Goldberg, D.E.: Multiobjective hBOA, clustering, and scalability. In: GECCO’05: Proceedings of the 2005 Conference on Genetic and Evolutionary Computation, ACM Press, New York, NY, USA, pp. 663–670 (2005). doi:10.1145/1068009.1068122

  108. Pelikan, M., Sastry, K., Cantú-Paz, E. (eds.) Scalable Optimization via Probabilistic Modeling: From Algorithms to Applications. Studies in Computational Intelligence, Springer (2006)

  109. Pelikan, M., Sastry, K., Goldberg, D.E. Multiobjective estimation ofdistribution algorithms. In: Pelikan, M., Sastry, K., Cantú-Paz, E. (eds.) Scalable Optimization via Probabilistic Modeling: From Algorithms to Applications, Studies in Computational Intelligence, Springer–Verlag, pp. 223–248 (2006)

  110. Praditwong, K., Yao, X.: How well do multi-objective evolutionary algorithms scale to large problems. In: 2007 IEEE Congress on Evolutionary Computation (CEC 2007), IEEE Press, Piscataway, New Jersey, pp. 3959–3966 (2007). doi:10.1109/CEC.2007.4424987

  111. Purshouse, R.C.: On the Evolutionary Optimisation of Many Objectives. PhD thesis, Department of Automatic Control and Systems Engineering, The University of Sheffield, Sheffield, UK (2003)

  112. Purshouse, R.C., Fleming, P.J.: Evolutionary multi-objective optimisation: an exploratory analysis. In: Proceedings of the 2003 Congress on Evolutionary Computation (CEC’2003), IEEE Press, Canberra, Australia, vol. 3, pp. 2066–2073 (2003)

  113. Purshouse, R.C., Fleming, P.J.: On the evolutionary optimization of many conflicting objectives. IEEE Trans. Evolut. Comput. 11(6), 770–784 (2007). doi:10.1109/TEVC.2007.910138

    Article  Google Scholar 

  114. Qin, A.K., Suganthan, P.N.: Robust growing neural gas algorithm with application in cluster analysis. Neural Netw. 17(8–9), 1135–1148 (2004). doi:10.1016/j.neunet.2004.06.013

    Article  MATH  Google Scholar 

  115. Rubinstein, R.Y.: Simulation and the Monte Carlo Method. Wiley, New York (1981)

    Book  MATH  Google Scholar 

  116. Schaffer, J.D.: Multiple objective optimization with vector evaluated genetic algorithms. In: Genetic Algorithms and their Applications: Proceedings of the First International Conference on Genetic Algorithms, Lawrence Erlbaum, pp. 93–100 (1985)

  117. Schervish, M.J.: Theory of Statistics, 2nd edn. Springer Series in Statistics, Springer, Berlin/Heidelberg (1997)

  118. Schütze, O., Mostaghim, S., Dellnitz, M., Teich, J.: Covering Pareto sets by multilevel evolutionary subdivision techniques. In: Fonseca, C.M., Fleming, P.J., Zitzler, E., Deb, K., Thiele, L. (eds.) Evolutionary Multi-Criterion Optimization. Second International Conference, EMO 2003, Springer. Lecture Notes in Computer Science, vol. 2632, Faro, Portugal, pp. 118–132 (2003)

  119. Schwarz, G.: Estimating the dimension of a model. Ann. Stat. 6(2), 461–464 (1978)

    Article  MathSciNet  MATH  Google Scholar 

  120. Shapiro, J.: Diversity loss in general estimation of distribution algorithms. In: Parallel Problem Solving from Nature—PPSN IX, pp. 92–101 (2006). doi:10.1007/11844297_10

  121. Srinivas, N., Deb, K.: Multiobjective optimization using nondominated sorting in genetic algorithms. Evolut. Comput. 2(3), 221–248 (1994)

    Article  Google Scholar 

  122. Stewart, T.J., Bandte, O., Braun, H., Chakraborti, N., Ehrgott, M., Göbelt, M., Jin, Y., Nakayama, H., Poles, S., Di Stefano. D.: Real-world applications of multiobjective optimization. In: Branke, J., Miettinen, K., Deb, K., Słowiǹski, R. (eds.) Multiobjective Optimization, Lecture Notes in Computer Science, vol. 5252, Springer–Verlag, Berlin/Heidelberg, pp. 285–327 (2008)

  123. Thierens, D.: Convergence time analysis for the multi-objective counting ones problem. In: Fonseca, C.M., Fleming, P.J., Zitzler, E., Deb, K., Thiele, L. (eds.) Evolutionary Multi-Criterion Optimization. Second International Conference, EMO 2003, Springer. Lecture Notes in Computer Science, vol. 2632, Faro, Portugal, pp. 355–364 (2003)

  124. Thierens, D., Bosman, P.A.N.: Multi-objective mixture-based iterated density estimation evolutionary algorithms. In: Spector, L., Goodman, E., Wu, A., Langdon, W., Voigt, H., Gen, M., Sen, S., Dorigo, M., Pezeshk, S., Garzon, M., Burke, E. (eds.) Proceedings of the Genetic and Evolutionary Computation Conference GECCO-2001, Morgan Kaufmann Publishers, San Francisco, CA, pp. 663–670 (2001)

  125. Thierens, D., Bosman, P.A.N.: Multi-objective optimization with iterated density estimation evolutionary algorithms using mixture models. In: Proceedings of the Third International Symposium on Adaptive Systems-Evolutionary Computation and Probabilistic Graphical Models, Institute of Cybernetics, Mathematics and Physics, Havana, Cuba, pp. 129–136 (2001)

  126. Timm, H., Borgelt, C., Doring, C., Kruse, R.: An extension to possibilistic fuzzy cluster analysis. Fuzzy Sets Syst. 147(1), 3–16 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  127. Vapnik, V.N.: An overview of statistical learning theory. IEEE Trans. Neural Netw. 10(5), 988–999 (1999). doi:10.1109/72.788640

    Article  Google Scholar 

  128. Wang, H., Zhang, Q., Jiao, L., Yao, X.: Regularity model for noisy multiobjective optimization. IEEE Trans. Cybern. PP(99), pp. 1–1 (2015). doi:10.1109/TCYB.2015.2459137

  129. While, L., Bradstreet, L., Barone, L., Hingston, P.: Heuristics for optimising the calculation of hypervolume for multi-objective optimization problems. In: 2005 IEEE Congress on Evolutionary Computation (CEC’2005), IEEE Service Center, Edinburgh, Scotland, vol. 3, pp. 2225–2232 (2005)

  130. While, L., Hingston, P., Barone, L., Huband, S.: A faster algorithm for calculating hypervolume. IEEE Trans. Evolut. Comput. 10(1), 29–38 (2006). doi:10.1109/TEVC.2005.851275

    Article  Google Scholar 

  131. Wilcoxon, F.: Individual comparisons by ranking methods. Biom. Bull. 1, 80–83 (1945)

    Article  Google Scholar 

  132. Xu, R., Wunsch II, D.: Survey of clustering algorithms. IEEE Trans. Neural Netw. 16(3), 645–678 (2005)

    Article  Google Scholar 

  133. Xu, R., Wunsch II, D.: Clustering, Illustrated edn. IEEE Press Series on Computational Intelligence, Wiley, IEEE Press, New York (2008)

  134. Yuan, B., Gallagher, M.: On the importance of diversity maintenance in estimation of distribution algorithms. In: GECCO’05: Proceedings of the 2005 Conference on Genetic and Evolutionary Computation, ACM Press, New York, NY, USA, pp. 719–726 (2005). doi:10.1145/1068009.1068129

  135. Zhang, Q., Zhou, A., Jin, Y.: RM-MEDA: a regularity model-based multiobjective estimation of distribution algorithm. IEEE Trans. Evolut. Comput. 12(1), 41–63 (2008). doi:10.1109/TEVC.2007.894202

    Article  Google Scholar 

  136. Zhou, A., Zhang, Q., Jin, Y., Tsang, E., Okabe, T.: A model-based evolutionary algorithm for bi-objective optimization. In: 2005 IEEE Congress on Evolutionary Computation (CEC’2005), IEEE Service Center, Edinburgh, Scotland, vol. 3, pp. 2568–2575 (2005)

  137. Zitzler, E., Künzli, S.: Indicator-based selection in multiobjective search. In: Yao, X. (ed.) Parallel Problem Solving from Nature—PPSN VIII, Springer-Verlag, Berlin/Heidelberg, Lecture Notes in Computer Science, vol. 3242, pp. 832–842 (2004)

  138. Zitzler, E., Thiele, L.: An Evolutionary Algorithm for Multiobjective Optimization: The Strength Pareto Approach. Tech. Rep. 43, Computer Engineering and Communication Networks Lab (TIK), Swiss Federal Institute of Technology (ETH), Zurich, Switzerland (1998)

  139. Zitzler, E., Deb, K., Thiele, L.: Comparison of multiobjective evolutionary algorithms on test functions of different difficulty. In: Wu, A.S. (ed.) Proceedings of the 1999 Genetic and Evolutionary Computation Conference. Workshop Program, Orlando, Florida, pp. 121–122 (1999)

  140. Zitzler, E., Laumanns, M., Thiele, L.: SPEA2: improving the strength Pareto evolutionary algorithm. In: Giannakoglou, K., Tsahalis, D., Periaux, J., Papailou, P., Fogarty, T. (eds.) EUROGEN 2001. Evolutionary Methods for Design, Optimization and Control with Applications to Industrial Problems, Athens, Greece, pp. 95–100(2002)

  141. Zitzler, E., Laumanns, M., Thiele, L., Fonseca, C.M., Grunert da Fonseca, V.: Why quality assessment of multiobjective optimizers is difficult. In: Langdon, W.B., Cantú-Paz, E., Mathias, K., Roy, R., Davis, D., Poli, R., Balakrishnan, K., Honavar, V., Rudolph, G., Wegener, J., Bull, L., Potter, M., Schultz, A., Miller, J., Burke, E., Jonoska, N. (eds.) Proceedings of the Genetic and Evolutionary Computation Conference (GECCO’2002), Morgan Kaufmann Publishers, San Francisco, California, pp. 666–673 (2002)

  142. Zitzler, E., Thiele, L., Laumanns, M., Fonseca, C.M., Grunert da Fonseca, V.: Performance assessment of multiobjective optimizers: an analysis and review. IEEE Trans. Evolut. Comput. 7(2), 117–132 (2003)

    Article  Google Scholar 

  143. Zitzler, E., Brockhoff, D., Thiele, L.: The hypervolume indicator revisited: on the design of pareto-compliant indicators via weighted integration. In: Obayashi, S., et al. (eds.) Conference on Evolutionary Multi-Criterion Optimization (EMO 2007), Springer, Berlin, LNCS, vol. 4403, pp. 862–876 (2007)

  144. Zitzler, E., Knowles, J., Thiele, L.: Quality assessment of pareto set approximations. In: Branke, J., Miettinen, K., Deb, K., Słowiǹski, R. (eds.) Multiobjective Optimization, Lecture Notes in Computer Science, vol. 5252, Springer–Verlag, Berlin/Heidelberg, pp. 373–404 (2008)

Download references

Acknowledgments

This work has been funded in part by projects CNPq BJT 407851/2012-7, FAPERJ APQ1 211.451/2015, MINECO TEC2014-57022-C2-2-R and TEC2012-37832-C02-01.

Author information

Authors and Affiliations

  1. Institute of Computing, Universidade Federal Flumense, Av. Gal. Milton Tavares de Souza, Niterói, Rio de Janerio, 24210-346, Brazil

    Luis Martí

  2. Group of Applied Artificial Intelligence, Department of Informatics, Universidad Carlos III de Madrid, Av. de la Universidad Carlos III, 22, Colmenarejo, 28270, Madrid, Spain

    Jesús García, Antonio Berlanga & José M. Molina

Authors
  1. Luis Martí
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jesús García
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Antonio Berlanga
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. José M. Molina
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Luis Martí.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Martí, L., García, J., Berlanga, A. et al. MONEDA: scalable multi-objective optimization with a neural network-based estimation of distribution algorithm. J Glob Optim 66, 729–768 (2016). https://doi.org/10.1007/s10898-016-0415-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10898-016-0415-7

Keywords

Search

Navigation