Skip to main content
SpringerLink
Log in

Phenotypic Niching Using Quality Diversity Algorithms

  • Chapter
  • First Online:
Metaheuristics for Finding Multiple Solutions

Part of the book series: Natural Computing Series ((NCS))

Abstract

Here we describe quality diversity algorithms, a recent and powerful class of evolutionary algorithms that produces a diverse set of high-performing solutions. The optimization paradigm emphasizes phenotypic niching and egalitarian treatment of quality and diversity. We ground quality diversity in ecology, describe the historical development, and give an intuition and formalization of the algorithms. We present a practical example that we refer to for engineers and laymen readers to understand how and why quality diversity can be used. The main insights from research of quality diversity, performance metrics, and benchmarks are discussed. Finally, the open challenges are presented.

An animal’s behaviour tends to maximize the survival of the genes “for” that behaviour, whether or not those genes happen to be in the body of the particular animal performing it.

- Richard Dawkins [13]

There is a power and utility to regarding the gene as the unit of selection, but equally there is value to seeing the organism as the unit of niche construction.

- Kevin Laland [46]

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 139.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 179.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 179.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Popular literature about computational evolution often use the active tense when talking about evolution, as though it is a driving force rather than an emergent property of life. Even natural selection could be seen as a misnomer, as selection implies an entity acting upon the world, whereas selection seems to be an emergent property of complex interactions in nature.

  2. 2.

    Laland takes Dawkins’ idea one step further and introduces the idea of niche construction. By influencing its environment, a creature can create its own niche. This introduces the idea of evolution being a causally cyclical process. A creature creates a niche by acting on its environment, its genome adapts to the niche, which in itself causes changes in the environment, and so forth [46].

  3. 3.

    As in diversity of qualities. Both quality diversity as well as illumination are used in the field, although one could view the deeper concept to be phenotypic niching.

  4. 4.

    The field of evolutionary algorithms does not always make a distinction between genetic or phenotypic niching. Confusingly, after Deb mentions the biological definition of species and niches, he defines niches as being artificial subpopulations and niching as a method to force population diversity [17]. There is a case to be made to use more rigor in the definition of niches and species. Niches are basins of attraction in the objective function in phenotypic space. Species are phenotypic solutions that tend to fill certain niches. The genomes of species tend to be similar or at least compatible. One could then argue that, especially when the phenotype is indirectly encoded in the genome, niching has to take place based on phenotypic characteristics. Speciation still takes place on a genetic level, to ensure compatibility between genes. Placing speciation on this level is compatible with Dawkins’ understanding of genes as a primary evolutionary unit [12].

  5. 5.

    One can also argue that the behavior of a neural robot controller in a particular maze is the extended phenotype’s embedding in its environment. When a controller was evolved to act in a specific environment, it is not of interest to describe its behavior in other environments.

  6. 6.

    Igel and Toussaint analyzed the effects of neutral encodings on computational evolution and showed they are necessary for self-adaptation while only marginally increasing the number of necessary function evaluations [42].

  7. 7.

    Classes translate to species as defined in [3].

References

  1. Alonso, D., Etienne, R.S., McKane, A.J.: The merits of neutral theory. Trends Ecol. Evol. 21(8), 451–457 (2006)

    Article  Google Scholar 

  2. Arulkumaran, K., Cully, A., Togelius, J.: Alphastar: an evolutionary computation perspective. In: Proceedings of the Genetic and Evolutionary Computation Conference Companion, pp. 314–315 (2019)

    Google Scholar 

  3. Basto-Fernandes, V., Yevseyeva, I., Deutz, A., Emmerich, M.: A survey of diversity oriented optimization: problems, indicators, and algorithms. In: EVOLVE–A Bridge Between Probability, Set Oriented Numerics and Evolutionary Computation, vol. 7, pp. 3–23. Springer (2017)

    Google Scholar 

  4. Basto-Fernandes, V., Yevseyeva, I., Emmerich, M.: A survey of diversity-oriented optimization. EVOLVE 2013-A Bridge Probab. Set Oriented Numer. Evol. Comput. 1(2013), 101–109 (2013)

    Google Scholar 

  5. Cully, A.: Autonomous skill discovery with quality-diversity and unsupervised descriptors. In: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 81–89 (2019)

    Google Scholar 

  6. Cully, A., Clune, J., Tarapore, D., Mouret, J.-B.: Robots that can adapt like animals. Nature 521(7553), 503–507 (2015)

    Article  Google Scholar 

  7. Cully, A., Demiris, Y.: Quality and diversity optimization: a unifying modular framework. In: IEEE Transactions on Evolutionary Computation, pp. 1–15 (2017)

    Google Scholar 

  8. Cully, A., Demiris, Y.: Hierarchical behavioral repertoires with unsupervised descriptors. Presented at the (2018)

    Google Scholar 

  9. Cully, A., Mouret, J.-B.: Learning to walk in every direction. Evol. Comput. 24(1), 59–88 (2013)

    Article  Google Scholar 

  10. Cully, A., Mouret, J.-B.: Evolving a behavioral repertoire for a walking robot. Evol. Comput. 24(1), 59–88 (2016)

    Article  Google Scholar 

  11. Cully, A., Pierre, U., Mouret, J.-B.: Behavioral repertoire learning in robotics. In: Proceeding of the Fifteenth Annual Conference on Genetic and Evolutionary Computation Conference - GECCO ’13, pp. 175–182 (2013)

    Google Scholar 

  12. Dawkins, R.: The Selfish Gene. Oxford University Press, Oxford (1976)

    Google Scholar 

  13. Dawkins, R.: The Extended Phenotype, vol. 8. Oxford University Press, Oxford (1982)

    Google Scholar 

  14. De Jong, K.A.: An analysis of the behavior of a class of genetic adaptive systems (1975)

    Google Scholar 

  15. Deb, K., Agrawal, S., Pratap, A., Meyarivan, T.: A fast elitist non-dominated sorting genetic algorithm for multi-objective optimization: Nsga-ii, pp. 849–858. Springer (2000)

    Google Scholar 

  16. Deb, K., Goldberg, D.E.: An Investigation of Niche and Species Formation in Genetic Function Optimization (1989)

    Google Scholar 

  17. Deb, K., Spears, W.: Speciation methods. Evol. Comput. 2, 93–100 (2010)

    Google Scholar 

  18. Deb, K., Srinivasan, A.: Innovization: innovating design principles through optimization. Presented at the (2006)

    Google Scholar 

  19. Deb, K., Tiwari, S.: Omni-optimizer: a generic evolutionary algorithm for single and multi-objective optimization. Eur. J. Oper. Res. 185(3), 1062–1087 (2008)

    Article  MathSciNet  Google Scholar 

  20. Doncieux, S., Coninx, A.: Open-ended evolution with multi-containers QD. In: Proceedings of the Genetic and Evolutionary Computation Conference Companion, pp. 107–108 (2018)

    Google Scholar 

  21. Duarte, M., Gomes, J., Oliveira, S.M., Christensen, A.L.: Evorbc: evolutionary repertoire-based control for robots with arbitrary locomotion complexity. In: Proceedings of the Genetic and Evolutionary Computation Conference 2016, pp. 93–100 (2016)

    Google Scholar 

  22. Ecoffet, A., Huizinga, J., Lehman, J., Stanley, K.O., Clune, J.: Go-explore: a new approach for hard-exploration problems (2019). arXiv:1901.10995

  23. Elman, J.L.: Finding structure in time. Cogn. Sci. 14(1990), 179–211 (1990)

    Article  Google Scholar 

  24. Gaier, A., Asteroth, A., Mouret, J.-B.: Aerodynamic design exploration through surrogate-assisted illumination. In: 18th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, AIAA AVIATION Forum, (AIAA 2017-3330) (2017)

    Google Scholar 

  25. Gaier, A., Asteroth, A., Mouret, J.-B.: Data-efficient exploration, optimization, and modeling of diverse designs through surrogate-assisted illumination. In: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 9–106 (2017)

    Google Scholar 

  26. Gaier, A., Asteroth, A., Mouret, J.-B.: Are quality diversity algorithms better at generating stepping stones than objective-based search? In: Proceedings of the Genetic and Evolutionary Computation Conference Companion, pp. 115–116 (2019)

    Google Scholar 

  27. Goldberg, D.E., Richardson, J.: In: Genetic algorithms with sharing for multimodal function optimization, pp. 41–49. Lawrence Erlbaum, Hillsdale, NJ (1987)

    Google Scholar 

  28. Gomes, J., Lyhne Christensen, A.: Comparing Approaches for Evolving High-level Robot Control based on Behaviour Repertoires. In: 2018 IEEE Congress on Evolutionary Computation (CEC), pp. 1–6 (2018)

    Google Scholar 

  29. Gomes, J., Oliveira, S.M., Christensen, A.L.: An approach to evolve and exploit repertoires of general robot behaviours. In: Swarm and Evolutionary Computation, pp. 265–283 (2018)

    Google Scholar 

  30. Gravina, D., Liapis, A., Yannakakis, G.N.: Surprise search for evolutionary divergence (2017). arXiv:1706.02556

  31. Gravina, D., Liapis, A., Yannakakis, G.N.: Quality diversity through surprise. IEEE Trans. Evol. Comput. PP(c):1 (2018)

    Google Scholar 

  32. Hagg, A., Asteroth, A., Bäck, T.: In: Prototype Discovery Using Quality-diversity, pp. 500–511. Springer, Berlin (2018)

    Google Scholar 

  33. Hagg, A., Asteroth, A., Bäck, T.: Modeling user selection in quality diversity. In: Proceedings of the 2019 on Genetic and Evolutionary Computation Conference - GECCO 2019 (2019)

    Google Scholar 

  34. Hagg, A., Asteroth, A., Thomas, B.: A deep dive into exploring the preference hypervolume. In: ICCC (2020)

    Google Scholar 

  35. Hagg, A., Preuss, M., Asteroth, A., Bäck, T.: An analysis of phenotypic diversity in multi-solution optimization. In: BIOMA 2020 (2020)

    Google Scholar 

  36. Hagg, A., Wilde, D., Asteroth, A., Bäck, T.: Designing air flow with surrogate-assisted phenotypic niching (2020)

    Google Scholar 

  37. Harik, G.R.: Finding multimodal solutions using restricted tournament selection. In: ICGA, pp. 24–31 (1995)

    Google Scholar 

  38. Hart, E., Steyven, A.S.W., Paechter, B.: Evolution of a functionally diverse swarm via a novel decentralised quality-diversity algorithm. In: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 101–108 (2018)

    Google Scholar 

  39. Holland, J.H.: Adaptation in Natural and Artificial Systems. MIT press (1975)

    Google Scholar 

  40. Howard, D., Eiben, A.E., Kennedy, D.F., Mouret, J.-B., Valencia, P., Winkler, D.: Evolving embodied intelligence from materials to machines. Nat. Mach. Intell. 1(1), 12–19 (2019)

    Article  Google Scholar 

  41. Huizinga, J., Clune, J.: Evolving multimodal robot behavior via many stepping stones with the combinatorial multi-objective evolutionary algorithm (2018). arXiv:1807.03392

  42. Igel, C., Toussaint, M.: Neutrality and self-adaptation. Nat. Comput. 2(2), 117–132 (2003)

    Article  MathSciNet  Google Scholar 

  43. Jegorova, M., Doncieux, S., Hospedales, T.: Generative adversarial policy networks for behavioural repertoire (2018). arXiv:1811.02945

  44. Kim, S., Doncieux, S.: Learning highly diverse robot throwing movements through quality diversity search. In: Proceedings of the Genetic and Evolutionary Computation Conference Companion, pp. 1177–1178 (2017)

    Google Scholar 

  45. Koos, S., Mouret, J.-B., Doncieux, S.: The transferability approach: crossing the reality gap in evolutionary robotics. In: IEEE Transactions on Evolutionary Computation, pp. 1–25 (2012)

    Google Scholar 

  46. Laland, K.N.: Extending the extended phenotype. Biol. Philos. 19(3), 313–325 (2004)

    Article  Google Scholar 

  47. Lehman, J., Miikkulainen, R.: Enhancing divergent search through extinction events. Presented at the (2015)

    Google Scholar 

  48. Lehman, J., Stanley, K.O.: Exploiting open-endedness to solve problems through the search for novelty. In: Alife, pp. 329–336 (2008)

    Google Scholar 

  49. Lehman, J., Stanley, K.O.: Abandoning objectives: evolution through the search for novelty alone. Evol. Comput. 19(2), 189–222 (2011)

    Article  Google Scholar 

  50. Lehman, J., Stanley, K.O.: Evolving a diversity of virtual creatures through novelty search and local competition. In: Proceedings of the 13th Annual Conference on Genetic and Evolutionary Computation, pp. 211–218 (2011)

    Google Scholar 

  51. Meyerson, E., Miikkulainen, R.: Discovering evolutionary stepping stones through behavior domination. In: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 139–146 (2017)

    Google Scholar 

  52. Mlot, N.J., Tovey, C.A., David, L.H.: Fire ants self-assemble into waterproof rafts to survive floods. Proc. Natl. Acad. Sci. 108(19), 7669–7673 (2011)

    Article  Google Scholar 

  53. Mouret, J.-B.: Novelty-based multiobjectivization. In: New Horizons in Evolutionary Robotics, pp. 139–154. Springer (2011)

    Google Scholar 

  54. Mouret, J.-B., Clune, J.: An algorithm to create phenotype-fitness maps. Proc. Artif. Life Conf. 375(2012), 593–594 (2012)

    Google Scholar 

  55. Mouret, J.-B., Clune, J.: Illuminating search spaces by mapping elites (2015). arXiv:1504.04909

  56. Mouret, J.-B., Doncieux, S.: Encouraging behavioral diversity in evolutionary robotics: an empirical study. Evol. Comput. 20(1), 91–133 (2012)

    Article  Google Scholar 

  57. Nguyen, A., Yosinski, J., Clune, J.: Deep neural networks are easily fooled: high confidence predictions for unrecognizable images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 427–436 (2015)

    Google Scholar 

  58. Nguyen, A., Yosinski, J., Clune, J.: Innovation engines: automated creativity and improved stochastic optimization via deep learning. In: Proceedings of the 2015 on Genetic and Evolutionary Computation Conference - GECCO ’15, pp. 959–966 (2015)

    Google Scholar 

  59. Nguyen, A., Yosinski, J., Clune, J.: Understanding innovation engines: automated creativity and improved stochastic optimization via deep learning. Evol. Comput. 24(3), 545–572 (2016)

    Article  Google Scholar 

  60. Nordmoen, J., Samuelsen, E., Ellefsen, K.O., Glette, K.: Dynamic Mutation in Map-Elites for Robotic Repertoire Generation, pp. 598–605. MIT Press (2018)

    Google Scholar 

  61. Pétrowski, A.: In: A clearing procedure as a niching method for genetic algorithms, pp. 798–803. IEEE (1996)

    Google Scholar 

  62. Preuss, M.: Niching prospects. In: Proceedings of Bioinspired Optimization Methods and their Applications (BIOMA 2006), pp. 25–34 (2006)

    Google Scholar 

  63. Preuss, M.: Niching the cma-es via nearest-better clustering. In: Proceedings of the 12thAnnual Conference Companion on Genetic and Evolutionary Computation, pp. 1711–1718 (2010)

    Google Scholar 

  64. Preuss, M.: Multimodal Optimization by Means of Evolutionary Algorithms. Springer (2015)

    Google Scholar 

  65. Pugh, J.K., Soros, L.B., Stanley, K.O.: An extended study of quality diversity algorithms. In: Proceedings of the 2016 on Genetic and Evolutionary Computation Conference Companion, pp. 19–20 (2016)

    Google Scholar 

  66. Pugh, J.K., Soros, L.B., Stanley, K.O.: Searching for quality diversity when diversity is unaligned with quality. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 9921 LNCS:880–889 (2016)

    Google Scholar 

  67. Pugh, J.K., Soros, L.B., Szerlip, P.A., Stanley, K.O.: Confronting the challenge of quality diversity. In: Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation, pp. 967–974 (2015)

    Google Scholar 

  68. Rastrigin, L.A.: Systems of extremal control. Nauka (1974)

    Google Scholar 

  69. Schaaf, L.J., John Odling-Smee, F., Laland, K.N., Feldman, M.W.: Niche Construction: The Neglected Process in Evolution. Princeton University Press, Princeton (2003)

    Google Scholar 

  70. Shir, O.M., Preuss, M., Naujoks, B., Emmerich, M.: Enhancing decision space diversity in evolutionary multiobjective algorithms. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 5467 LNCS:95–109 (2009)

    Google Scholar 

  71. Sims, K.: Evolution of Virtual Creatures. In: Proceedings of the 21st Annual Conference on Computer Graphics and Interactive Techniques (1994)

    Google Scholar 

  72. Skinner, B.F.: Reinforcement today. Am. Psychol. 13(3), 94 (1958)

    Google Scholar 

  73. Smith, D., Tokarchuk, L., Wiggins, G.: Rapid phenotypic landscape exploration through hierarchical spatial partitioning. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 9921 LNCS:911–920 (2016)

    Google Scholar 

  74. Stanley, K.O., D’Ambrosio, D.B., Gauci, J.: A hypercube-based encoding for evolving large-scale neural networks. Artif. Life 15(2), 185–212 (2009)

    Article  Google Scholar 

  75. Tarapore, D., Clune, J., Cully, A., Mouret, J.-B.: How do different encodings influence the performance of the map-elites algorithm? In: Proceedings of the Genetic and Evolutionary Computation Conference 2016, pp. 173–180 (2016)

    Google Scholar 

  76. Toffolo, A., Benini, E.: Genetic diversity as an objective in multi-objective evolutionary algorithms. Evol. Comput. 11(2), 151–167 (2003)

    Article  Google Scholar 

  77. Ulrich, T.: Integrating decision space diversity into hypervolume-based multiobjective search categories and subject descriptors. In: GECCO 2010, pp. 455–462 (2010)

    Google Scholar 

  78. Urquhart, N., Hart, E.: Optimisation and illumination of a real-world workforce scheduling and routing application (WSRP) via map-elites. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 11101 LNCS:488–499 (2018)

    Google Scholar 

  79. Vassiliades, V., Mouret, J.-B.: Discovering the elite hypervolume by leveraging interspecies correlation. In: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 149–156 (2018)

    Google Scholar 

  80. Vassiliades, V., Chatzilygeroudis, K., Mouret, J.-B.: A comparison of illumination algorithms in unbounded spaces. In: Proceedings of the Genetic and Evolutionary Computation Conference Companion, pp. 1578–1581 (2017)

    Google Scholar 

  81. Vassiliades, V., Chatzilygeroudis, K., Mouret, J.-B.: Using centroidal voronoi tessellations to scale up the multidimensional archive of phenotypic elites algorithm. IEEE Trans. Evol. Comput. 22(4), 623–630 (2017)

    Google Scholar 

  82. Vergnon, R., Dulvy, N.K., Freckleton, R.P.: Niches versus neutrality: uncovering the drivers of diversity in a species-rich community. Ecol. Lett. 12(10), 1079–1090 (2009)

    Article  Google Scholar 

  83. Wessing, S.: Two-stage methods for multimodal optimization. PhD thesis, Universitätsbibliothek Dortmund (2015)

    Google Scholar 

  84. Wright, N.A., Steadman, D.W., Witt, C.C.: Predictable evolution toward flightlessness in volant island birds. Proc. Natl. Acad. Sci. 113(17), 4765–4770 (2016)

    Article  Google Scholar 

Download references

Acknowledgements

I would like to thank Alexander Asteroth, Adam Gaier, and Jörg Stork for their feedback. This work received funding from the German Federal Ministry of Education and Research and the Ministry for Culture and Science of the state of North Rhine-Westfalia (research grants 03FH012PX5 and 13FH156IN6).

Author information

Authors and Affiliations

  1. Leiden Institute of Advanced Computer Science, Leiden, The Netherlands

    Alexander Hagg

  2. Bonn-Rhein-Sieg University of Applied Sciences, Sankt Augustin, Germany

    Alexander Hagg

Authors
  1. Alexander Hagg
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alexander Hagg .

Editor information

Editors and Affiliations

  1. LIACS, Universiteit Leiden, Leiden, The Netherlands

    Mike Preuss

  2. The Signal Group, Athens, Greece

    Michael G. Epitropakis

  3. Computer Science and Software Engineering, RMIT University, Melbourne, VIC, Australia

    Xiaodong Li

  4. Department of Computer Science, University of Exeter, Exeter, UK

    Jonathan E. Fieldsend

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Hagg, A. (2021). Phenotypic Niching Using Quality Diversity Algorithms. In: Preuss, M., Epitropakis, M.G., Li, X., Fieldsend, J.E. (eds) Metaheuristics for Finding Multiple Solutions. Natural Computing Series. Springer, Cham. https://doi.org/10.1007/978-3-030-79553-5_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-79553-5_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-79552-8

  • Online ISBN: 978-3-030-79553-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation