Advertisement

Novelty Search and the Problem with Objectives

  • Joel Lehman
  • Kenneth O. Stanley
Chapter
Part of the Genetic and Evolutionary Computation book series (GEVO)

Abstract

By synthesizing a growing body ofwork in search processes that are not driven by explicit objectives, this paper advances the hypothesis that there is a fundamental problem with the dominant paradigm of objective-based search in evolutionary computation and genetic programming: Most ambitious objectives do not illuminate a path to themselves. That is, the gradient of improvement induced by ambitious objectives tends to lead not to the objective itself but instead to deadend local optima. Indirectly supporting this hypothesis, great discoveries often are not the result of objective-driven search. For example, the major inspiration for both evolutionary computation and genetic programming, natural evolution, innovates through an open-ended process that lacks a final objective. Similarly, large-scale cultural evolutionary processes, such as the evolution of technology, mathematics, and art, lack a unified fixed goal. In addition, direct evidence for this hypothesis is presented from a recently-introduced search algorithm called novelty search. Though ignorant of the ultimate objective of search, in many instances novelty search has counter-intuitively outperformed searching directly for the objective, including a wide variety of randomly-generated problems introduced in an experiment in this chapter. Thus a new understanding is beginning to emerge that suggests that searching for a fixed objective, which is the reigning paradigm in evolutionary computation and even machine learning as a whole, may ultimately limit what can be achieved. Yet the liberating implication of this hypothesis argued in this paper is that by embracing search processes that are not driven by explicit objectives, the breadth and depth of what is reachable through evolutionary methods such as genetic programming may be greatly expanded.

Keywords

Novelty search objective-based search non-objective search deception evolutionary computation 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Barnett, Lionel (2001). Netcrawling - optimal evolutionary search with neutral networks. In Proc. of the 2001 IEEE Intl. Conf. on Evol. Comp., pages 30–37. IEEE Press.Google Scholar
  2. Brockhoff, Dimo, Friedrich, Tobias, Hebbinghaus, Nils, Klein, Christian, Neumann, Frank, and Zitzler, Eckart (2007). Do additional objectives make a problem harder? In GECCO ’07: Proc. of the 9th Annual Conf. on Genetic and Evol. Comp., pages 765–772, New York, NY, USA. ACM.Google Scholar
  3. Cliff, Dave andMiller, Geoffrey (1995). Tracking the red queen:Measurements of adaptive progress in co-evolutionary simulations. Advances in Artificial Life, pages 200–218.Google Scholar
  4. Deb, Kalyanmoy (1999). Multi-objective genetic algorithms: Problem difficulties and construction of test problems. Evol. Comp., 7:205–230.CrossRefGoogle Scholar
  5. Doucette, John (2010).Novelty-based fitnessmeasures in genetic programming. Master of science in computer science, Dalhouise University.Google Scholar
  6. Drexler, K.E. and Minsky, M. (1986). Engines of creation. Anchor Press.Google Scholar
  7. Ficici, Sevan and Pollack, Jordan B. (1998). Challenges in coevolutionary learning: Arms-race dynamics, open-endedness, and mediocre stable states. In Proc. of the Sixth Intl. Conf. on Art. Life, pages 238–247. MIT Press.Google Scholar
  8. Goldberg, David E. (1987). Simple genetic algorithms and the minimal deceptive problem. In Davis, L. D., editor, Genetic Algorithms and SimulatedAnnealing, Re- search Notes in Artificial Intelligence. Morgan Kaufmann.Google Scholar
  9. Goldsby, H.J. and Cheng, B.H.C. (2010). Automatically Discovering Properties that Specify the Latent Behavior of UML Models. In Proceedings of MODELS 2010.Google Scholar
  10. Gomez, Faustino and Miikkulainen, Risto (1997). Incremental evolution of complex general behavior. Adaptive Behavior, 5:317–342.CrossRefGoogle Scholar
  11. Gould, Steven Jay (1996). Full House: The Spread of Excellence from Plato to Darwin. Harmony Books.Google Scholar
  12. Holland, John H. (1975). Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control and Artificial Intelligence. University of Michigan Press, Ann Arbor, MI.Google Scholar
  13. Kelly, K. (2010). What technology wants. Viking Press.Google Scholar
  14. Koza, John R., Keane, Martin A., Streeter, Matthew J., Mydlowec, William, Yu, Jessen, and Lanza, Guido (2003). Genetic Programming IV: Routine Human-Competitive Machine Intelligence. Kluwer Academic Publishers.Google Scholar
  15. Lehman, Joel and Stanley, Kenneth O. (2008). Exploiting open-endedness to solve problems through the search for novelty. In Proc. of the Eleventh Intl. Conf. on Artificial Life (ALIFE XI), Cambridge, MA. MIT Press.Google Scholar
  16. Lehman, Joel and Stanley, Kenneth O. (2010a). Efficiently evolving programs through the search for novelty. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2010). ACM.Google Scholar
  17. Lehman, Joel and Stanley, Kenneth O. (2010b). Revising the evolutionary computationGoogle Scholar
  18. abstraction: Minimal criteria novelty search. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2010). ACM.Google Scholar
  19. Lehman, Joel and Stanley, Kenneth O. (2011). Abandoning objectives: Evolution through the search for novelty alone. Evol. Comp., 19(2):189–223.CrossRefGoogle Scholar
  20. Lynch, Michael (2007). The frailty of adaptive hypotheses for the origins of organismal complexity. In Proc Natl Acad SciUSA, volume104, pages 8597– 8604.Google Scholar
  21. Mahfoud, SamirW. (1995). Nichingmethods for genetic algorithms. PhD thesis, University of Illinois at Urbana-Champaign, Champaign, IL, USA.Google Scholar
  22. Mill, John Stuart (1846). A System of Logic, Ratiocinative and Inductive. John W. Parker and Son.Google Scholar
  23. Mouret, Jean-Baptiste (2009). Novelty-based multiobjectivization. In Proc. of the Workshop on Exploring New Horizons in Evol. Design of Robots,2009 IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems.Google Scholar
  24. Pelikan, Martin, Pelikan, Martin, Goldberg, David E., and Goldberg, David E. (2001). Escaping hierarchical traps with competent genetic algorithms. In Proc. of the Genetic and Evolutionary Computation Conference (GECCO- 2001), pages 511–518. Morgan Kaufmann.Google Scholar
  25. Reil, Torsten and Husbands, Phil (2002). Evolution of central pattern generators for bipedal walking in a real-time physics environment. IEEE Transactions on Evolutionary Computation, 6(2):159–168.CrossRefGoogle Scholar
  26. Reynolds, AM(2010). Maze-solving by chemotaxis. Physical Review E, 81(6).Google Scholar
  27. Risi, S., Hughes, C.E., and Stanley, K.O. (2010). Evolving plastic neural networks with novelty search. Adaptive Behavior.Google Scholar
  28. Schmidhuber, J. (2006). Developmental robotics, optimal artificial curiosity, creativity, music, and the fine arts. Connection Science, 18(2):173–187.CrossRefGoogle Scholar
  29. Secretan, J.,Beato,N.,D’Ambrosio,D.B.,Rodriguez,A.,Campbell,A., Folsom-Google Scholar
  30. Kovarik, J.T., and Stanley, K.O. (2011). Picbreeder: A case study in collaborativeGoogle Scholar
  31. evolutionary exploration of design space. Evol. Comp. To appear.Google Scholar
  32. Spector,Lee,Barnum,Howard,Bernstein,Herbert J., and Swamy,Nikhil (1999). Quantum computing applications of genetic programming. In Spector, Lee, Langdon, William B., O’Reilly, Una-May, and Angeline, Peter J., editors, Advances in Genetic Programming 3, chapter 7, pages 135–160. MIT Press, Cambridge, MA, USA.Google Scholar
  33. Stanley, Kenneth O. andMiikkulainen, Risto (2002). Evolving neural networks through augmenting topologies. Evolutionary Computation, 10:99–127.CrossRefGoogle Scholar
  34. Stewart, T. C. (2001). Extrema selection: Accelerated evolution on neutral networks. In Proc. of the 2001 IEEE Intl. Conf. on Evol. Comp. IEEE Press.Google Scholar
  35. Veldhuizen, David A. Van and Lamont, Gary B. (2000). Multiobjective evolutionary algorithms: Analyzing the state-of-the-art. Evolutionary Computation, 8(2):125–147.CrossRefGoogle Scholar
  36. Weinberger, Edward (1990). Correlated and uncorrelated fitness landscapes and how to tell the difference. Biological Cybernetics, 63(5):325–336.zbMATHCrossRefGoogle Scholar
  37. Wolpert, David H. and Macready, William (1995). No free lunch theorems for search. Technical Report SFI-TR-95-01-010, The Santa Fe Institute, Santa Fe, NM.Google Scholar
  38. Woolley, Brian G. and Stanley, Kenneth O. (2011). On the deleterious effectsGoogle Scholar
  39. of a priori objectives on evolution and representation. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2011). ACM. Yao, Xin (1999). Evolving artificial neural networks. Proceedings of the IEEE, 87(9):1423–1447.CrossRefGoogle Scholar
  40. Zaera, N., Cliff, D., and Bruten, J. (1996). (Not) evolving collective behaviours in synthetic fish. In From Animals to Animats 4: Proc. of the Fourth Intl. Conf. on Simulation of Adaptive Behavior. MIT Press Bradford Books.Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  • Joel Lehman
    • 1
  • Kenneth O. Stanley
    • 1
  1. 1.Department of EECSUniversity of Central FloridaOrlandoUSA

Personalised recommendations