Skip to main content
SpringerLink
Log in

Neuromodulated multiobjective evolutionary neurocontrollers without speciation

  • Research Paper
  • Published:
Evolutionary Intelligence Aims and scope Submit manuscript

Abstract

Neuromodulation is a biologically-inspired technique that can adapt the per-connection learning rates of synaptic plasticity. Neuromodulation has been used to facilitate unsupervised learning by adapting neural network weights. Multiobjective evolution of neural network topology and weights has been used to design neurocontrollers for autonomous robots. This paper presents a novel multiobjective evolutionary neurocontroller with unsupervised learning for robot navigation. Multiobjective evolution of network weights and topologies (NEAT-MODS) is augmented with neuromodulated learning. NEAT-MODS is an NSGA-II based multiobjective neurocontroller that uses two conflicting objectives. The first rewards the robot when it moves in a direct manner with minimal turning; the second objective is to reach as many targets as possible. NEAT-MODS uses speciation, a selection process that aims to ensure Pareto-optimal genotypic diversity and elitism. The effectiveness of the design is demonstrated using a series of experiments with a simulated robot traversing a simple maze containing target goals. It is shown that when neuromodulated learning is combined with multiobjective evolution, better-performing neural controllers are synthesized than by evolution alone. Secondly, it is demonstrated that speciation is unnecessary in neuromodulated neuroevolution, as neuromodulation preserves topological innovation. The proposed neuromodulated approach is found to be statistically superior to NEAT-MODS alone when applied to solve a multiobjective navigation problem.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21

Similar content being viewed by others

References

  1. Abramovich O, Moshaiov A (2016) Multi-objective topology and weight evolution of neuro-controllers. In: 2016 IEEE congress on evolutionary computation (CEC), pp 670–677. https://doi.org/10.1109/CEC.2016.7743857

  2. Deb K (2001) Multi objective optimization using evolutionary algorithms. Wiley, New York

    MATH  Google Scholar 

  3. Deb K, Pratap A, Agarwal S, Meyarivan T (2002) A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans Evol Comput 6(2):182–197. https://doi.org/10.1109/4235.996017

    Article  Google Scholar 

  4. Doya K (2002) Metalearning and neuromodulation. Neural Netw 15(4–6):495–506. https://doi.org/10.1016/S0893-6080(02)00044-8

    Article  Google Scholar 

  5. Floreano D, Dürr P, Mattiussi C (2008) Neuroevolution: from architectures to learning. Evol Intell 1(1):47–62. https://doi.org/10.1007/s12065-007-0002-4

    Article  Google Scholar 

  6. Hebb DO (1949) The organization of behavior, a neuropsychological theory. Wiley, New York

    Google Scholar 

  7. Katz PS, Harris-Warrick RM (1999) The evolution of neuronal circuits underlying species-specific behavior. Curr Opin Neurobiol 9(5):628–633. https://doi.org/10.1016/S0959-4388(99)00012-4

    Article  Google Scholar 

  8. Knowles JD, Watson RA, Corne DW (2001) Reducing local optima in single-objective problems by multi-objectivization. In: International conference on evolutionary multi-criterion optimization. Springer, pp 269–283

  9. Mandziuk J, Rajkiewicz P (2016) Neuro-evolutionary system for FOREX trading. In: 2016 IEEE congress on evolutionary computation, CEC 2016, pp 4654–4661. https://doi.org/10.1109/CEC.2016.7744384

  10. Nayak SC, Misra BB (2018) Estimating stock closing indices using a GA-weighted condensed polynomial neural network. Financial Innov 4(1):1–22. https://doi.org/10.1186/s40854-018-0104-2

    Article  Google Scholar 

  11. Niv Y, Joel D, Meilijson I, Ruppin E (2002) Evolution of reinforcement learning in uncertain environments: a simple explanation for complex foraging behaviors. Adapt Behav 10(1):5–24. https://doi.org/10.1177/1059-712302-010001-01

    Article  MATH  Google Scholar 

  12. Norouzzadeh MS, Clune J (2016) Neuromodulation improves the evolution of forward models. In: Proceedings of the 2016 on genetic and evolutionary computation conference—GECCO ’16, pp 157–164. https://doi.org/10.1145/2908812.2908837

  13. Prados D, Kak S (1989) Neural network capacity using delta rule. Electron Lett 25(3):197–199. https://doi.org/10.1049/el:19890142

    Article  Google Scholar 

  14. Reisinger J, Bahceci E, Karpov I, Miikkulainen R (2007) Coevolving strategies for general game playing. In: 2007 IEEE symposium on computational intelligence and games, pp 320–327. https://doi.org/10.1109/CIG.2007.368115

  15. Richard K. Belew, McInerney J, Schraudolph NN (1991) Evolving networks: using the genetic algorithm with connectionist learning. In: Proceedings of the second artificial life conference, pp 511—-547

  16. Showalter I, Schwartz HM (2004) A growing and pruning method for a history stack neural network based adaptive controller. In: Proceedings of the IEEE conference on decision and control, vol 5, pp 4946–4951. https://doi.org/10.1109/CDC.2004.1429590

  17. Silva F, Urbano P, Christensen AL (2014) Online evolution of adaptive robot behaviour. IGI Global, Hershey, pp 59–77

    Google Scholar 

  18. Silva F, Urbano P, Oliveira S, Christensen AL (2012) odNEAT: an algorithm for distributed online, onboard evolution of robot behaviours. Artif Life 13:251–258. https://doi.org/10.7551/978-0-262-31050-5-ch034

    Article  Google Scholar 

  19. Soltoggio A, Bullinaria JA, Mattiussi C, Dürr P, Floreano D (2008) Evolutionary advantages of neuromodulated plasticity in dynamic, reward-based scenarios. In: Artificial life XI: proceedings of the 11th international conference on simulation and synthesis of living systems (ALIFE 2008), vol 2, pp 569–576. https://doi.org/10.1016/S0269-7491(01)00278-0

  20. Stanley KO, Bryant BD, Miikkulainen R (2005) Real-time learning in the NERO video game. IEEE Trans Evol Comput 9(6):653–668

    Article  Google Scholar 

  21. Stanley KO, Miikkulainen R (2002) Evolving neural networks through augmenting topologies. Evol Comput 10(2):99–127. https://doi.org/10.1162/106365602320169811

    Article  Google Scholar 

  22. Stanley KO, Miikkulainen R (2004) Competitive coevolution through evolutionary complexification. J Artif Intell Res 21:63–100. https://doi.org/10.1613/jair.1338

    Article  Google Scholar 

  23. van Willigen W, Haasdijk E, Kester L (2013) A multi-objective approach to evolving platooning strategies in intelligent transportation systems. In: Proceeding of the fifteenth annual conference on Genetic and evolutionary computation conference—GECCO ’13. ACM Press, New York, NY, USA, pp 1397–1404. https://doi.org/10.1145/2463372.2463534

Download references

Author information

Authors and Affiliations

  1. Department of Systems and Computer Engineering, Carleton University, 4456 Mackenzie, 1125 Colonel By Drive, Ottawa, ON, K1S 5B6, Canada

    Ian Showalter & Howard M. Schwartz

Authors
  1. Ian Showalter
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Howard M. Schwartz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ian Showalter.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Showalter, I., Schwartz, H.M. Neuromodulated multiobjective evolutionary neurocontrollers without speciation. Evol. Intel. 14, 1415–1430 (2021). https://doi.org/10.1007/s12065-020-00394-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12065-020-00394-9

Keywords

Search

Navigation