Abstract
Taking neuromodulation as a mechanism underlying emotions, this paper investigates how such a mechanism can bias an artificial neural network towards exploration of new courses of action, as seems to be the case in positive emotions, or exploitation of known possibilities, as in negative emotions such as predatory fear. We use neural networks of spiking leaky integrate-and-fire neurons acting as minimal disturbance systems, and test them with continuous actions. The networks have to balance the activations of all their output neurons concurrently. We have found that having the middle layer modulate the output layer helps balance the activations of the output neurons. A second discovery is that when the network is modulated in this way, it performs better at tasks requiring the exploitation of actions that are found to be rewarding. This is complementary to previous findings where having the input layer modulate the middle layer biases the network towards exploration of alternative actions. We conclude that a network can be biased towards either exploration of exploitation depending on which layers are being modulated.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Koch, C.: Biophysics of Computation. Oxford University Press, Oxford (1999)
Fellous, J.M.: The neuromodulatory basis of emotion. The neuroscientist 5(5), 283–294 (1999)
Kelley, A.E.: 3. In: Who needs emotions? The brain meets the robot, pp. 29–77. Oxford University Press, Oxford (2005)
Damasio, A.: Descartes’ Error: Emotion, Reason, and the Human Brain. Quill (1994)
Evans, D.: The search hypothesis of emotion. British Journal for the Philosophy of Science 53(4), 497–509 (2002)
Nesse, R.: Evolutionary explanations of emotion. Human Nature 1(30), 261–289 (1990)
LeDoux, J.E.: The Emotional Brain. Simon & Schuster (1998)
Avila-García, O., Cañamero, L.: Hormonal modulation of perception in motivation-based action selection architectures. In: Avila-García, O. (ed.) Proceedings of the Symposium on Agents that Want and Like: Motivational and Emotional roots of Cognition and Action at the AISB-05 conference, The society for the study of artificial intelligence and the simulation of behaviour, pp. 9–16 (2005)
Blanchard, A., Cañamero, L.: Developing affect-modulated behaviors: Stability, exploration, exploitation, or imitation? In: Kaplan, F. (ed.) Proc. 6th Intl. Workshop on Epigenetic Robotics, vol. 128, Lund University Cognitive Studies (2006)
Wehmeier, U., Dong, D., Koch, C., van Essen, D.: Modeling the mammalian visual system. In: Koch, C., Segev, I. (eds.) Methods in Neuronal Modeling: From synapses to networks, pp. 335–360. MIT Press, Cambridge (1989)
Wörgötter, F., Porr, B.: Temporal sequence learning, prediction and control - a review of different models and their relation to biological mechanisms. Neural Computation 17, 1–75 (2004)
Karmarkar, U.R., Najariana, M.T., Buonomano, D.V.: Mechanisms and significance of spike-timing dependent synaptic plasticity. Biological Cybernetics 87, 373–382 (2002)
Parussel, K.M.: A bottom-up approach to emulating emotions using neuromodulation in agents. PhD thesis, University of Stirling (2006)
Parussel, K., Smith, L.: Cost minimisation and reward maximisation. a neuromodulating minimal disturbance system using anti-hebbian spike timing-dependent plasticity. In: Proceedings of the Symposium on Agents that Want and Like: Motivational and Emotional roots of Cognition and Action at the AISB-05 conference, The society for the study of artificial intelligence and the simulation of behaviour, pp. 98–101 (2005)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Parussel, K., Cañamero, L. (2007). Biasing Neural Networks Towards Exploration or Exploitation Using Neuromodulation. In: de Sá, J.M., Alexandre, L.A., Duch, W., Mandic, D. (eds) Artificial Neural Networks – ICANN 2007. ICANN 2007. Lecture Notes in Computer Science, vol 4669. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74695-9_91
Download citation
DOI: https://doi.org/10.1007/978-3-540-74695-9_91
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-74693-5
Online ISBN: 978-3-540-74695-9
eBook Packages: Computer ScienceComputer Science (R0)