On Affect and Self-adaptation: Potential Benefits of Valence-Controlled Action-Selection
Psychological studies have shown that emotion and affect influence learning. We employ these findings in a machine-learning meta-parameter context, and dynamically couple an adaptive agent’s artificial affect to its action-selection mechanism (Boltzmann β). The agent’s performance on two important learning problems is measured. The first consists of learning to cope with two alternating goals. The second consists of learning to prefer a later larger reward (global optimum) for an earlier smaller one (local optimum). Results show that, compared to several control conditions, coupling positive affect to exploitation and negative affect to exploration has several important benefits. In the alternating-goal task, it significantly reduces the agent’s “goal-switch search peak”. The agent finds its new goal faster. In the second task, artificial affect facilitates convergence to a global instead of a local optimum, while permitting to exploit that local optimum. We conclude that affect-controlled action-selection has adaptation benefits.
KeywordsLocal Optimum Reinforcement Learning Markov Decision Process Goal Location Large Reward
Unable to display preview. Download preview PDF.
- 2.Damasio, A.R.: Descartes’ error: Emotion, reason, and the human brain. Gosset/Putnam Press, New York (1994)Google Scholar
- 3.Forgas, J.P.: Feeling is believing? The role of processing strategies in mediating affective influences in beliefs. In: Frijda, N., et al. (eds.) Emotions and Beliefs, pp. 108–143. Cambridge University Press, Cambridge (2000)Google Scholar
- 6.Dreisbach, G., Goschke, T.: How positive affect modulates cognitive control: Reduced perseveration at the cost of increased distractibility. Journal of Experimental Psychology 30, 343–353 (2004)Google Scholar
- 8.Craig, S.D., Graesser, A.C., Sullins, J., Gholson, B.: Affect and learning: An exploratory look into the role of affect in learning with Autotutor. Journal of Educational Media 29, 241–250 (2004)Google Scholar
- 9.Sutton, R., Barto, A.: Reinforcement learning, An introduction. MIT Press, Cambridge (1998)Google Scholar
- 10.Csikszentmihalyi, M.: Flow: The psychology of optimal experience. Harper Row, New York (1990)Google Scholar
- 13.Tyrrell, T.: Computational mechanisms for action selection, PhD thesis. University of Edinburgh (1993)Google Scholar
- 14.Avila-Garcia, O., Cañamero, L.: Using hormonal feedback to modulate action selection in a competitive scenario. In: Proceedings of the 8th Intl. Conf. on Simulation of Adaptive Behavior, pp. 243–252 (2004)Google Scholar
- 15.Belavkin, R.V.: On relation between emotion and entropy. In: Proceedings of the AISB’04 Symposium on Emotion, Cognition and Affective Computing, Leeds, UK, pp. 1–8 (2004)Google Scholar
- 16.Botelho, L.M., Coelho, H.: Information processing, motivation and decision making. In: Proc. 4th International Workshop on Artificial Intelligence in Economics and Management (1998)Google Scholar