Attentional Action Selection Using Reinforcement Learning

  • Dario Di Nocera
  • Alberto Finzi
  • Silvia Rossi
  • Mariacarla Staffa
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7426)


Reinforcement learning is typically used to model and optimize action selection strategies, in this work we deploy it to optimize attentional allocation strategies while action selection is obtained as a side effect. We present a reinforcement learning approach to attentional allocation and action selection in a behavior-based robotic systems. We detail our attentional allocation mechanisms describing the reinforcement learning problem and analysing its performance in a survival domain.


attention allocation reinforcement learning action selection 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bandera, C., Vico, F.J., Bravo, J.M., Harmon, M.E., Iii, L.C.B.: Residual q-learning applied to visual attention. In: ICML 1996, pp. 20–27 (1996)Google Scholar
  2. 2.
    Burattini, E., Rossi, S.: Periodic adaptive activation of behaviors in robotic system. IJPRAI 22(5), 987–999 (2008)Google Scholar
  3. 3.
    Burattini, E., Rossi, S., Finzi, A., Staffa, M.: Attentional Modulation of Mutually Dependent Behaviors. In: Doncieux, S., Girard, B., Guillot, A., Hallam, J., Meyer, J.-A., Mouret, J.-B. (eds.) SAB 2010. LNCS, vol. 6226, pp. 283–292. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  4. 4.
    Houk, J.C., Adams, J.L., Barto, A.G.: A model of how the basal ganglia generate and use neural signals that predict reinforcement. In: Houk, J.C., Davis, J.L., Beiser, D.G. (eds.) Models of Information Processing in the Basal Ganglia, pp. 249–270. MIT Press, Cambridge (1995)Google Scholar
  5. 5.
    Kahneman, D.: Attention and Effort. Prentice-Hall, Englewood Cliffs (1973)Google Scholar
  6. 6.
    Montague, P.R., Dayan, P., Sejnowskw, T.J.: A framework for mesencephalic dopamine systems based on predictive hebbian learning. J. Neur. 1936–1947 (1996)Google Scholar
  7. 7.
    Norman, D., Shallice, T.: Attention in action: willed and automatic control of behaviour. Consciousness and Self-Regulation: Advances in Research and Theory 4, 1–18 (1986)Google Scholar
  8. 8.
    O’Reilly, R., Frank, M.: Making working memory work: A computational model of learning in the frontal cortex and basal ganglia. Neural Computation 18, 283–328 (2006)MathSciNetzbMATHCrossRefGoogle Scholar
  9. 9.
    Paletta, L., Fritz, G., Seifert, C.: Q-learning of sequential attention for visual object recognition from informative local descriptors. In: ICML 2005 (2005)Google Scholar
  10. 10.
    Posner, M.I., Presti, D.E.: Selective attention and cognitive control. TINS 10, 13–17 (1987)Google Scholar
  11. 11.
    Senders, J.: The human operator as a monitor and controller of multidegree of freedom systems, pp. 2–6 (1964)Google Scholar
  12. 12.
    Sutton, R., Barto, A.: Reinforcement learning: An introduction, vol. 1. Cambridge Univ. Press (1998)Google Scholar
  13. 13.
    Tan, M.: Multi-agent reinforcement learning: Independent vs. cooperative agents. In: ICML 1993, pp. 330–337. Morgan Kaufmann (1993)Google Scholar
  14. 14.
    Watkins, C., Dayan, P.: Q-learning. Machine Learning 8(3), 279–292 (1992)zbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Dario Di Nocera
    • 1
  • Alberto Finzi
    • 1
  • Silvia Rossi
    • 1
  • Mariacarla Staffa
    • 2
  1. 1.Dipartimento di Scienze FisicheUniversity of Naples “Federico II”NaplesItaly
  2. 2.Dipartimento di Informatica e SistemisticaUniversity of Naples “Federico II”NaplesItaly

Personalised recommendations