Reinforcement Learning in MirrorBot

  • Cornelius Weber
  • David Muse
  • Mark Elshaw
  • Stefan Wermter
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3696)


For this special session of EU projects in the area of NeuroIT, we will review the progress of the MirrorBot project with special emphasis on its relation to reinforcement learning and future perspectives. Models inspired by mirror neurons in the cortex, while enabling a system to understand its actions, also help in the solving of the curse of dimensionality problem of reinforcement learning. Reinforcement learning, which is primarily linked to the basal ganglia, is a powerful method to teach an agent such as a robot a goal-directed action strategy. Its limitation is mainly that the perceived situation has to be mapped to a state space, which grows exponentially with input dimensionality. Cortex-inspired computation can alleviate this problem by pre-processing sensory information and supplying motor primitives that can act as modules for a superordinate reinforcement learning scheme.


State Space Motor Cortex Motor Unit Reinforcement Learning Target Object 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Rizzolatti, G., Fogassi, L., Gallese, V.: Motor and cognitive functions of the ventral premotor cortex. Current Opinion in Neurobiology 12, 149–154 (2002)CrossRefGoogle Scholar
  2. 2.
    Touzet, C.: Neural reinforcement learning for behaviour synthesis. Robotics and Autonomous Systems 22, 251–281 (1997)CrossRefGoogle Scholar
  3. 3.
    Weber, C., Wermter, S., Zochios, A.: Robot docking with neural vision and reinforcement. Knowledge-Based Systems 17, 165–172 (2004)CrossRefGoogle Scholar
  4. 4.
    Jog, M., Kubota, Y., Connolly, C., Hillegaart, V., Graybiel, A.: Building neural representations of habits. Science 286, 1745–1749 (1999)CrossRefGoogle Scholar
  5. 5.
    Elshaw, M., Weber, C., Zochios, A., Wermter, S.: A mirror neuron inspired hierarchical network for action selection. In: Proc. NeuroBotics, pp. 89–97 (2004)Google Scholar
  6. 6.
    Kohonen, T.: Self-Organizing Maps. Springer, Heidelberg (2001)zbMATHGoogle Scholar
  7. 7.
    Pulvermüller, F.: The Neuroscience of Language. On Brain Circuits of Words and Serial Order. Cambridge University Press, Cambridge (2003)CrossRefGoogle Scholar
  8. 8.
    Kondo, T., Ito, K.: A reinforcement learning with evolutionary state recruitment strategy for autonomous mobile robots control. Robotics and Autonomous Systems 46, 111–124 (2004)CrossRefGoogle Scholar
  9. 9.
    Lee, I., Lau, Y.: Adaptive state space partitioning for reinforcement learning. Engineering Applications of Artificial Intelligence 17, 577–588 (2004)CrossRefGoogle Scholar
  10. 10.
    Barto, A., Mahadevan, S.: Recent advances in hierarchical reinforcement learning. Discrete Event Dynamic Systems 13, 41–77 (2003)zbMATHCrossRefMathSciNetGoogle Scholar
  11. 11.
    Kalmár, Z., Szepesvári, C., Lörincz, A.: Module-based reinforcement learning: Experiments with a real robot. Machine Learning 31, 55–85 (1998)zbMATHCrossRefGoogle Scholar
  12. 12.
    Humphrys, M.: W-learning: A simple RL-based society of mind. In: 3rd European Conference on Artificial Life, p. 30 (1995)Google Scholar
  13. 13.
    Knoblauch, A., Markert, H., Palm, G.: An associative model of cortical language and action processing. In: Proc. 9th Neural Comp. and Psych. Workshop (2004)Google Scholar
  14. 14.
    Vitay, J., Rougier, N., Alexandre, F.: A distributed model of visual spatial attention. In: Biomimetic Neural Learning for Intelligent Robotics. Springer, Heidelberg (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Cornelius Weber
    • 1
  • David Muse
    • 1
  • Mark Elshaw
    • 1
  • Stefan Wermter
    • 1
  1. 1.Hybrid Intelligent Systems, SCATUniversity of SunderlandUK

Personalised recommendations