Guiding Exploration by Combining Individual Learning and Imitation in Societies of Autonomous Robots

  • Willi Richert
  • Oliver Niehörster
  • Florian Klompmaker
Conference paper
Part of the IFIP – The International Federation for Information Processing book series (IFIPAICT, volume 268)


Robots have a powerful means to drastically cut down the exploration space with imitation. However, as existing imitation approaches usually require repetitive demonstrations of the skill to learn in order to be useful, those are typically not applicable in groups of robots. In these settings usually each robot has its own task to accomplish and should not be disturbed by teaching others. As a result an imitating robot most of the time has only one observation of a specific skill from which it can learn.

We present an approach that allows an individually learning robot to make use of such cases of sporadic imitation which is the normal case in groups of robots. Thereby, a robot can use imitation in order to guide its exploration efforts towards more rewarding areas in the exploration space. This is inspired by imitation often found in nature where animals or humans try to map observations into their own capability space. We show the feasibility by realistic simulation of Pioneer robots.


Hide Markov Model Reinforcement Learn Exploration Space Viterbi Algorithm Strategy Layer 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    E. Alpaydin. Introduction To Machine Learning. MIT Press, 2004.Google Scholar
  2. 2.
    R. Bellman. Dynamic Programming. Courier Dover Publications, 2003.Google Scholar
  3. 3.
    Y. Bengio. Markovian models for sequential data. Neural Computing Surveys, 2:129-162, 1999.Google Scholar
  4. 4.
    A. Billard, Y. Epars, S. Calinon, G. Cheng, and S. Schaal. Discovering optimal imitation strategies. Robotics and Autonomous Systems, 47(2-3):69-77, 2004.CrossRefGoogle Scholar
  5. 5.
    Billard, A. Learning motor skills by imitation: a biologically inspired robotic model, 2000.Google Scholar
  6. 6.
    Borenstein, E. and Ruppin, E. Enhancing autonomous agents evolution with learning by imitation. In Second International Symposium on Imitation in Animals and Artifacts, 2003.Google Scholar
  7. 7.
    T. M. Cover and P. E. Hart. Nearest neighbor pattern classification. IEEE Transactions on Information Theory, 13:21-27, 1967.zbMATHCrossRefGoogle Scholar
  8. 8.
    Demiris, J. and Hayes, G. Imitation as a dual-route process featuring predictive and learning components: a biologically-plausible computational model. In K. Dautenhahn and C. Nehaniv, editors, Imitation in animals and artifacts, pages 327-361, Cambridge, MA, USA, 2002. MIT Press.Google Scholar
  9. 9.
    Y. Gatsoulis, G. Maistros, Y. Marom, and G. Hayes. Learning to forage through imitation. In Proceedings of the Second IASTED International Conference on Artificial Intelligence and Applications (AIA2002), pages 485-491, Sept. 2002.Google Scholar
  10. 10.
    A. Ijspeert, J. Nakanishi, and S. Schaal. Movement imitation with nonlinear dynamical systems in humanoid robots. In International Conference on Robotics and Automation (ICRA2002), 2002.Google Scholar
  11. 11.
    T. Inamura, Y. Nakamura, H. Ezaki, and I. Toshima. Imitation and primitive symbol acquisition of humanoids by the integrated mimesis loop. Robotics and Automation, 2001. Proceedings 2001 ICRA. IEEE International Conference on, 4, 2001.Google Scholar
  12. 12.
    T. Inamura, I. Toshima, Y. Nakamura, and J. Saitama. Acquiring Motion Elements for Bidi-rectional Computation of Motion Recognition and Generation. Experimental Robotics VIII, 2003.Google Scholar
  13. 13.
    M. J. Kochenderfer. Adaptive Modelling and Planning for Learning Intelligent Behaviour. PhD thesis, School of Informatics, University of Edinburgh, 2006.Google Scholar
  14. 14.
    W. Richert, O. L üke, B. Nordmeyer, and B. Kleinjohann. Increasing the autonomy of mobile robots by on-line learning simultaneously at different levels of abstraction. In IEEE International Conference on Autonomic and Autonomous Systems (ICAS’08), March 2008.Google Scholar
  15. 15.
    R. S. Sutton and A. G. Barto. Reinforcement Learning: An Introduction. MIT Press, Cambridge, 1998.Google Scholar
  16. 16.
    A. Viterbi. Error bounds for convolutional codes and an asymptotically optimum decoding algorithm. Information Theory, IEEE Transactions on, 13(2):260-269, 1967.zbMATHCrossRefGoogle Scholar

Copyright information

© International Federation for Information Processing 2008

Authors and Affiliations

  • Willi Richert
    • 1
  • Oliver Niehörster
    • 1
  • Florian Klompmaker
    • 1
  1. 1.Intelligent Mobile SystemsUniversity of Paderborn / C-LABGermany

Personalised recommendations