Learning to Interpret Pointing Gestures: Experiments with Four-Legged Autonomous Robots

  • Verena V. Hafner
  • Frédéric Kaplan
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3575)

Abstract

In order to bootstrap shared communication systems, robots must have a non-verbal way to influence the attention of one another. This chapter presents an experiment in which a robot learns to interpret pointing gestures of another robot. We show that simple feature-based neural learning techniques permit reliably to discriminate between left and right pointing gestures. This is a first step towards more complex attention coordination behaviour. We discuss the results of this experiment in relation to possible developmental scenarios about how children learn to interpret pointing gestures.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Steels, L., Kaplan, F.: Situated grounded word semantics. In: Dean, T. (ed.) Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence IJCAI 1999, pp. 862–867. Morgan Kaufmann Publishers, San Francisco (1999)Google Scholar
  2. 2.
    Steels, L., Kaplan, F.: Collective learning and semiotic dynamics. In: Floreano, D., Mondada, F. (eds.) ECAL 1999. LNCS (LNAI), vol. 1674, pp. 679–688. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  3. 3.
    Steels, L., Kaplan, F.: Bootstrapping grounded word semantics. In: Briscoe, T. (ed.) Linguistic evolution through language acquisition: formal and computational models, pp. 53–73. Cambridge University Press, Cambridge (2002)CrossRefGoogle Scholar
  4. 4.
    Steels, L.: The Talking Heads Experiment. Words and Meanings, vol. 1. Antwerpen (1999)Google Scholar
  5. 5.
    Vogt, P.: Lexicon grounding on mobile robots. PhD thesis, Vrije Universiteit Brussel (2000)Google Scholar
  6. 6.
    Kaplan, F.: La naissance d’une langue chez les robots. Hermes Science (2001)Google Scholar
  7. 7.
    Steels, L., Kaplan, F., McIntyre, A., Van Looveren, J.: Crucial factors in the origins of word-meaning. In: Wray, A. (ed.) The Transition to Language, pp. 252–271. Oxford University Press, Oxford (2002)Google Scholar
  8. 8.
    Scassellati, B.: Imitation and mechanisms of joint attention: A developmental structure for building social skills on a humanoid robot. In: Nehaniv, C.L. (ed.) CMAA 1998. LNCS (LNAI), vol. 1562, p. 176. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  9. 9.
    Kozima, H., Yano, H.: A robot that learns to communicate with human caregivers. In: First International Workshop on Epigenetic Robotics, Lund, Sweden (2001)Google Scholar
  10. 10.
    Imai, M., Ono, T., Ishiguro, H.: Physical relation and expression: Joint attention for human-robot interaction. In: Proceedings of the 10th IEEE International Workshop on Robot and Human Communication (2001)Google Scholar
  11. 11.
    Nagai, Y., Asada, M., Hosoda, K.: A developmental approach accelerates learning of joint attention. In: Proceedings of the second international conference of development and learning (2002)Google Scholar
  12. 12.
    Nagai, Y., Hosoda, K., Morita, A., Asada, M.: A constructive model for the development of joint attention. Connection Science 15, 211–229 (2003)CrossRefGoogle Scholar
  13. 13.
    Nickel, K., Stiefelhagen, R.: Real-time recognition of 3d-pointing gestures for human-machine-interaction. In: Michaelis, B., Krell, G. (eds.) DAGM 2003. LNCS, vol. 2781, pp. 557–565. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  14. 14.
    Baillie, J.C.: Grounding symbols in perception with two interacting autonomous robots. In: Berthouze, L., Kozima, H., Prince, C., Sandini, G., Stojanov, G., Metta, G., Balkenius, C. (eds.) Proceedings of the 4th International Workshop on Epigenetic Robotics: Modeling Cognitive Development in Robotic System, Lund University Cognitive Studies 117 (2004)Google Scholar
  15. 15.
    Kaplan, F., Oudeyer, P.Y.: Maximizing learning progress: an internal reward system for development. In: Iida, F., Pfeifer, R., Steels, L., Kuniyoshi, Y. (eds.) Embodied Artificial Intelligence. LNCS (LNAI), vol. 3139, pp. 259–270. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  16. 16.
    Carlson, E., Triesch, J.: A computational model of the emergence of gaze following. In: Proceedings of the 8th Neural Computation Workshop, NCPW8 (2003)Google Scholar
  17. 17.
    Jasso, H., Triesch, J., Teuscher, C.: Gaze following in the virtual living room. In: Palm, G., Wermter, S. (eds.) Proceedings of the KI 2004 Workshop on Neurobotics (2004)Google Scholar
  18. 18.
    Bell, A.J., Sejnowski, T.J.: Edges are the independent components of natural scenes. In: Advances in Neural Information Processing Systems (NIPS), pp. 831–837 (1996)Google Scholar
  19. 19.
    Dudek, G., Jenkin, M.: Computational principles of mobile robotics. Cambridge University Press, Cambridge (2000)MATHGoogle Scholar
  20. 20.
    Witten, I., Eibe, F.: Data mining. Morgan Kaufmann Publishers, San Francisco (2000)Google Scholar
  21. 21.
    Butterworth, G.: Origins of mind in perception and action. In: Moore, C., Dunham, P. (eds.) Joint attention: its origins and role in development, pp. 29–40. Lawrence Erlbaum Associates, Mahwah (1995)Google Scholar
  22. 22.
    Baron-Cohen, S.: Mindblindness: an essay on autism and theory of mind. MIT Press, Boston (1997)Google Scholar
  23. 23.
    Tomasello, M.: Joint attention as social cognition. In: Moore, C., Dunham, P. (eds.) Joint attention: its origins and role in development, pp. 103–130. Lawrence Erlbaum Associates, Mahwah (1995)Google Scholar
  24. 24.
    Hobson, P.: The craddle of thought. Macmillan, Basingstoke (2002)Google Scholar
  25. 25.
    Kaplan, F., Hafner, V.: The challenges of joint attention. In: Berthouze, L., Kozima, H., Prince, C., Sandini, G., Stojanov, G., Metta, G., Balkenius, C. (eds.) Proceedings of the 4th International Workshop on Epigenetic Robotics: Modeling Cognitive Development in Robotic System, Lund University Cognitive Studies 117, pp. 67–74 (2004)Google Scholar
  26. 26.
    Brighton, H., Kirby, S., Smith, K.: Cultural selection for learnability: Three hypotheses underlying the view that language adapts to be learnable. In: Tallerman, M. (ed.) Language Origins: Perspective on Evolution. Oxford University Press, Oxford (2005)Google Scholar
  27. 27.
    Elshaw, M., Weber, C., Zochios, A., Wermter, S.A.: A mirror neuron inspired hierarchical network for action selection. In: Palm, G., Wermter, S. (eds.) Proceedings of the KI 2004 Workshop on NeuroBotics, Ulm, Germany, pp. 98–105 (2004)Google Scholar
  28. 28.
    Desrochers, S., Morisette, P., Ricard, M.: Two perspectives on pointing in infancy. In: Moore, C., Dunham, P. (eds.) Joint Attention: its origins and role in development, pp. 85–101. Lawrence Erlbaum Associates, Mahwah (1995)Google Scholar
  29. 29.
    Liszkowski, U., Carpenter, M., Henning, A., Striano, T., Tomasello, M.: Twelve-month-olds point to share attention and interest. Developmental Science 7, 297–307 (2004)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Verena V. Hafner
    • 1
  • Frédéric Kaplan
    • 1
  1. 1.Sony CSL ParisParisFrance

Personalised recommendations