Learning Adjectives and Nouns from Affordances on the iCub Humanoid Robot

  • Onur Yürüten
  • Kadir Fırat Uyanık
  • Yiğit Çalışkan
  • Asil Kaan Bozcuoğlu
  • Erol Şahin
  • Sinan Kalkan
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7426)


This article studies how a robot can learn nouns and adjectives in language. Towards this end, we extended a framework that enabled robots to learn affordances from its sensorimotor interactions, to learn nouns and adjectives using labeling from humans. Specifically, an iCub humanoid robot interacted with a set of objects (each labeled with a set of adjectives and a noun) and learned to predict the effects (as labeled with a set of verbs) it can generate on them with its behaviors. Different from appearance-based studies that directly link the appearances of objects to nouns and adjectives, we first predict the affordances of an object through a set of Support Vector Machine classifiers which provided a functional view of the object. Then, we learned the mapping between these predicted affordance values and nouns and adjectives. We evaluated and compared a number of different approaches towards the learning of nouns and adjectives on a small set of novel objects.

The results show that the proposed method provides better generalization than the appearance-based approaches towards learning adjectives whereas, for nouns, the reverse is the case. We conclude that affordances of objects can be more informative for (a subset of) adjectives describing objects in language.


affordances nouns adjectives 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Harnad, S.: The symbol grounding problem. Physica D: Nonlinear Phenomena 42(1-3), 335–346 (1990)CrossRefGoogle Scholar
  2. 2.
    Borghi, A.M.: Object concepts and embodiment: Why sensorimotor and cognitive processes cannot be separated. La Nuova Critica 49(50), 90–107 (2007)Google Scholar
  3. 3.
    Gibson, J.J.: The ecological approach to visual perception. Lawrence Erlbaum (1986)Google Scholar
  4. 4.
    Dag, N., Atil, I., Kalkan, S., Sahin, E.: Learning affordances for categorizing objects and their properties. In: IEEE ICPR. IEEE (2010)Google Scholar
  5. 5.
    Atil, I., Dag, N., Kalkan, S., Şahin, E.: Affordances and emergence of concepts. In: Epigenetic Robotics (2010)Google Scholar
  6. 6.
    Yu, C., Ballard, D.H.: On the integration of grounding language and learning objects. In: Proc. 19th Int. Conf. on Artifical Intelligence, AAAI 2004, pp. 488–493. AAAI Press (2004)Google Scholar
  7. 7.
    Carbonetto, P., de Freitas, N.: Why can’t jose read? the problem of learning semantic associations in a robot environment. In: Proc. the HLT-NAACL 2003 Workshop on Learning Word Meaning from Non-Linguistic Data, pp. 54–61 (2003)Google Scholar
  8. 8.
    Nehaniv, C.L., Saunders, J., Lyon, C.: Robot learning of lexical semantics from sensorimotor interaction and the unrestricted speech of human tutors. In: Proc. 2nd Int. Symp. New Frontiers Human-Robot Interact. ASIB Convent. (2010)Google Scholar
  9. 9.
    Cangelosi, A.: Evolution of communication and language using signals, symbols, and words. IEEE Tran. on Evolutionary Computation 5(2), 93–101 (2001)CrossRefGoogle Scholar
  10. 10.
    Cangelosi, A.: Grounding language in action and perception: From cognitive agents to humanoid robots. Physics of Life Reviews 7(2), 139–151 (2010)CrossRefGoogle Scholar
  11. 11.
    Şahin, E., Çakmak, M., Doğar, M.R., Uğur, E., Üçoluk, G.: To afford or not to afford: A new formalization of affordances toward affordance-based robot control. Adaptive Behavior 15(4), 447–472 (2007)CrossRefGoogle Scholar
  12. 12.
    Montesano, L., Lopes, M., Bernardino, A., Santos-Victor, J.: Learning object affordances: From sensory–motor coordination to imitation. IEEE Tran. on Robotics 24(1), 15–26 (2008)CrossRefGoogle Scholar
  13. 13.
    Stoytchev, A.: Learning the Affordances of Tools Using a Behavior-Grounded Approach. In: Rome, E., Hertzberg, J., Dorffner, G. (eds.) Towards Affordance-Based Robot Control. LNCS (LNAI), vol. 4760, pp. 140–158. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  14. 14.
    Kraft, D., Pugeault, N., Baseski, E., Popovic, M., Kragic, D., Kalkan, S., Wörgötter, F., Krüger, N.: Birth of the object: Detection of objectness and extraction of object shape through object action complexes. International Journal of Humanoid Robotics 5(2), 247–265 (2008)CrossRefGoogle Scholar
  15. 15.
    Ugur, E., Şahin, E.: Traversability: A case study for learning and perceiving affordances in robots. Adaptive Behavior 18(3-4), 258–284 (2010)CrossRefGoogle Scholar
  16. 16.
    Fitzpatrick, P., Metta, G., Natale, L., Rao, S., Sandini, G.: Learning about objects through action - initial steps towards artificial cognition. In: IEEE ICRA (2003)Google Scholar
  17. 17.
    Detry, R., Kraft, D., Buch, A.G., Kruger, N., Piater, J.: Refining grasp affordance models by experience. In: IEEE ICRA, pp. 2287–2293 (2010)Google Scholar
  18. 18.
    Ugur, E., Oztop, E., Şahin, E.: Goal emulation and planning in perceptual space using learned affordances. Robotics and Autonomous Systems 59(7-8) (2011)Google Scholar
  19. 19.
    Ugur, E., Şahin, E., Oztop, E.: Affordance learning from range data for multi-step planning. In: Int. Conf. on Epigenetic Robotics (2009)Google Scholar
  20. 20.
    Montesano, L., Lopes, M., Bernardino, A., Santos-Victor, J.: A computational model of object affordances. In: Advances in Cognitive Systems. IET (2009)Google Scholar
  21. 21.
    Sinapov, J., Stoytchev, A.: Learning and generalization of behavior-grounded tool affordances. In: IEEE 6th International Conference on Development and Learning, ICDL 2007, pp. 19–24 (July 2007)Google Scholar
  22. 22.
    Sinapov, J., Stoytchev, A.: Detecting the functional similarities between tools using a hierarchical representation of outcomes. In: 7th IEEE International Conference on Development and Learning, ICDL 2008, pp. 91–96 (August 2008)Google Scholar
  23. 23.
    Montesano, L., Lopes, M., Bernardino, A., Santos-Victor, J.: Learning object affordances: From sensory–motor coordination to imitation. IEEE Tran. on Robotics 24(1), 15–26 (2008)CrossRefGoogle Scholar
  24. 24.
    Montesano, L., Lopes, M., Melo, F., Bernardino, A., Santos-Victor, J.: A computational model of object affordances. Advances in Cognitive Systems 54, 258 (2009)Google Scholar
  25. 25.
    Rusu, R.B., Cousins, S.: 3d is here: Point cloud library (pcl). In: IEEE ICRA, pp. 1–4. IEEE (2011)Google Scholar
  26. 26.
    Kira, K., Rendell, L.A.: A practical approach to feature selection. In: Proc. 9th Int. Workshop on Machine Learning, pp. 249–256 (1992)Google Scholar
  27. 27.
    Qin, A.K., Suganthan, P.N.: Robust growing neural gas algorithm with application in cluster analysis. Neural Networks 17(8-9), 1135–1148 (2004)zbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Onur Yürüten
    • 1
  • Kadir Fırat Uyanık
    • 1
    • 3
  • Yiğit Çalışkan
    • 1
    • 2
  • Asil Kaan Bozcuoğlu
    • 1
  • Erol Şahin
    • 1
  • Sinan Kalkan
    • 1
  1. 1.Kovan Res. Lab, Dept. of Computer Eng.Middle East Technical UniversityTurkey
  2. 2.Dept. of Computer Eng.Bilkent UniversityTurkey
  3. 3.Dept. of Electrical and Electronics Eng.Middle East Technical UniversityTurkey

Personalised recommendations