Nadine: A Social Robot that Can Localize Objects and Grasp Them in a Human Way

  • Nadia Magnenat ThalmannEmail author
  • Li Tian
  • Fupin Yao
Conference paper
Part of the Lecture Notes in Electrical Engineering book series (LNEE, volume 433)


What makes a social humanoid robot behave like a human? It needs to understand and show emotions, has a chat box, a memory and also a decision-making process. However, more than that, it needs to recognize objects and be able to grasp them in a human way. To become an intimate companion, social robots need to behave the same way as real humans in all areas and understand real situations in order they can react properly. In this chapter, we describe our ongoing research on social robotics. It includes the making of articulated hands of Nadine Robot, the recognition of objects and their signification, as well as how to grasp them in a human way. State of the art is presented as well as some early results.


Robotic hand 3D printing Object recognition Trajectories planning Natural grasp 



This research is supported by the BeingTogether Centre, a collaboration between Nanyang Technological University (NTU) Singapore and University of North Carolina (UNC) at Chapel Hill. The BeingTogether Centre is supported by the National Research Foundation, Prime Minister’s Office, Singapore under its International Research Centres in Singapore Funding Initiative.


  1. 1.
    Uncanny valley, in Wikipedia. Accessed 28 Nov 2016
  2. 2.
  3. 3.
    Nadine social robot, in Wikipedia. Accessed 28 Nov 2016
  4. 4.
    Atlas (robot), in Wikipedia. Accessed 30 Nov 2016
  5. 5.
    Y. Sakagami, R. Watanabe, C. Aoyama et al., The intelligent ASIMO: system overview and integration, in IEEE/RSJ International Conference on Intelligent Robots and Systems, 2002, vol. 3, (IEEE, 2002), pp. 2478-2483Google Scholar
  6. 6.
    G. Metta, G. Sandini, D. Vernon, L. Natale, F. Nori, The iCub humanoid robot: an open platform for research in embodied cognition, in PerMIS’08 Proceedings of the 8th Workshop on Performance Metrics for Intelligent Systems, pp. 50–56Google Scholar
  7. 7.
    Actroid, in Wikipedia. Accessed 28 Nov 2016
  8. 8.
    T. Times, Meet Jiajia, china’s new interactive robot. Accessed 30 Nov 2016
  9. 9.
    Home, Accessed 30 Nov 2016
  10. 10.
    R. Girshick, J. Donahue, T. Darrell, J. Malik, Rich feature hierarchies for accurate object detection and semantic segmentation, in CVPR (2014)Google Scholar
  11. 11.
    R. Girshick, Fast r-cnn, in Proceedings of the IEEE International Conference on Computer Vision (2015), pp. 1440–1448Google Scholar
  12. 12.
    S. Ren, K. He, R. Girshick, J. Sun, Faster r-cnn: towards real-time object detection with region proposal networks, in Advances in Neural Information Processing Systems (2015), pp. 91–99Google Scholar
  13. 13.
    J. Redmon, S. Divvala, R. Girshick, A. Farhadi, You only look once: unified, real-time object detection, in CVPR (2016)Google Scholar
  14. 14.
    W. Liu, D. Anguelov, D. Erhan et al., SSD: single shot multibox detector (2015). arXiv:1512.02325
  15. 15.
    A. Saxena, J. Driemeyer, A.Y. Ng, Robotic grasping of novel objects using vision. Int. J. Rob. Res. 27(2), 157–173 (2008)Google Scholar
  16. 16.
    Q.V. Le, D. Kamm, A.F. Kara, A.Y. Ng, Learning to grasp objects with multiple contact points, in IEEE International Conference on Robotics and Automation (ICRA) (2010), pp. 5062–5069Google Scholar
  17. 17.
    Ian Lenz, Honglak Lee, Ashutosh Saxena, Deep learning for detecting robotic grasps. Int. J. Rob. Res. 34(4–5), 705–724 (2015)CrossRefGoogle Scholar
  18. 18.
    Y. Jiang, S. Moseson, A. Saxena, Efficient grasping from rgbd images: learning using a new rectangle representation, in IEEE International Conference on Robotics and Automation (ICRA) (2011), pp. 3304–3311Google Scholar
  19. 19.
    J. Redmon, A. Angelova, Real-time grasp detection using convolutional neural networks, in IEEE International Conference on Robotics and Automation (ICRA) (2015), pp. 1316–1322Google Scholar
  20. 20.
    D. Guo, T. Kong, F. Sun et al., Object discovery and grasp detection with a shared convolutional neural network, in 2016 IEEE International Conference on Robotics and Automation (ICRA). (IEEE, 2016), pp. 2038–2043Google Scholar
  21. 21.
  22. 22.
    G. Schreiber, A. Stemmer, R. Bischoff, The fast research interface for the kuka lightweight robot, in IEEE Workshop on Innovative Robot Control Architectures for Demanding (Research) Applications How to Modify and Enhance Commercial Controllers (ICRA 2010) (2010), pp. 15–21Google Scholar
  23. 23.
    S. Rueckhaus, MH24. Accessed 28 Nov 2016
  24. 24.
    Inmoov project, Accessed 1 Dec 2016
  25. 25.
    P. Slade et al., Tack: design an performance of an open-source, affordable, myoelectric prosthetic hand, in 2015 IEEE ICRA Google Scholar
  26. 26.
    J.T. Belter et al., Mechanical design and performance specifications of anthropomorphic prosthetic hands: a review. J. Rehabil. Res. Dev. 50(5) 2013Google Scholar
  27. 27.
    A. Krizhevsky, I. Sutskever, G.E. Hinton, Imagenet classification with deep convolutional neural networks, in Advances in Neural Information Processing Systems (2012), pp. 1097–1105Google Scholar
  28. 28.
    Layer catalogue, Accessed 28 Dec 2016
  29. 29.
    Theano 0.8.2 documentation (2008), Accessed 21 Dec 2016
  30. 30.
    Deep learning framework (no date), Accessed 21 Dec 2016
  31. 31.
    A.M.R Agur, M.J. Lee, Grant’s Atlas of Anatomy 10th ed (1999)Google Scholar
  32. 32.
    J. Lin, Y. Wu, T.S. Huang, Modeling the constraints of human hand motion, in Workshop on Human Motion, 2000. Proceedings. (IEEE, 2000), pp. 121–126Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2017

Authors and Affiliations

  1. 1.Institute for Media Innovation (IMI)Nanyang Technological University (NTU)SingaporeSingapore

Personalised recommendations