Visual Interaction Including Biometrics Information for a Socially Assistive Robotic Platform

  • Pierluigi Carcagnì
  • Dario Cazzato
  • Marco Del CocoEmail author
  • Cosimo Distante
  • Marco Leo
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8927)


This work introduces biometrics as a way to improve human-robot interaction. In particular, gender and age estimation algorithms are used to provide awareness of the user biometrics to a humanoid robot (Aldebaran NAO), in order to properly react with a specific gender/age behavior. The system can also manage multiple persons at the same time, recognizing the age and gender of each participant. All the estimation algorithms employed have been validated through a k-fold test and successively practically tested in a real human-robot interaction environment, allowing for a better natural interaction. Our system is able to work at a frame rate of 13 fps with 640\(\times \)480 images taken from NAO’s embedded camera. The proposed application is well-suited for all assisted environments that consider the presence of a socially assistive robot like therapy with disable people, dementia, post-stroke rehabilitation, Alzheimer disease or autism.


Linear Discriminant Analysis Local Binary Pattern Humanoid Robot Face Detection Active Appearance Model 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
  2. 2.
  3. 3.
    Belhumeur, P., Hespanha, J., Kriegman, D.: Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence 19(7), 711–720 (1997)CrossRefGoogle Scholar
  4. 4.
    Bemelmans, R., Gelderblom, G.J., Jonker, P., De Witte, L.: Socially assistive robots in elderly care: A systematic review into effects and effectiveness. Journal of the American Medical Directors Association 13(2), 114–120 (2012)CrossRefGoogle Scholar
  5. 5.
    Boser, B.E., Guyon, I.M., Vapnik, V.N.: A training algorithm for optimal margin classifiers. In: Proceedings of the Fifth Annual Workshop on Computational Learning Theory, COLT 1992, pp. 144–152. ACM, New York (1992).
  6. 6.
    Brey, P.: Freedom and privacy in ambient intelligence. Ethics and Information Technology 7(3), 157–166 (2005)CrossRefGoogle Scholar
  7. 7.
    Brunelli, R., Poggio, T.: Hyberbf networks for gender classification (1995)Google Scholar
  8. 8.
    Carcagnì, P., Del Coco, M., Mazzeo, P.L., Testa, A., Distante, C.: Features descriptors for demographic estimation: a comparative study. In: Distante, C., Battiato, S., Cavallaro, A. (eds.) VAAM 2014. LNCS, vol. 8811, pp. 66–85. Springer, Heidelberg (2014) CrossRefGoogle Scholar
  9. 9.
    Castrillón, M., Déniz, O., Guerra, C., Hernández, M.: Encara2: real-time detection of multiple faces at different resolutions in video streams. Journal of Visual Communication and Image Representation 18(2), 130–140 (2007)CrossRefGoogle Scholar
  10. 10.
    Chang, C.C., Lin, C.J.: LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology 2, 27:1–27:27 (2011).
  11. 11.
    Chen, J., Shan, S., He, C., Zhao, G., Pietikainen, M., Chen, X., Gao, W.: Wld: A robust local image descriptor. IEEE Transactions on Pattern Analysis and Machine Intelligence 32(9), 1705–1720 (2010)CrossRefGoogle Scholar
  12. 12.
    Cortes, C., Vapnik, V.: Support-vector networks. Machine Learning 20(3), 273–297 (1995).
  13. 13.
    Cottrell, G.W., Metcalfe, J.: Empath: face, emotion, and gender recognition using holons. In: Advances in Neural Information Processing Systems, pp. 564–571 (1990)Google Scholar
  14. 14.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, vol. 1, pp. 886–893, June 2005Google Scholar
  15. 15.
    Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. Syst. Sci. 55(1), 119–139 (1997)CrossRefzbMATHMathSciNetGoogle Scholar
  16. 16.
    Fukai, H., Nishie, Y., Abiko, K., Mitsukura, Y., Fukumi, M., Tanaka, M.: An age estimation system on the aibo. In: International Conference on Control, Automation and Systems, ICCAS 2008, pp. 2551–2554, October 2008Google Scholar
  17. 17.
    Golomb, B.A., Lawrence, D.T., Sejnowski, T.J.: Sexnet: a neural network identifies sex from human faces. In: NIPS, pp. 572–579 (1990)Google Scholar
  18. 18.
    Abdi, H., Valentine, D., Edelman, B., O’Toole, A.J.: More about the difference between men and women: evidence from linear neural networks and the principal-component approach. Neural Comput. 7(6), 1160–1164 (1995)CrossRefGoogle Scholar
  19. 19.
    Hsu, C.W., Chang, C.C., Lin, C.J., et al.: A practical guide to support vector classification (2003)Google Scholar
  20. 20.
    Hsu, C.W., Lin, C.J.: A comparison of methods for multiclass support vector machines. IEEE Transactions on Neural Networks 13(2), 415–425 (2002)CrossRefGoogle Scholar
  21. 21.
    Jain, A.K., Dass, S.C., Nandakumar, K.: Soft biometric traits for personal recognition systems. In: Zhang, D., Jain, A.K. (eds.) ICBA 2004. LNCS, vol. 3072, pp. 731–738. Springer, Heidelberg (2004) CrossRefGoogle Scholar
  22. 22.
    Knerr, S., Personnaz, L., Dreyfus, G.: Single-layer learning revisited: a stepwise procedure for building and training a neural network. In: Soulié, F., Hérault, J. (eds.) Neurocomputing, NATO ASI Series, vol. 68, pp. 41–50. Springer, Berlin Heidelberg (1990).
  23. 23.
    Lee, M.W., Kwak, K.C.: Performance comparison of gender and age group recognition for human-robot interaction. International Journal of Advanced Computer Science & Applications 3(12) (2012)Google Scholar
  24. 24.
    Liu, L., Liu, J., Cheng, J.: Age-group classification of facial images. In: 2012 11th International Conference on Machine Learning and Applications (ICMLA), vol. 1, pp. 693–696, December 2012Google Scholar
  25. 25.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60(2), 91–110 (2004)CrossRefGoogle Scholar
  26. 26.
    Luo, R., Chang, L.W., Chou, S.C.: Human age classification using appearance images for human-robot interaction. In: Industrial Electronics Society, IECON 2013–39th Annual Conference of the IEEE, pp. 2426–2431, November 2013Google Scholar
  27. 27.
    Luo, R.C., Wu, X.: Real-time gender recognition based on 3d human body shape for human-robot interaction. In: Proceedings of the 2014 ACM/IEEE International Conference on Human-Robot Interaction, pp. 236–237. ACM (2014)Google Scholar
  28. 28.
    Lyons, M.J., Budynek, J., Plante, A., Akamatsu, S.: Classifying facial attributes using a 2-d gabor wavelet representation and discriminant analysis. In: Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 202–207 (2000)Google Scholar
  29. 29.
    Mäkinen, E., Raisamo, R.: An experimental comparison of gender classification methods. Pattern Recognition Letters 29(10), 1544–1556 (2008). CrossRefGoogle Scholar
  30. 30.
    McColl, D., Zhang, Z., Nejat, G.: Human body pose interpretation and classification for social human-robot interaction. International Journal of Social Robotics 3(3), 313–332 (2011)CrossRefGoogle Scholar
  31. 31.
    Moore, D.: Computers and people with autism. Asperger Syndrome, pp. 20–21 (1998)Google Scholar
  32. 32.
    Phillips, P., Moon, H., Rizvi, S., Rauss, P.: The feret evaluation methodology for face-recognition algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(10), 1090–1104 (2000)CrossRefGoogle Scholar
  33. 33.
    Saatci, Y., Town, C.: Cascaded classification of gender and facial expression using active appearance models. In: 7th International Conference on Automatic Face and Gesture Recognition, FGR 2006, pp. 393–398, April 2006Google Scholar
  34. 34.
    Sakarkaya, M., Yanbol, F., Kurt, Z.: Comparison of several classification algorithms for gender recognition from face images. In: 2012 IEEE 16th International Conference on Intelligent Engineering Systems (INES), pp. 97–101, June 2012Google Scholar
  35. 35.
    Sun, N., Zheng, W., Sun, C., Zou, C., Zhao, L.: Gender classification based on boosting local binary pattern. In: Wang, J., Yi, Z., Żurada, J.M., Lu, B.-L., Yin, H. (eds.) ISNN 2006. LNCS, vol. 3972, pp. 194–201. Springer, Heidelberg (2006) CrossRefGoogle Scholar
  36. 36.
    Sun, Z., Bebis, G., Yuan, X., Louis, S.J.: Genetic feature subset selection for gender classification: a comparison study. In: IEEE Workshop on Applications of Computer Vision, pp. 165–170 (2002)Google Scholar
  37. 37.
    Tapus, A., Maja, M., et al.: Towards socially assistive robotics. International Journal of the Robotics Society of Japan (JRSJ) 24(5), 576–578 (2006)CrossRefGoogle Scholar
  38. 38.
    Tapus, A., Ţăpuş, C., Matarić, M.J.: User-robot personality matching and assistive robot behavior adaptation for post-stroke rehabilitation therapy. Intelligent Service Robotics 1(2), 169–183 (2008)CrossRefGoogle Scholar
  39. 39.
    Tapus, A., Tapus, C., Mataric, M.J.: The use of socially assistive robots in the design of intelligent cognitive therapies for people with dementia. In: IEEE International Conference on Rehabilitation Robotics, ICORR 2009, pp. 924–929. IEEE (2009)Google Scholar
  40. 40.
    Ullah, I., Hussain, M., Muhammad, G., Aboalsamh, H., Bebis, G., Mirza, A.: Gender recognition from face images with local wld descriptor. In: 2012 19th International Conference on Systems, Signals and Image Processing (IWSSIP), pp. 417–420, April 2012Google Scholar
  41. 41.
    Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2001, vol. 1, pp. I-511. IEEE (2001)Google Scholar
  42. 42.
    Walker, J.H., Sproull, L., Subramani, R.: Using a human face in an interface. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 85–91. ACM (1994)Google Scholar
  43. 43.
    Wayman, J.L.: Large-scale civilian biometrics system - issues and feasibility. In: Proceedings of the CardTech/SecureTech Government, Washington DC (1997)Google Scholar
  44. 44.
    Ylioinas, J., Hadid, A., Pietikainen, M.: Age classification in unconstrained conditions using lbp variants. In: 2012 21st International Conference on Pattern Recognition (ICPR), pp. 1257–1260, November 2012Google Scholar
  45. 45.
    Ylioinas, J., Hadid, A., Hong, X., Pietikäinen, M.: Age estimation using local binary pattern kernel density estimate. In: Petrosino, A. (ed.) ICIAP 2013, Part I. LNCS, vol. 8156, pp. 141–150. Springer, Heidelberg (2013) CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Pierluigi Carcagnì
    • 1
  • Dario Cazzato
    • 1
  • Marco Del Coco
    • 1
    Email author
  • Cosimo Distante
    • 1
  • Marco Leo
    • 1
  1. 1.National Research Council of Italy - Institute of OpticsArnesanoItaly

Personalised recommendations