Automatic Initialization for Facial Analysis in Interactive Robotics

  • Ahmad Rabie
  • Christian Lang
  • Marc Hanheide
  • Modesto Castrillón-Santana
  • Gerhard Sagerer
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5008)


The human face plays an important role in communication as it allows to discern different interaction partners and provides non-verbal feedback. In this paper, we present a soft real-time vision system that enables an interactive robot to analyze faces of interaction partners not only to identify them, but also to recognize their respective facial expressions as a dialog-controlling non-verbal cue. In order to assure applicability in real world environments, a robust detection scheme is presented which detects faces and basic facial features such as the position of the mouth, nose, and eyes. Based on these detected features, facial parameters are extracted using active appearance models (AAMs) and conveyed to support vector machine (SVM) classifiers to identify both persons and facial expressions. This paper focuses on four different initialization methods for determining the initial shape for the AAM algorithm and their particular performance in two different classification tasks with respect to either the facial expression DaFEx database and to the real world data obtained from a robot’s point of view.


facial analysis initialization aam face detection 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Darwin, C., Ekman, P.: The Expression of the Emotions in Man and Animals, 3rd edn. Oxford University Press, Oxford (1998)Google Scholar
  2. 2.
    Barkhuysen, P., Krahmer, E., Swerts, M.: Problem detection in human-machine interactions based on facial expressions of users. Speech Communication 45(3), 343–359 (2005)CrossRefGoogle Scholar
  3. 3.
    Haasch, A., Hohenner, S., Hüwel, S., Kleinehagenbrock, M., Lang, S., Toptsis, I., Fink, G.A., Fritsch, J., Wrede, B., Sagerer, G.: Biron – the bielefeld robot companion, May 2004, pp. 27–32. Fraunhofer IRB Verlag, Stuttgart, Germany (2004)Google Scholar
  4. 4.
    Fritsch, J., Kleinehagenbrock, M., Lang, S., Plötz, T., Fink, G.A., Sagerer, G.: Multi-modal anchoring for human-robot-interaction. Robotics and Autonomous Systems 43(2–3), 133–147 (2003)CrossRefGoogle Scholar
  5. 5.
    Li, S., Wrede, B.: Why and how to model multi-modal interaction for a mobile robot companion. In: AAAI Technical Report SS-07-04: Interaction Challenges for Intelligent Assistants, pp. 71–79. Stanford, AAAI Press, Menlo Park (2007)Google Scholar
  6. 6.
    Chellappa, R., Wilson, C., Sirohey, S.: Human and machine recognition of faces: A survey. Proceedings IEEE 83(5), 705–740 (1995)CrossRefGoogle Scholar
  7. 7.
    Zhao, W., Chellappa, R., Phillips, P.J., Rosenfeld, A.: Face recognition: A literature survey. Association for Computing Machinery 35(4), 399–458 (2003)Google Scholar
  8. 8.
    Phillips, P.J., Flynn, P.J., Scruggs, T., Bowyer, K.W., Chang, J., Hoffman, K., Marques, J., Min, J., Worek, W.: Overview of the face recognition grand challenge. In: IEEE Conference on Computer Vision and Pattern Recognition (2005)Google Scholar
  9. 9.
    Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. PAMI 23(6), 681–685 (2001)Google Scholar
  10. 10.
    Sattar, A., Aidarous, Y., Gallou, S.L., Seguier, R.: Face alignment by 2.5d active appearance model optimized by simplex. In: ICVS (2007)Google Scholar
  11. 11.
    Wong, C., Kortenkamp, D., Speich, M.: A mobile robot that recognizes people. In: Proc. Int. Conf. on Tools with Artificial Intelligence, p. 346. IEEE Computer Society, Washington, DC, USA (1995)CrossRefGoogle Scholar
  12. 12.
    Matsusaka, Y., Tojo, T., Kubota, S., Furukawa, K., Tamiya, D., Fujie, S., Koabyashi, T.: Multi-person conversation via multi-modal interface: A robot who communicate with multi-user. In: Proc. Eurospeech, pp. 1723–1726 (1999)Google Scholar
  13. 13.
    Sakaue, F., Kobayashi, M., Migita, T., Shakunaga, T.: A real-life test of face recognition system for dialogue interface robot in ubiquitous environments. In: ICPR 2006: Proceedings of the 18th International Conference on Pattern Recognition, pp. 1155–1160. IEEE Computer Society, Washington, DC, USA (2006)Google Scholar
  14. 14.
    Huang, X., Li, S.Z., Wang, Y.: Statistical learning of evaluation function for ASM/AAM image alignment. In: Maltoni, D., Jain, A.K. (eds.) BioAW 2004. LNCS, vol. 3087, Springer, Heidelberg (2004)Google Scholar
  15. 15.
    Vapnik, V.: The nature of statistical learning theory. Springer, New York (1995)zbMATHGoogle Scholar
  16. 16.
    Li, S.Z., Zhu, L., Zhang, Z., Blake, A., Zhang, H., Shum, H.: Statistical learning of multi-view face detection. In: European Conference Computer Vision, pp. 67–81 (2002)Google Scholar
  17. 17.
    Schneiderman, H., Kanade, T.: A statistical method for 3d object detection applied to faces and cars. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1746–1759 (2000)Google Scholar
  18. 18.
    Viola, P., Jones, M.J.: Robust real-time face detection. International Journal of Computer Vision 57(2), 151–173 (2004)CrossRefGoogle Scholar
  19. 19.
    Lienhart, R., Kuranov, A., Pisarevsky, V.: Empirical analysis of detection cascades of boosted classifiers for rapid object detection. In: Michaelis, B., Krell, G. (eds.) DAGM 2003. LNCS, vol. 2781, pp. 297–304. Springer, Heidelberg (2003)Google Scholar
  20. 20.
    Kruppa, H., Castrillón Santana, M., Schiele, B.: Fast and robust face finding via local context. In: Joint IEEE Internacional Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance (VS-PETS), October 2003, pp. 157–164 (2003)Google Scholar
  21. 21.
    Battocchi, A., Pianesi, F., Goren-Bar, D.: Dafex, a database of kinetic facial expression. In: ICMI 2005 Doctoral Spotlight and Demo Proceedings, pp. 49–51 (2005)Google Scholar
  22. 22.
    Castrillón Santana, M., Déniz Suárez, O., Hernández Tejera, M., Guerra Artal, C.: ENCARA2: Real-time detection of multiple faces at different resolutions in video streams. Journal of Visual Communication and Image Representation, 130–140 (April 2007)Google Scholar
  23. 23.
    Spacek, L.: Collection of Facial Images WWW (September 2007), Available:

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Ahmad Rabie
    • 1
  • Christian Lang
    • 1
  • Marc Hanheide
    • 1
  • Modesto Castrillón-Santana
    • 2
  • Gerhard Sagerer
    • 1
  1. 1.Applied Computer Science Group, Fac. of Techn.Bielefeld UniversityGermany
  2. 2.SIANIUniversity of Las Palmas de Gran CanariaSpain

Personalised recommendations