Smile Detection for User Interfaces

  • O. Deniz
  • M. Castrillon
  • J. Lorenzo
  • L. Anton
  • G. Bueno
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5359)


Perceptual User Interfaces (PUIs) aim at facilitating human-computer interaction with the aid of human-like capacities (computer vision, speech recognition, etc.). In PUIs, the human face is a central element, since it conveys not only identity but also other important information, particularly with respect to the user’s mood or emotional state. This paper describes both a face detector and a smile detector for PUIs. Both are suitable for real-time interaction. The face detector provides eye, mouth and nose locations in frontal or nearly-frontal poses, whereas the smile detector is able to give a smile intensity measure. Experiments confirm that they are competitive with respect to extant detectors. These two detectors are used in an unobtrusive application that allows to interact with an Instant Messaging (IM) client.


Facial Expression Face Detection Facial Expression Recognition Instant Messaging Duchenne Smile 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Hjelmas, E., Low, B.K.: Face detection: A survey. Computer Vision and Image Understanding 83, 236–274 (2001)CrossRefzbMATHGoogle Scholar
  2. 2.
    Yang, M.H., Kriegman, D., Ahuja, N.: Detecting faces in images: A survey. Transactions on Pattern Analysis and Machine Intelligence 24, 34–58 (2002)CrossRefGoogle Scholar
  3. 3.
    Castrillón, M., Déniz, O., Hernández, M., Guerra, C.: ENCARA2: Real-time detection of multiple faces at different resolutions in video streams. Journal of Visual Communication and Image Representation, 130–140 (2007)Google Scholar
  4. 4.
    Viola, P., Jones, M.: Robust real-time face detection. IJCV 57, 151–173 (2004)CrossRefGoogle Scholar
  5. 5.
    Hewitt, R.: Seeing with OpenCV, part 2. Servo, 48–52 (2007)Google Scholar
  6. 6.
    Reimondo, A.: OpenCV Swiki. Last accessed: January 2008 (2008)Google Scholar
  7. 7.
    Battocchi, A., Pianesi, F.: Dafex: Un database di espressioni facciali dinamiche. In: SLI-GSCP Workshop Comunicazione Parlata e Manifestazione delle Emozioni (2004)Google Scholar
  8. 8. Smile detection technology in camera. Last accessed: January 2008 (2008)Google Scholar
  9. 9. Sony’s Cyber-shot T200 gets its first review. Last accessed: January 2008 (2008)Google Scholar
  10. 10.
    Omron Corp.: Omron OKAO vision system. Last accessed: January 2008 (2008)Google Scholar
  11. 11.
    Fasel, B., Luettin, J.: Automatic facial expression analysis: a survey. Pattern Recognition 36, 259–275 (2003)CrossRefzbMATHGoogle Scholar
  12. 12.
    Pantic, M., Member, S., Rothkrantz, L.J.M.: Automatic analysis of facial expressions: The state of the art. IEEE Transactions on Pattern Analysis and Machine Intelligence 22, 1424–1445 (2000)CrossRefGoogle Scholar
  13. 13.
    Ito, A., Wang, X., Suzuki, M., Makino, S.: Smile and laughter recognition using speech processing and face recognition from conversation video. In: Procs. of the 2005 IEEE Int. Conf. on Cyberworlds, CW 2005 (2005)Google Scholar
  14. 14.
    Shinohara, Y., Otsu, N.: Facial expression recognition using Fisher weight maps. In: Procs. of the IEEE Int. Conf. on AFGR (2004)Google Scholar
  15. 15.
    Kowalik, U., Aoki, T., Yasuda, H.: Broaference - a next generation multimedia terminal providing direct feedback on audience’s satisfaction level. In: INTERACT, pp. 974–977 (2005)Google Scholar
  16. 16.
    Kowalik, U., Aoki, T., Yasuda, H.: Using automatic facial expression classification for contents indexing based on the emotional component. In: EUC, pp. 519–528 (2006)Google Scholar
  17. 17.
    Lyons, M., Akamatsu, S., Kamachi, M., Gyoba, J.: Coding facial expressions with gabor wavelets. In: Procs. of the Third IEEE International Conference on Automatic Face and Gesture Recognition (1998)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • O. Deniz
    • 1
  • M. Castrillon
    • 2
  • J. Lorenzo
    • 2
  • L. Anton
    • 2
  • G. Bueno
    • 1
  1. 1.Universidad de Castilla-La Mancha, E.T.S.I.ISpain
  2. 2.Dpto. Informatica y Sistemas,Edificio de InformaticaUniversidad de Las Palmas de Gran Canaria35017 Las PalmasSpain

Personalised recommendations