An Affective Inference Model Based on Facial Expression Analysis

  • Paula Andrea Lago
  • Claudia Lucía Jiménez-Guarín
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8276)


Ubiquitous computing aims to reduce the complexity of interacting with computing devices. Analyzing psychological user states helps in this task. In this work we propose a computational model for analyzing psychological user states that takes into account three emotions that have not been explored deeply: interest, boredom and confusion. The model was constructed based on a video analysis of 35 engineering students during two class activities, all of whom reported the emotions they were feeling as they performed the activity. From the video, facial expressions features were extracted and matched with the emotion reported. This allowed us to construct patterns of facial expression and emotion inference rules. Our model is based on distances and indicators of change with respect to a user baseline, which allows the model to adapt to different users, moods and personal manners. Recognizing these emotions can be used as an implicit feedback in different systems.


analysis of psychological user states affective computing facial expression implicit feedback computational model of emotion interest boredom confusion 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ekman, P.: Basic Emotions. de Handbook of Cognition and Emotion, pp. 45–60. John Wiley & Sons, Ltd. (2005)Google Scholar
  2. 2.
    Lago, P., Jimenez, C.: MAGPIE:Pervasive Environment with Emotional Awareness for Managing Information in the Context of Collaborative Communities. In: Conference Proceedings Ucami 2011 – 5th International Symposium on Ubiquitous Computing And Ambient Intelligence, Riviera Maya, Mexico (2011) ISBN 978-84-694-9677-0Google Scholar
  3. 3.
    Höök, K.: Affective Computing. In: Soegaard, M., Dam, R.F. (eds.) The Encyclopedia of Human-Computer Interaction, 2nd edn. The Interaction Design Foundation (2013)Google Scholar
  4. 4.
    Picard, R.W.: Affective Computing: From Laughter to IEEE. IEEE Transactions on Affective Computing 1(1), 11–17 (2010)CrossRefGoogle Scholar
  5. 5.
    Bradley, M.M., Lang, P.J.: Measuring emotion: behavior, feeling, and physiology. In: Lane, R.D., Nadel, L. (eds.) Cognitive Neuroscience of Emotion, pp. 242–276. Oxford University Press (2000)Google Scholar
  6. 6.
    Ekman, P.: Emotions Revealed. Times Books, New York (2003)Google Scholar
  7. 7.
    Ekman, P.: Facial Action Coding System: A Technique for the Measurement of Facial Movement. Consulting Psychologists Press (1978)Google Scholar
  8. 8.
    Warren, G., Schertler, E., Bull, P.: Detecting Deception from Emotional and Unemotional Cues. Journal of Nonverbal BehaviorGoogle Scholar
  9. 9.
    Fasel, B., Luettin, J.: Automatic facial expression analysis: a survey. Pattern Recognition 36(1), 259–275 (2003)CrossRefGoogle Scholar
  10. 10.
    Beristain, A., Graña, M.: Emotion Recognition Based on The Analysis of Facial Expresions: A Survey. New Mathematics and Natural Computation (NMNC) 5(2), 513–534 (2009)CrossRefGoogle Scholar
  11. 11.
    Chang, C.-Y., Tsai, J.-S., Wang, C.-J., Chung, P.-C.: Emotion recognition with consideration of facial expression and physiological signals (2009)Google Scholar
  12. 12.
    Cerezo, E., Hupont, I., Baldassarri, S., Ballano, S.: Emotional facial sensing and multimodal fusion in a continuous 2D affective space. Journal of Ambient Intelligence and Humanized Computing, 1–16Google Scholar
  13. 13.
    Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I.: The Extended Cohn-Kande Dataset (CK+): A complete facial expression dataset for action unit and emotion-specified expression. In: Third IEEE Workshop on CVPR for Human Communicative Behavior Analysis (2010)Google Scholar
  14. 14.
    Aifanti, N., Papachristou, C., Delopoulos, A.: The MUG Facial Expression Database. In: 11th Int. Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS), Proc. 11th Int. Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS), Desenzano, Italy (2010)Google Scholar
  15. 15.
    Sim, T., Baker, S., Bsat, M.: The CMU Pose, Illumination, and Expression Database. IEEE Transactions on Pattern Analysis and Machine Intelligence 25(12), 1615–1618 (2003)CrossRefGoogle Scholar
  16. 16.
    Valstar, M.F., Pantic, M.: Induced Disgust, Happiness and Surprise: an Addition to the MMI Facial Expression Database. In: Workshop on Emotional Corpora (2010)Google Scholar
  17. 17.

Copyright information

© Springer International Publishing Switzerland 2013

Authors and Affiliations

  • Paula Andrea Lago
    • 1
  • Claudia Lucía Jiménez-Guarín
    • 1
  1. 1.Department of Systems Engineering and ComputationUniversidad de Los AndesBogotáColombia

Personalised recommendations