CONTROLO 2016 pp 385-395 | Cite as

Happiness and Sadness Recognition System—Preliminary Results with an Intel RealSense 3D Sensor

  • Vinícius Silva
  • Filomena Soares
  • João Sena EstevesEmail author
  • Joana Figueiredo
  • Cristina Santos
  • Ana Paula Pereira
Conference paper
Part of the Lecture Notes in Electrical Engineering book series (LNEE, volume 402)


Systems and devices that can recognize human affects have been in development for a considerable time. Facial features are usually extracted by using video data or a Microsoft Kinect sensor. The present paper proposes an emotion recognition system that uses the recent Intel RealSense 3D sensor, whose reliability and validity in the field of emotion recognition has not yet been studied. This preliminary work focus on happiness and sadness. The system extracts the user’s facial Action Units and head motion data. Then, it uses a Support Vector Machine to automatically classify the emotion expressed by the user. The results point out the adequacy of Intel RealSense for facial features extraction in emotion recognition systems as well as the importance of determining head motion when recognizing sadness.


Happiness and sadness recognition system Intel RealSense Support vector machine 



The authors would like to express their acknowledgments to all students for their voluntary co-operation. This work has been supported by COMPETE: POCI-01-0145-FEDER-007043 and FCT—Fundação para a Ciência e Tecnologia within the Project Scope: UID/CEC/00319/2013.


  1. 1.
    Jham, V., Puri, S.: Cases on consumer-centric marketing management. IGI Global (2014)Google Scholar
  2. 2.
    D’Mello, S., Calvo, R.: Beyond the basic emotions: what should affective computing compute? Abstr. Hum. Factors Comput. 2287–2294 (2013)Google Scholar
  3. 3.
    Rani, P., Sarkar, N.: Operator engagement detection for robot behavior adaptation. Int. J. Adv. Robot. Syst. 4, 1–12 (2007)CrossRefGoogle Scholar
  4. 4.
    Michel, P., El Kaliouby, R.: Real time facial expression recognition in video using support vector machines. In: Proceedings of 5th International Conference on Multimodal Interfaces—ICMI’03, p. 258 (2003)Google Scholar
  5. 5.
    Youssef, E., Aly, S.F., Ibrahim, A.S., Abbott, L.: Auto-optimized multimodal expression recognition framework using 3D kinect data for ASD therapeutic aid. Int. J. Model. Optim. 3(2), 112–115 (2013)Google Scholar
  6. 6.
    Alabbasi, H.A., Moldoveanu, P., Moldoveanu, A.: Real time facial emotion recognition using kinect V2 sensor. IOSR J. Comput. Eng. Ver. II 17(3), 2278–2661 (2015)Google Scholar
  7. 7.
  8. 8.
    RealSense—Get Started with RealSense Technology|Intel® Developer Zone. Accessed 15 Jan 2016
  9. 9.
    Ekman, P., Rosenberg, E.: What the face reveals (2005)Google Scholar
  10. 10.
    FACS (Facial Action Coding System). Accessed 25 Jan 2016
  11. 11.
    Sacavém, A., Wezowski, K., Wezowski, P., Linguagem, A.: Corporal Revela o que as Palavras Escondem. Top Books (2014) (in Portuguese)Google Scholar
  12. 12.
  13. 13.
    camera2.png (imagem PNG, 384 x 213 pixeis). Accessed 04 Feb 2016
  14. 14.
  15. 15.
    Burges, C.C.: A tutorial on support vector machines for pattern recognition. Data Min. Knowl. Discov. 2(2), 121–167 (1998)CrossRefGoogle Scholar
  16. 16.
    Support Vector Machines (SVM)—MATLAB & Simulink. Accessed 20 Jan 2016
  17. 17.
    Michel, P., El Kaliouby, R.: Facial expression recognition using support vector machines (2000)Google Scholar
  18. 18.
    Cross-Validation—MATLAB. Accessed 20 Jan 2016

Copyright information

© Springer International Publishing Switzerland 2017

Authors and Affiliations

  • Vinícius Silva
    • 1
  • Filomena Soares
    • 2
  • João Sena Esteves
    • 2
    Email author
  • Joana Figueiredo
    • 1
  • Cristina Santos
    • 3
  • Ana Paula Pereira
    • 4
  1. 1.Industrial Electronics Department, School of EngineeringUniversity of MinhoGuimarãesPortugal
  2. 2.Industrial Electronics Department, R&D Centre Algoritmi, School of EngineeringUniversity of MinhoGuimarãesPortugal
  3. 3.Industrial Electronics Department, R&D Centre MEMS, School of EngineeringUniversity of MinhoGuimarãesPortugal
  4. 4.Education Research Center, Institute of EducationUniversity of MinhoBragaPortugal

Personalised recommendations