Neural Computing and Applications

, Volume 25, Issue 6, pp 1467–1477 | Cite as

Fuzzy model of dominance emotions in affective computing

  • Kaveh BakhtiyariEmail author
  • Hafizah Husain
Original Article


To date, most of the human emotion recognition systems are intended to sense the emotions and their dominance individually. This paper discusses a fuzzy model for multilevel affective computing based on the dominance dimensional model of emotions. This model can detect any other possible emotions simultaneously at the time of recognition. One hundred and thirty volunteers from various countries with different cultural backgrounds were selected to record their emotional states. These volunteers have been selected from various races and different geographical locations. Twenty-seven different emotions with their strengths in a scale of 5 were questioned through a survey. Recorded emotions were analyzed with the other possible emotions and their levels of dominance to build the fuzzy model. Then this model was integrated into a fuzzy emotion recognition system using three input devices of mouse, keyboard and the touch screen display. Support vector machine classifier detected the other possible emotions of the users along with the directly sensed emotion. The binary system (non-fuzzy) sensed emotions with an incredible accuracy of 93 %. However, it only could sense limited emotions. By integrating this model, the system was able to detect more possible emotions at a time with slightly lower recognition accuracy of 86 %. The recorded false positive rates of this model for four emotions were measured at 16.7 %. The resulted accuracy and its false positive rate are among the top three accurate human emotion recognition (affective computing) systems.


Fuzzy emotion Dominance emotion Multilevel emotion Human–computer interaction Affective computing 



We thank the students of Universiti Kebangsaan Malaysia (The National University of Malaysia) and University of Duisburg-Essen for their collaborations and participations in this research and emotions’ survey. Special thanks to Mona Taghavi for her initial review and comments on this paper.


  1. 1.
    Cowie R, Douglas-Cowie E, Tsapatsoulis N, Votsis G, Kollias S, Fellenz W, Taylor JG (2001) Emotion recognition in human-computer interaction. IEEE Signal Process Mag 18(1):32–80CrossRefGoogle Scholar
  2. 2.
    Epp C, Lippold M, Mandryk RL (2011) Identifying emotional states using keystroke dynamics. Paper presented at the Proceedings of the 2011 annual conference on Human factors in computing systems, Vancouver, BC, CanadaGoogle Scholar
  3. 3.
    Liu Y, Sourina O, Nguyen MK (2010) Real-time EEG-based human emotion recognition and visualization. In: International conference on cyberworlds, Alberta, Canada. IEEE, pp 262–269Google Scholar
  4. 4.
    Milanova M, Sirakov N (2008) Recognition of emotional states in natural human–computer interaction. In: IEEE international symposium on signal processing and information technology (ISSPIT), Ho Chi Minh City, Vietnam. IEEE, pp 186–191Google Scholar
  5. 5.
    Bashir MG, Nagarajan R, Hazry D (2010) Facial emotion detection using GPSO and Lucas-Kanade algorithms. Paper presented at the international conference on computer and communication engineering (ICCCE)Google Scholar
  6. 6.
    Kao ECC, Liu CC, Yang TH, Hsieh CT, Soo VW (2009) Towards text-based emotion detection. In: International conference on information management and engineering (ICIME), Kuala Lumpur, Malaysia. IEEE, pp 70–74Google Scholar
  7. 7.
    Amarakeerthi S, Ranaweera R, Cohen M (2010) Speech-based emotion characterization using postures and gestures in CVEs. In: International conference on cyberworlds, Alberta, Canada. IEEE, pp 72–76Google Scholar
  8. 8.
    Bakhtiyari K, Taghavi M, Husain H (2014) Implementation of emotional-aware computer systems using typical input devices. In: Nguyen NT, Attachoo B, Trawiński B, Somboonviwat K (eds) 6th Asian conference on intelligent information and database systems, Bangkok, Thailand, 7-9 April 2014. Springer International Publishing, pp 364–374. doi: 10.1007/978-3-319-05476-6_37
  9. 9.
    Strongman KT (2003) The psychology of emotion. Department of Psychology, University of Canterbury, 5th edn. Wiley, ChristchurchGoogle Scholar
  10. 10.
    Derks D, Fischer AH, Bos AER (2008) The role of emotion in computer-mediated communication: a review. Comput Hum Behav 24(3):766–785. doi: 10.1016/j.chb.2007.04.004 CrossRefGoogle Scholar
  11. 11.
    Mehrabian A (1996) Pleasure-arousal-dominance: a general framework for describing and measuring individual differences in temperament. Curr Psychol 14(4):261–292. doi: 10.1007/BF02686918 MathSciNetCrossRefGoogle Scholar
  12. 12.
    Griffiths PE (2003) Basic emotions, complex emotions, Machiavellian emotions. R Inst Philos Suppl 52:28. doi: 10.1017/S1358246100007888 Google Scholar
  13. 13.
    Gendron M, Barrett LF (2009) Reconstructing the past: a century of ideas about emotion in psychology. Int Soc Res Emotion 1(4):24. doi: 10.1177/1754073909338877 Google Scholar
  14. 14.
    Hartel CEJ, Ashkanasy NM, Zerbe WJ (2007) Functionality, intentionality and morality, vol 3. Research on emotion in organization, 1st edn. Elsevier, AmsterdamGoogle Scholar
  15. 15.
    Lewis M, Haviland-Jones JM, Barrett LF (2010) The cultural psychology of the emotions—ancient and renewed. In: Richard A. Shweder JH, Horton R, Joseph C (eds) Handbook of emotions, 3rd edn. Guilford Press, New York, p 19Google Scholar
  16. 16.
    Kao CY, Fahn CS (2012) A design of face detection and facial expression recognition techniques based on boosting schema. Appl Mech Mater 121:617–621Google Scholar
  17. 17.
    Konar A, Chakraborty A, Halder A, Mandal R, Janarthanan R (2012) Interval type-2 fuzzy model for emotion recognition from facial expression. Percept Mach Intell 114–121Google Scholar
  18. 18.
    Glowinski D, Camurri A, Volpe G, Dael N, Scherer K (2008) Technique for automatic emotion recognition by body gesture analysis. In: IEEE computer society conference on computer vision and pattern recognition workshops, Providence, RI, USAGoogle Scholar
  19. 19.
    Monrose F, Rubin AD (2000) Keystroke dynamics as a biometric for authentication. Futur Gener Comput Syst 16(4):351–359CrossRefGoogle Scholar
  20. 20.
    Li H, Pang N, Guo S, Wang H (2008) Research on textual emotion recognition incorporating personality factor. In: IEEE international conference on robotics and biomimetics, Sanya, China. IEEE, pp 2222–2227Google Scholar
  21. 21.
    Xiao Z, Dellandrea E, Dou W, Chen L (2007) Automatic hierarchical classification of emotional speech. In: Ninth IEEE international symposium on multimedia workshops, pp 291–296Google Scholar
  22. 22.
    Klecková J (2009) Important nonverbal attributes for spontaneous speech recognition. In: Fourth international conference on systems. IEEE, pp 13–16Google Scholar
  23. 23.
    Pittermann J, Schmitt A, El Sayed NF (2008) Integrating linguistic cues into speech-based emotion recognition. In: 4th International Conference on Intelligent Environments, University of Washington, Seattle, USA. IET, pp 1–4Google Scholar
  24. 24.
    Wang Y, Guan L (2008) Recognizing human emotional state from audiovisual signals*. IEEE Trans Multimed 10(5):936–946CrossRefGoogle Scholar
  25. 25.
    Huang T (2008) Audio-visual human computer interface. In: IEEE international symposium on consumer electronics, University of Illinois, USA. IEEEGoogle Scholar
  26. 26.
    Guangying Y, Shanxiao Y (2010) Emotion recognition of electromyography based on support vector machine. In: Third international symposium on intelligent information technology and security informatics, Jinggangshan, China. IEEE, pp 298–301Google Scholar
  27. 27.
    Schaaff K, Schultz T (2009) Towards emotion recognition from electroencephalographic signals. In: Affective computing and intelligent interaction and workshops (ACII), Memphis, TN, USA. IEEE, pp 1–6Google Scholar
  28. 28.
    Schuller B, Rigoll G, Lang M (2004) Emotion recognition in the manual interaction with graphical user interfaces. In: IEEE international conference on multimedia and expo. IEEE, pp 1215–1218Google Scholar
  29. 29.
    Seniuk A, Blostein D (2009) Pen acoustic emissions for text and gesture recognition. In: 10th international conference on document analysis and recognition. ACM, pp 872–876Google Scholar
  30. 30.
    Wang R, Fang B (2008) Affective computing and biometrics based HCI surveillance system. In: International symposium on information science and engineering, China. IEEE, pp 192–195Google Scholar
  31. 31.
    Hertenstein MJ, Keltner D, App B, Bulleit BA, Jaskolka AR (2006) Touch communicates distinct emotions. Emotion 6(3):528CrossRefGoogle Scholar
  32. 32.
    Jaimes A, Sebe N (2007) Multimodal human–computer interaction: a survey. Comput Vis Image Underst 108(1–2):116–134. doi: 10.1016/j.cviu.2006.10.019
  33. 33.
    Watson D, Clark LA, Tellegen A (1988) Development and validation of brief measures of positive and negative affect: the PANAS scales. J Personal Soc Psychol 54(6):1063–1070. doi: 10.1037/0022-3514.54.6.1063 CrossRefGoogle Scholar
  34. 34.
    Yusoff N, Salim S (2010) SCOUT and affective interaction design: Evaluating physiological signals for usability in emotional processing. In: 2nd international conference on computer engineering and technology (ICCET). IEEE, pp V1-201-V201-205Google Scholar
  35. 35.
    Kong X, Yang Y (2009) Measuring emotions in interactive contexts. In: IEEE 10th international conference on computer-aided industrial design and conceptual design. IEEE, pp 526–530Google Scholar
  36. 36.
    Mower E, Mataric M, Narayanan S (2010) A framework for automatic human emotion classification using emotion profiles. IEEE Trans Audio Speech Lang Process 99:1Google Scholar
  37. 37.
    Rajaraman A, Ullman J (2011) Recommendation systems. In: Mining of massive datasets. Cambridge University Press, CambridgeGoogle Scholar
  38. 38.
    Taghavi M, Bakhtiyari K, Scavino E (2013) Agent-based computational investing recommender system. Paper presented at the Proceedings of the 7th ACM conference on recommender systems, Hong Kong, China. doi: 10.1145/2507157.2508072

Copyright information

© Springer-Verlag London 2014

Authors and Affiliations

  1. 1.Department of Computer and Cognitive Science, Faculty of EngineeringUniversity of Duisburg-EssenDuisburgGermany
  2. 2.Department of Electrical, Electronics and Systems Engineering, Faculty of Engineering and Built EnvironmentUniversiti Kebangsaan Malaysia (UKM)BangiMalaysia

Personalised recommendations