Real-Time Psychophysiological Emotional State Estimation in Digital Gameplay Scenarios

  • Pedro A. Nogueira
  • Rui Rodrigues
  • Eugénio Oliveira
Part of the Communications in Computer and Information Science book series (CCIS, volume 383)


Despite the rising number of emotional state detection methods motivated by the popularity increase in affective computing techniques in recent years, they are yet faced with subject and domain transferability issues. In this paper, we present an improved methodology for modelling individuals’ emotional states in multimedia interactive environments. Our method relies on a two-layer classification process to classify Arousal and Valence based on four distinct physiological sensor inputs. The first classification layer uses several regression models to normalize each of the sensor inputs across participants and experimental conditions, while also correlating each input to either Arousal or Valence – effectively addressing the aforementioned transferability issues. The second classification layer then employs a residual sum of squares-based weighting scheme to merge the various regression outputs into one optimal Arousal/Valence classification in real-time, while maintaining a smooth prediction output. The presented method exhibited convincing accuracy ratings – 85% for Arousal and 78% for Valence –, which are only marginally worse than our previous non-real-time approach.


Affect recognition regression analysis affective computing games physiology galvanic skin response heart rate electromyography 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ermi, L., Mäyrä, F.: Fundamental components of the gameplay experience: Analysing immersion. In: Digital Games Research Association Conference: Changing Views - Worlds in Play (2005)Google Scholar
  2. 2.
    Chanel, G., Kronegg, J., Grandjean, D., Pun, T.: Emotion assessment: Arousal Evaluation using EEG’s and Peripheral Physiological Signals. In: Gunsel, B., Jain, A.K., Tekalp, A.M., Sankur, B. (eds.) MRCS 2006. LNCS, vol. 4105, pp. 530–537. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  3. 3.
    Mandryk, R., Atkins, M.: A fuzzy physiological approach for continuously modeling emotion during interaction with play technologies. International Journal of Human-Computer Studies 65, 329–347 (2007)CrossRefGoogle Scholar
  4. 4.
    Vinhas, V., Silva, D., Oliveira, E., Reis, L.: Biometric Emotion Assessment and Feedback in an Immersive Digital Environment. Social Robotics, 307–317 (2009)Google Scholar
  5. 5.
    Leon, E., Clarke, G., Callaghan, V., Sepulveda, F.: A user-independent real-time emotion recognition system for software agents in domestic environments. Engineering Applications of Artificial Intelligence 20, 337–345 (2007)CrossRefGoogle Scholar
  6. 6.
    Russel, J.A.: A Circumplex Model of Affect. Journal of Personality and Social Psychology 39, 1161–1178 (1980)CrossRefGoogle Scholar
  7. 7.
    Plutchik, R.: A General Psychoevolutionary Theory of Emotion. Emotion: Theory, Research, and Experience 1, 3–33 (1980)Google Scholar
  8. 8.
    Haag, A., Goronzy, S., Schaich, P., Williams, J.: Emotion recognition using bio-sensors: First steps towards an automatic system. In: André, E., Dybkjær, L., Minker, W., Heisterkamp, P. (eds.) ADS 2004. LNCS (LNAI), vol. 3068, pp. 36–48. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  9. 9.
    Lang, P.J., Bradley, M.M., Cuthbert, B.N.: International affective picture system (IAPS): Affective ratings of pictures and instruction manual. University of Florida, Gainesville (2008)Google Scholar
  10. 10.
    Stern, R.M., Ray, W.J., Quigley, K.S.: Psychophysiological recording. Oxford University Press, New York (2001)Google Scholar
  11. 11.
    Nogueira, P.A., Rodrigues, R., Oliveira, E., Nacke, L.E.: A Regression-based Method for Lightweight Emotional State Detection in Interactive Environments. In: Proceedings of the XVI Portuguese Conference on Artificial Intelligence (EPIA), Angra do Heroísmo, Açores, Portugal (to appear, 2013)Google Scholar
  12. 12.
    Drachen, A., Nacke, L.E., Yannakakis, G., Pedersen, A.L.: Correlation between Heart Rate, Electrodermal Activity and Player Experience in First-Person Shooter Games. In: Proc. of the 5th ACM SIGGRAPH Symposium on Video Games, pp. 49–54 (2010)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Pedro A. Nogueira
    • 1
  • Rui Rodrigues
    • 2
  • Eugénio Oliveira
    • 2
  1. 1.LIACC – Artificial Intelligence and Computer Science Lab.University of PortoPortugal
  2. 2.INESC TEC and Faculty of EngineeringUniversity of PortoPortugal

Personalised recommendations