Real-Time Psychophysiological Emotional State Estimation in Digital Gameplay Scenarios
Despite the rising number of emotional state detection methods motivated by the popularity increase in affective computing techniques in recent years, they are yet faced with subject and domain transferability issues. In this paper, we present an improved methodology for modelling individuals’ emotional states in multimedia interactive environments. Our method relies on a two-layer classification process to classify Arousal and Valence based on four distinct physiological sensor inputs. The first classification layer uses several regression models to normalize each of the sensor inputs across participants and experimental conditions, while also correlating each input to either Arousal or Valence – effectively addressing the aforementioned transferability issues. The second classification layer then employs a residual sum of squares-based weighting scheme to merge the various regression outputs into one optimal Arousal/Valence classification in real-time, while maintaining a smooth prediction output. The presented method exhibited convincing accuracy ratings – 85% for Arousal and 78% for Valence –, which are only marginally worse than our previous non-real-time approach.
KeywordsAffect recognition regression analysis affective computing games physiology galvanic skin response heart rate electromyography
Unable to display preview. Download preview PDF.
- 1.Ermi, L., Mäyrä, F.: Fundamental components of the gameplay experience: Analysing immersion. In: Digital Games Research Association Conference: Changing Views - Worlds in Play (2005)Google Scholar
- 4.Vinhas, V., Silva, D., Oliveira, E., Reis, L.: Biometric Emotion Assessment and Feedback in an Immersive Digital Environment. Social Robotics, 307–317 (2009)Google Scholar
- 7.Plutchik, R.: A General Psychoevolutionary Theory of Emotion. Emotion: Theory, Research, and Experience 1, 3–33 (1980)Google Scholar
- 9.Lang, P.J., Bradley, M.M., Cuthbert, B.N.: International affective picture system (IAPS): Affective ratings of pictures and instruction manual. University of Florida, Gainesville (2008)Google Scholar
- 10.Stern, R.M., Ray, W.J., Quigley, K.S.: Psychophysiological recording. Oxford University Press, New York (2001)Google Scholar
- 11.Nogueira, P.A., Rodrigues, R., Oliveira, E., Nacke, L.E.: A Regression-based Method for Lightweight Emotional State Detection in Interactive Environments. In: Proceedings of the XVI Portuguese Conference on Artificial Intelligence (EPIA), Angra do Heroísmo, Açores, Portugal (to appear, 2013)Google Scholar
- 12.Drachen, A., Nacke, L.E., Yannakakis, G., Pedersen, A.L.: Correlation between Heart Rate, Electrodermal Activity and Player Experience in First-Person Shooter Games. In: Proc. of the 5th ACM SIGGRAPH Symposium on Video Games, pp. 49–54 (2010)Google Scholar