Advertisement

Abstract

This study aims at recognizing the affective states of players from non-acted, non-repeated body movements in the context of a video game scenario. A motion capture system was used to collect the movements of the participants while playing a Nintendo Wii tennis game. Then, a combination of body movement features along with a machine learning technique was used in order to automatically recognize emotional states from body movements. Our system was then tested for its ability to generalize to new participants and to new body motion data using a sub-sampling validation technique. To train and evaluate our system, online evaluation surveys were created using the body movements collected from the motion capture system and human observers were recruited to classify them into affective categories. The results showed that observer agreement levels are above chance level and the automatic recognition system achieved recognition rates comparable to the observers’ benchmark.

Keywords

Body movement automatic emotion recognition exertion game 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Kim, J.H., Gunn, D.V., Schuh, E., Phillips, B., Pagulayan, R.J., Wixon, D.: Tracking real-time user experience (TRUE): a comprehensive instrumentation solution for complex systems. In: Proceedings of the 26th Annual SIGCHI Conference On Human Factors In Computing Systems, pp. 443–452. ACM, New York (2008)Google Scholar
  2. 2.
    Pollick, F., Paterson, H., Bruderlin, A., Sanford, A.: Perceiving affect from arm movement. Cognition 82, 51–61 (2001)CrossRefGoogle Scholar
  3. 3.
    Mehrabian, A., Friar, J.: Encoding of attitude by a seated communicator via posture and position cues. Journal of Consulting and Clinical Psychology 33, 330–336 (1969)CrossRefGoogle Scholar
  4. 4.
    Mandler, G.: History of Psychology. Emotion, vol. 1, ch. 8. Wiley (2002) Google Scholar
  5. 5.
    Bernhardt, D., Robinson, P.: Detecting Affect from Non-stylised Body Motions. In: Paiva, A.C.R., Prada, R., Picard, R.W. (eds.) ACII 2007. LNCS, vol. 4738, pp. 59–70. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  6. 6.
    Castellano, G., Villalba, S., Camurri, A.: Recognising Human Emotions from Body Movement and Gesture Dynamics. In: Paiva, A., Prada, R., Picard, R.W. (eds.) ACII 2007. LNCS, vol. 4738, pp. 71–82. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  7. 7.
    Kleinsmith, A., Fushimi, T., Bianchi-Berthouze, N.: An incremental and interactive affective posture recognition system. In: Carberry, S., De Rosis, F. (eds.) International Workshop on Adapting the Interaction Style to Affective Factors, in conjunction with the International Conference on User Modeling (2005)Google Scholar
  8. 8.
    Kleinsmith, A., Bianchi-Berthouze, N.: Recognizing Affective Dimensions from Body Posture. In: Paiva, A.C.R., Prada, R., Picard, R.W. (eds.) ACII 2007. LNCS, vol. 4738, pp. 48–58. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  9. 9.
    Coulson, M.: Attributing emotion to static body postures: Recognition accuracy, confusions, and viewpoint dependence. Journal of Nonverbal Behavior 28, 117–139 (2004)CrossRefGoogle Scholar
  10. 10.
    Kleinsmith, A., De Silva, R., Bianchi-Berthouze, N.: Cross-cultural differences in recognizing affect from body posture. Interacting with Computers 18(6), 1371–1389 (2006)CrossRefGoogle Scholar
  11. 11.
    Camurri, A., Mazzarino, B., Ricchetti, M., Timmers, R., Volpe, G.: Multimodal Analysis of Expressive Gesture in Music and Dance Performances. In: Camurri, A., Volpe, G. (eds.) GW 2003. LNCS (LNAI), vol. 2915, pp. 20–39. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  12. 12.
    Mandler, G.: History of Psychology. Emotion, vol. 1, ch. 8. Wiley (2002)Google Scholar
  13. 13.
    Kitagawa, M., Windsor, B.: MoCap for Artists: Workflow and Techniques for Motion Capture, pp. 190–194. Focal Press (2008)Google Scholar
  14. 14.
    Roether, C., Omlor, L., Christensen, A., Giese, M.A.: Critical features for the perception of emotion from gait. Journal of Vision 8(6), 15, 1–32 (2009)Google Scholar
  15. 15.
    Elman, J.L.: Finding Structure in Time. Cognitive Science 14, 179–211 (1990)CrossRefGoogle Scholar
  16. 16.
    Haykin, S.: Neural Networks: A Comprehensive Foundation, 2nd edn., pp. 754–777. Prentice-Hall (1999)Google Scholar
  17. 17.
    Bodén, M.: A guide to recurrent neural networks and backpropagation, in The DALLAS project. Report from the NUTEK-supported project AIS-8: Application of Data Analysis with Learning Systems, 1999-2001. Holst, A. (ed.), SICS Technical Report T2002:03, SICS, Kista, Sweden (2001)Google Scholar
  18. 18.
    Storm, C., Storm, T.: A taxonomic study of the vocabulary of emotions. Journal of Personality and Social Psychology 53(4), 805–816 (1987)CrossRefGoogle Scholar
  19. 19.
    Kleinsmith, A., Bianchi-Berthouze, N., Steed, A.: Automatic Recognition of Non-Acted Affective Postures. IEEE Transactions on Systems, Man and Cybernetics, Part B (2011)Google Scholar
  20. 20.
    Gunes, H., Piccardi, M.: Bi-modal emotion recognition from expressive face and body gestures. Journal of Network and Computer Applications 30, 1334–1345 (2007)CrossRefGoogle Scholar
  21. 21.
    Muller, F., Bianchi-Berthouze, N.: Evaluating Exertion Games Experiences from Investigating Movement Based. Human-Computer Interaction Series, Part 4, pp. 187–207. Springer, Heidelberg (2010)Google Scholar
  22. 22.
    Pasch, M., Bianchi-Berthouze, N., van Dijk, B., Nijholt, A.: Movement-based Sports Video Games: Investigating Motivation and Gaming Experience. Entertainment Computing 9(2), 169–180 (2009)Google Scholar
  23. 23.
    De Silva, R., Bianchi-Berthouze, N.: Modeling human affective postures: An information theoretic characterization of posture features. Journal of Computational Animation and Virtual Worlds 15(3-4), 269–276 (2004)CrossRefGoogle Scholar
  24. 24.
    Kleinsmith, A., de Silva, R., Bianchi-Berthouze, N.: Cross-cultural differences in recognizing affect from body posture. Interacting with Computers 18, 1371–1389 (2006)CrossRefGoogle Scholar
  25. 25.
    Kleinsmith, A., Bianchi-Berthouze, N.: Recognizing Affective Dimensions from Body Posture. In: Paiva, A.C.R., Prada, R., Picard, R.W. (eds.) ACII 2007. LNCS, vol. 4738, pp. 48–58. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  26. 26.
    Russell, J.A., Feldman-Barrett, L.: Core affect, prototypical emotional episodes, and other things called emotion: Dissecting the elephant. J. Pers. Social Psychol. 76, 805–819 (1999)CrossRefGoogle Scholar

Copyright information

© ICST Institute for Computer Science, Social Informatics and Telecommunications Engineering 2012

Authors and Affiliations

  • Nikolaos Savva
    • 1
  • Nadia Bianchi-Berthouze
    • 1
  1. 1.UCLICUniversity College LondonLondonUK

Personalised recommendations