Journal on Multimodal User Interfaces

, Volume 9, Issue 1, pp 43–54 | Cite as

An analysis of player affect transitions in survival horror games

  • Vanus Vachiratamporn
  • Roberto Legaspi
  • Koichi Moriyama
  • Ken-ichi Fukui
  • Masayuki Numao
Original Paper


The trend of multimodal interaction in interactive gaming has grown significantly as demonstrated for example by the wide acceptance of the Wii Remote and the Kinect as tools not just for commercial games but for game research as well. Furthermore, using the player’s affective state as an additional input for game manipulation has opened the realm of affective gaming. In this paper, we analyzed the affective states of players prior to and after witnessing a scary event in a survival horror game. Player affect data were collected through our own affect annotation tool that allows the player to report his affect labels while watching his recorded gameplay and facial expressions. The affect data were then used for training prediction models with the player’s brainwave and heart rate signals, as well as keyboard–mouse activities collected during gameplay. Our results show that (i) players are likely to get more fearful of a scary event when they are in the suspense state and that (ii) heart rate is a good candidate for detecting player affect. Using our results, game designers can maximize the fear level of the player by slowly building tension until the suspense state and showing a scary event after that. We believe that this approach can be applied to the analyses of different sets of emotions in other games as well.


Affective gaming Physiological signal EEG EKG Anxiety Suspense Fear 



This work was partly supported by JSPS Core-to-Core Program, A. Advanced Research Networks.


  1. 1.
    Dekker A, Champion E (2007) Please biofeed the zombies: enhancing the gameplay and display of a horror game using biofeedback. In: Proceedings of digital games research association (DiGRA) 2007 conference, pp 550–558Google Scholar
  2. 2.
    D’Mello S, Taylor R, Graesser A (2007) Monitoring affective trajectories during complex learning. In: Proceedings of the 29th annual conference of cognitive science society, pp 203–208Google Scholar
  3. 3.
    Drachen A, Nacke LE, Yannakakis G, Pedersen AL (2010) Correlation between heart rate, electrodermal activity and player experience in first-person shooter games. In: Proceedings of the 5th ACM SIGGRAPH symposium on video games, pp 49–54. doi: 10.1145/1836135.1836143
  4. 4.
    Fanselow MS (1994) Neural organization of the defensive behavior system responsible for fear. Psychon Bull Rev 1(4):429–438. doi: 10.3758/BF03210947 CrossRefGoogle Scholar
  5. 5.
    Garner T, Grimshaw M (2011) A climate of fear: considerations for designing a virtual acoustic ecology of fear. In: Proceedings of the 6th audio mostly—a conference on interaction with sound, pp 31–38. doi: 10.1145/2095667.2095672
  6. 6.
    Garner T, Grimshaw M, Abdel Nabi D (2010) A preliminary experiment to assess the fear value of preselected sound parameters in a survival horror game. In: Proceedings of the 5th audio mostly—a conference on interaction with sound. doi: 10.1145/1859799.1859809
  7. 7.
    Giakoumis D, Tzovaras D, Moustakas K, Hassapis G (2011) Automatic recognition of boredom in video games using novel biosignal moment-based features. IEEE Trans Affect Comput 2(3):119–133. doi: 10.1109/T-AFFC.2011.4 CrossRefGoogle Scholar
  8. 8.
    Gilleade KM, Dix A, Allanson J (2005) Affective videogames and modes of affective gaming: assist me, challenge me, emote me. In: Proceedings of digital games research association (DiGRA) 2005 conference, pp 547–554Google Scholar
  9. 9.
    Hall M, Frank E, Holmes G, Pfahringer B, Reutemann P, Witten IH (2009) The weka data mining software: an update. SIGKDD Explor Newsl 11(1):10–18. doi: 10.1145/1656274.1656278 CrossRefGoogle Scholar
  10. 10.
    Higuchi T (1988) Approach to an irregular time series on the basis of the fractal theory. Phys D 31(2):277–283. doi: 10.1016/0167-2789(88)90081-4 CrossRefzbMATHMathSciNetGoogle Scholar
  11. 11.
    Hudlicka E (2009) Affective game engines: motivation and requirements. In: Proceedings of the 4th international conference on foundations of digital games, pp 299–306. doi: 10.1145/1536513.1536565
  12. 12.
    Jennett C, Cox AL, Cairns P, Dhoparee S, Epps A, Tijs T, Walton A (2008) Measuring and defining the experience of immersion in games. Int J Hum Comput Stud 66(9):641–661. doi: 10.1016/j.ijhcs.2008.04.004 CrossRefGoogle Scholar
  13. 13.
    Khanna P, Sasikumar M (2010) Recognising emotions from keyboard stroke pattern. Int J Comput Appl 11(9):1–5. doi: 10.5120/1614-2170 Google Scholar
  14. 14.
    Krzywinska T (2002) Hands-on horror. Screenplay: cinema/videogames/interfaces. Wallflower Press, London, pp 206–223Google Scholar
  15. 15.
    Landwehr N, Hall M, Frank E (2005) Logistic model trees. Mach Learn 59(1–2):161–205CrossRefzbMATHGoogle Scholar
  16. 16.
    Lane JS (2012) The effect of performance experience on vocal music major’s perception of musical tension. J Res Music Perform. ISSN: 2326–1722Google Scholar
  17. 17.
    Laurans G, Desmet PM, Hekkert P (2012) Assessing emotion in human–product interaction: an overview of available methods and a new approach. Int J Prod Dev 16(3–4):225–242. doi: 10.1504/IJPD.2012.049835 CrossRefGoogle Scholar
  18. 18.
    Mandryk RL, Atkins MS (2007) A fuzzy physiological approach for continuously modeling emotion during interaction with play technologies. Int J Hum Comput Stud 65(4):329–347. doi: 10.1016/j.ijhcs.2006.11.011 CrossRefGoogle Scholar
  19. 19.
    Martinez HP, Garbarino M, Yannakakis GN (2011) Generic physiological features as predictors of player experience. In: Proceedings of the 4th international conference on affective computing and intelligent interaction, part I, pp 267–276Google Scholar
  20. 20.
    Metallinou A, Narayanan S (2013) Annotation and processing of continuous emotional attributes: challenges and opportunities. In: Proceedings of the 10th IEEE international conference and workshops on automatic face and gesture recognition. doi: 10.1109/FG.2013.6553804
  21. 21.
    Nacke L, Lindley CA (2008) Flow and immersion in first-person shooters: measuring the player’s gameplay experience. In: Proceedings of the 2008 conference on future play, pp 81–88. doi: 10.1145/1496984.1496998
  22. 22.
    Nacke LE, Kalyn M, Lough C, Mandryk RL (2011), Biofeedback game design: using direct and indirect physiological control to enhance game interaction. In: Proceedings of the SIGCHI conference on human factors in computing systems, pp 103–112. doi: 10.1145/1978942.1978958
  23. 23.
    Öhman A (2007) Anxiety. Encyclopedia of stress, 2nd edn. Academic Press, New York, pp 236–239CrossRefGoogle Scholar
  24. 24.
    Onyett C (2012) Slender is pure horror. Accessed 22 Jan 2014
  25. 25.
    Parker JR, Heerema J (2008) Audio interaction in computer mediated games. Int J Comput Games Technol 2008:178923. doi: 10.1155/2008/178923 CrossRefGoogle Scholar
  26. 26.
    Perron B (2004) Sign of a threat: the effects of warning systems in survival horror games. In: Proceedings of the 4th international conference on computational semiotics, SplitGoogle Scholar
  27. 27.
    Prieto-Pablos JA (1998) The paradox of suspense. Poetics 26(2):99–113. doi: 10.1016/S0304-422X(98)00014-X CrossRefGoogle Scholar
  28. 28.
    Quinlan JR (1993) C4.5: programs for machine learning. Morgan Kaufmann, San FranciscoGoogle Scholar
  29. 29.
    Roux-Girard G (2011) Listening to fear: a study of sound in horror computer games. In: Grimshaw M (ed) Game sound technology and player interaction. IGI Global, Hershey, pp 192–212Google Scholar
  30. 30.
    Schröder M, Cowie R, Douglas-Cowie E, Savvidou S, McMahon E, Sawey M (2000) ’FEELTRACE’: an instrument for recording perceived emotion in real time. Proceedings of the ISCA workshop on speech and emotion. Textflow, Belfast, pp 19–24Google Scholar
  31. 31.
    Sourina O, Liu Y, Nguyen MK (2012) Real-time EEG-based emotion recognition for music therapy. J Multimodal User Interfaces 5(1–2):27–35. doi: 10.1007/s12193-011-0080-6 CrossRefGoogle Scholar
  32. 32.
    Task Force of the European Society of Cardiology the North American Society of Pacing Electrophysiology (1996) Heart rate variability: standards of measurement, physiological interpretation, and clinical use. Circulation 93(5):1043–1065. doi: 10.1161/01.CIR.93.5.1043
  33. 33.
    Toprac P, Abdel-Meguid A (2011) Causing fear, suspense, and anxiety using sound design in computer games. In: Grimshaw M (ed) Game sound technology and player interaction. IGI Global, Hershey, pp 176–191CrossRefGoogle Scholar
  34. 34.
    Truong KP, Neerincx MA, van Leeuwen DA (2008) Assessing agreement of observer- and self-annotations in spontaneous multimodal emotion data. In: Proceedings of the 9th Annual Conference of the International Speech Communication Association, pp 318–321.Google Scholar
  35. 35.
    Tsui WH, Lee P, Hsiao TC (2013) The effect of emotion on keystroke: an experimental study using facial feedback hypothesis. In: Proceedings of the 35th annual international conference of the IEEE engineering in medicine and biology society, pp 2870–2873. doi: 10.1109/EMBC.2013.6610139
  36. 36.
    Wang Q, Sourina O, Nguyen MK (2011) Fractal dimension based neurofeedback in serious games. Vis Comput 27(4):299–309. doi: 10.1007/s00371-011-0551-5 CrossRefGoogle Scholar
  37. 37.
    Weber R, Behr KM, Tamborini R, Ritterfeld U, Mathiak K (2009) What do we really know about first-person-shooter games? An event-related, high-resolution content analysis. J Comput Mediat Commun 14(4):1016–1037. doi: 10.1111/j.1083-6101.2009.01479.x CrossRefGoogle Scholar
  38. 38.
    Witten IH, Frank E, Hall MA (2011) Data mining: practical machine learning tools and techniques, 3rd edn. Morgan Kaufmann, AmsterdamGoogle Scholar
  39. 39.
    Yannakakis GN, Hallam J (2008) Entertainment modeling through physiology in physical play. Int J Hum Comput Stud 66(10):741–755. doi: 10.1016/j.ijhcs.2008.06.004 CrossRefGoogle Scholar

Copyright information

© OpenInterface Association 2014

Authors and Affiliations

  • Vanus Vachiratamporn
    • 1
  • Roberto Legaspi
    • 2
  • Koichi Moriyama
    • 1
  • Ken-ichi Fukui
    • 1
  • Masayuki Numao
    • 1
  1. 1.The Institute of Scientific and Industrial ResearchOsaka UniversityOsakaJapan
  2. 2.The Institute of Statistical MathematicsResearch Organization of Information and SystemsTokyoJapan

Personalised recommendations