Abstract

Nowadays, the human computer interfaces can be designed to be adaptive and emotion-enabled. The recognized emotions of the user can help make the user’s experience more complete, more engaging, less stressful or more stressful depending on the target of the applications. Such affective human-computer interfaces are getting more attention from researchers and engineers. EEG signals are used to recognize emotions of the user in real time. We describe a real-time emotion recognition algorithm that is used to personalize different applications according to the user’s current emotions. The algorithm is subject-dependent and needs a training session before running the application. Two EEG-enabled games and one adaptive advertisement based on the algorithm are designed and implemented. One game is the “Bar” game where a difficulty level of the game is adapted based on the player’s score and emotions. Another game is the “Girl Twins” one where the player’s emotions are monitored in real time, and an emotional companion is implemented as the girl twins avatars whose behaviour changes according to the user’s emotions. An adaptive advertising movie is designed and implemented as well. Here, the real-time emotion recognition algorithm is used to adjust the scenes of the advertisement based on the current emotion recognized.

Keywords

EEG adaptive interfaces emotion recognition BCI affective computing 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Mauss, I.B., Robinson, M.D.: Measures of emotion: A review. Cognition and Emotion 23(2), 209–237 (2009)CrossRefGoogle Scholar
  2. 2.
    Tsai, T.W., Lo, H.Y., Chen, K.S.: An affective computing approach to develop the game-based adaptive learning material for the elementary students. In: Proceedings of the 2012 Joint International Conference on Human-Centered Computer Environments, pp. 8–13 (2012)Google Scholar
  3. 3.
    Tijs, T.J.W., Brokken, D., IJsselsteijn, W.A.: Dynamic game balancing by recognizing affect. In: Markopoulos, P., de Ruyter, B., IJsselsteijn, W.A., Rowland, D. (eds.) Fun and Games 2008. LNCS, vol. 5294, pp. 88–93. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  4. 4.
    Tijs, T.J.W., Brokken, D., IJsselsteijn, W.A.: Creating an emotionally adaptive game. In: Stevens, S.M., Saldamarco, S.J. (eds.) ICEC 2008. LNCS, vol. 5309, pp. 122–133. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  5. 5.
    Saari, T., Turpeinen, M., Kuikkaniemi, K., Kosunen, I., Ravaja, N.: Emotionally adapted games – An example of a first person shooter. In: Jacko, J.A. (ed.) HCI International 2009, Part IV. LNCS, vol. 5613, pp. 406–415. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  6. 6.
    Gilleade, K.M., Dix, A.: Using frustration in the design of adaptive videogames. In: Proceedings of the 2004 ACM SIGCHI International Conference on Advances in Computer Entertainment Technology, pp. 228–232 (2004)Google Scholar
  7. 7.
    Poels, K., Dewitte, S.: How to capture the heart? Reviewing 20 years of emotion measurement in advertising. Journal of Advertising Research 46(1), 18–37 (2006)CrossRefGoogle Scholar
  8. 8.
    Liu, Y., Sourina, O.: EEG Databases for Emotion Recognition. In: Proc. 2013 Int. Conf. on Cyberworlds, pp. 302–309 (2013)Google Scholar
  9. 9.
    Mehrabian, A.: Framework for a comprehensive description and measurement of emotional states. Genetic, Social, and General Psychology Monographs 121(3), 339–361 (1995)Google Scholar
  10. 10.
    Mehrabian, A.: Pleasure-Arousal-Dominance: A general framework for describing and measuring individual differences in temperament. Current Psychology 14(4), 261–292 (1996)CrossRefMathSciNetGoogle Scholar
  11. 11.
    Bolls, P.D., Lang, A., Potter, R.F.: The effects of message valence and listener arousal on attention, memory, and facial muscular responses to radio advertisements. Communication Research 28(5), 627–651 (2001)CrossRefGoogle Scholar
  12. 12.
    Liu, Y., Sourina, O., Nguyen, M.K.: Real-time EEG-based Human Emotion Recognition and Visualization. In: Proceedings of the 2010 International Conference on Cyberworlds, pp. 262–269 (2010)Google Scholar
  13. 13.
    Liu, Y., Sourina, O., Nguyen, M.K.: Real-Time EEG-Based Emotion Recognition and Its Applications. In: Gavrilova, M.L., Tan, C.J.K., Sourin, A., Sourina, O. (eds.) Trans. on Comput. Sci. XII. LNCS, vol. 6670, pp. 256–277. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  14. 14.
    Koelstra, S., Muhl, C., Soleymani, M., Lee, J.-S., Yazdani, A., Ebrahimi, T., Pun, T., Nijholt, A., Patras, I.: DEAP: A Database for Emotion Analysis; Using Physiological Signals. IEEE Transactions on Affective Computing 3(1), 18–31 (2012)CrossRefGoogle Scholar
  15. 15.
    Bechara, A., Damasio, H., Damasio, A.R.: Emotion, decision making and the orbitofrontal cortex. Cerebral Cortex 10(3), 295–307 (2000)CrossRefGoogle Scholar
  16. 16.
    Kringelbach, M.L.: The human orbitofrontal cortex: Linking reward to hedonic experience. Nature Reviews Neuroscience 6(9), 691–702 (2005)CrossRefGoogle Scholar
  17. 17.
    Dolcos, F., Labar, K.S., Cabeza, R.: Dissociable effects of arousal and valence on prefrontal activity indexing emotional evaluation and subsequent memory: An event-related fMRI study. NeuroImage 23(1), 64–74 (2004)CrossRefGoogle Scholar
  18. 18.
    Demaree, H.A., Everhart, D.E., Youngstrom, E.A., Harrison, D.W.: Brain lateralization of emotional processing: Historical roots and a future incorporating “dominance”. Behavioral and Cognitive Neuroscience Reviews 4(1), 3–20 (2005)CrossRefGoogle Scholar
  19. 19.
    Petrides, M., Pandya, D.N.: Projections to the frontal cortex from the posterior parietal region in the rhesus monkey. Journal of Comparative Neurology 228(1), 105–116 (1984)CrossRefGoogle Scholar
  20. 20.
    Banks, S.J., Eddy, K.T., Angstadt, M., Nathan, P.J., Phan, K.L.: Amygdala–frontal connectivity during emotion regulation. Social Cognitive and Affective Neuroscience 2(4), 303–312 (2007)CrossRefGoogle Scholar
  21. 21.
    Chanel, G., Kierkels, J.J.M., Soleymani, M., Pun, T.: Short-term emotion assessment in a recall paradigm. International Journal of Human Computer Studies 67(8), 607–627 (2009)CrossRefGoogle Scholar
  22. 22.
    Moddemeijer, R.: On estimation of entropy and mutual information of continuous distributions. Signal Processing 16(3), 233–248 (1989)CrossRefMathSciNetGoogle Scholar
  23. 23.
  24. 24.
    American electroencephalographic society guidelines for standard electrode position nomenclature. Journal of Clinical Neurophysiology 8(2), 200–202 (1991)Google Scholar
  25. 25.
    Ranky, G.N., Adamovich, S.: Analysis of a commercial EEG device for the control of a robot arm. In: Proceedings of the 2010 IEEE 36th Annual Northeast Bioengineering Conference, pp. 1–2 (2010)Google Scholar
  26. 26.
    Stytsenko, K., Jablonskis, E., Prahm, C.: Evaluation of consumer EEG device Emotiv EPOC. Poster session presented at MEi:CogSci Conference 2011 (2011)Google Scholar
  27. 27.
    Duvinage, M., Castermans, T., Dutoit, T., Petieau, M., Hoellinger, T., Saedeleer, C.D., Seetharaman, K., Cheron, G.: A P300-based quantitative comparison between the emotiv epoc headset and a medical EEG device. In: Proceedings of the IASTED International Conference Biomedical Engineering, pp. 37–42 (2012)Google Scholar
  28. 28.
    Liu, Y., Sourina, O., Hafiyyandi, M.R.: EEG-Based Emotion-Adaptive Advertising. In: 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction (ACII), pp. 843–848 (2013)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Olga Sourina
    • 1
  • Yisi Liu
    • 1
  1. 1.Fraunhofer IDM@NTUNanyang Technological UniversitySingapore

Personalised recommendations