Real-time Brain Assessment for Adaptive Virtual Reality Game : A Neurofeedback Approach

  • Hamdi Ben Abdessalem
  • Claude Frasson
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10512)


Humans’ cognitive and affective states are constantly subject to regular and sudden changes. The origins of these changes are multiple and unpredictable. Virtual Reality (VR) game environments could represent an immersive unconstrained experimental context in which game designers could control a wide range of parameters that act on these states. In this paper, we propose to track and adapt to individuals’ frustration and excitement levels in real time while interacting with a VR environment. We developed “AmbuRun”, a VR game designed to modify the speed and the difficulty in real time. A neural agent was created to control these parameters within the game using an intervention strategy that was intended to induce appropriate modifications of the players ‘excitement and frustration level. An experimental study involving 20 participants was conducted to evaluate our neurofeedback approach. Results showed that intelligent control through neurofeedback of speed and difficulty affected excitement and frustration before and after the agent action.


Neurofeedback Intelligent agent Virtual reality Adaptive game Emotional intelligence 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Yates, M., Kelemen, A., Sik Lanyi, C.: Virtual reality gaming in the rehabilitation of the upper extremities post-stroke. Brain Inj. 30, 855–863 (2016)CrossRefGoogle Scholar
  2. 2.
    Biocca, F.: The Cyborg’s Dilemma: Progressive Embodiment in Virtual Environments [1]. J. Comput.-Mediat. Commun. 3 (2006)Google Scholar
  3. 3.
    Alvarez, R.P., Johnson, L., Grillon, C.: Contextual-specificity of short-delay extinction in humans: Renewal of fear-potentiated startle in a virtual environment. Learn. Mem. 14, 247–253 (2007)CrossRefGoogle Scholar
  4. 4.
    Bohil, C.J., Alicea, B., Biocca, F.A.: Virtual reality in neuroscience research and therapy. Nat. Rev., Neurosci. (2011)Google Scholar
  5. 5.
    Benlamine, M.S, Chaouachi, M., Frasson, C.: Dufresne, Aude: Predicting Spontaneous Facial Expressions from EEG. Intell. Tutoring Syst. (2016)Google Scholar
  6. 6.
    Chaouachi, M., Jraidi, I., Frasson, C.: MENTOR: a physiologically controlled tutoring system. In: Ricci, F., Bontcheva, K., Conlan, O., Lawless, S. (eds.) User Modeling, Adaptation and Personalization, pp. 56–67. Springer International Publishing, Cham (2015)CrossRefGoogle Scholar
  7. 7.
    Lecuyer, A., Lotte, F., Reilly, R.B., Leeb, R., Hirose, M., Slater, M.: Brain-Computer Interfaces, Virtual Reality, and Videogames. Computer 41, 66–72 (2008)CrossRefGoogle Scholar
  8. 8.
    McFarland, D.J., Sarnacki, W.A., Wolpaw, J.R.: Electroencephalographic (EEG) control of three-dimensional movement. J. Neural Eng. 7, 036007 (2010)CrossRefGoogle Scholar
  9. 9.
    Gorini, A., Riva, G.: Virtual reality in anxiety disorders: the past and the future. Expert Rev. Neurother. 8, 215–233 (2008)CrossRefGoogle Scholar
  10. 10.
    Rose, F.D., Brooks, B.M., Rizzo, A.A.: Virtual Reality in Brain Damage Rehabilitation: Review. Cyberpsychol. Behav. 8, 241–262 (2005)CrossRefGoogle Scholar
  11. 11.
    D’Mello, S., Olney, A., Williams, C., Hays, P.: Gaze tutor: A gaze-reactive intelligent tutoring system. Int. J. Hum.-Comput. Stud. 70, 377–398 (2012)CrossRefGoogle Scholar
  12. 12.
    Chaouachi, M., Jraidi, I., Frasson, C.: Adapting to learners’ mental states using a physiological computing approach. In: FLAIRS 2015 Twenty-Eighth Int. Flairs Conf. (2015)Google Scholar
  13. 13.
    Ghali, R., Ben Abdessalem, H., Frasson, C.: Improving Intuitive Reasoning Through Assistance Strategies in a Virtual Reality Game (2017)Google Scholar
  14. 14.
    Sherlin, L.H., Arns, M., Lubar, J., Heinrich, H., Kerson, C., Strehl, U., Sterman, M.B.: Neurofeedback and Basic Learning Theory: Implications for Research and Practice. J. Neurother. 15, 292–304 (2011)CrossRefGoogle Scholar
  15. 15.
    Van Doren, J., Heinrich, H., Bezold, M., Reuter, N., Kratz, O., Horndasch, S., Berking, M., Ros, T., Gevensleben, H., Moll, G.H., Studer, P.: Theta/beta neurofeedback in children with ADHD: Feasibility of a short-term setting and plasticity effects. Int. J. Psychophysiol. 112, 80–88 (2017)CrossRefGoogle Scholar
  16. 16.
    Heinrich, H., Gevensleben, H., Strehl, U.: Annotation: Neurofeedback ? train your brain to train behaviour. J. Child Psychol. Psychiatry 48, 3–16 (2007)CrossRefGoogle Scholar
  17. 17.
    Duric, N.S., Elgen, I., Assmus, J.: Self-reported efficacy of neurofeedback treatment in a clinical randomized controlled study of ADHD children and adolescents. Neuropsychiatr. Dis. Treat., 1645 (2014)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Département d’Informatique et de Recherche OpérationnelleUniversité de MontréalMontréalCanada

Personalised recommendations