Games for BCI Skill Learning

  • Reinhold Scherer
  • Gernot Müller-Putz
  • Elisabeth V C Friedrich
  • Viktoria Pammer-Schindler
  • Karin Wilding
  • Stephan Keller
  • Johanna Pirker
Living reference work entry

Abstract

A brain–computer interface (BCI) is a device that translates the users’ thoughts directly into action. Brain signal patterns used to encode messages are user specific. However, experimental paradigms used to collect neurophysiological trials from individuals are typically data-centered and not user-centered. This means that experimental paradigms are tuned to collect as many trials as possible – which is indeed important for reliable calibration of pattern recognition – and are generally rather demanding and not very motivating or engaging for individuals. Subject cooperation and their compliance with the task may decrease over time. This leads in turn to a high variability of the collected brain signals and thus results in unreliable pattern recognition. One solution to this issue might be the implementation of engaging games instead of the use of standard paradigms to gain and maintain BCI control. This chapter first reviews basic principles and standard experimental paradigms used in BCI training that detect messages expressed by spontaneous electroencephalogram (EEG) rhythms. Users can independently modulate oscillations by performing appropriate mental tasks. Then, requirements for successful connection of games and these BCI paradigms are outlined in order to provide users with engaging methods to acquire the BCI skill. Last, a novel training concept for BCI in the framework of games is proposed. A recently introduced communication board for users with cerebral palsy is described as example to illustrate game-inspired training paradigms.

Keywords

Brain–computer interface User-centered design Man–machine learning People with special needs Skill learning 

Recommended Reading

  1. B.Z. Allison, C. Neuper, Could anyone use a BCI? in Brain-Computer Interfaces (Springer, London, 2010), pp. 35–54CrossRefGoogle Scholar
  2. B. Blankertz, C. Sannelli, S. Halder, E.M. Hammer, A. Kübler et al., Neurophysiological predictor of SMR-based BCI performance. Neuroimage 51, 1303–1309 (2010)CrossRefGoogle Scholar
  3. C.J. Bell, P. Shenoy, R. Chalodhorn, R.P.N. Rao, Control of a humanoid robot by a noninvasive brain-computer interface in humans. J. Neural Eng. 5(2), 214–220 (2008). doi:10.1088/1741-2560/5/2/012CrossRefGoogle Scholar
  4. C.M. Bishop, Pattern Recognition and Machine Learning (Springer, New York, 2006)MATHGoogle Scholar
  5. J. Breuer, G. Brente, J. Comput. Game Culture 4(1), 7–24 (2010)Google Scholar
  6. M. Carter, J. Downs, B. Nansen, M. Harrop, M. Gibbs, Paradigms of games research in HCI: a review of 10 years of research at CHI, in Proceedings of the First ACM SIGCHI Annual Symposium on Computer-Human Interaction in Play (CHI PLAY’14) (ACM, 2014)Google Scholar
  7. M. Csikszentmihalyi, Flow: The Psychology of Optimal Experience, vol. 41 (Harper Perennial, New York, 1991)Google Scholar
  8. S. Deterding, D. Dixon, R. Khaled, L. Nacke, From game design elements to gamefulness: defining gamification, in Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments (ACM, 2011), pp. 9–15Google Scholar
  9. J. Faller, C. Vidaurre, T. Solis Escalante, C. Neuper, R. Scherer, Autocalibration and recurrent adaptation: towards a plug and play online ERD-BCI. IEEE Trans. Neural Syst. Rehabil. Eng. (2012). doi:10.1109/TNSRE.2012.2189584Google Scholar
  10. M. Fatourechi, A. Bashashati, R.K. Ward, G.E. Birch, EMG and EOG artifacts in brain computer interface systems: a survey. Clin. Neurophysiol. 118(3), 480–494 (2006)CrossRefGoogle Scholar
  11. E.V.C. Friedrich, R. Scherer, C. Neuper, The effect of distinct mental strategies on classification performance for brain-computer interfaces. Int. J. Psychophysiol. 84, 86–94 (2012)CrossRefGoogle Scholar
  12. E.V.C. Friedrich, C. Neuper, R. Scherer, Whatever works: a systematic user-centered training protocol to optimize brain-computer interfacing individually. PLoS One 8(9), e76214 (2013)CrossRefGoogle Scholar
  13. E.V.C. Friedrich, N. Suttie, A. Sivanathan, T. Lim, S. Louchart, A. Pineda, Brain-computer interface game applications for combined neurofeedback and biofeedback treatment for children on the autism spectrum. Front. Neuroeng. 7, 21 (2014). doi:10.3389/fneng.2014.00021CrossRefGoogle Scholar
  14. V.A. Holm, The causes of cerebral palsy: a contemporary perspective. JAMA 247(10), 1473–1477 (1982)CrossRefGoogle Scholar
  15. A. Isaksen, D. Gopstein, A. Nealen, Exploring game space using survival analysis. Foundations of Digital Games. Best Paper in Artificial Intelligence and Game Technology (2015)Google Scholar
  16. E.R. Kandel, J.H. Schwartz, T.M. Jessel, S.A. Siegelbaum, J. Hudspeth, Principles of Neural Science, 5th edn. (McGraw-Hill Medical, New York, 2014)Google Scholar
  17. S.E. Kober, C. Neuper, Using auditory event-related EEG potentials to assess presence in virtual reality. Int. J. Hum. Comput. Stud. 70, 577–587 (2012)CrossRefGoogle Scholar
  18. R. Koster, Theory of Fun for Game Design (“O”Reilly Media, Sebastopol, 2013)Google Scholar
  19. G. Krausz, R. Scherer, G. Korisek, G. Pfurtscheller, Critical decision-speed and information transfer in the Graz brain–computer interface. Appl. Psychophysiol. Biofeedback 28(3), 233–240 (2003)CrossRefGoogle Scholar
  20. A. Lecuyer, F. Lotte, R.B. Reilly, R. Leeb, M. Hirose, M. Slater, Brain-computer interfaces, virtual reality, and videogames. Computer 41(10), 66–72 (2008). doi:10.1109/MC.2008.410CrossRefGoogle Scholar
  21. F. Lotte, M. Congedo, A. Lécuyer, F. Lamarche, B. Arnaldi et al., A review of classification algorithms for EEG-based brain–computer interfaces. J. Neural Eng. 4, R1–R13 (2007)CrossRefGoogle Scholar
  22. F. Lotte, F. Larrue, C. Mühl, Flaws in current human training protocols for spontaneous brain-computer interfaces: lessons learned from instructional design. Front. Hum. Neurosci. 7, 568 (2013)CrossRefGoogle Scholar
  23. S. Mason, A. Bashashati, M. Fatourechi, K. Navarro, G. Birch, A comprehensive survey of brain interface technology designs. Ann. Biomed. Eng. 35, 137–169 (2007)CrossRefGoogle Scholar
  24. J.D. Millán, R. Rupp, G.R. Müller-Putz, R. Murray-Smith, C. Giugliemma, M. Tangermann, C. Vidaurre, F. Cincotti, A. Kübler, R. Leeb, et al., Combining brain–computer interfaces and assistive technologies: state-of-the-art and challenges. Front. Neurosci. 4, 161 (2010). doi:10.3389/fnins.2010.00161Google Scholar
  25. K. Müller, C.W. Anderson, G.E. Birch, Linear and nonlinear methods for brain-computer interfaces. IEEE Trans. Neural Syst. Rehabil. Eng. 11(2), 165–169 (2003). doi:10.1109/TNSRE.2003.814484CrossRefGoogle Scholar
  26. G.R. Müller-Putz, R. Scherer, C. Brauneis, G. Pfurtscheller, Steady-state visual evoked potential (SSVEP)-based communication: impact of harmonic frequency components. J. Neural Eng. 2(4), 123–130 (2005). doi:10.1088/1741-2560/2/4/008CrossRefGoogle Scholar
  27. G.R. Müller-Putz, R. Scherer, G. Pfurtscheller, C. Neuper, Temporal coding of brain patterns for direct limp control in humans. Front. Neurosci. 4, 34 (2010). doi:10.3389/fnins.2010.00034Google Scholar
  28. C. Neuper, G.R. Müller, A. Kübler, N. Birbaumer, G. Pfurtscheller, Clinical application of an EEG-based brain-computer interface: a case study in a patient with severe motor impairment. Clin. Neurophysiol. 114(3), 399–409 (2003)CrossRefGoogle Scholar
  29. C. Neuper, R. Scherer, M. Reiner, G. Pfurtscheller, Imagery of motor actions: differential effects of kinesthetic and visual-motor mode of imagery in single-trial EEG. Brain Res. Cogn. Brain Res. 25(3), 668–677 (2005). doi:10.1016/j.cogbrainres.2005.08.014CrossRefGoogle Scholar
  30. M. Ninaus, S.E. Kober, E.V.C. Friedrich, I. Dunwell, S. Freitas et al., Neurophysiological methods for monitoring brain activity in serious games and virtual environments: a review. Int. J. Technol. Enhanc. Learn. 6(1), 78 (2014). doi:10.1504/IJTEL.2014.060022CrossRefGoogle Scholar
  31. E. Niedermeyer, The normal EEG of the waking adult, in Electroencephalography: Basic Principles, Clinical Applications and Related Fields (1999), Williams & Wilkins, Baltimore, pp. 149–173Google Scholar
  32. A. Nijholt, D. Tan, Playing with your brain: brain-computer interfaces and games. In Proceedings of the international conference on Advances in computer entertainment technology (ACE ‘07). ACM, New York, pp 305–306 (2007)Google Scholar
  33. P.L. Nunez, R. Srinivasan, Electric Fields of the Brain: The Neurophysics of EEG, 2nd edn. (Oxford University Press, New York, 2006)CrossRefGoogle Scholar
  34. D.G. Oblinger, Games and learning. Educ. Q. 29(3), 5–7 (2006)Google Scholar
  35. G. Pfurtscheller, F.H. Lopes da Silva, Event-related EEG/MRG synchronization and desynchronization: basic principles. Clin. Neurophysiol. 110(11), 1842–1857 (1999). doi:10.1016/ S1388-2457(99)00141-8CrossRefGoogle Scholar
  36. G. Pfurtscheller, C. Neuper, Motor imagery and direct brain-computer communication. Proc. IEEE 89(7), 1123–1134 (2001)CrossRefGoogle Scholar
  37. G. Pfurtscheller, G.R. Müller-Putz, R. Scherer, C. Neuper, Rehabilitation with brain-computer interface systems. Computer 41(10), 58–65 (2008). doi:10.1109/MC.2008.432CrossRefGoogle Scholar
  38. J. Pirker, S. Berger, C. Gütl, J. Belcher, P.H. Bailey, Understanding physical concepts using an immersive virtual learning environment, in Proceedings of the 2nd European Immersive Education Summit (2012)Google Scholar
  39. W. Samek, F.C. Meinecke, K.R. Müller, Transferring subspaces between subjects in brain–computer interfacing. IEEE Trans. Biomed. Eng. 60(8), 2289–2298 (2013). doi:10.1109/TBME.2013.2253608CrossRefGoogle Scholar
  40. J. Schell, The Art of Game Design: A Book of Lenses (CRC Press, Boca Raton, 2014)CrossRefGoogle Scholar
  41. R. Scherer, A. Schlögl, F.Y. Lee, H. Bischof, D. Grassi, G. Pfurtscheller, The self-paced Graz brain-computer interface: methods and applications. Comput. Intell. Neurosci. 2007, 79826 (2007)CrossRefGoogle Scholar
  42. R. Scherer, J. Faller, D. Balderas, E.V. Friedrich, M. Pröll, B. Allison, G. Müller-Putz, Brain–computer interfacing: more than the sum of its parts. Soft Comput. 17(2), 317–331 (2013a)CrossRefGoogle Scholar
  43. R. Scherer, G. Moitzi, I. Daly, G.R. Müller-Putz, On the use of games for noninvasive EEG-based functional brain mapping. IEEE Trans. Comput. Intell. AI Games 5(2), 155–163 (2013b). doi:10.1109/TCIAIG.2013.2250287CrossRefGoogle Scholar
  44. R. Scherer, J. Faller, E.V.C. Friedrich, E. Opisso, U. Costa, A. Kübler, G.R. Müller-Putz, Individually adapted imagery improves brain-computer interface performance in end-users with disability. PLoS One 10(5), e0123727 (2015a). doi:10.1371/journal.pone.0123727CrossRefGoogle Scholar
  45. R. Scherer, M. Billinger, J. Wagner, A. Schwarz, D.T. Hettich, E. Bolinger, M. Lloria Garcia, J. Navarro, G.R. Müller-Putz, Thought-based row-column scanning communication board for individuals with cerebral palsy. Ann. Phys. Rehabil. Med. (2015b). doi:10.1016/j.rehab.2014.11.005Google Scholar
  46. R. Scherer, A. Schwarz, G.R. Müller-Putz, V. Pammer-Schindler, M. Lloria Garcia, Game-based BCI training: interactive design for individuals with cerebral palsy, in Proceedings of the IEEE SMC (2015c), in pressGoogle Scholar
  47. M. Slater, V. Linakis, M. Usoh, R. Kooper, in Immersion, Presence and Performance in Virtual Environments: An Experiment with Tri-Dimensional Chess, ed. G. Mark, in Proceedings of the ACM symposium on Virtual reality software and technology (VRST), 1996, pp. 163–172Google Scholar
  48. W.O. Tatum, B.A. Dworetzky, D.L. Schomer, Artifact and recording concepts in EEG. J. Clin. Neurophysiol. 28(3), 252–263 (2011)CrossRefGoogle Scholar
  49. C. Vidaurre, C. Sannelli, K.R. Müller, B. Blankertz, Co-adaptive calibration to improve BCI efficiency. J. Neural Eng. 8(2), 025009 (2011)CrossRefGoogle Scholar
  50. J. Wolpaw, N. Birbaumer, D.J. McFarland, G. Pfurtscheller, T.M. Vaughan, Brain-computer interfaces for communication and control. Clin. Neurophysiol. 113, 767–791 (2002). doi:10.1016/S1388-2457(02)00057-3CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Singapore 2015

Authors and Affiliations

  • Reinhold Scherer
    • 1
  • Gernot Müller-Putz
    • 1
  • Elisabeth V C Friedrich
    • 2
  • Viktoria Pammer-Schindler
    • 3
    • 5
  • Karin Wilding
    • 3
  • Stephan Keller
    • 3
  • Johanna Pirker
    • 4
  1. 1.Institute for Knowledge DiscoveryGraz University of TechnologyGrazAustria
  2. 2.Department of Cognitive ScienceUniversity of California San DiegoLa JollaUSA
  3. 3.Knowledge Technologies InstituteGraz University of TechnologyGrazAustria
  4. 4.Institute of Information Systems and Computer MediaGraz University of TechnologyGrazAustria
  5. 5.Know-Center GmbHGrazAustria

Personalised recommendations