Skip to main content
SpringerLink
Log in

Developing and evaluating a BCI video game for neurofeedback training: the case of autism

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

BCI video games are making brain training increasingly popular and available; yet scientific evidence to support its efficacy is lacking. Real-life descriptions of BCI video games deployments in concrete scenarios are urgently needed. In this paper, we report a use case of the development and pilot-testing of a BCI video game designed to support children with autism when attending to Neurofeedback training sessions, called FarmerKeeper. Caring for children with autism may impose new cognitive, motor, behavioral, and attention challenges that current solutions targeted for other populations may not address. The goal of the game is to maintain children’s attention above a threshold to control a runner who is seeking for lost farm animals. FarmerKeeper uses a consumer-grade BCI headset to read user’s attention. We evaluated FarmerKeeper’s usability and user experience through a 4-weeks deployment study with 12 children with autism. Our quantitative results show FarmerKeeper outperforms a commercial BCI video game used for neurofeedback training, and qualitatively, FarmerKeeper could successfully support children with autism when attending to neurofeedback training sessions by possibly improving their attention and reducing their anxiety. We close reflecting on our design aspects and discussing directions for future work.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

Notes

  1. Or BCI Serious game. The domain boundaries of the definition of Serious Games is still subject to debate [64, 83]; and although, the concept was originally proposed to support education [2], research has been exploring the use of serious games in other contexts and fields [64] including health-care [23, 62], training [57] and military [43].

  2. The EEG data and a transcript of our videos is available in our websites or could be directly requested by email to the first author.

References

  1. Aamodt S, Wang S (2007) Exercise on the brain. New York Times

  2. Abt CC (1970) Serious games: the art and science of games that simulate life. New Yorks Viking, New York, p 6

    Google Scholar 

  3. Alves S, Marques A, Queirós C, Orvalho V (2013) Lifeisgame: a serious game about emotions for children with autism spectrum disorders. PsychNology J 11:191–211

    Google Scholar 

  4. American Psychiatric Association (2013) Diagnostic and statistical manual of mental disorders, (DSM-5®). American Psychiatric Pub

  5. Antshel KM, Zhang-James Y, Wagner KE et al (2016) An update on the comorbidity of ADHD and ASD: a focus on clinical management. Expert Rev Neurother 16:279–293

    Article  Google Scholar 

  6. Autism, Investigators DDMNSY 2010 P (2014) Prevalence of autism spectrum disorder among children aged 8 years—autism and developmental disabilities monitoring network, 11 sites, United States, 2010. Morb Mortal Wkly Rep Surveill Summ 63:1–21

    Google Scholar 

  7. Bailey R, Wise K, Bolls P (2009) How avatar customizability affects children ’ s arousal and subjective presence during junk food – sponsored online video games. Cyberpsychology Behav 12:277–283. https://doi.org/10.1089/cpb.2008.0292

    Article  Google Scholar 

  8. Bakhshayesh AR, Hänsch S, Wyschkon A et al (2011) Neurofeedback in ADHD: a single-blind randomized controlled trial. Eur Child Adolesc Psychiatry 20:481–491. https://doi.org/10.1007/s00787-011-0208-y

    Article  Google Scholar 

  9. Bernardini S, Porayska-Pomsta K (2013) Planning-based social partners for children with autism. Twenty-Third Int Conf …, pp 362–370

  10. Beyer H, Holtzblatt K (1999) Contextual design. Interactions 6:32–42

    Article  Google Scholar 

  11. Birk MV, Atkins C, Bowey JT, Mandryk RL (2016) Fostering intrinsic motivation through avatar identification in digital games. Proc 2016 CHI Conf Hum Factors Comput Syst - CHI ’16, pp 2982–2995. https://doi.org/10.1145/2858036.2858062

  12. Blandón DZ, Munoz JE, Lopez DS, Gallo OH (2016) Influence of a BCI neurofeedback videogame in children with ADHD. Quantifying the brain activity through an EEG signal processing dedicated toolbox. In: Computing Conference (CCC), 2016 IEEE 11th Colombian, pp 1–8

  13. Brain S (2016) Attention span statistics – statistic brain. http://www.statisticbrain.com/attention-span-statistics/. Accessed 16 Feb 2018

  14. Brooke J (2013) SUS : a retrospective. J Usability Stud 8:29–40

    Google Scholar 

  15. Choon Guan Lim B, Lee T-S, Guan C et al (2010) Effectiveness of a brain-computer Interface based Programme for the treatment of ADHD: a pilot study. Psychopharmacol Bull 4343:73–82

    Google Scholar 

  16. Cibrian FL, Mercado J, Escobedo L, Tentori M (2018) A step towards identifying the sound preferences of children with autism. In: Proceedings of the 12th EAI international conference on pervasive computing technologies for healthcare, pp 158–167

    Google Scholar 

  17. Cmiel V, Janousek O, Kolarova J (2011) EEG biofeedback. Proc 4th Int Symp Appl Sci Biomed Commun Technol - ISABEL ’11, pp 1–5. https://doi.org/10.1145/2093698.2093752

  18. Coben R, Linden M, Myers TE (2010) Neurofeedback for autistic spectrum disorder: a review of the literature. Appl Psychophysiol Biofeedback 35:83–105. https://doi.org/10.1007/s10484-009-9117-y

    Article  Google Scholar 

  19. Dickey MD (2006) Game design narrative for learning: appropriating adventure game design narrative devices and techniques for the design of interactive learning environments. Educ Technol Res Dev 54:245–263. https://doi.org/10.1007/s11423-006-8806-y

    Article  Google Scholar 

  20. Engine U-G (2015) Unity technologies. Available Unity Technol

  21. Escobedo L, Tentori M, Quintana E et al (2014) Using augmented reality to help children with autism stay focused. IEEE Pervasive Comput 13:38–46. https://doi.org/10.1109/MPRV.2014.19

    Article  Google Scholar 

  22. Friese S (2014) Qualitative data analysis with ATLAS. ti. Sage, London

    Google Scholar 

  23. Fuchslocher A, Niesenhaus J, Krämer N (2011) Serious games for health: an empirical study of the game “balance” for teenagers with diabetes mellitus. Entertain Comput 2:97–101

    Article  Google Scholar 

  24. Garzotto F, Gelsomini M, Pappalardo A et al (2016) Using brain signals in adaptive smart spaces for disabled children. Proc 2016 CHI Conf Ext Abstr Hum Factors Comput Syst - CHI EA ’16, pp 1684–1690. https://doi.org/10.1145/2851581.2892533

  25. Goldstein G, Johnson CR, Minshew NJ (2001) Attentional processes in autism. J Autism Dev Disord 31:433–440. https://doi.org/10.1023/A:1010620820786

    Article  Google Scholar 

  26. Green CS, Bavelier D (2008) Exercising your brain: a review of human brain plasticity and training-induced learning. Psychol Aging 23:692

    Article  Google Scholar 

  27. Hallford N, Hallford J (2001) Swords and circuitry: a designer’s guide to computer role-playing games. Premier Press, Incorporated, Boston

    Google Scholar 

  28. Hammond DC (2011) What is neurofeedback: an update. J Neurother 15:305–336. https://doi.org/10.1080/10874208.2011.623090

    Article  Google Scholar 

  29. Hardy J, Scanlon M (2009) The science behind lumosity. San Fr CA Lumos Labs, San Francisco

    Google Scholar 

  30. Hayes GR, Hirano S, Marcu G et al (2010) Interactive visual supports for children with autism. Pers Ubiquit Comput 14:663–680. https://doi.org/10.1007/s00779-010-0294-8

    Article  Google Scholar 

  31. Hefner D, Klimmt C, Vorderer P (2007) Identification with the player character as determinant of video game enjoyment. In: Entertainment computing--ICEC 2007. Springer, pp 39–48

  32. Heinrich H, Gevensleben H, Strehl U (2007) Annotation: neurofeedback - train your brain to train behaviour. J Child Psychol Psychiatry Allied Discip 48:3–16. https://doi.org/10.1111/j.1469-7610.2006.01665.x

    Article  Google Scholar 

  33. Heyvaert M, Saenen L, Campbell JM et al (2014) Efficacy of behavioral interventions for reducing problem behavior in persons with autism: an updated quantitative synthesis of single-subject research. Res Dev Disabil 35:2463–2476

    Article  Google Scholar 

  34. Holtzblatt K, Wendell JB, Wood S (2004) Rapid contextual design: a how-to guide to key techniques for user-centered design. Elsevier, San Francisco

    Google Scholar 

  35. IJsselsteijn W, Van Den Hoogen W, Klimmt C et al (2008) Measuring the experience of digital game enjoyment. Proceedings of measuring behavior, pp 88–89

  36. Keay-Bright W (2007) Can computers create relaxation? Designing ReacTickles© software with children on the autistic spectrum. CoDesign 3:97–110. https://doi.org/10.1080/15710880601143443

    Article  Google Scholar 

  37. Keehn B, Müller RA, Townsend J (2013) Atypical attentional networks and the emergence of autism. Neurosci Biobehav Rev 37:164–183. https://doi.org/10.1016/j.neubiorev.2012.11.014

    Article  Google Scholar 

  38. Keizer AW, Verment RS, Hommel B (2010) Enhancing cognitive control through neurofeedback: a role of gamma-band activity in managing episodic retrieval. Neuroimage 49:3404–3413. https://doi.org/10.1016/j.neuroimage.2009.11.023

    Article  Google Scholar 

  39. Kim SK, Yoo EY, Lee JS et al (2014) The effects of neurofeedback training on concentration in children with attention deficit / hyperactivity disorder. Int J Bio-Science Bio-Technology 6:13–23

    Article  Google Scholar 

  40. Kouijzer MEJ, van Schie HT, de Moor JMH et al (2010) Neurofeedback treatment in autism. Preliminary findings in behavioral, cognitive, and neurophysiological functioning. Res Autism Spectr Disord 4:386–399. https://doi.org/10.1016/j.rasd.2009.10.007

    Article  Google Scholar 

  41. Laugwitz B, Held T, Schrepp M (2008) Construction and evaluation of a user experience questionnaire. HCI Usability Educ Work, pp 63–76. https://doi.org/10.1007/978-3-540-89350-9_6

  42. Lee K (2009) Evaluation of attention and relaxation levels of archers in shooting process using brain wave signal analysis algorithms. Sci Sensit 12:341–350

    Google Scholar 

  43. Lim C-W, Jung H-W (2013) A study on the military serious game. Adv Sci Technol Lett 39:73–77

    Google Scholar 

  44. Lim CG, Lee TS, Guan CT et al (2012) A brain-computer Interface based attention training program for treating attention deficit hyperactivity disorder. PLoS One 7:8. https://doi.org/10.1371/journal.pone.0046692

    Article  Google Scholar 

  45. Lubar JF (1995) Neurofeedback for the management of attention-deficit/hyperactivity disorders. In: Chapter in Schwartz MS (ed) Biofeedback: a practitioner’s guide. Guilford Press, New York, pp 493–522

  46. Lubar JF, Bahler WW (1976) Behavioral management of epileptic seizures following EEG biofeedback training of the sensorimotor rhythm. Biofeedback Self Regul 1:77–104. https://doi.org/10.1007/BF00998692

    Article  Google Scholar 

  47. Mandryk RL, Kalyn M, Dang Y et al (2012) Turning off-the-shelf games into biofeedback games. Proc 14th Int ACM SIGACCESS Conf Comput Access - ASSETS ’12, pp 199. https://doi.org/10.1145/2384916.2384952

  48. Mandryk RL, Dielschneider S, Kalyn MR et al (2013) Games as neurofeedback training for children with FASD. Proc 12th Int Conf Interact Des Child - IDC ’13, pp 165–172. https://doi.org/10.1145/2485760.2485762

  49. Mandryk RL, Nacke LE, Mandryk RL (2016) Biometrics in gaming and entertainment technologies. In: Biometrics a data driven world trends, Technol challenges, pp 191–224

    Chapter  Google Scholar 

  50. May T, Rinehart N, Wilding J, Cornish K (2013) The role of attention in the academic attainment of children with autism spectrum disorder. J Autism Dev Disord 43:2147–2158. https://doi.org/10.1007/s10803-013-1766-2

    Article  Google Scholar 

  51. Mihajlovic V, Grundlehner B, Vullers R, Penders J (2015) Wearable, wireless EEG solutions in daily life applications: what are we missing? IEEE J Biomed Heal Informatics 19:6–21. https://doi.org/10.1109/JBHI.2014.2328317

    Article  Google Scholar 

  52. Nouchi R, Taki Y, Takeuchi H et al (2012) Brain training game improves executive functions and processing speed in the elderly: a randomized controlled trial. PLoS One 7:e29676. https://doi.org/10.1371/journal.pone.0029676

    Article  Google Scholar 

  53. Ochs E, Kremer-Sadlik T, Sirota KG, Solomon O (2004) Autism and the social world: an anthropological perspective. Discourse Stud 6:147–183

    Article  Google Scholar 

  54. Owen AM, Hampshire A, Grahn JA et al (2010) Putting brain training to the test. Nature 465:775–778. https://doi.org/10.1038/nature09042

    Article  Google Scholar 

  55. Patsis G, Sahli H, Verhelst W, De Troyer O (2013) Evaluation of attention levels in a Tetris game using a brain computer interface. In: Carberry S, Weibelzahl S, Micarelli A, Semeraro G (eds) User modeling, adaptation, and personalization. Springer Berlin Heidelberg, Berlin, pp 127–138

    Chapter  Google Scholar 

  56. Pineda JA, Brang D, Hecht E et al (2008) Positive behavioral and electrophysiological changes following neurofeedback training in children with autism. Res Autism Spectr Disord 2:557–581. https://doi.org/10.1016/j.rasd.2007.12.003

    Article  Google Scholar 

  57. Planchon J, Vacher A, Comblet J et al (2018) Serious game training improves performance in combat life-saving interventions. Injury 49:86–92

    Article  Google Scholar 

  58. Pope AT, Palsson OS (2001) Helping video games “rewire our minds”. Int J Soc Psychiatry 54:370–382

    Google Scholar 

  59. Rauschenberger M, Schrepp M, Perez-Cota M et al (2013) Efficient measurement of the user experience of interactive products. How to use the user experience questionnaire (UEQ). Example: Spanish language version. Int J Interact Multimed Artif Intell 2:39–45. https://doi.org/10.9781/ijimai.2013.215

    Google Scholar 

  60. Read JC, MacFarlane S (2006) Using the fun toolkit and other survey methods to gather opinions in child computer interaction. In: Proceedings of the 2006 conference on Interaction design and children, pp 81–88

  61. Rebolledo-Mendez G, Dunwell I, Martínez-Mirón EA et al (2009) Assessing NeuroSky’s usability to detect attention levels in an assessment exercise. In: Jacko JA (ed) Human-computer interaction. New trends. Springer Berlin Heidelberg, Berlin, pp 149–158

    Chapter  Google Scholar 

  62. Rego P, Moreira PM, Reis LP (2010) Serious games for rehabilitation: a survey and a classification towards a taxonomy. In: Information Systems and Technologies (CISTI), 2010 5th Iberian Conference on, pp 1–6

  63. Renard Y, Lotte F, Gibert G et al (2010) OpenViBE : an open-source software platform to design , test , and use brain – computer interfaces in real and virtual environments. Presence 19:35–53. https://doi.org/10.1162/pres.19.1.35

    Article  Google Scholar 

  64. Sawyer B, Smith P (2008) Serious games taxonomy. In: Slides from the serious games summit at the game developers conference

  65. Schoneveld EA, Malmberg M, Lichtwarck-Aschoff A et al (2016) A neurofeedback video game (MindLight) to prevent anxiety in children: a randomized controlled trial. Comput Hum Behav 63:321–333. https://doi.org/10.1016/j.chb.2016.05.005

    Article  Google Scholar 

  66. Schwartz MS, Andrasik F (2017) Biofeedback: a practitioner’s guide. Guilford Publications, New York

    Google Scholar 

  67. Shaphiro SS, Wilk MB (1965) An analysis of variance test for normality. Biometrika 52:591–611

    Article  MathSciNet  Google Scholar 

  68. Sitaram R, Ros T, Stoeckel L et al (2017) Closed-loop brain training: the science of neurofeedback. Nat Rev Neurosci 18:86–100. https://doi.org/10.1038/nrn.2016.164

    Article  Google Scholar 

  69. Southam-Gerow MA, Kendall PC (2002) Emotion regulation and understanding: implications for child psychopathology and therapy. Clin Psychol Rev 22:189–222. https://doi.org/10.1016/S0272-7358(01)00087-3

    Article  Google Scholar 

  70. Spence C (2011) Crossmodal correspondences: a tutorial review. Atten Percept Psychophysiol 73:971–995. https://doi.org/10.3758/s13414-010-0073-7

    Article  Google Scholar 

  71. Steiner NJ, Frenette EC, Rene KM et al (2014) Neurofeedback and cognitive attention training for children with attention-deficit hyperactivity disorder in schools. J Dev Behav Pediatr 35:18–27. https://doi.org/10.1097/DBP.0000000000000009

    Article  Google Scholar 

  72. Strauss A, Corbin J (1998) Basics of qualitative research techniques. Sage publications, Thousand Oaks

    Google Scholar 

  73. Student (1908) The probable error of a mean. Biometrika 6:1–25

    Article  Google Scholar 

  74. Tan D, Nijholt A (2010) Brain-computer interfaces and human-computer interaction. In: Brain-computer interfaces. Springer, pp 3–19

  75. Tang ST, McCorkle R (2002) Use of family proxies in quality of life research for cancer patients at the end of life: a literature review. Cancer Investig 20:1086–1104

    Article  Google Scholar 

  76. Tang YY, Posner MI (2014) Training brain networks and states. Trends Cogn Sci 18:345–350. https://doi.org/10.1016/j.tics.2014.04.002

    Article  Google Scholar 

  77. Turkay S, Kinzer CK (2014) The effects of avatar-based customization on player identification. Int J Gaming Comput Simulations 6:1–25. https://doi.org/10.4018/ijgcms.2014010101

    Article  Google Scholar 

  78. van Steensel FJ, Bögels SM, Perrin S (2011) Anxiety disorders in children and adolescents with autistic spectrum disorders: a meta-analysis. Clin Child Fam Psychol Rev 14:302–317. https://doi.org/10.1007/s10567-011-0097-0

    Article  Google Scholar 

  79. Vernon D, Egner T, Cooper N et al (2003) The effect of training distinct neurofeedback protocols on aspects of cognitive performance. Int J Psychophysiol 47:75–85. https://doi.org/10.1016/S0167-8760(02)00091-0

    Article  Google Scholar 

  80. Wang Q, Sourina O, Nguyen MK (2010) EEG-based “serious” games design for medical applications. Proc - 2010 Int Conf Cyberworlds, CW 2010, pp 270–276. https://doi.org/10.1109/CW.2010.56

  81. Whyte EM, Smyth JM, Scherf KS (2015) Designing serious game interventions for individuals with autism. J Autism Dev Disord 45:3820–3831. https://doi.org/10.1007/s10803-014-2333-1

    Article  Google Scholar 

  82. Wilcoxon F (1946) Individual comparisons of grouped data by ranking methods. J Econ Entomol 39:269. https://doi.org/10.1093/jee/39.2.269

    Article  Google Scholar 

  83. Wilkinson P (2016) A brief history of serious games Phil. Entertain Comput Serious Games 9970, pp 17–41. https://doi.org/10.1007/978-3-319-46152-6

  84. Wilkinson N, Ang RP, Goh DH (2008) Online video game therapy for mental health concerns: a review. Int J Soc Psychiatry 54:370–382

    Article  Google Scholar 

  85. Yoon H, Park S-W, Lee Y-K, Jang J-H (2013) Emotion recognition of serious game players using a simple brain computer interface. 2013 Int Conf Inf Commun Technol Converg ICTC 2013, Oct 14, 2013 - Oct 16, 2013, pp 783–786. https://doi.org/10.1109/ICTC.2013.6675478

  86. Zalapa R, Tentori M (2013) Movement-based and tangible interactions to offer body awareness to children with autism. In: Urzaiz G, Ochoa SF, Bravo J et al (eds) Ubiquitous computing and ambient intelligence. Context-awareness and context-driven interaction. Springer International Publishing, Cham, pp 127–134

    Chapter  Google Scholar 

  87. Zickefoose S, Hux K, Brown J, Wulf K (2013) Let the games begin: a preliminary study using attention process training-3 and lumosity™ brain games to remediate attention deficits following traumatic brain injury. Brain Inj 27:707–716. https://doi.org/10.3109/02699052.2013.775484

    Article  Google Scholar 

  88. Zoefel B, Huster RJ, Herrmann CS (2011) Neurofeedback training of the upper alpha frequency band in EEG improves cognitive performance. Neuroimage 54:1427–1431. https://doi.org/10.1016/j.neuroimage.2010.08.078

    Article  Google Scholar 

Download references

Acknowledgments

We thank all the participants enrolled in this study and the researchers and reviewers who provide helpful comments on previous versions of this document. We also thank CONACYT for the first author fellowship and we thank to the CONACYT project #2209 of the fourth author for their financial support.

Author information

Authors and Affiliations

  1. Department of Computer Science, CICESE, Ensenada, B.C., Mexico

    Jose Mercado & Monica Tentori

  2. CICESE-UT3, Tepic, Nayarit, Mexico

    Ismael Espinosa-Curiel

  3. School of Engineering, CETYS Universidad, Tijuana, B.C., Mexico

    Lizbeth Escobedo

Authors
  1. Jose Mercado
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ismael Espinosa-Curiel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lizbeth Escobedo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Monica Tentori
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jose Mercado.

Ethics declarations

Conflict of interest

Authors declare that they have no conflict of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mercado, J., Espinosa-Curiel, I., Escobedo, L. et al. Developing and evaluating a BCI video game for neurofeedback training: the case of autism. Multimed Tools Appl 78, 13675–13712 (2019). https://doi.org/10.1007/s11042-018-6916-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-6916-2

Keywords

Search

Navigation