The current feasibility study examined the adherence, reliability, and assessment potential of an evidence-based game-like mobile Monitoring Tool (Akili Interactive Labs), to monitor 100 participants’ cognition for eight sessions at a summer camp for children with special needs. A validated measure of attention was administered at baseline. In the last session, participants completed an exit questionnaire. The Monitoring Tool was found to be enjoyable, and showed a high rate of adherence. No Monitor-related adverse events were reported. Monitor metrics showed good reliability across repeated measurements, indicating it is stable over long-term cognitive monitoring. There was evidence that the Monitoring Tool was able to detect differences in cognition between the children diagnosed with attention deficit hyperactivity disorder and autism spectrum disorders.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
Aggarwal, A., & Lillystone, D. (2000). A follow up pilot study of objective measures in children with attention deficit hyperactivity disorder. Journal of Pediatrics and Child Health, 36(2), 134–138. https://doi.org/10.1046/j.1440-1754.2000.00464.x.
American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th edn.). Arlington: American Psychiatric Publishing.
Anderson-Hanley, C., Tureck, K., & Schneiderman, R. (2011). Autism and exergaming: Effects on repetitive behaviors and cognition. Psychology Research and Behavior Management, 4, 129–137. https://doi.org/10.2147/PRBM.S24016.
Anguera, J. A., Boccanfuso, J., Rintoul, J. L., Al-Hashimi, O., Faraji, F., Janowich, J., Kong, E., Larraburo, Y., Johnston, E., & Gazzaley, A. (2013). Video game training enhances cognitive control in older adults. Nature, 501(7465), 97–101. https://doi.org/10.1038/nature12486.
Anguera, J. A., Brandes-Aitken, A. N., Rolle, C. E., Skinner, S. N., Desai, S. S., Bower, J. D., & Marco, E. J. (2016). Characterizing cognitive control abilities in children with 16p11. 2 deletion using adaptive ‘video game’ technology: a pilot study. Translational Psychiatry, 6(9), e893.
Anguera, J. A., Jordan, J. T., Castaneda, D., Gazzaley, A., & Areán, P. A. (2016). Conducting a fully mobile and randomized clinical trial for depression: Access, engagement and expense. BMJ Innovations, 2(1), 14–21. https://doi.org/10.1136/bmjinnov-2015-000098.
Atlis, M. M., Hahn, J., & Butcher, J. N. (2006). Computer-based assessment with the MMPI-2. In J. N. Butcher (Ed.), MMPI-2: The practitioner’s handbook (pp. 445–476). Washington, DC: American Psychological Association.
Bates, D., Maechler, M., Bolker, B., & Walker, S. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48. https://doi.org/10.18637/jss.v067.i01.
Bellotti, F., Kapralos, B., Lee, K., Moreno-Ger, P., & Berta, R. (2013). Assessment in and of serious games: An overview. Advances in Human-Computer Interaction. 2013, https://doi.org/10.1155/2013/136864.
Bioulac, S., Lallemand, S., Rizzo, A., Philip, P., Fabrigoule, C., & Bouvard, M. P. (2012). Impact of time on task on ADHD patient’s performances in a virtual classroom. European Journal of Pediatric Neurology, 16(5), 514–521. https://doi.org/10.1016/j.ejpn.2012.01.006.
Butcher, J. N. (2006). Assessment in clinical psychology: A perspective on the past, present challenges, and future prospects. Clinical Psychology: Science and Practice, 13(3), 205–209. https://doi.org/10.1111/j.1468-2850.2006.00025.x.
Centers for Disease Control and Prevention. (2010). Increasing prevalence of parent-reported attention-deficit/hyperactivity disorder among children - United States, 2003 and 2007. Morbidity and Mortality Weekly Report, 59(44), 1439–1443.
Centers for Disease Control and Prevention. (2014). Prevalence of autism spectrum disorder among children aged 8 years - Autism and developmental disabilities monitoring network, United States, 2010. Morbidity and Mortality Weekly Report, 63(2), 1–21.
Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment., 6(4), 284–290. https://doi.org/10.1037/1040-35220.127.116.114.
Corbett, B. A., Constantine, L. J., Hendren, R., Rocke, D., & Ozonoff, S. (2009). Examining executive functioning in children with autism spectrum disorder, attention deficit hyperactivity disorder and typical development. Psychiatry Research, 166(2), 210–222. https://doi.org/10.1016/j.psychres.2008.02.005.
Craig, F., Margari, F., Legrottaglie, A. R., Palumbi, R., De Giambattista, C., & Margari, L. (2016). A review of executive function deficits in autism spectrum disorder and attention-deficit/hyperactivity disorder. Neuropsychiatric Disease and Treatment, 12, 1191.
De Vries, M., Prins, P. J. M., Schmand, B. A., & Geurts, H. M. (2015). Working memory and cognitive flexibility-training for children with an autism spectrum disorder: A randomized controlled trial. Journal of Child Psychology and Psychiatry, 56, 566–576. https://doi.org/10.1111/jcpp.12324.
Deveau, J., Jaeggi, S. M., Zordan, V., Phung, C., & Seitz, A. R. (2015). How to build better memory training games. Frontiers in Systems Neuroscience, 8, 1–6. https://doi.org/10.3389/fnsys.2014.00243.
Diamond, A., & Liang, D. S. (2016). Conclusions about interventions, programs, and approaches for improving executive functions that appear justified and those that, despite much hype, do not. Developmental Cognitive Neuroscience, 18, 34–48. https://doi.org/10.1016/j.dcn.2015.11.005.
Dolan, R., Hall, T. E., Banerjee, M., Chun, E., & Strangman, N. (2005). Applying principles of universal design to test delivery: The effect of computer-based read-aloud on test performance of high school students with learning disabilities. The Journal of Technology, Learning and Assessment, 3(7), 3–32.
Dovis, S., Van der Oord, S., Wiers, R. W., & Prins, P. J. M. (2015). Improving executive functioning in children with ADHD: Training multiple executive functions within the context of a computer game. A randomized double-blind placebo controlled trial. PLoS ONE, 10(4), e0121651. https://doi.org/10.1371/journal.pone.0121651.
Frazier, T. W., Youngstrom, E. A., Glutting, J. J., & Watkins, M. W. (2007). ADHD and achievement: Meta-analysis of the child, adolescent, and adult literatures and a concomitant study with college students. Journal of Learning Disabilities, 40(1), 49–65. https://doi.org/10.1177/00222194070400010401.
Gamberini, L., Barresi, G., Maier, A., & Scarpetta, F. (2008). A game a day keeps the doctor away: A short review of computer games in mental healthcare. Journal of CyberTherapy and Rehabilitation, 1(2), 127–145.
Hanson, E., Bernier, R., Porche, K., Jackson, F. I., Goin-Kochel, R. P., Snyder, L. G., & Chen, Q. (2015). The cognitive and behavioral phenotype of the 16p11. 2 deletion in a clinically ascertained population. Biological Psychiatry, 77(9), 785–793.
Happé, F., Booth, R., Charlton, R., & Hughes, C. (2006). Executive function deficits in autism spectrum disorders and attention-deficit/hyperactivity disorder: Examining profiles across domains and ages. Brain and Cognition, 61(1), 25–39.
Kazmi, S., Ugail, H., Lesk, V., & Palmer, I. (2014). Interactive digital serious games for the assessment, rehabilitation, and prediction of dementia. International Journal of Computer Games Technology, 2014, 12.
Klingberg, T., Fernell, E., Olesen, P. J., Johnson, M., Gustafsson, P., Dahlstrçm, K., Gillberg, C. G., Forssberg, H., & Westerberg, H. (2005). Computerized training of working memory in children with ADHD – a randomized, controlled trial. Journal of the American Academy of Child and Adolescent Psychiatry, 44(2), 177–186.
Lange, B., Koenig, S., McConnell, E., Chang, C. Y., Juang, R., Suma, E., Bolas, M., & Rizzo, A. (2012). Interactive game-based rehabilitation using the Microsoft Kinect. In Virtual Reality Short Papers and Posters (VRW), 2012 IEEE (pp. 171–172). IEEE.
Lawson, R. A., Papadakis, A. A., Higginson, C. I., Barnett, J. E., Wills, M. C., Strang, J. F., Wallace, G. L., & Kenworthy, L. (2015). Everyday executive function impairments predict comorbid psychopathology in autism spectrum and attention deficit hyperactivity disorders. Neuropsychology, 29(3), 445. https://doi.org/10.1037/neu0000145.
Leek, M. R. (2001). Adaptive procedures in psychophysical research. Perception & psychophysics, 63(8), 1279–1292.
Lumsden, J., Edwards, E. A., Lawrence, N. S., Coyle, D., & Munafò, M. R. (2016a). Gamification of cognitive assessment and cognitive training: A systematic review of applications and efficacy. JMIR Serious Games. https://doi.org/10.2196/games.5888.
Matthews, T., King, M. L., Kupzyk, K., & Lake, C. (2014). Findings and implications of developmental screening for highrisk children referred to a tertiary developmental disability center. Journal of Pediatric Health Care, 28(6), 507–515. https://doi.org/10.1016/j.pedhc.2014.03.002.
Miyake, A., Friedman, N. P., Emerson, M. J., Witzki, A. H., Howerter, A., & Wager, T. D. (2000). The unity and diversity of executive functions and their contributions to complex “frontal lobe” tasks: A latent variable analysis. Cognitive Psychology, 41(1), 49–100. https://doi.org/10.1006/cogp.1999.0734.
Piotrowski, C. (1999). Assessment practices in the era of managed care: Current status and future directions. Journal of Clinical Psychology, 55(7), 787–796. https://doi.org/10.1002/(SICI)1097-4679(199907)55:7%3C787::AID-JCLP2%3E3.0.CO;2-U.
Rapport, M. D., Orban, S. A., Kofler, M. J., & Friedman, L. M. (2013). Do programs designed to train working memory, other executive functions, and attention benefit children with ADHD? A meta-analytic review of cognitive, academic, and behavioral outcomes. Clinical Psychology Review, 33, 1237–1252.
Rideout, V. (2016). Measuring time spent with media: The common sense census of media use by US 8- to 18-year-olds. Journal of Children and Media, 10(1), 138–144. https://doi.org/10.1080/17482798.2016.1129808.
Robinson, K. E., Kaizar, E., Catroppa, C., Godfrey, C., & Yeates, K. O. (2014). Systematic review and meta-analysis of cognitive interventions for children with central nervous system disorders and neurodevelopmental disorders. Journal of Pediatric Psychology, 39(8), 846–865.
Sansosti, F. J., & Powell-Smith, K. A. (2008). Using computer-presented social stories and video models to increase the social communication skills of children with high-functioning autism spectrum disorders. Journal of Positive Behavior Interventions, 10(3), 162–178. https://doi.org/10.1177/1098300708316259.
Tong, T., Chignell, M., Tierney, M. C., & Lee, J. (2016). A serious game for clinical assessment of cognitive status: Validation study. JMIR Serious Games, 4(1), e7. https://doi.org/10.2196/games.5006.
Tseng, H. M., Tiplady, B., Macleod, H. A., & Wright, P. (1998). Computer anxiety: A comparison of pen-based personal digital assistants, conventional computer and paper assessment of mood and performance. British Journal of Psychology, 89(4), 599–610. https://doi.org/10.1177/1098300708316259.
Watters, C., Oore, S., Shepherd, M., Abouzied, A., Cox, A., Kellar, M., & Otley, A. (2006). Extending the use of games in health care. In System Sciences, 2006. HICSS’06. Proceedings of the 39th Annual Hawaii International Conference on (Vol. 5, pp. 88b). IEEE.
Yerys, B. E., Wallace, G. L., Sokoloff, J. L., Shook, D. A., James, J. D., & Kenworthy, L. (2009). Attention deficit/hyperactivity disorder symptoms moderate cognition and behavior in children with autism spectrum disorders. Autism Research, 2(6), 322–333. https://doi.org/10.1002/aur.103.
Zelazo, P. D., Qu, L., Müller, U., Schneider, W., Schumann-Hengsteler, R., & Sodian, B. (2005). Hot and cool aspects of executive function: Relations in early development. In Young children’s cognitive development: Interrelationships among executive functioning, working memory, verbal ability, and theory of mind (pp. 71–93). New York: Psychology Press.
We would like to recognize the children, staff and leadership at Ramapo for Children for their participation. We would especially like to recognize the research assistants who conducted the study Laurel Kiyabu Morgan, Juliana Demicco, Gabi Verdoni, Adriana Gonzalez, Beatrice Baker, Angela Law, Dottie Bradshaw, Reeti Pal, Brianna Morales, and Gabbi Gaddy. In addition, we would like to thank Dr. Mary McKay for her mentorship on this project. This work was presented as a poster in October 2017 at the annual meeting for the American Academy of Child & Adolescent Psychiatry, Washington, D.C.
Conflict of interest
Rachel Flynn and Nirmaliz Colón-Acosta both were employees of Ramapo for Children, the data collection site, at various points of time. Jeffrey Bower is an employee of Akili Interactive whose product was used. In addition, Jimmy Zhou was an employee of Akili Interactive.
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Informed consent was obtained from all individual participants included in the study.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Flynn, R.M., Colón-Acosta, N., Zhou, J. et al. A Game-Based Repeated Assessment for Cognitive Monitoring: Initial Usability and Adherence Study in a Summer Camp Setting. J Autism Dev Disord 49, 2003–2014 (2019). https://doi.org/10.1007/s10803-019-03881-w
- Autism spectrum disorders
- Attention deficit hyperactivity disorder
- Learning disorders
- Clinical assessment
- Video game