A Game-Based Repeated Assessment for Cognitive Monitoring: Initial Usability and Adherence Study in a Summer Camp Setting
The current feasibility study examined the adherence, reliability, and assessment potential of an evidence-based game-like mobile Monitoring Tool (Akili Interactive Labs), to monitor 100 participants’ cognition for eight sessions at a summer camp for children with special needs. A validated measure of attention was administered at baseline. In the last session, participants completed an exit questionnaire. The Monitoring Tool was found to be enjoyable, and showed a high rate of adherence. No Monitor-related adverse events were reported. Monitor metrics showed good reliability across repeated measurements, indicating it is stable over long-term cognitive monitoring. There was evidence that the Monitoring Tool was able to detect differences in cognition between the children diagnosed with attention deficit hyperactivity disorder and autism spectrum disorders.
KeywordsAutism spectrum disorders Attention deficit hyperactivity disorder Learning disorders Cognition Clinical assessment Video game
We would like to recognize the children, staff and leadership at Ramapo for Children for their participation. We would especially like to recognize the research assistants who conducted the study Laurel Kiyabu Morgan, Juliana Demicco, Gabi Verdoni, Adriana Gonzalez, Beatrice Baker, Angela Law, Dottie Bradshaw, Reeti Pal, Brianna Morales, and Gabbi Gaddy. In addition, we would like to thank Dr. Mary McKay for her mentorship on this project. This work was presented as a poster in October 2017 at the annual meeting for the American Academy of Child & Adolescent Psychiatry, Washington, D.C.
RF conceived of the study, planned its design, coordinated data collection, participated in data interpretation, and drafted the manuscript; NC participated in the study design, coordinated data collection and helped draft the manuscript; JB planned the study design, performed the statistical analysis, and participated in data interpretation; JZ performed the statistical analysis. All authors read, edited and approved the final manuscript.
Compliance with Ethical Standards
Conflict of interest
Rachel Flynn and Nirmaliz Colón-Acosta both were employees of Ramapo for Children, the data collection site, at various points of time. Jeffrey Bower is an employee of Akili Interactive whose product was used. In addition, Jimmy Zhou was an employee of Akili Interactive.
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Informed consent was obtained from all individual participants included in the study.
- Anguera, J. A., Boccanfuso, J., Rintoul, J. L., Al-Hashimi, O., Faraji, F., Janowich, J., Kong, E., Larraburo, Y., Johnston, E., & Gazzaley, A. (2013). Video game training enhances cognitive control in older adults. Nature, 501(7465), 97–101. https://doi.org/10.1038/nature12486.CrossRefPubMedCentralGoogle Scholar
- Anguera, J. A., Brandes-Aitken, A. N., Rolle, C. E., Skinner, S. N., Desai, S. S., Bower, J. D., & Marco, E. J. (2016). Characterizing cognitive control abilities in children with 16p11. 2 deletion using adaptive ‘video game’ technology: a pilot study. Translational Psychiatry, 6(9), e893.CrossRefPubMedCentralGoogle Scholar
- Atlis, M. M., Hahn, J., & Butcher, J. N. (2006). Computer-based assessment with the MMPI-2. In J. N. Butcher (Ed.), MMPI-2: The practitioner’s handbook (pp. 445–476). Washington, DC: American Psychological Association.Google Scholar
- Centers for Disease Control and Prevention. (2010). Increasing prevalence of parent-reported attention-deficit/hyperactivity disorder among children - United States, 2003 and 2007. Morbidity and Mortality Weekly Report, 59(44), 1439–1443.Google Scholar
- Centers for Disease Control and Prevention. (2014). Prevalence of autism spectrum disorder among children aged 8 years - Autism and developmental disabilities monitoring network, United States, 2010. Morbidity and Mortality Weekly Report, 63(2), 1–21.Google Scholar
- Corbett, B. A., Constantine, L. J., Hendren, R., Rocke, D., & Ozonoff, S. (2009). Examining executive functioning in children with autism spectrum disorder, attention deficit hyperactivity disorder and typical development. Psychiatry Research, 166(2), 210–222. https://doi.org/10.1016/j.psychres.2008.02.005.CrossRefPubMedCentralGoogle Scholar
- De Vries, M., Prins, P. J. M., Schmand, B. A., & Geurts, H. M. (2015). Working memory and cognitive flexibility-training for children with an autism spectrum disorder: A randomized controlled trial. Journal of Child Psychology and Psychiatry, 56, 566–576. https://doi.org/10.1111/jcpp.12324.CrossRefGoogle Scholar
- Diamond, A., & Liang, D. S. (2016). Conclusions about interventions, programs, and approaches for improving executive functions that appear justified and those that, despite much hype, do not. Developmental Cognitive Neuroscience, 18, 34–48. https://doi.org/10.1016/j.dcn.2015.11.005.CrossRefGoogle Scholar
- Dolan, R., Hall, T. E., Banerjee, M., Chun, E., & Strangman, N. (2005). Applying principles of universal design to test delivery: The effect of computer-based read-aloud on test performance of high school students with learning disabilities. The Journal of Technology, Learning and Assessment, 3(7), 3–32.Google Scholar
- Dovis, S., Van der Oord, S., Wiers, R. W., & Prins, P. J. M. (2015). Improving executive functioning in children with ADHD: Training multiple executive functions within the context of a computer game. A randomized double-blind placebo controlled trial. PLoS ONE, 10(4), e0121651. https://doi.org/10.1371/journal.pone.0121651.CrossRefPubMedCentralGoogle Scholar
- Frazier, T. W., Youngstrom, E. A., Glutting, J. J., & Watkins, M. W. (2007). ADHD and achievement: Meta-analysis of the child, adolescent, and adult literatures and a concomitant study with college students. Journal of Learning Disabilities, 40(1), 49–65. https://doi.org/10.1177/00222194070400010401.CrossRefGoogle Scholar
- Gamberini, L., Barresi, G., Maier, A., & Scarpetta, F. (2008). A game a day keeps the doctor away: A short review of computer games in mental healthcare. Journal of CyberTherapy and Rehabilitation, 1(2), 127–145.Google Scholar
- Kazmi, S., Ugail, H., Lesk, V., & Palmer, I. (2014). Interactive digital serious games for the assessment, rehabilitation, and prediction of dementia. International Journal of Computer Games Technology, 2014, 12.Google Scholar
- Klingberg, T., Fernell, E., Olesen, P. J., Johnson, M., Gustafsson, P., Dahlstrçm, K., Gillberg, C. G., Forssberg, H., & Westerberg, H. (2005). Computerized training of working memory in children with ADHD – a randomized, controlled trial. Journal of the American Academy of Child and Adolescent Psychiatry, 44(2), 177–186.CrossRefGoogle Scholar
- Lange, B., Koenig, S., McConnell, E., Chang, C. Y., Juang, R., Suma, E., Bolas, M., & Rizzo, A. (2012). Interactive game-based rehabilitation using the Microsoft Kinect. In Virtual Reality Short Papers and Posters (VRW), 2012 IEEE (pp. 171–172). IEEE.Google Scholar
- Lawson, R. A., Papadakis, A. A., Higginson, C. I., Barnett, J. E., Wills, M. C., Strang, J. F., Wallace, G. L., & Kenworthy, L. (2015). Everyday executive function impairments predict comorbid psychopathology in autism spectrum and attention deficit hyperactivity disorders. Neuropsychology, 29(3), 445. https://doi.org/10.1037/neu0000145.CrossRefGoogle Scholar
- Matthews, T., King, M. L., Kupzyk, K., & Lake, C. (2014). Findings and implications of developmental screening for highrisk children referred to a tertiary developmental disability center. Journal of Pediatric Health Care, 28(6), 507–515. https://doi.org/10.1016/j.pedhc.2014.03.002.CrossRefGoogle Scholar
- Miyake, A., Friedman, N. P., Emerson, M. J., Witzki, A. H., Howerter, A., & Wager, T. D. (2000). The unity and diversity of executive functions and their contributions to complex “frontal lobe” tasks: A latent variable analysis. Cognitive Psychology, 41(1), 49–100. https://doi.org/10.1006/cogp.1999.0734.CrossRefGoogle Scholar
- Piotrowski, C. (1999). Assessment practices in the era of managed care: Current status and future directions. Journal of Clinical Psychology, 55(7), 787–796. https://doi.org/10.1002/(SICI)1097-4679(199907)55:7%3C787::AID-JCLP2%3E3.0.CO;2-U.CrossRefGoogle Scholar
- Rapport, M. D., Orban, S. A., Kofler, M. J., & Friedman, L. M. (2013). Do programs designed to train working memory, other executive functions, and attention benefit children with ADHD? A meta-analytic review of cognitive, academic, and behavioral outcomes. Clinical Psychology Review, 33, 1237–1252.CrossRefGoogle Scholar
- Sansosti, F. J., & Powell-Smith, K. A. (2008). Using computer-presented social stories and video models to increase the social communication skills of children with high-functioning autism spectrum disorders. Journal of Positive Behavior Interventions, 10(3), 162–178. https://doi.org/10.1177/1098300708316259.CrossRefGoogle Scholar
- Tseng, H. M., Tiplady, B., Macleod, H. A., & Wright, P. (1998). Computer anxiety: A comparison of pen-based personal digital assistants, conventional computer and paper assessment of mood and performance. British Journal of Psychology, 89(4), 599–610. https://doi.org/10.1177/1098300708316259.CrossRefGoogle Scholar
- Watters, C., Oore, S., Shepherd, M., Abouzied, A., Cox, A., Kellar, M., & Otley, A. (2006). Extending the use of games in health care. In System Sciences, 2006. HICSS’06. Proceedings of the 39th Annual Hawaii International Conference on (Vol. 5, pp. 88b). IEEE.Google Scholar
- Yerys, B. E., Wallace, G. L., Sokoloff, J. L., Shook, D. A., James, J. D., & Kenworthy, L. (2009). Attention deficit/hyperactivity disorder symptoms moderate cognition and behavior in children with autism spectrum disorders. Autism Research, 2(6), 322–333. https://doi.org/10.1002/aur.103.PubMedCentralGoogle Scholar
- Zelazo, P. D., Qu, L., Müller, U., Schneider, W., Schumann-Hengsteler, R., & Sodian, B. (2005). Hot and cool aspects of executive function: Relations in early development. In Young children’s cognitive development: Interrelationships among executive functioning, working memory, verbal ability, and theory of mind (pp. 71–93). New York: Psychology Press.Google Scholar