Abstract
Commercial brain games are home- and computer-based cognitive trainings that are industrially offered and promise to enhance cognitive functioning by repeating cognitive tasks. Despite compelling evidence for the effectiveness of cognitive trainings in various domains and populations, the assumption of brain games’ effects on people’s minds has been challenged. However, there are only very few attempts to systematically evaluate the effectiveness of such games under ecologically valid training conditions. To approach this gap in the literature, we applied commercially available training tasks assumed to tap into working memory updating and capacity. The effectiveness of this training was measured by utilizing pre- and post-tests in trained tasks (criterion tasks), untrained transfer tasks from the assumed training domains (near-transfer tasks), as well as from the domains processing speed, shifting, inhibition, reasoning, and self-reported cognitive failures (far-transfer tasks). Training as well as pre-post-tests were completely administered home-based. In contrast to an active control group, a training group improved performance in the criterion tasks and near-transfer tasks. Improved performance was also evident in processing speed and shifting tasks (i.e., far-transfer tasks), but these improvements were not as conclusive as those in near-transfer tasks. Further, the number of reported cognitive failures was reduced in the training in contrast to the control group at post-test. Performance improvements were more pronounced for high-performing participants (i.e., magnification effects). In general, this study provides an evaluation of the effectiveness of a particular set of working-memory training tasks in an ecologically valid setting in the context of brain games.
Similar content being viewed by others
Notes
Since the score measures of task performance underlying most of the reports presented here represent rather arbitrary values (reflecting a performance compound related to speed, accuracy, difficulty level, etc.), a probably more psychologically interpretable variable is the difficulty level achieved by participants in each particular task. For example, the difficulty level in working memory tasks is typically defined by the amount of information that needs to be stored and manipulated (objects, numbers etc., see detailed task descriptions in the method section). We therefore additionally ran a MANOVA for the matched sample (excluding the two criterion tasks for a most conservative assessment of transfer effects) in which we used the achieved difficulty level (instead of scores) as a dependent measure for the tasks Restorer, Turning tops, Turnabout, IQube, and Missing Link. The remaining tasks (for which no difficulty level existed as a dependent variable) remained the same as in the previous analysis (Digit span, Stroop error effect, Stroop RT effect, d2, TMT 1, TMT 2). As a result, the MANOVA still revealed the crucial significant group*time interaction, F(11, 138) = 4.036, p < .001, ηp 2 = 243. The pattern of significant effects in the subsequent ANOVAs were the same as in the previous analysis: Again, there were significant group*time interactions for Restorer, F(1148) = 5.612, p = .019, ηp 2 = .037 (increase in difficulty level from 3.73 to 4.22 (SE = 0.08) in the training group vs. from 3.66 to 3.92 (SE = 0.08) in the control group), Turning tops, F(1148) = 34.527, p < .001, ηp 2 = .189 (increase in difficulty level from 1.05 to 2.36 (SE = 0.12) in the training group vs. from 1.20 to 1.67 (SE = 0.12) in the control group), TMT 1, F(1148) = 5.110, p = .025, ηp2 = .033, and TMT 2, F(1148) = 6.212, p = .014, ηp2 = .040. Thus, the increase in difficulty level from pre- to post-training for the training group was about two times the size of the corresponding increase in the control group for the two variables Restorer (where the difficulty level is directly associated with the number of objects to be remembered) and Turning Tops. There was neither a significant group*time interaction regarding the variable Turnabout, F(11, 138) = 3.443, p = .066, ηp 2 = .023, nor regarding the remainder of dependent variables (Digit span, Stroop error effect, Stroop RT effect, d2, IQube, Missing link), all F < 1.
To rule out the objection that our final matching procedure affected our results, we additionally followed up on the MANOVA regarding the full sample of participants that contributed pre- and post-test data. These analyses revealed (strikingly similar) post-hoc ANOVA results when compared to the matched sample. Specifically, significant group*time interactions emerged for the two variables that were part of both the pre-/post tests and the training sessions, Shuffler and Memory interrupted, indicating a direct training effect, F(1171) = 26.554, p < .001, ηp 2 = .134, and F(1171) = 141.867, p < .001, ηp 2 = .453, respectively. Post-hoc contrasts revealed a significant pre-training advantage for the control group (p = .010 and p = .005), but a significant post-test advantage of the training group (p = .035 and p < .001, respectively).
Furthermore, significant group*time interactions emerged for four dependent transfer variables: Restorer, F(1171) = 10.247, p = .002, ηp 2 = .057 (pre-test advantage of the control group, p = .009, but no significant post-test group difference, p = .359), Turning tops, F(1171) = 36.338, p < .001, ηp 2 = .175 (pre-test advantage of the control group, p = .022, post-test advantage of the training group, p = .014), TMT 1, F(1171) = 5.702, p = .018, ηp 2 = .032 (pre-test advantage of the control group, p = .008, no significant post-test group difference, p = .742), and TMT 2, F(1171) = 4.980, p = .027, ηp 2 = .028 (pre-test advantage of the control group, p = .003, no significant post-test group difference, p = .531). There were no significant interactions regarding the remainder of dependent variables (Turnabout, Digit span, Stroop error effect, Stroop RT effect, d2, IQube, Missing link), all Fs < 1. Taken together, it is important to note that the final (matched) selection of participants (in order to achieve comparable pre-training performance levels in the criterion tasks between groups) actually worked against our hypothesis of finding significantly greater training effects in the training group. When analyzing the complete set of pre-post data, in many cases a performance disadvantage of the training group in the pre-training assessment was turned into a performance advantage at post-test assessment.
References
A consensus on the brain training industry from the scientific community. (2014). Retrieved from http://longevity3.stanford.edu/blog/2014/10/15/the-consensus-on-the-brain-training-industry-from-the-scientific-community-2/ .
Anguera, J. A., Boccanfuso, J., Rintoul, J. L., Al-Hashimi, O., Faraji, F., Janowich, J., et al. (2013). Video game training enhances cognitive control in older adults. Nature, 501(7465), 97–101.
Arnett, J. A., & Labovitz, S. S. (1995). Effect of physical layout in performance of the trail making test. Psychological Assessment, 7(2), 220–221.
Au, J., Sheehan, E., Tsai, N., Duncan, G. J., Buschkuehl, M., & Jaeggi, S. M. (2015). Improving fluid intelligence with training on working memory: a meta-analysis. Psychonomic Bulletin & Review, 22(2), 366–377.
Baddeley, A. (1986). Working memory. Oxford: Clarendon Press.
Baddeley, A. (2003). Working memory: looking back and looking forward. Nature Reviews Neuroscience, 4(10), 829–839.
Baddeley, A. (2012). Working memory: theories, models, and controversies. Annual Review of Psychology, 63, 1–29.
Blomqvist, N. (1977). On the relation between change and initial value. Journal of the American Statistical Association, 72(360a), 746–749.
Boot, W. R., Blakely, D. P., & Simons, D. J. (2011). Do action video games improve perception and cognition? Frontiers in Psychology, 2, 226.
Brickenkamp, R. (1962). Test d2: Aufmerksamkeits-Belastungs-Test. Göttingen: Hogrefe.
Broadbent, D. E., Cooper, P. F., FitzGerald, P., & Parkes, K. R. (1982). The cognitive failures questionnaire (CFQ) and its correlates. British Journal of Clinical Psychology, 21(1), 1–16.
Buckley, D., Codina, C., Bhardwaj, P., & Pascalis, O. (2010). Action video game players and deaf observers have larger Goldmann visual fields. Vision Research, 50(5), 548–556.
Campbell, D. T., & Stanley, J. (1966). Experimental and quasi-experimental designs for research. Chicago: Rand McNally.
Carpenter, P. A., Just, M. A., & Shell, P. (1990). What one intelligence test measures: a theoretical account of the processing in the raven progressive matrices test. Psychological Review, 97(3), 404–431.
Caruso, J. C. (2004). A comparison of the reliabilities of four types of difference scores for five cognitive assessment batteries. European Journal of Psychological Assessment, 20(3), 166–171.
Castel, A. D., Pratt, J., & Drummond, E. (2005). The effects of action video game experience on the time course of inhibition of return and the efficiency of visual search. Acta Psychologica, 119(2), 217–230.
Chein, J. M., & Morrison, A. B. (2010). Expanding the mind’s workspace: training and transfer effects with a complex working memory span task. Psychonomic Bulletin & Review, 17(2), 193–199.
Cronbach, L. J., & Furby, L. (1970). How we should measure" change": or should we? Psychological Bulletin, 74(1), 68–80.
Dahlin, K. I. E. (2013). Working memory training and the effect on mathematical achievement in children with attention deficits and special needs. Journal of Education and Learning, 2(1), 118–133.
Dahlin, E., Neely, A. S., Larsson, A., Bäckman, L., & Nyberg, L. (2008a). Transfer of learning after updating training mediated by the striatum. Science, 320(5882), 1510–1512.
Dahlin, E., Nyberg, L., Bäckman, L., & Neely, A. S. (2008b). Plasticity of executive functioning in young and older adults: Immediate training gains, transfer, and long-term maintenance. Psychology and Aging, 23(4), 720–730.
Daneman, M., & Carpenter, P. A. (1980). Individual differences in working memory and reading. Journal of Verbal Learning and Verbal Behavior, 19(4), 450–466.
Diamond, A. (2013). Executive functions. Annual Review of Psychology, 64, 135–168.
Dye, M. W., Green, C. S., & Bavelier, D. (2009). Increasing speed of processing with action video games. Current Directions in Psychological Science, 18(6), 321–326.
Engle, R. W., Carullo, J. J., & Collins, K. W. (1991). Individual differences in working memory for comprehension and following directions. The Journal of Educational Research, 84(5), 253–262.
Green, C. S., & Bavelier, D. (2003). Action video game modifies visual selective attention. Nature, 423(6939), 534–537.
Green, C. S., & Bavelier, D. (2006a). Effect of action video games on the spatial distribution of visuospatial attention. Journal of Experimental Psychology: Human Perception and Performance, 32(6), 1465–1478.
Green, C. S., & Bavelier, D. (2006b). Enumeration versus multiple object tracking: the case of action video game players. Cognition, 101(1), 217–245.
Green, C. S., & Bavelier, D. (2007). Action-video-game experience alters the spatial resolution of vision. Psychological Science, 18(1), 88–94.
Green, C. S., Strobach, T., & Schubert, T. (2014). On methodological standards in training and transfer experiments. Psychological Research, 78(6), 756–772.
Halford, G. S., Cowan, N., & Andrews, G. (2007). Separating cognitive capacity from knowledge: a new hypothesis. Trends in Cognitive Sciences, 11(6), 236–242.
Hardy, J. L., Nelson, R. A., Thomason, M. E., Sternberg, D. A., Katovich, K., Farzin, F., & Scanlon, M. (2015). Enhancing cognitive abilities with comprehensive training: a large, online, randomized, active-controlled trial. PLoS One, 10(9), e0134467.
Hedge, C., Powell, G., Sumner, P. (2017). The reliability paradox: Why robust cognitive tasks do not produce reliable individual differences. Behavior Research Methods.(In press)
Holmes, J., Gathercole, S. E., Dunning, D. L. (2009). Adaptive training leads to sustained enhancement of poor working memory in children. Developmental science, 12(4), 9–15.
Jaeggi, S. M., Buschkuehl, M., Jonides, J., & Perrig, W. J. (2008). Improving fluid intelligence with training on working memory. Proceedings of the National Academy of Sciences, 105(19), 6829–6833.
Jaeggi, S. M., Studer-Luethi, B., Buschkuehl, M., Su, Y.-F., Jonides, J., & Perrig, W. J. (2010). The relationship between n-back performance and matrix reasoning—Implications for training and transfer. Intelligence, 38(6), 625–635.
Karbach, J., & Kray, J. (2009). How useful is executive control training? Age differences in near and far transfer of task-switching training. Developmental Science, 12(6), 978–990.
Karbach, J., Strobach, T., & Schubert, T. (2015). Adaptive working-memory training benefits reading, but not mathematics in middle childhood. Child Neuropsychology, 21(3), 285–301.
Kiesel, A., Steinhauser, M., Wendt, M., Falkenstein, M., Jost, K., Philipp, A. M., & Koch, I. (2010). Control and interference in task switching—a review. Psychological Bulletin, 136(5), 849–874.
Klingberg, T. (2010). Training and plasticity of working memory. Trends in Cognitive Sciences, 14(7), 317–324.
Klingberg, T., Fernell, E., Olesen, P. J., Johnson, M., Gustafsson, P., Dahlström, K., et al. (2005). Computerized training of working memory in children with ADHD-a randomized, controlled trial. Journal of the American Academy of Child & Adolescent Psychiatry, 44(2), 177–186.
Koenen, T., Strobach, T., & Karbach, J. (2016). Working memory training. In T. Strobach & K. J. (Eds.), Cognitive training: An overview of features and applications. New York: Springer.
Kristjánsson, Á. (2013). The case for causal influences of action videogame play upon vision and attention. Attention, Perception, & Psychophysics, 75(4), 667–672.
Li, R., Polat, U., Makous, W., & Bavelier, D. (2009). Enhancing the contrast sensitivity function through action video game training. Nature Neuroscience, 12(5), 549–551.
MacLeod, C. M. (1991). Half a century of research on the Stroop effect: an integrative review. Psychological Bulletin, 109(2), 163–203.
Mahncke, H. W., Connor, B. B., Appelman, J., Ahsanuddin, O. N., Hardy, J. L., Wood, R. A., et al. (2006). Memory enhancement in healthy older adults using a brain plasticity-based training program: a randomized, controlled study. Proceedings of the National Academy of Sciences, 103(33), 12523–12528.
Melby-Lervåg, M., & Hulme, C. (2013). Is working memory training effective? A meta-analytic review. Developmental Psychology, 49(2), 270–291.
Morrison, A. B., & Chein, J. M. (2011). Does working memory training work? The promise and challenges of enhancing cognition by training working memory. Psychonomic Bulletin & Review, 18(1), 46–60.
Nouchi, R., Taki, Y., Takeuchi, H., Hashizume, H., Akitsuki, Y., Shigemune, Y., et al. (2012). Brain training game improves executive functions and processing speed in the elderly: a randomized controlled trial. PLoS One, 7(1), e29676.
Oberauer, K. (2009). Design for a working memory. Psychology of Learning and Motivation, 51, 45–100.
Oldham, P. (1962). A note on the analysis of repeated measurements of the same subjects. Journal of Chronic Diseases, 15(10), 969–977.
Olesen, P. J., Westerberg, H., & Klingberg, T. (2004). Increased prefrontal and parietal activity after training of working memory. Nature Neuroscience, 7(1), 75–79.
Owen, A. M., Hampshire, A., Grahn, J. A., Stenton, R., Dajani, S., Burns, A. S., et al. (2010). Putting brain training to the test. Nature, 465(7299), 775–778.
Rabipour, S., & Raz, A. (2012). Training the brain: fact and fad in cognitive and behavioral remediation. Brain and Cognition, 79(2), 159–179.
Rebok, G. W., Carlson, M. C., & Langbaum, J. B. (2007). Training and maintaining memory abilities in healthy older adults: traditional and novel approaches. The Journals of Gerontology Series B: Psychological Sciences and Social Sciences, 62(1), 53–61.
Richmond, L. L., Morrison, A. B., Chein, J. M., & Olson, I. R. (2011). Working memory training and transfer in older adults. Psychology and Aging, 26(4), 813–822.
Rogosa, D. (1995). Myths and methods: “myths about longitudinal research” plus supplemental questions. In J. M. Gottman (Ed.), The analysis of change (pp. 3–66). Hillsdale: Lawrence Ealbaum Associates.
Salminen, T., Frensch, P., Strobach, T., & Schubert, T. (2016a). Age-specific differences of dual n-back training. Aging, Neuropsychology, and Cognition, 23(1), 18–39.
Salminen, T., Kühn, S., Frensch, P. A., & Schubert, T. (2016b). Transfer after dual n-back training depends on striatal activation change. Journal of Neuroscience, 36(39), 10198–10213.
Schmiedek, F. (2016). Methods and designs. In T. Strobach & J. Karbach (Eds.), Cognitive training: An overview of features and applications. New York: Springer.
Schmiedek, F., Bauer, C., Lövdén, M., Brose, A., & Lindenberger, U. (2010a). Cognitive enrichment in old age: web-based training programs. GeroPsych: The Journal of Gerontopsychology and Geriatric Psychiatry, 23(2), 59–67.
Schmiedek, F., Lövdén, M., & Lindenberger, U. (2010b). Hundred days of cognitive training enhance broad cognitive abilities in adulthood: findings from the COGITO study. Frontiers in Aging Neuroscience, 2, 27.
Schubert, T., Finke, K., Redel, P., Kluckow, S., Müller, H., & Strobach, T. (2015). Video game experience and its influence on visual attention parameters: an investigation using the framework of the theory of visual attention (TVA). Acta Psychologica, 157, 200–214.
Shipstead, Z., Redick, T. S., & Engle, R. W. (2012). Is working memory training effective? Psychological Bulletin, 138(4), 628–654.
Simons, D. J., Boot, W. R., Charness, N., Gathercole, S. E., Chabris, C. F., Hambrick, D. Z., & Stine-Morrow, E. A. (2016). Do “brain-training” programs work? Psychological Science in the Public Interest, 17(3), 103–186.
Spence, I., Yu, J. J., Feng, J., & Marshman, J. (2009). Women match men when learning a spatial skill. Journal of Experimental Psychology: Learning, Memory, and Cognition, 35(4), 1097–1103.
Stanovich, K. E. (1986). Matthew effects in reading: Some consequences of individual differences in the acquisition of literacy. Reading research quarterly 21(4), 360–407.
Steinborn, M., Langner, R., Flehmig, H. C., & Huestegge, L. (2016). Everyday life cognitive instability predicts simple reaction time variability: analysis of reaction time distributions and delta plots. Applied Cognitive Psychology, 30(1), 92–102.
Steinborn, M., Langner, R., Flehmig, H., Huestegge, L. (2017). Methodology of performance scoring in the d2 sustained-attention test: Cumulative-reliability functions and practical guidelines. Psychological Assessment (In press).
Strobach, T., & Karbach, J. (2016). Cognitive training: an overview of features and applications. New York: Springer.
Strobach, T., Frensch, P. A., & Schubert, T. (2012a). Video game practice optimizes executive control skills in dual-task and task switching situations. Acta Psychologica, 140(1), 13–24.
Strobach, T., Frensch, P. A., Soutschek, A., & Schubert, T. (2012b). Investigation on the improvement and transfer of dual-task coordination skills. Psychological Research, 76(6), 794–811.
Stroop, J. R. (1935). Studies of interference in serial verbal reactions. Journal of Experimental Psychology, 18(6), 643–662.
Taatgen, N. A. (2013). The nature and transfer of cognitive skills. Psychological Review, 120(3), 439–471.
Thorndike, E. L. (1924). The influence of the chance imperfections of measures upon the relation of initial score to gain or loss. Journal of Experimental Psychology, 7(3), 225–232.
Torous, J., Staples, P., Fenstermacher, E., Dean, J., & Keshavan, M. (2016). Barriers, benefits, and beliefs of brain training smartphone apps: an internet survey of younger US consumers. Frontiers in Human Neuroscience, 10, 180.
Tu, Y. K., & Gilthorpe, M. S. (2007). Revisiting the relation between change and initial value: A review and evaluation. Statistics in Medicine, 26(2), 443–457.
van Muijden, J., Band, G. P., & Hommel, B. (2012). Online games training aging brains: Limited transfer to cognitive control functions. Frontiers in Human Neuroscience, 6, 221.
von Bastian, C. C., & Oberauer, K. (2013). Distinct transfer effects of training different facets of working memory capacity. Journal of Memory and Language, 69(1), 36–58.
Walberg, H. J., & Tsai, S.-L. (1983). Matthew effects in education. American Educational Research Journal, 20(3), 359–373.
Wechsler, D. (2008). Wechsler adult intelligence scale-fourth. San Antonio: Pearson.
Westerberg, H., Jacobaeus, H., Hirvikoski, T., Clevberger, P., Östensson, M.-L., Bartfai, A., & Klingberg, T. (2007). Computerized working memory training after stroke—a pilot study. Brain Injury, 21(1), 21–29.
Wilhelm, O., Hildebrandt, A., & Oberauer, K. (2013). What is working memory, and how can we measure it? Frontiers in Psychology, 4, 433.
Willett, J. B. (1988). Questions and answers in the measurement of change. Review of Research in Education, 15(1), 345–422.
Acknowledgements
We would like to thank www.neuronation.com and Rouwen Hirth for their technical support during data collection. Correspondence concerning this article should be addressed to Tilo Strobach, Medical School Hamburg, Department of Psychology, Am Kaiserkai 1, 20457 Hamburg, Germany. E-mail may be sent to tilo.strobach@medicalschool-hamburg.de.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interest
TS and LH have full positions at Medical School Hamburg and at University of Würzburg, respectively. The authors have no financial interest in this study and have no financial disclosure to the participating company. They do not hold shares in this company.
Appendices
Appendix A
Appendix B
Rights and permissions
About this article
Cite this article
Strobach, T., Huestegge, L. Evaluating the Effectiveness of Commercial Brain Game Training with Working-Memory Tasks. J Cogn Enhanc 1, 539–558 (2017). https://doi.org/10.1007/s41465-017-0053-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s41465-017-0053-0