Advertisement

Retest effects in working memory capacity tests: A meta-analysis

Article

Abstract

The repeated administration of working memory capacity tests is common in clinical and research settings. For cognitive ability tests and different neuropsychological tests, meta-analyses have shown that they are prone to retest effects, which have to be accounted for when interpreting retest scores. Using a multilevel approach, this meta-analysis aims at showing the reproducibility of retest effects in working memory capacity tests for up to seven test administrations, and examines the impact of the length of the test-retest interval, test modality, equivalence of test forms and participant age on the size of retest effects. Furthermore, it is assessed whether the size of retest effects depends on the test paradigm. An extensive literature search revealed 234 effect sizes from 95 samples and 68 studies, in which healthy participants between 12 and 70 years repeatedly performed a working memory capacity test. Results yield a weighted average of g = 0.28 for retest effects from the first to the second test administration, and a significant increase in effect sizes was observed up to the fourth test administration. The length of the test-retest interval and publication year were found to moderate the size of retest effects. Retest effects differed between the paradigms of working memory capacity tests. These findings call for the development and use of appropriate experimental or statistical methods to address retest effects in working memory capacity tests.

Keywords

Meta-analysis Retest effect Practice effect Working memory 

Notes

Acknowledgements

We thank David Darby, Susanne M. Jaeggi, Thomas W. Kaminski, Shu-Chen Li, Florian Schmiedek, Nash Unsworth, and Barbara A. Wilson for providing data on eligible studies. This work was partly supported by grant HO 1286/6-4 of the Deutsche Forschungsgemeinschaft.

Supplementary material

13423_2018_1461_MOESM1_ESM.pdf (81 kb)
(PDF 80.6 KB)
13423_2018_1461_MOESM2_ESM.docx (69 kb)
(DOCX 68.6 KB)

References

  1. Ackerman, P. L. (1987). Individual differences in skill learning: An integration of psychometric and information processing perspectives. Psychological Bulletin, 102(1), 3–27.  https://doi.org/10.1037/0033-2909.102.1.3 CrossRefGoogle Scholar
  2. Ackerman, P. L., Beier, M. E., & Boyle, M. O. (2005). Working memory and intelligence: the same or different constructs? Psychological Bulletin, 131(1), 30–60.  https://doi.org/10.1037/0033-2909.131.1.30 PubMedCrossRefGoogle Scholar
  3. Anastasi, A. (1981). Coaching, test sophistication, and developed abilities. American Psychologist, 36(10), 1086–1093.  https://doi.org/10.1037/0003-066X.36.10.1086 CrossRefGoogle Scholar
  4. Arendasy, M. E., & Sommer, M. (2013). Quantitative differences in retest effects across different methods used to construct alternate test forms. Intelligence, 41(3), 181–192.  https://doi.org/10.1016/j.intell.2013.02.004 CrossRefGoogle Scholar
  5. Arendasy, M. E., & Sommer, M. (2017). Reducing the effect size of the retest effect: Examining different approaches. Intelligence, 62, 89–98.  https://doi.org/10.1016/j.intell.2017.03.003 CrossRefGoogle Scholar
  6. Au, J., Sheehan, E., Tsai, N., Duncan, G. J., Buschkühl, M., & Jaeggi, S. M. (2015). Improving fluid intelligence with training on working memory: A meta-analysis. Psychonomic Bulletin & Review, 22(2), 366–377.  https://doi.org/10.3758/s13423-014-0699-x CrossRefGoogle Scholar
  7. Au, J., Buschkühl, M., Duncan, G. J., & Jaeggi, S. M. (2016). There is no convincing evidence that working memory training is not effective: A reply to Melby-Lervåg and Hulme (2015). Psychonomic Bulletin & Review, 23(1), 331–337.  https://doi.org/10.3758/s13423-015-0967-4 CrossRefGoogle Scholar
  8. Baddeley, A. D. (2000). The episodic buffer: A new component of working memory? Trends in Cognitive Sciences, 4(11), 417–423.  https://doi.org/10.1016/S1364-6613(00)01538-2 PubMedCrossRefGoogle Scholar
  9. Baddeley, A. D., & Hitch, G. (1974). Working memory. In G. H. Bower (Ed.),Psychology of learning and motivation (Vol. 8, pp. 47–89). New York: Academic Press.  https://doi.org/10.1016/S0079-7421(08)60452-1
  10. Ball, K., Edwards, J. D., & Ross, L. A. (2007). The impact of speed of processing training on cognitive and everyday functions. Journals of Gerontology: Series B, 62B(I), 19–31.CrossRefGoogle Scholar
  11. Baltes, P. B., & Kliegl, R. (1992). Further testing of limits of cognitive plasticity: Negative age differences in a mnemonic skill are robust. Developmental Psychology, 28(1), 121–125.CrossRefGoogle Scholar
  12. Bartels, C., Wegrzyn, M., Wiedl, A., Ackermann, V., & Ehrenreich, H. (2010). Practice effects in healthy adults: A longitudinal study on frequent repetitive cognitive testing. BMC Neuroscience, 11(1), 118.  https://doi.org/10.1186/1471-2202-11-118 PubMedPubMedCentralCrossRefGoogle Scholar
  13. Basso, M. R., Bornstein, R. A., & Lang, J. M. (1999). Practice effects on commonly used measures of executive function across twelve months. The Clinical Neuropsychologist, 13 (3), 283–292.  https://doi.org/10.1076/clin.13.3.283.1743 PubMedCrossRefGoogle Scholar
  14. Beckmann, B., Holling, H., & Kuhn, J. T. (2007). Reliability of verbal-numerical working memory tasks. Personality and Individual Differences, 43(4), 703–714.  https://doi.org/10.1016/j.paid.2007.01.011 CrossRefGoogle Scholar
  15. Beglinger, L. J., Gaydos, B., Tangphao-Daniels, O., Duff, K., Kareken, D. A., Crawford, J., & Siemers, E. R. (2005). Practice effects and the use of alternate forms in serial neuropsychological testing. Archives of Clinical Neuropsychology, 20(4), 517–529.  https://doi.org/10.1016/j.acn.2004.12.003 PubMedCrossRefGoogle Scholar
  16. Benedict, R. H. B., & Zgaljardic, D. J. (1998). Practice effects during repeated administrations of memory tests with and without alternate forms. Journal of Clinical and Experimental Neuropsychology, 20(3), 339–352.  https://doi.org/10.1076/jcen.20.3.339.822 PubMedCrossRefGoogle Scholar
  17. Borenstein, M. (2009). Effect sizes for continuous data. In H. Cooper, L. V. Hedges, & J. C. Valentine (Eds.), The handbook of research synthesis and meta-analysis (pp. 221–235). New York: Russell Sage Foundation.Google Scholar
  18. Braver, T. S., & Barch, D. M. (2002). A theory of cognitive control, aging cognition, and neuromodulation. Neuroscience & Biobehavioral Reviews, 26(7), 809–817.  https://doi.org/10.1016/S0149-7634(02)00067-2 CrossRefGoogle Scholar
  19. Brunoni, A. R., & Vanderhasselt, M. A. (2014). Working memory improvement with non-invasive brain stimulation of the dorsolateral prefrontal cortex: A systematic review and meta-analysis. Brain and Cognition, 86, 1–9.  https://doi.org/10.1016/j.bandc.2014.01.008 PubMedCrossRefGoogle Scholar
  20. Buschkühl, M. (2007). Arbeitsgedächtnistraining: Untersuchungen mit jungen und älteren Erwachsenen [Working memory trainings: Studies on younger and older adults] (Doctoral Dissertation). University of Bern, Bern.Google Scholar
  21. Calamia, M., Markon, K., & Tranel, D. (2012). Scoring higher the second time around: Meta-analyses of practice effects in neuropsychological assessment. The Clinical Neuropsychologist, 26(4), 543–570.  https://doi.org/10.1080/13854046.2012.680913 PubMedCrossRefGoogle Scholar
  22. Calamia, M., Markon, K., & Tranel, D. (2013). The robust reliability of neuropsychological measures: Meta-analyses of test–retest correlations. The Clinical Neuropsychologist, 27 (7), 1077–1105.  https://doi.org/10.1080/13854046.2013.809795 PubMedCrossRefGoogle Scholar
  23. Cattell, R. B. (1987) Intelligence: Its structure growth and action. Amsterdam: North-Holland.Google Scholar
  24. Chelune, G. J., Naugle, R. I., Lüders, H., Sedlak, J., & Awad, I. A. (1993). Individual change after epilepsy surgery: Practice effects and base-rate information. Neuropsychology, 7(1), 41–52.  https://doi.org/10.1037/0894-4105.7.1.41 CrossRefGoogle Scholar
  25. Cohen, J. (1988) Statistical power analysis for the behavioral sciences. Hilsdale: Erlbaum.Google Scholar
  26. Cohen, J. (1992). Statistical power analysis. Current Directions in Psychological Science, 1(3), 98–101.  https://doi.org/10.1111/1467-8721.ep10768783 CrossRefGoogle Scholar
  27. Collie, A., Maruff, P., Darby, D. G., & McStephen, M. (2003). The effects of practice on the cognitive test performance of neurologically normal individuals assessed at brief test-retest intervals. Journal of the International Neuropsychological Society, 9(3), 419–428.  https://doi.org/10.1017/S1355617703930074 PubMedCrossRefGoogle Scholar
  28. Conway, A. R. A., & Kane, M. J. (2001). Capacity, control and conflict: an individual differences perspective on attentional capture. In C. L. Folk, & B. S. Gibson (Eds.), Attraction, disctraction and action: Multiple perspectives on attentional capture (pp. 349–372). New York: Elsevier Science.  https://doi.org/10.1016/S0166-4115(01)80016-9
  29. Conway, A. R. A., Kane, M. J., Bunting, M. F., Hambrick, D. Z., Wilhelm, O., & Engle, R. W. (2005). Working memory span tasks: A methodological review and user’s guide. Psychonomic Bulletin & Review, 12 (5), 769–786.  https://doi.org/10.3758/BF03196772 CrossRefGoogle Scholar
  30. Cowan, N. (1999). An embedded-processes model of working memory. In A. E. Miyake, & P. E. Shah (Eds.), Models of working memory: Mechanisms of active maintenance and executive control (pp. 62–101). New York: Cambridge University Press.  https://doi.org/10.1017/CBO9781139174909.006
  31. Craik, F. I. M. (1986). A functional account of age differences in memory. In F. Klix, & H. Hagendorf (Eds.), Human memory and cognitive capabilities (pp. 409–422). Amsterdam: Elsevier Science.Google Scholar
  32. Cysique, L. A., Franklin, D., Abramson, I., Ellis, R. J., Letendre, S., Collier, A., … HNRC group (2011). Normative data and validation of a regression based summary score for assessing meaningful neuropsychological change. Journal of Clinical and Experimental Neuropsychology, 33(5), 505–522.  https://doi.org/10.1080/13803395.2010.535504
  33. Daneman, M., & Carpenter, P. A. (1980). Individual differences in working memory and reading. Journal of Verbal Learning and Verbal Behavior, 19 (4), 450–466.  https://doi.org/10.1016/S0022-5371(80)90312-6 CrossRefGoogle Scholar
  34. de Oliveira, R. S., Trezza, B. M., Busse, A. L., & Filho, W. J. (2014). Learning effect of computerized cognitive tests in older adults. Einstein (São Paulo), 12(2), 149–153.  https://doi.org/10.1590/s1679-45082014ao2954 CrossRefGoogle Scholar
  35. D’Esposito, M., Postle, B. R., & Rypma, B. (2000). Prefrontal cortical contributions to working memory: Evidence from event-related fMRI studies. Experimental Brain Research, 133(1), 3–11.  https://doi.org/10.1007/s002210000395 PubMedCrossRefGoogle Scholar
  36. Dunlap, W. P., Cortina, J. M., Vaslow, J. B., & Burke, M. J. (1996). Meta-analysis of experiments with matched groups or repeated measures designs. Psychological Methods, 1(2), 170–177.  https://doi.org/10.1037/1082-989X.1.2.170 CrossRefGoogle Scholar
  37. Dunning, D. L., & Holmes, J. (2014). Does working memory training promote the use of strategies on untrained working memory tasks? Memory & Cognition, 42(6), 854–862.  https://doi.org/10.3758/s13421-014-0410-5 CrossRefGoogle Scholar
  38. Engle, R. W. (2001). What is working memory capacity? In H. L. Roediger III (Ed.), The nature of remembering: Essays in honor of Robert G. Crowder (pp. 297–314). Washington, DC: American Psychological Association.  https://doi.org/10.1037/10394-016
  39. Freund, P. A., & Holling, H. (2011). How to get really smart: Modeling retest and training effects in ability testing using computer-generated figural matrix items. Intelligence, 39(4), 233–243.  https://doi.org/10.1016/j.intell.2011.02.009 CrossRefGoogle Scholar
  40. Friedman, N. P., & Miyake, A. (2004). The reading span test and its predictive power for reading comprehension ability. Journal of Memory and Language, 51(1), 136–158.  https://doi.org/10.1016/j.jml.2004.03.008 CrossRefGoogle Scholar
  41. Glisky, E. L. (2007). Brain aging: Models, methods, and mechanisms. In D. Riddle (Ed.), Changes in cognitive function in human aging (pp. 3–21). Boca Raton: CRC Press.Google Scholar
  42. Goldberg, T. E., Harvey, P. D., Wesnes, K. A., Snyder, P. J., & Schneider, L. S. (2015). Practice effects due to serial cognitive assessment: Implications for preclinical Alzheimer’s disease randomized controlled trials. Assessment & Disease Monitoring, 1(1), 103–111.  https://doi.org/10.1016/j.dadm.2014.11.003 CrossRefGoogle Scholar
  43. González, H. M., Tarraf, W., Bowen, M. E., Johnson-Jennings, M. D., & Fisher, G. G. (2013). What do parents have to do with my cognitive reserve? Life course perspectives on twelve-year cognitive decline. Neuroepidemiology, 41(2), 101–109.  https://doi.org/10.1159/000350723 PubMedCrossRefGoogle Scholar
  44. Green, C. S., Strobach, T., & Schubert, T. (2014). On methodological standards in training and transfer experiments. Psychological Research, 78(6), 756–772.  https://doi.org/10.1007/s00426-013-0535-3 PubMedCrossRefGoogle Scholar
  45. Guye, S., Simoni, C. D., & von Bastian, C. C. (2017). Do individual differences predict change in cognitive training performance? A latent growth curve modeling approach. Journal of Cognitive Enhancement, 1(4), 374–393.  https://doi.org/10.1007/s41465-017-0049-9 CrossRefGoogle Scholar
  46. Hausknecht, J. P., Halpert, J. A., Di Paolo, N. T., & Moriarty Gerrard, M. O. (2007). Retesting in selection: A meta-analysis of coaching and practice effects for tests of cognitive ability. Journal of Applied Psychology, 92(2), 373–385.  https://doi.org/10.1037/0021-9010.92.2.373 PubMedCrossRefGoogle Scholar
  47. Hedges, L. V. (1981). Distribution theory for Glass’s estimator of effect size and related estimators. Journal of Educational Statistics, 6(2), 107–128.  https://doi.org/10.2307/1164588 CrossRefGoogle Scholar
  48. Heilbronner, R. L., Sweet, J. J., Attix, D. K., Krull, K. R., Henry, G. K., & Hart, R. P. (2010). Official position of the American Academy of Clinical Neuropsychology on serial neuropsychological assessments: The utility and challenges of repeat test administrations in clinical and forensic contexts. The Clinical Neuropsychologist, 24(8), 1267–1278.  https://doi.org/10.1080/13854046.2010.526785 PubMedCrossRefGoogle Scholar
  49. Hunter, J. E., & Schmidt, F. L. (2004) Methods of meta-analysis: Correcting error and bias in research finding (2nd ed.). Thousand Oaks: Sage Publications.CrossRefGoogle Scholar
  50. Iddekinge, C. H. V., Morgeson, F. P., Schleicher, D. J., & Campion, M. A. (2011). Can I retake it? Exploring subgroup differences and criterion-related validity in promotion retesting. Journal of Applied Psychology, 96(5), 941–955.  https://doi.org/10.1037/a0023562 PubMedCrossRefGoogle Scholar
  51. Jaeggi, S. M., Buschkühl, M., Jonides, J., & Perrig, W. J. (2008). Improving fluid intelligence with training on working memory. Proceedings of the National Academy of Sciences of the United States of America, 105 (19), 6829–6833.  https://doi.org/10.1073/pnas.0801268105 PubMedPubMedCentralCrossRefGoogle Scholar
  52. Jensen, A. R. (1998) The g factor: The science of mental ability. Westport: Praeger Publishers/Greenwood Publishing Group.Google Scholar
  53. Jolles, D. D., Grol, M. J., van Buchem, M. A., Rombouts, S. A., & Crone, E. A. (2010). Practice effects in the brain: Changes in cerebral activation after working memory practice depend on task demands. NeuroImage, 52 (2), 658–668.  https://doi.org/10.1016/j.neuroimage.2010.04.028 PubMedCrossRefGoogle Scholar
  54. Kane, M. J., & Engle, R. W. (2002). The role of prefrontal cortex in working-memory capacity, executive attention, and general fluid intelligence: An individual-differences perspective. Psychonomic Bulletin & Review, 9(4), 637–671.  https://doi.org/10.3758/BF03196323 CrossRefGoogle Scholar
  55. Karbach, J., & Verhaeghen, P. (2014). Making working memory work: A meta-analysis of executive-control and working memory training in older adults. Psychological Science, 25(11), 2027–2037.  https://doi.org/10.1177/0956797614548725 PubMedPubMedCentralCrossRefGoogle Scholar
  56. Karch, D., Albers, L., Renner, G., Lichtenauer, N., & von Kries, R. (2013). The efficacy of cognitive training programs in children and adolescents. Deutsches Ärzteblatt International, 110(39), 543–652.Google Scholar
  57. Kelly, M. E., Loughrey, D., Lawlor, B. A., Robertson, I. H., Walsh, C., & Brennan, S. (2014). The impact of cognitive training and mental stimulation on cognitive and everyday functioning of healthy older adults: A systematic review and meta-analysis. Ageing Research Reviews, 15, 28–43.  https://doi.org/10.1016/j.arr.2014.02.004 PubMedCrossRefGoogle Scholar
  58. Kirchner, W. K. (1958). Age differences in short-term retention of rapidly changing information. Journal of Experimental Psychology, 55(4), 352–358.  https://doi.org/10.1037/h0043688 PubMedCrossRefGoogle Scholar
  59. Kliegl, R., & Baltes, P. B. (1987). Theory-guided analysis of mechanisms of development and aging through testing-the-limits and research on expertise. In C. Schooler (Ed.), Cognitive functioning and social structure over the life course (p. 95–119). Norwood: Ablex Publishing.Google Scholar
  60. Kliegl, R., Smith, J., & Baltes, P. B. (1989). Testing-the-limits and the study of adult age differences in cognitive plasticity of a mnemonic skill. Developmental Psychology, 25(2), 247–256.  https://doi.org/10.1037/0012-1649.25.2.247 CrossRefGoogle Scholar
  61. Kliegl, R., Maayr, U., & Krampe, R. (1994). Time-accuracy functions for determining process and person differences: An application to cognitive aging. Cognitive Psychology, 26(2), 134–164.  https://doi.org/10.1006/cogp.1994.1005 PubMedCrossRefGoogle Scholar
  62. Knight, R. G., McMahon, J., Skeaff, C. M., & Green, T. J. (2007). Reliable change index scores for persons over the age of 65 tested on alternate forms of the Rey AVLT. Archives of Clinical Neuropsychology, 22(4), 513–518.  https://doi.org/10.1016/j.acn.2007.03.005 PubMedCrossRefGoogle Scholar
  63. Kulik, J. A., Kulik, C.-l. C., & Bangert, R. L. (1984). Effects of practice on aptitude and achievement test scores. American Educational Research Journal, 21 (2), 435–447.  https://doi.org/10.2307/1162453 CrossRefGoogle Scholar
  64. Kyllonen, P. C., & Christal, R. E. (1990). Reasoning ability is (little more than) working-memory capacity?! Intelligence, 14(4), 389–433.  https://doi.org/10.1016/S0160-2896(05)80012-1 CrossRefGoogle Scholar
  65. Lampit, A., Hallock, H., & Valenzuela, M. (2014). Computerized cognitive training in cognitively healthy older adults: A systematic review and meta-analysis of effect modifiers. PLoS Medicine, 11(11), e1001756.  https://doi.org/10.1371/journal.pmed.1001756 PubMedPubMedCentralCrossRefGoogle Scholar
  66. Lange, S. (2013). Transfer von kognitivem Training in den Alltag bei älteren Erwachsenen [Transfer of cognitive training into everyday lives of elderly adults] (Unpublished doctoral dissertation). University of Magdeburg.Google Scholar
  67. Lievens, F., Buyse, T., & Sackett, P. R. (2005). Retest effects in operational selection settings: Development and test of a framework. Personnel Psychology, 58(4), 981–1007.  https://doi.org/10.1111/j.1744-6570.2005.00713.x CrossRefGoogle Scholar
  68. Lievens, F., Reeve, C. L., & Heggestad, E. D. (2007). An examination of psychometric bias due to retesting on cognitive ability tests in selection settings. Journal of Applied Psychology, 92(6), 1672–1682.  https://doi.org/10.1037/0021-9010.92.6.1672 PubMedCrossRefGoogle Scholar
  69. Lubinski, D. (2000). Scientific and social significance of assessing individual differences: ‘Sinking shafts at a few critical points’. Annual Review of Psychology, 51, 405–444.  https://doi.org/10.1146/annurev.psych.51.1.405 PubMedCrossRefGoogle Scholar
  70. Matton, N., Vautier, S., & Raufaste, É. (2009). Situational effects may account for gain scores in cognitive ability testing: A longitudinal SEM approach. Intelligence, 37(4), 412–421.  https://doi.org/10.1016/j.intell.2009.03.011 CrossRefGoogle Scholar
  71. Matton, N., Vautier, S., & Raufaste, É. (2011). Test-specificity of the advantage of retaking cognitive ability tests. International Journal of Selection and Assessment, 19(1), 11–17.  https://doi.org/10.1111/j.1468-2389.2011.00530.x CrossRefGoogle Scholar
  72. Mayr, U., & Kliegl, R. (1993). Sequential and coordinative complexity: Age-based processing limitations in figural transformations. Journal of Experimental Psychology: Learning, Memory, and Cognition, 19(6), 1297–1320.  https://doi.org/10.1037/0278-7393.19.6.1297 PubMedGoogle Scholar
  73. McCabe, D. P. (2008). The role of covert retrieval in working memory span tasks: Evidence from delayed recall tests. Journal of Memory and Language, 58(2), 480–494.  https://doi.org/10.1016/j.jml.2007.04.004 PubMedPubMedCentralCrossRefGoogle Scholar
  74. McCaffrey, R. J., Duff, K., & Westervelt, H. J. (2000) Practitioner’s guide to evaluating change with neuropsychological assessment instruments. New York: Springer Science & Business Media.Google Scholar
  75. Melby-Lervåg, M., & Hulme, C. (2013). Is working memory training effective? A meta-analytic review. Developmental Psychology, 49(2), 270–291.  https://doi.org/10.1037/a0028228 PubMedCrossRefGoogle Scholar
  76. Melby-Lervåg, M., & Hulme, C. (2016). There is no convincing evidence that working memory training is effective: A reply to Au et al. (2014) and Karbach and Verhaeghen (2014). Psychonomic Bulletin & Review, 23(1), 324–330.  https://doi.org/10.3758/s13423-015-0862-z CrossRefGoogle Scholar
  77. Morley, M. E., Bridgeman, B., & Lawless, R. R. (2004). Transfer between variants of quantitative items. ETS Research Report Series, 2004 (2), 1–27.  https://doi.org/10.1002/j.2333-8504.2004.tb01963.x CrossRefGoogle Scholar
  78. Morris, S. B. (2000). Distribution of the standardized mean change effect size for meta–analysis on repeated measures. British Journal of Mathematical and Statistical Psychology, 53(1), 17–29.  https://doi.org/10.1348/000711000159150 PubMedCrossRefGoogle Scholar
  79. Morris, S. B., & DeShon, R. P. (2002). Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs. Psychological Methods, 7(1), 105.  https://doi.org/10.1037/1082-989X.7.1.105 PubMedCrossRefGoogle Scholar
  80. Morrison, A. B., & Chein, J. M. (2011). Does working memory training work? The promise and challenges of enhancing cognition by training working memory. Psychonomic Bulletin & Review, 18(1), 46–60.  https://doi.org/10.3758/s13423-010-0034-0 CrossRefGoogle Scholar
  81. Oberauer, K. (1993). Die Koordination kognitiver Operationen—eine Studie über die Beziehung zwischen Intelligenz und ‘working memory.’ [The coordination of cognitive operations: A study on the relation between intelligence and ‘working memory’]. Zeitschrift für Psychologie mit Zeitschrift für angewandte Psychologie, 201(1), 57–84.Google Scholar
  82. Oberauer, K. (2009). Design for a working memory. In B. H. Ross (Ed.), The psychology of learning (Vol. 51, pp. 45–100). New York: Academic Press.  https://doi.org/10.1016/S0079-7421(09)51002-X
  83. Oberauer, K., & Kliegl, R. (2001). Beyond resources: Formal models of complexity effects and age differences in working memory. European Journal of Cognitive Psychology, 13(1-2), 187–215.  https://doi.org/10.1080/09541440042000278 CrossRefGoogle Scholar
  84. Oberauer, K., & Kliegl, R. (2006). A formal model of capacity limits in working memory. Journal of Memory and Language, 55(4), 601–626.  https://doi.org/10.1016/j.jml.2006.08.009 CrossRefGoogle Scholar
  85. Oberauer, K., Süß, H. M., Schulze, R., Wilhelm, O., & Wittmann, W. W. (2000). Working memory capacity—facets of a cognitive ability construct. Personality and Individual Differences, 29(6), 1017–1045.  https://doi.org/10.1016/S0191-8869(99)00251-2 CrossRefGoogle Scholar
  86. Oberauer, K., Süß, H. M., Wilhelm, O., & Wittman, W. W. (2003). The multiple faces of working memory: Storage, processing, supervision, and coordination. Intelligence, 31(2), 167–193.  https://doi.org/10.1016/S0160-2896(02)00115-0 CrossRefGoogle Scholar
  87. Oberauer, K., Schulze, R., Wilhelm, O., & Süß, H.-M. (2005). Working memory and intelligence—their correlation and their relation: Comment on Ackerman, Beier, and Boyle (2005). Psychological Bulletin, 131(1), 61–65.PubMedCrossRefGoogle Scholar
  88. Oberauer, K., Farrell, S., Jarrold, C., & Lewandowsky, S. (2016). What limits working memory capacity? Psychological Bulletin, 142(7), 758–799.  https://doi.org/10.1037/bul0000046 PubMedCrossRefGoogle Scholar
  89. Olkin, I., & Gleser, L. (2009). Stochastically dependent effect sizes. In H. Cooper, L.V. Hedges, & J. C. Valentine (Eds.), The handbook of research synthesis and meta-analysis (pp. 357–376). New York: Russell Sage Foundation.Google Scholar
  90. Peng, P., Namkung, J., Barnes, M., & Sun, C. (2015). A meta-analysis of mathematics and working memory: Moderating effects of working memory domain, type of mathematics skill, and sample characteristics. Journal of Educational Psychology, 108(4), 455–473.CrossRefGoogle Scholar
  91. Pollack, I., Johnson, L. B., & Knaff, P. R. (1959). Running memory span. Journal of Experimental Psychology, 57(3), 137–146.  https://doi.org/10.1037/h0046137 PubMedCrossRefGoogle Scholar
  92. Powers, K. L., Brooks, P. J., Aldrich, N. J., Palladino, M. A., & Alfieri, L. (2013). Effects of video-game play on information processing: A meta-analytic investigation. Psychonomic Bulletin & Review, 20(6), 1055–1079.  https://doi.org/10.3758/s13423-013-0418-z CrossRefGoogle Scholar
  93. R Core Team (2017). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna: Austria. http://www.R-project.org
  94. Randall, J. G., & Villado, A. J. (2017). Take two: Sources and deterrents of score change in employment retesting. Human Resource Management Review, 27, 536–553.CrossRefGoogle Scholar
  95. Redick, T. S. (2015). Working memory training and interpreting interactions in intelligence interventions. Intelligence, 50, 14–20.  https://doi.org/10.1016/j.intell.2015.01.014 CrossRefGoogle Scholar
  96. Roediger, H. L. III, & Karpicke, J. D. (2006). Test-enhanced learning: Taking memory tests improves long-term retention. Psychological Science, 17(3), 249–255.  https://doi.org/10.1111/j.1467-9280.2006.01693.x PubMedCrossRefGoogle Scholar
  97. Salthouse, T. A. (2010). Influence of age on practice effects in longitudinal neurocognitive change. Neuropsychology, 24(5), 563–572.  https://doi.org/10.1037/a0019026 PubMedPubMedCentralCrossRefGoogle Scholar
  98. Salthouse, T. A. (2011). Effects of age on time-dependent cognitive change. Psychological Science, 22(5), 682–688.  https://doi.org/10.1177/0956797611404900 PubMedPubMedCentralCrossRefGoogle Scholar
  99. Salthouse, T. A. (2015). Test experience effects in longitudinal comparisons of adult cognitive functioning. Developmental Psychology, 51(9), 1262–1270.  https://doi.org/10.1037/dev0000030 PubMedCrossRefGoogle Scholar
  100. Salthouse, T. A., Babcock, R. L., & Shaw, R. J. (1991). Effects of adult age on structural and operational capacities in working memory. Psychology and Aging, 6(1), 118.  https://doi.org/10.1037/0882-7974.6.1.118 PubMedCrossRefGoogle Scholar
  101. Salthouse, T. A., Schroeder, D. H., & Ferrer, E. (2004). Estimating retest effects in longitudinal assessments of cognitive functioning in adults between 18 and 60 years of age. Developmental Psychology, 40(5), 813–822.  https://doi.org/10.1037/0012-1649.40.5.813 PubMedCrossRefGoogle Scholar
  102. Salthouse, T. A., & Tucker-Drob, E. M. (2008). Implications of short-term retest effects for the interpretation of longitudinal change. Neuropsychology, 22(6), 800–811.  https://doi.org/10.1037/a0013091 PubMedPubMedCentralCrossRefGoogle Scholar
  103. Scharfen, J., Blum, D., & Holling, H. (2018). Response time reduction due to retesting in mental speed tests: A meta-analysis. Journal of Intelligence, 6(6).  https://doi.org/10.3390/jintelligence6010006
  104. Scharfen, J., Peters, J. M., & Holling, H. (2018). Retest effects in cognitive ability tests: A meta-analysis. Intelligence, 67, 44–66.  https://doi.org/10.1016/j.intell.2018.01.003 CrossRefGoogle Scholar
  105. Schleicher, D. J., Iddekinge, C. H. V., Morgeson, F. P., & Campion, M. A. (2010). If at first you don’t succeed, try, try again: Understanding race, age, and gender differences in retesting score improvement. Journal of Applied Psychology, 95(4), 603–617.  https://doi.org/10.1037/a0018920 PubMedCrossRefGoogle Scholar
  106. Schmidt, P. J., Keenan, P. A., Schenkel, L. A., Berlin, K., Gibson, C., & Rubinow, D. R. (2013). Cognitive performance in healthy women during induced hypogonadism and ovarian steroid addback. Archives of Women’s Mental Health, 16(1), 47–58.  https://doi.org/10.1007/s00737-012-0316-9 PubMedCrossRefGoogle Scholar
  107. Schmiedek, F., Lövdén, M., & Lindenberger, U. (2014). A task is a task is a task: Putting complex span, n-back, and other working memory indicators in psychometric context. Frontiers in Psychology, 5.  https://doi.org/10.3389/fpsyg.2014.01475
  108. Schuerger, J., & Witt, A. C. (1989). The temporal stability of individually tested intelligence. Journal of Clinical Psychology, 45(2), 294–302.CrossRefGoogle Scholar
  109. Shaffer, D. R., & Kipp, K. (2010) Developmental psychology: Childhood and adolescence (8th ed.). Belmont: Thomson Brooks/Cole Publishing Co.Google Scholar
  110. Shing, Y. L., Schmiedek, F., Lövdén, M., & Lindenberger, U. (2012). Memory updating practice across 100 days in the COGITO study. Psychology and Aging, 27(2), 451–461.  https://doi.org/10.1037/a0025568 PubMedCrossRefGoogle Scholar
  111. Shipstead, Z., Redick, T. S., & Engle, R. W. (2012). Is working memory training effective? Psychological Bulletin, 138(4), 628–654.  https://doi.org/10.1037/a0027473 PubMedCrossRefGoogle Scholar
  112. Smith, P. J., Blumenthal, J. A., Hoffman, B. M., Cooper, H., Strauman, T. A., Welsh-Bohmer, K., … Sherwood, A. (2010). Aerobic exercise and neurocognitive performance: A meta-analytic review of randomized controlled trials. Psychosomatic Medicine, 72(3), 239–252.  https://doi.org/10.1097/PSY.0b013e3181d14633
  113. Soveri, A., Antfolk, J., Karlsson, L., Salo, B., & Laine, M. (2017). Working memory training revisited: A mulitlevel meta-analysis of N-back training studies. Psychonomic Bulletin and Review.  https://doi.org/10.3758/s13423-016-1217-0
  114. Sterne, J. A., & Egger, M. (2001). Funnel plots for detecting bias in meta-analysis: Guidelines on choice of axis. Journal of Clinical Epidemiology, 54(10), 1046–1055.  https://doi.org/10.1016/S0895-4356(01)00377-8 PubMedCrossRefGoogle Scholar
  115. te Nijenhuis, J., van Vianen, A. E. M., & van der Flier, H. (2007). Score gains on g-loaded tests: No g. Intelligence, 35(3), 283–300.  https://doi.org/10.1016/j.intell.2006.07.006 CrossRefGoogle Scholar
  116. Thorgusen, S. R., Suchy, Y., Chelune, G. J., & Baucom, B. R. (2016). Neuropsychological practice effects in the context of cognitive decline: Contributions from learning and task novelty. Journal of the International Neuropsychological Society, 22(4), 453–466.  https://doi.org/10.1017/S1355617715001332 PubMedCrossRefGoogle Scholar
  117. Tombaugh, T. N. (2006). A comprehensive review of the Paced Auditory Serial Addition Test (PASAT). Archives of Clinical Neuropsychology, 21(1), 53–76.  https://doi.org/10.1016/j.acn.2005.07.006 PubMedCrossRefGoogle Scholar
  118. Toril, P., Reales, J. M., & Ballesteros, S. (2014). Video game training enhances cognition of older adults: A meta-analytic study. Psychology and Aging, 29(3), 706–716.  https://doi.org/10.1037/a0037507 PubMedCrossRefGoogle Scholar
  119. Towse, J. N., Hitch, G. J., & Hutton, U. (2000). On the interpretation of working memory span in adults. Memory & Cognition, 28(3), 341–348.  https://doi.org/10.3758/bf03198549 CrossRefGoogle Scholar
  120. Turley-Ames, K. J., & Whitfield, M. M. (2003). Strategy training and working memory task performance. Journal of Memory and Language, 49(4), 446–468.  https://doi.org/10.1016/S0749-596X(03)00095-0 CrossRefGoogle Scholar
  121. Turner, M. L., & Engle, R. W. (1989). Is working memory capacity task dependent? Journal of Memory and Language, 28(2), 127–154.  https://doi.org/10.1016/0749-596X(89)90040-5 CrossRefGoogle Scholar
  122. Unsworth, N., & Engle, R. W. (2007). The nature of individual differences in working memory capacity: Active maintenance in primary memory and controlled search from secondary memory. Psychological Review, 114 (1), 104–132.  https://doi.org/10.1037/0033-295X.114.1.104 PubMedCrossRefGoogle Scholar
  123. Unsworth, N., Fukuda, K., Awh, E., & Vogel, E. K. (2014). Working memory and fluid intelligence: Capacity, attention control, and secondary memory retrieval. Cognitive Psychology, 71, 1–26.  https://doi.org/10.1016/j.cogpsych.2014.01.003 PubMedPubMedCentralCrossRefGoogle Scholar
  124. van den Noortgate, W., López-López, J. A., Marín-Martínez, F., & Sánchez-Meca, J. (2015). Meta-analysis of multiple outcomes: A multilevel approach. Behavior Research Methods, 47(4), 1274–1294.  https://doi.org/10.3758/s13428-014-0527-2 CrossRefGoogle Scholar
  125. Verhaeghen, P., Cerella, J., & Basak, C. (2004). A working memory workout: How to expand the focus of serial attention from one to four items in 10 hours or less. Journal of Experimental Psychology: Learning, Memory, and Cognition, 30(6), 1322–1337.  https://doi.org/10.1037/0278-7393.30.6.1322 PubMedGoogle Scholar
  126. Viechtbauer, W. (2010). Conducting meta-analyses in R with the metafor package. Journal of Statistical Software, 36(3), 1–48.  https://doi.org/10.18637/jss.v036.i03.CrossRefGoogle Scholar
  127. Wang, P., Liu, H. H., Zhu, X. T., Meng, T., Li, H. J., & Zuo, X. N. (2016). Action video game training for healthy adults: A meta-analytic study. Frontiers in Psychology, 7.  https://doi.org/10.3389/fpsyg.2016.00907
  128. Wechsler, D. (2008) Wechsler Adult Intelligence Scale – Fourth Edition (WAIS–IV). San Antonio: Pearson Assessments.Google Scholar
  129. Wilhelm, O., Hildebrandt, A., & Oberauer, K. (2013). What is working memory capacity, and how can we measure it? Frontiers in Psychology, 4, 433.  https://doi.org/10.3389/fpsyg.2013.00433 PubMedPubMedCentralCrossRefGoogle Scholar
  130. Wilson, R. S., Li, Y., Bienias, L., & Bennett, D. A. (2006). Cognitive decline in old age: Separating retest effects from the effects of growing older. Psychology and Aging, 21(4), 774–789.  https://doi.org/10.1037/0882-7974.21.4.774 PubMedCrossRefGoogle Scholar
  131. Zehnder, F., Martin, M., Altgassen, M., & Clare, L. (2009). Memory training effects in old age as markers of plasticity: A meta-analysis. Restorative Neurology and Neuroscience, 27(5), 507–520.  https://doi.org/10.3233/RNN-2009-0491 PubMedGoogle Scholar

Copyright information

© Psychonomic Society, Inc. 2018

Authors and Affiliations

  1. 1.Institute of PsychologyWestfälische Wilhelms-Universität MünsterMuensterGermany

Personalised recommendations