The reliability and stability of visual working memory capacity

Article

Abstract

Because of the central role of working memory capacity in cognition, many studies have used short measures of working memory capacity to examine its relationship to other domains. Here, we measured the reliability and stability of visual working memory capacity, measured using a single-probe change detection task. In Experiment 1, the participants (N = 135) completed a large number of trials of a change detection task (540 in total, 180 each of set sizes 4, 6, and 8). With large numbers of both trials and participants, reliability estimates were high (α > .9). We then used an iterative down-sampling procedure to create a look-up table for expected reliability in experiments with small sample sizes. In Experiment 2, the participants (N = 79) completed 31 sessions of single-probe change detection. The first 30 sessions took place over 30 consecutive days, and the last session took place 30 days later. This unprecedented number of sessions allowed us to examine the effects of practice on stability and internal reliability. Even after much practice, individual differences were stable over time (average between-session r = .76).

Keywords

Visual working memory Reliability Change detection 

References

  1. Beckmann, B., Holling, H., & Kuhn, J.-T. (2007). Reliability of verbal–numerical working memory tasks. Personality and Individual Differences, 43, 703–714. doi:10.1016/j.paid.2007.01.011 CrossRefGoogle Scholar
  2. Brainard, D. H. (1997). The psychophysics toolbox. Spatial Vision, 10, 433–436. doi:10.1163/156856897X00357 PubMedCrossRefGoogle Scholar
  3. Brown, W. (1910). Some experimental results in the correlation of mental abilities. British Journal of Psychology, 1904–1920(3), 296–322. doi:10.1111/j.2044-8295.1910.tb00207.x Google Scholar
  4. Buschman, T. J., Siegel, M., Roy, J. E., & Miller, E. K. (2011). Neural substrates of cognitive capacity limitations. Proceedings of the National Academy of Sciences, 108, 11252–11255. doi:10.1073/pnas.1104666108 CrossRefGoogle Scholar
  5. Cowan, N. (2001). The magical number 4 in short-term memory: A reconsideration of mental storage capacity. Behavioral and Brain Sciences 24, 87–114–185. doi:10.1017/S0140525X01003922
  6. Cowan, N., Fristoe, N. M., Elliott, E. M., Brunner, R. P., & Saults, J. S. (2006). Scope of attention, control of attention, and intelligence in children and adults. Memory & Cognition, 34, 1754–1768. doi:10.3758/BF03195936 CrossRefGoogle Scholar
  7. Cramer, D. (1997). Basic statistics for social research: Step-by-step calculations and computer techniques using Minitab. London: Routledge.CrossRefGoogle Scholar
  8. Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297–334. doi:10.1007/BF02310555 CrossRefGoogle Scholar
  9. Cuthbert, B. N., & Kozak, M. J. (2013). Constructing constructs for psychopathology: The NIMH research domain criteria. Journal of Abnormal Psychology, 122, 928–937. doi:10.1037/a0034028 PubMedCrossRefGoogle Scholar
  10. Elmore, L. C., Magnotti, J. F., Katz, J. S., & Wright, A. A. (2012). Change detection by rhesus monkeys (Macaca mulatta) and pigeons (Columba livia). Journal of Comparative Psychology, 126, 203–212. doi:10.1037/a0026356 PubMedPubMedCentralCrossRefGoogle Scholar
  11. Engle, R. W., Tuholski, S. W., Laughlin, J. E., & Conway, A. R. A. (1999). Working memory, short-term memory, and general fluid intelligence: A latent-variable approach. Journal of Experimental Psychology: General, 128, 309–331. doi:10.1037/0096-3445.128.3.309 CrossRefGoogle Scholar
  12. Foster, J. L., Shipstead, Z., Harrison, T. L., Hicks, K. L., Redick, T. S., & Engle, R. W. (2015). Shortened complex span tasks can reliably measure working memory capacity. Memory & Cognition, 43, 226–236. doi:10.3758/s13421-014-0461-7 CrossRefGoogle Scholar
  13. Fukuda, K., Vogel, E., Mayr, U., & Awh, E. (2010). Quantity, not quality: The relationship between fluid intelligence and working memory capacity. Psychonomic Bulletin & Review, 17, 673–679. doi:10.3758/17.5.673 CrossRefGoogle Scholar
  14. Fukuda, K., Woodman, G. F., & Vogel, E. K. (2015). Individual differences in visual working memory capacity: Contributions of attentional control to storage. In P. Jolicœur, C. Lefebvre, & J. Martinez-Trujillo (Eds.), Mechanisms of sensory working memory: Attention and performance XXV (pp. 105–119). San Diego: Academic Press Elsevier.Google Scholar
  15. Gibson, B., Wasserman, E., & Luck, S. J. (2011). Qualitative similarities in the visual short-term memory of pigeons and people. Psychonomic Bulletin & Review, 18, 979–984. doi:10.3758/s13423-011-0132-7 CrossRefGoogle Scholar
  16. Gold, J. M., Wilk, C. M., McMahon, R. P., Buchanan, R. W., & Luck, S. J. (2003). Working memory for visual features and conjunctions in schizophrenia. Journal of Abnormal Psychology, 112, 61–71. doi:10.1037/0021-843X.112.1.61 PubMedCrossRefGoogle Scholar
  17. Hockey, A., & Geffen, G. (2004). The concurrent validity and test-retest reliability of a visuospatial working memory task. Intelligence, 32, 591–605. doi:10.1016/j.intell.2004.07.009
  18. Johnson, M. K., McMahon, R. P., Robinson, B. M., Harvey, A. N., Hahn, B., Leonard, C. J., & Gold, J. M. (2013). The relationship between working memory capacity and broad measures of cognitive ability in healthy adults and people with schizophrenia. Neuropsychology, 27, 220–229. doi:10.1037/a0032060 PubMedPubMedCentralCrossRefGoogle Scholar
  19. Klein, K., & Fiss, W. H. (1999). The reliability and stability of the Turner and Engle working memory task. Behavior Research Methods, Instruments, & Computers, 31, 429–432. doi:10.3758/BF03200722 CrossRefGoogle Scholar
  20. Lee, E.-Y., Cowan, N., Vogel, E. K., Rolan, T., Valle-Inclan, F., & Hackley, S. A. (2010). Visual working memory deficits in patients with Parkinson’s disease are due to both reduced storage capacity and impaired ability to filter out irrelevant information. Brain, 133, 2677–2689. doi:10.1093/brain/awq197 PubMedPubMedCentralCrossRefGoogle Scholar
  21. Luria, R., Balaban, H., Awh, E., & Vogel, E. K. (2016). The contralateral delay activity as a neural measure of visual working memory. Neuroscience & Biobehavioral Reviews, 62, 100–108. doi:10.1016/j.neubiorev.2016.01.003 CrossRefGoogle Scholar
  22. Mason, W., & Suri, S. (2012). Conducting behavioral research on Amazon’s Mechanical Turk. Behavior Research Methods, 44, 1–23. doi:10.3758/s13428-011-0124-6 PubMedCrossRefGoogle Scholar
  23. Melby-Lervåg, M., & Hulme, C. (2013). Is working memory training effective? A meta-analytic review. Developmental Psychology, 49, 270–291. doi:10.1037/a0028228 PubMedCrossRefGoogle Scholar
  24. Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York: McGraw-Hill.Google Scholar
  25. Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349, aac4716. doi:10.1126/science.aac4716 CrossRefGoogle Scholar
  26. Pailian, H., & Halberda, J. (2015). The reliability and internal consistency of one-shot and flicker change detection for measuring individual differences in visual working memory capacity. Memory & Cognition, 43, 397–420. doi:10.3758/s13421-014-0492-0 CrossRefGoogle Scholar
  27. Pashler, H. (1988). Familiarity and visual change detection. Perception & Psychophysics, 44, 369–378. doi:10.3758/BF03210419 CrossRefGoogle Scholar
  28. Reinhart, R. M. G., Heitz, R. P., Purcell, B. A., Weigand, P. K., Schall, J. D., & Woodman, G. F. (2012). Homologous mechanisms of visuospatial working memory maintenance in macaque and human: properties and sources. Journal of Neuroscience, 32, 7711–7722. doi:10.1523/JNEUROSCI.0215-12.2012 PubMedPubMedCentralCrossRefGoogle Scholar
  29. Rodebaugh, T. L., Scullin, R. B., Langer, J. K., Dixon, D. J., Huppert, J. D., Bernstein, A., & Lenze, E. J. (2016). Unreliability as a threat to understanding psychopathology: The cautionary tale of attentional bias. Journal of Abnormal Psychology, 125, 840–851. doi:10.1037/abn0000184 PubMedCrossRefGoogle Scholar
  30. Rouder, J. N. (n.d.). Applications and source code. Retrieved June 22, 2016, from http://pcl.missouri.edu/apps
  31. Rouder, J. N., Morey, R. D., Cowan, N., Zwilling, C. E., Morey, C. C., & Pratte, M. S. (2008). An assessment of fixed-capacity models of visual working memory. Proceedings of the National Academy of Sciences, 105, 5975–5979. doi:10.1073/pnas.0711295105 CrossRefGoogle Scholar
  32. Rouder, J. N., Morey, R. D., Morey, C. C., & Cowan, N. (2011). How to measure working memory capacity in the change detection paradigm. Psychonomic Bulletin & Review, 18, 324–330. doi:10.3758/s13423-011-0055-3 CrossRefGoogle Scholar
  33. Shipstead, Z., Redick, T. S., & Engle, R. W. (2012). Is working memory training effective? Psychological Bulletin, 138, 628–654. doi:10.1037/a0027473 PubMedCrossRefGoogle Scholar
  34. Sochat, V. V., Eisenberg, I. W., Enkavi, A. Z., Li, J., Bissett, P. G., & Poldrack, R. A. (2016). The experiment factory: Standardizing behavioral experiments. Frontiers in Psychology, 7, 610. doi:10.3389/fpsyg.2016.00610 PubMedPubMedCentralCrossRefGoogle Scholar
  35. Spearman, C. (1910). Correlation calculated from faulty data. British Journal of Psychology, 1904–1920(3), 271–295. doi:10.1111/j.2044-8295.1910.tb00206.x Google Scholar
  36. Todd, J. J., & Marois, R. (2004). Capacity limit of visual short-term memory in human posterior parietal cortex. Nature, 428, 751–754. doi:10.1038/nature02466 PubMedCrossRefGoogle Scholar
  37. Unsworth, N., Fukuda, K., Awh, E., & Vogel, E. K. (2014). Working memory and fluid intelligence: Capacity, attention control, and secondary memory retrieval. Cognitive Psychology, 71, 1–26. doi:10.1016/j.cogpsych.2014.01.003 PubMedPubMedCentralCrossRefGoogle Scholar
  38. Van Snellenberg, J. X., Conway, A. R. A., Spicer, J., Read, C., & Smith, E. E. (2014). Capacity estimates in working memory: Reliability and interrelationships among tasks. Cognitive, Affective, & Behavioral Neuroscience, 14, 106–116. doi:10.3758/s13415-013-0235-x CrossRefGoogle Scholar
  39. Vogel, E. K., & Machizawa, M. G. (2004). Neural activity predicts individual differences in visual working memory capacity. Nature, 428, 748–751. doi:10.1038/nature02447 PubMedCrossRefGoogle Scholar
  40. Waters, G. S., & Caplan, D. (1996). The measurement of verbal working memory capacity and its relation to reading comprehension. Quarterly Journal of Experimental Psychology, 49A, 51–75. doi:10.1080/713755607 CrossRefGoogle Scholar
  41. Wood, G., Hartley, G., Furley, P. A., & Wilson, M. R. (2016). Working memory capacity, visual attention and hazard perception in driving. Journal of Applied Research in Memory and Cognition, 5, 454–462. doi:10.1016/j.jarmac.2016.04.009 CrossRefGoogle Scholar

Copyright information

© Psychonomic Society, Inc. 2017

Authors and Affiliations

  1. 1.School of PsychologySouthwest UniversityChongqingChina
  2. 2.Department of PsychologyUniversity of ChicagoChicagoUSA

Personalised recommendations