Attention, Perception, & Psychophysics

, Volume 76, Issue 7, pp 1962–1974 | Cite as

Working memory resources are shared across sensory modalities

  • V. R. Salmela
  • M. Moisala
  • K. Alho


A common assumption in the working memory literature is that the visual and auditory modalities have separate and independent memory stores. Recent evidence on visual working memory has suggested that resources are shared between representations, and that the precision of representations sets the limit for memory performance. We tested whether memory resources are also shared across sensory modalities. Memory precision for two visual (spatial frequency and orientation) and two auditory (pitch and tone duration) features was measured separately for each feature and for all possible feature combinations. Thus, only the memory load was varied, from one to four features, while keeping the stimuli similar. In Experiment 1, two gratings and two tones—both containing two varying features—were presented simultaneously. In Experiment 2, two gratings and two tones—each containing only one varying feature—were presented sequentially. The memory precision (delayed discrimination threshold) for a single feature was close to the perceptual threshold. However, as the number of features to be remembered was increased, the discrimination thresholds increased more than twofold. Importantly, the decrease in memory precision did not depend on the modality of the other feature(s), or on whether the features were in the same or in separate objects. Hence, simultaneously storing one visual and one auditory feature had an effect on memory precision equal to those of simultaneously storing two visual or two auditory features. The results show that working memory is limited by the precision of the stored representations, and that working memory can be described as a resource pool that is shared across modalities.


Working memory Attention Load Precision Psychophysics Discrimination threshold Audio–visual Auditory features Visual features Cross-modal 


Author Note

This work was supported by the Academy of Finland (Grant No. 260054). We thank Daryl Fougnie and two anonymous reviewers for their helpful comments.


  1. Anderson, D. E., & Awh, E. (2012). The plateau in mnemonic resolution across large set sizes indicates discrete resource limits in visual working memory. Attention, Perception, & Psychophysics, 74, 891–910. doi: 10.3758/s13414-012-0292-1 CrossRefGoogle Scholar
  2. Anderson, D. E., Vogel, E. K., & Awh, E. (2011). Precision in visual working memory reaches a stable plateau when individual item limits are exceeded. Journal of Neuroscience, 31, 1128–1138. doi: 10.1523/JNEUROSCI.4125-10.2011 PubMedCrossRefGoogle Scholar
  3. Baddeley, A. (1986). Working memory. Oxford, UK: Oxford University Press, Clarendon Press.Google Scholar
  4. Baddeley, A. (2010). Working memory. Current Biology, 20, R136–R140. doi: 10.1016/j.cub.2009.12.014 PubMedCrossRefGoogle Scholar
  5. Baddeley, A. D., & Hitch, G. J. (1974). Working memory. In G. H. Bower (Ed.), The psychology of learning and motivation: Advances in research and theory (Vol. 8, pp. 47–89). New York, NY: Academic Press.Google Scholar
  6. Bays, P. M. (2014). Noise in neural populations accounts for errors in working memory. Journal of Neuroscience, 34, 3632–3645. doi: 10.1523/JNEUROSCI.3204-13.2014 PubMedCrossRefPubMedCentralGoogle Scholar
  7. Bays, P. M., Catalao, R. F., & Husain, M. (2009). The precision of visual working memory is set by allocation of a shared resource. Journal of Vision, 9(10), 7. doi: 10.1167/9.10.7. 1–11.PubMedCrossRefPubMedCentralGoogle Scholar
  8. Bays, P. M., & Husain, M. (2008). Dynamic shifts of limited working memory resources in human vision. Science, 321, 851–854. doi: 10.1126/science.1158023 PubMedCrossRefPubMedCentralGoogle Scholar
  9. Brooks, L. R. (1968). Spatial and verbal components of the act of recall. Canadian Journal of Experimental Psychology, 22, 349–368. doi: 10.1037/h0082775 CrossRefGoogle Scholar
  10. Carandini, M., & Heeger, D. J. (2012). Normalization as a canonical neural computation. Nature Reviews Neuroscience, 13, 51–62. doi: 10.1038/nrn3136 CrossRefGoogle Scholar
  11. Cocchini, G., Logie, R. H., Della Sala, S., MacPherson, S. E., & Baddeley, A. D. (2002). Concurrent performance of two memory tasks: Evidence for domain-specific working memory systems. Memory & Cognition, 30, 1086–1095. doi: 10.3758/BF03194326 CrossRefGoogle Scholar
  12. Cowan, N. (1997). Attention and memory. New York, NY: Oxford University Press.Google Scholar
  13. Cowan, N. (2001). The magical number 4 in short-term memory: A reconsideration of mental storage capacity. Behavioral and Brain Sciences, 24, 87–114. doi: 10.1017/S0140525X01003922. disc. 114–185.PubMedCrossRefGoogle Scholar
  14. Cowan, N. (2011). The focus of attention as observed in visual working memory tasks: Making sense of competing claims. Neuropsychologia, 49, 1401–1406. doi: 10.1016/j.neuropsychologia.2011.01.035 PubMedCrossRefPubMedCentralGoogle Scholar
  15. Degerman, A., Rinne, T., Pekkola, J., Autti, T., Jaaskelainen, I. P., Sams, M., & Alho, K. (2007). Human brain activity associated with audiovisual perception and attention. NeuroImage, 34, 1683–1691. doi: 10.1016/j.neuroimage.2006.11.019 PubMedCrossRefGoogle Scholar
  16. Fougnie, D., Asplund, C. L., & Marois, R. (2010). What are the units of storage in visual working memory? Journal of Vision, 10(12), 27. doi: 10.1167/10.12.27 PubMedCrossRefPubMedCentralGoogle Scholar
  17. Fougnie, D., Cormiea, S. M., & Alvarez, G. A. (2013). Object-based benefits without object-based representations. Journal of Experimental Psychology: General, 142, 621–626. doi: 10.1037/a0030300 CrossRefGoogle Scholar
  18. Fougnie, D., & Marois, R. (2006). Distinct capacity limits for attention and working memory: Evidence from attentive tracking and visual working memory paradigms. Psychological Science, 17, 526–534. doi: 10.1111/j.1467-9280.2006.01739.x PubMedCrossRefGoogle Scholar
  19. Fougnie, D., & Marois, R. (2011). What limits working memory capacity? Evidence for modality-specific sources to the simultaneous storage of visual and auditory arrays. Journal of Experimental Psychology: Learning, Memory, and Cognition, 37, 1329–1341. doi: 10.1037/a0024834 PubMedPubMedCentralGoogle Scholar
  20. Fougnie, D., Suchow, J. W., & Alvarez, G. A. (2012). Variability in the quality of visual working memory. Nature Communication, 3, 1229. doi: 10.1038/ncomms2237 CrossRefGoogle Scholar
  21. Franconeri, S. L., Alvarez, G. A., & Cavanagh, P. (2013). Flexible cognitive resources: Competitive content maps for attention and memory. Trends in Cognitive Sciences, 17, 134–141. doi: 10.1016/j.tics.2013.01.010 PubMedCrossRefGoogle Scholar
  22. Huang, L. (2010a). Visual working memory is better characterized as a distributed resource rather than discrete slots. Journal of Vision, 10(14), 8. doi: 10.1167/10.14.8 PubMedCrossRefGoogle Scholar
  23. Huang, L. (2010b). What is the unit of visual attention? Object for selection, but Boolean map for access. Journal of Experimental Psychology: General, 139, 162–179. doi: 10.1037/a0018034 CrossRefGoogle Scholar
  24. Kroll, N. E., Parks, T., Parkinson, S. R., Bieber, S. L., & Johnson, A. L. (1970). Short-term memory while shadowing: Recall of visually and of aurally presented letters. Journal of Experimental Psychology, 85, 220–224. doi: 10.1037/h0029544 PubMedCrossRefGoogle Scholar
  25. Kumar, S., Joseph, S., Pearson, B., Teki, S., Fox, Z. V., Griffiths, T. D., & Husain, M. (2013). Resource allocation and prioritization in auditory working memory. Cognitive Neuroscience, 4, 12–20. doi: 10.1080/17588928.2012.716416 PubMedCrossRefPubMedCentralGoogle Scholar
  26. Luck, S. J., & Vogel, E. K. (1997). The capacity of visual working memory for features and conjunctions. Nature, 390, 279–281. doi: 10.1038/36846 PubMedCrossRefGoogle Scholar
  27. Luck, S. J., & Vogel, E. K. (1998). Response from Luck and Vogel. Trends in Cognitive Sciences, 2, 78–79.PubMedCrossRefGoogle Scholar
  28. Machizawa, M. G., Goh, C. C., & Driver, J. (2012). Human visual short-term memory precision can be varied at will when the number of retained items is low. Psychological Science, 23, 554–559. doi: 10.1177/0956797611431988 PubMedCrossRefGoogle Scholar
  29. Morey, C. C., & Cowan, N. (2004). When visual and verbal memories compete: Evidence of cross-domain limits in working memory. Psychonomic Bulletin & Review, 11, 296–301. doi: 10.3758/BF03196573 CrossRefGoogle Scholar
  30. Morey, C. C., & Cowan, N. (2005). When do visual and verbal memories conflict? The importance of working-memory load and retrieval. Journal of Experimental Psychology: Learning, Memory, and Cognition, 31, 703–713. doi: 10.1037/0278-7393.31.4.703 PubMedPubMedCentralGoogle Scholar
  31. Morey, C. C., Cowan, N., Morey, R. D., & Rouder, J. N. (2011). Flexible attention allocation to visual and auditory working memory tasks: Manipulating reward induces a trade-off. Attention, Perception, & Psychophysics, 73, 458–472. doi: 10.3758/s13414-010-0031-4 CrossRefGoogle Scholar
  32. Murray, A. M., Nobre, A. C., Astle, D. E., & Stokes, M. G. (2012). Lacking control over the trade-off between quality and quantity in visual short-term memory. PLoS ONE, 7, e41223. doi: 10.1371/journal.pone.0041223 PubMedCrossRefPubMedCentralGoogle Scholar
  33. Oberauer, K., & Eichenberger, S. (2013). Visual working memory declines when more features must be remembered for each object. Memory & Cognition, 41, 1212–1227. doi: 10.3758/s13421-013-0333-6 CrossRefGoogle Scholar
  34. Olson, I. R., & Jiang, Y. (2002). Is visual short-term memory object based? Rejection of the “strong-object” hypothesis. Perception & Psychophysics, 64, 1055–1067. doi: 10.3758/BF03194756 CrossRefGoogle Scholar
  35. Palmer, J. (1990). Attentional limits on the perception and memory of visual information. Journal of Experimental Psychology: Human Perception and Performance, 16, 332–350. doi: 10.1037/0096-1523.16.2.332 PubMedGoogle Scholar
  36. Reynolds, J. H., & Heeger, D. J. (2009). The normalization model of attention. Neuron, 61, 168–185. doi: 10.1016/j.neuron.2009.01.002 PubMedCrossRefPubMedCentralGoogle Scholar
  37. Salmela, V. R., Lähde, M., & Saarinen, J. (2012). Visual working memory for amplitude-modulated shapes. Journal of Vision, 12(6), 2. doi: 10.1167/12.6.2 PubMedCrossRefGoogle Scholar
  38. Salmela, V. R., Mäkelä, T., & Saarinen, J. (2010). Human working memory for shapes of radial frequency patterns. Vision Research, 50, 623–629. doi: 10.1016/j.visres.2010.01.014 PubMedCrossRefGoogle Scholar
  39. Salmela, V. R., & Saarinen, J. (2013). Detection of small orientation changes and the precision of visual working memory. Vision Research, 76, 17–24. doi: 10.1016/j.visres.2012.10.003 PubMedCrossRefGoogle Scholar
  40. Saults, J. S., & Cowan, N. (2007). A central capacity limit to the simultaneous storage of visual and auditory arrays in working memory. Journal of Experimental Psychology: General, 136, 663–684. doi: 10.1037/0096-3445.136.4.663 CrossRefGoogle Scholar
  41. Scarborough, D. L. (1972). Stimulus modality effects on forgetting in short-term memory. Journal of Experimental Psychology, 95, 285–289. doi: 10.1037/h0033667 PubMedCrossRefGoogle Scholar
  42. Sims, C. R., Jacobs, R. A., & Knill, D. C. (2012). An ideal observer analysis of visual working memory. Psychological Review, 119, 807–830. doi: 10.1037/a0029856 PubMedCrossRefPubMedCentralGoogle Scholar
  43. van den Berg, R., Shin, H., Chou, W. C., George, R., & Ma, W. J. (2012). Variability in encoding precision accounts for visual short-term memory limitations. Proceedings of the National Academy of Sciences, 109, 8780–8785. doi: 10.1073/pnas.1117465109 CrossRefGoogle Scholar
  44. Van der Burg, E., Awh, E., & Olivers, C. N. (2013). The capacity of audiovisual integration is limited to one item. Psychological Science, 24, 345–351. doi: 10.1177/0956797612452865 PubMedCrossRefGoogle Scholar
  45. Vergauwe, E., Barrouillet, P., & Camos, V. (2010). Do mental processes share a domain-general resource? Psychological Science, 21, 384–390. doi: 10.1177/0956797610361340 PubMedCrossRefGoogle Scholar
  46. Wheeler, M. E., & Treisman, A. M. (2002). Binding in short-term visual memory. Journal of Experimental Psychology: General, 131, 48–64. doi: 10.1037/0096-3445.131.1.48 CrossRefGoogle Scholar
  47. Wilken, P., & Ma, W. J. (2004). A detection theory account of change detection. Journal of Vision, 4(12), 1120–1135. doi: 10.1167/4.12.11 PubMedCrossRefGoogle Scholar
  48. Xu, Y. (2002). Encoding color and shape from different parts of an object in visual short-term memory. Perception & Psychophysics, 64, 1260–1280. doi: 10.3758/BF03194770 CrossRefGoogle Scholar
  49. Zhang, W., & Luck, S. J. (2008). Discrete fixed-resolution representations in visual working memory. Nature, 453, 233–235. doi: 10.1038/nature06860 PubMedCrossRefPubMedCentralGoogle Scholar
  50. Zhang, W., & Luck, S. J. (2011). The number and quality of representations in working memory. Psychological Science, 22, 1434–1441. doi: 10.1177/0956797611417006 PubMedCrossRefPubMedCentralGoogle Scholar

Copyright information

© The Psychonomic Society, Inc. 2014

Authors and Affiliations

  1. 1.Institute of Behavioural Sciences, Division of Cognitive and NeuropsychologyUniversity of HelsinkiHelsinkiFinland
  2. 2.Department of Teacher Education, Institute of Behavioural SciencesUniversity of HelsinkiHelsinkiFinland
  3. 3.Helsinki Collegium for Advanced StudiesUniversity of HelsinkiHelsinkiFinland

Personalised recommendations