Commonalities of visual and auditory working memory in a spatial-updating task

Abstract

Although visual and auditory inputs are initially processed in separate perception systems, studies have built on the idea that to maintain spatial information these modalities share a component of working memory. The present study used working memory navigation tasks to examine functional similarities and dissimilarities in the performance of updating tasks. Participants mentally updated the spatial location of a target in a virtual array in response to sequential pictorial and sonant directional cues before identifying the target’s final location. We predicted that if working memory representations are modality-specific, mixed-modality cues would demonstrate a cost of modality switching relative to unimodal cues. The results indicate that updating performance using visual unimodal cues positively correlated with that using auditory unimodal cues. Task performance using unimodal cues was comparable to that using mixed modality cues. The results of a subsequent experiment involving updating of target traces were consistent with those of the preceding experiments and support the view of modality-nonspecific memory.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

References

  1. Alain, C., He, Y., & Grady, C. (2008). The contribution of the inferior parietal lobe to auditory spatial working memory. Journal of Cognitive Neuroscience, 20(2), 285–295. https://doi.org/10.1162/jocn.2008.20014

    Article  PubMed  Google Scholar 

  2. Alain, C., McDonald, K. L., Kovacevic, N., & McIntosh, A. R. (2009). Spatiotemporal analysis of auditory “what” and “where” working memory. Cerebral Cortex, 19(2), 305–314. https://doi.org/10.1093/cercor/bhn082

    Article  PubMed  Google Scholar 

  3. Alais, D., & Burr, D. (2004). The ventriloquist effect results from near-optimal bimodal integration. Current Biology, 14(3), 257–262. https://doi.org/10.1016/S0960-9822(04)00043-0

    Article  PubMed  Google Scholar 

  4. Attneave, F., & Curlee, T. E. (1983). Locational representation in imagery: A moving spot task. Journal of Experimental Psychology: Human Perception and Performance, 9(1), 20–30. https://doi.org/10.1037/0096-1523.9.1.20

    Article  PubMed  Google Scholar 

  5. Awh, E., Jonides, J., & Reuter-Lorenz, P. A. (1998). Rehearsal in spatial working memory. Journal of Experimental Psychology: Human Perception and Performance, 24(3), 780–790. https://doi.org/10.1037/0096-1523.24.3.780

    Article  PubMed  Google Scholar 

  6. Baddeley, A. D. (2002). Is working memory still working? European Psychologist, 7(2), 85–97. https://doi.org/10.1027//1016-9040.7.2.85

    Article  Google Scholar 

  7. Baddeley, A. D. (2012). Working memory: Theories, models, and controversies. Annual Review of Psychology, 63, 1–29. https://doi.org/10.1146/annurev-psych-120710-100422

    Article  PubMed  Google Scholar 

  8. Baddeley, A. D., Allen, R. J., & Hitch, G. J. (2011). Binding in visual working memory: The role of the episodic buffer. Neuropsychologia, 49(6), 1393–1400. https://doi.org/10.1016/j.neuropsychologia.2010.12.042

    Article  PubMed  Google Scholar 

  9. Bledowski, C., Prvulovic, D., Hoechstetter, K., Scherg, M., Wibral, M., Goebel, R., & Linden, D. E. J. (2004). Localizing P300 generators in visual target and distractor processing: A combined event-related potential and functional magnetic resonance imaging study. The Journal of Neuroscience, 24(42), 9353–9360. https://doi.org/10.1523/JNEUROSCI.1897-04.2004

    Article  PubMed  PubMed Central  Google Scholar 

  10. Bushara, K. O., Weeks, R. A., Ishii, K., Catalan, M.-J., Tian, B., Rauschecker, J. P., & Hallett, M. (1999). Modality-specific frontal and parietal areas for auditory and visual spatial localization in humans. Nature Neuroscience, 2(8), 759–766. https://doi.org/10.1038/11239

    Article  PubMed  Google Scholar 

  11. Cohen, M. A., Evans, K. K., Horowitz, T. S., & Wolfe, J. M. (2011). Auditory and visual memory in musicians and nonmusicians. Psychonomic Bulletin & Review, 18(3), 586–591. https://doi.org/10.3758/s13423-011-0074-0

    Article  Google Scholar 

  12. Cohen, M. A., Horowitz, T. S., & Wolfe, J. M. (2009). Auditory recognition memory is inferior to visual recognition memory. PNAS Proceedings of the National Academy of Sciences of the United States of America, 106(14), 6008–6010. https://doi.org/10.1073/pnas.0811884106

    Article  Google Scholar 

  13. Colby, C. L., & Goldberg, M. E. (1999). Space and attention in parietal cortex. Annual Review of Neuroscience, 22, 319–349. https://doi.org/10.1146/annurev.neuro.22.1.319

    Article  PubMed  Google Scholar 

  14. Cornoldi, C., Cortesi, A., & Preti, D. (1991). Individual differences in the capacity limitations of visuospatial short-term memory: Research on sighted and totally congenitally blind people. Memory & Cognition, 19(5), 459–468. https://doi.org/10.3758/BF03199569

    Article  Google Scholar 

  15. Cornoldi, C., Tinti, C., Mammarella, I. C., Re, A. M., & Varotto, D. (2009). Memory for an imagined pathway and strategy effects in sighted and in totally congenitally blind individuals. Acta Psychologica, 130(1), 11–16. https://doi.org/10.1016/j.actpsy.2008.09.012

    Article  PubMed  Google Scholar 

  16. Cowan, N. (1999). An Embedded-Processes Model of working memory. In A. Miyake & P. Shah (Eds.), Models of working memory: Mechanisms of active maintenance and executive control (p. 62–101). Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9781139174909.006

    Google Scholar 

  17. Cowan, N. (2016). Working memory capacity. New York: Routledge, https://doi.org/10.4324/9781315625560

    Google Scholar 

  18. Craik, F. I. (1969). Modality effects in short-term storage. Journal of Verbal Learning and Verbal Behavior, 8(5), 658–664. https://doi.org/10.1016/S0022-5371(69)80119-2

    Article  Google Scholar 

  19. Delogu, F., Nijboer, T. C. W., & Postma, A. (2012). Encoding location and serial order in auditory working memory: Evidence for separable processes. Cognitive Processing, 13(3), 267–276. https://doi.org/10.1007/s10339-012-0442-3

    Article  PubMed  PubMed Central  Google Scholar 

  20. Dienes, Z., & Mclatchie, N. (2018). Four reasons to prefer Bayesian analyses over significance testing. Psychonomic Bulletin & Review, 25(1), 207–218. https://doi.org/10.3758/s13423-017-1266-z

    Article  Google Scholar 

  21. Driver, J., & Spence, C. (2004). Crossmodal spatial attention: Evidence from human performance. In C. Spence & J. Driver (Eds.), Crossmodal space and crossmodal attention. Oxford: Oxford University Press. https://doi.org/10.1093/acprof:oso/9780198524861.003.0008

    Google Scholar 

  22. Goossens, H. H. L. M., & Van Opstal, A. J. (1999). Influence of head position on the spatial representation of acoustic targets. Journal of Neurophysiology, 81(6), 2720–2736.

    Article  Google Scholar 

  23. Hall, J. L. (1974) PEST: Note on the reduction of variance of threshold estimates. The Journal of the Acoustical Society of America, 55(5), 1090–1091. https://doi.org/10.1121/1.1914656

    Article  PubMed  Google Scholar 

  24. Hitch, G. J., Allen, R. J., & Baddeley, A. D. (2020). Attention and binding in visual working memory: Two forms of attention and two kinds of buffer storage. Attention, Perception, & Psychophysics 82(1), 280–293. https://doi.org/10.3758/s13414-019-01837-x

    Article  Google Scholar 

  25. JASP Team. (2019). JASP (Version 0.11.1) [Computer software], Retrieved from https://jasp-stats.org/. Accessed 6 Nov 2019

  26. Javitt, D. C., Strous, R. D., Grochowski, S., Ritter, W., & Cowan, N. (1997). Impaired precision, but normal retention, of auditory sensory (“echoic”) memory information in schizophrenia. Journal of Abnormal Psychology, 106(2), 315–324. https://doi.org/10.1037/0021-843X.106.2.315

    Article  PubMed  Google Scholar 

  27. Jeffreys, H. (1961). Theory of probability, 3rd edn. Oxford: Oxford University Press.

    Google Scholar 

  28. Kaiser, J. (2015). Dynamics of auditory working memory. Frontiers in Psychology, 6, Article 613.

    PubMed  Google Scholar 

  29. Kerr, N. H. (1993). Rate of imagery processing in two versus three dimensions. Memory & Cognition, 21(4), 467–476. https://doi.org/10.3758/BF03197178

    Article  Google Scholar 

  30. Kiesel, A., Steinhauser, M., Wendt, M., Falkenstein, M., Jost, K., Philipp, A. M., & Koch, I. (2010). Control and interference in task switching—A review. Psychological Bulletin, 136(5), 849–874. https://doi.org/10.1037/a0019842

    Article  PubMed  Google Scholar 

  31. Kleiner, M., Brainard, D., & Pelli, D. (2007). What’s new in Psychtoolbox-3? Perception, 36(14), 1–16, ECVP Abstract Supplement.

    Google Scholar 

  32. Kong, L., Michalka, S. W., Rosen, M. L., Sheremata, S. L., Swisher, J. D., Shinn-Cunningham, B. G., & Somers, D. C. (2014). Auditory spatial attention representations in the human cerebral cortex. Cerebral Cortex, 24(3), 773–784. https://doi.org/10.1093/cercor/bhs359

    Article  PubMed  Google Scholar 

  33. Kreutzfeldt, M. Stephan, D. N., Willmes, K., & Koch, I. (2017). Modality-specific preparatory influences on the flexibility of cognitive control in task switching. Journal of Cognitive Psychology, 29(5), 607–617. https://doi.org/10.1080/20445911.2017.1293064

    Article  Google Scholar 

  34. Krumbholz, K., Nobis, E. A., Weatheritt, R. J., & Fink, G. R. (2009). Executive control of spatial attention shifts in the auditory compared to the visual modality. Human Brain Mapping, 30(5), 1457–1469. https://doi.org/10.1002/hbm.20615

    Article  PubMed  Google Scholar 

  35. Lehnert, G., & Zimmer, H. D. (2006). Auditory and visual spatial working memory. Memory & Cognition, 34(5), 1080–1090. https://doi.org/10.3758/BF03193254

    Article  Google Scholar 

  36. Lehnert, G., & Zimmer, H. D. (2008a). Common coding of auditory and visual spatial information in working memory. Brain Research, 1230, 158–167. https://doi.org/10.1016/j.brainres.2008.07.005

    Article  PubMed  Google Scholar 

  37. Lehnert, G., & Zimmer, H. D. (2008b). Modality and domain specific components in auditory and visual working memory tasks. Cognitive Processing, 9(1), 53–61. https://doi.org/10.1007/s10339-007-0187-6

    Article  PubMed  Google Scholar 

  38. Lewald, J., & Ehrenstein, W. H. (2001). Spatial coordinates of human auditory working memory. Cognitive Brain Research, 12(1), 153–159. https://doi.org/10.1016/S0926-6410(01)00042-8

    Article  PubMed  Google Scholar 

  39. Logie, R. H. (1995). Essays in cognitive psychology. Visuo-spatial working memory. Hove: Lawrence Erlbaum Associates, Inc.

    Google Scholar 

  40. Logie, R. H. (2011). The functional organization and capacity limits of working memory. Current Directions in Psychological Science, 20(4), 240–245. https://doi.org/10.1177/0963721411415340

    Article  Google Scholar 

  41. Loomis, J. M., Klatzky, R. L., McHugh, B., & Giudice, N. A. (2012). Spatial working memory for locations specified by vision and audition: Testing the amodality hypothesis. Attention, Perception, & Psychophysics, 74(6), 1260–1267. https://doi.org/10.3758/s13414-012-0311-2

    Article  Google Scholar 

  42. Luck, S. J., & Vogel, E. K. (1997). The capacity of visual working memory for features and conjunctions. Nature, 390(6657), 279–281. https://doi.org/10.1038/36846

    Article  PubMed  Google Scholar 

  43. Luck, S. J., & Vogel, E. K. (2013). Visual working memory capacity: From psychophysics and neurobiology to individual differences. Trends in Cognitive Sciences, 17(8), 391–400. https://doi.org/10.1016/j.tics.2013.06.006

    Article  PubMed  PubMed Central  Google Scholar 

  44. Mance, I., & Vogel, E. K. (2013). Visual working memory. WIREs Cognitive Science, 4(2), 179–190. https://doi.org/10.1002/wcs.1219

    Article  PubMed  Google Scholar 

  45. Martinkauppi, S., Räma, P., Aronen, H. J., Korvenoja, A., & Carlson, S. (2000). Working memory of auditory localization. Cerebral Cortex, 10(9), 889–898. https://doi.org/10.1093/cercor/10.9.889

    Article  PubMed  Google Scholar 

  46. Michalka, S. W., Rosen, M. L., Kong, L., Shinn-Cunningham, B. G., & Somers, D. C. (2015). Auditory spatial coding flexibly recruits anterior, but not posterior, visuotopic parietal cortex. Cerebral Cortex, 26(3), 1302–1308. https://doi.org/10.1093/cercor/bhv303

    Article  PubMed  Google Scholar 

  47. Monsell, S. (2003). Task switching. Trends in Cognitive Sciences, 7(3), 134–140. https://doi.org/10.1016/S1364-6613(03)00028-7

    Article  PubMed  Google Scholar 

  48. Morey, R. D., & Rouder, J. N. (2011). Bayes factor approaches for testing interval null hypotheses. Psychological Methods, 16(4), 406–419. https://doi.org/10.1037/a0024377

    Article  PubMed  Google Scholar 

  49. Morey, R. D., Rouder, J. N., Jamil, T., Urbanek, S., Forner, K., & Ly, A. (2018). BayesFactor: Computation of Bayes factors for common designs (R package, version 0.9.12-4.2) [Computer software]. Retrieved from https://CRAN.R-project.org/package=BayesFactor. Accessed 6 Nov 2019

  50. Polich, J. (2007). Updating p300: An integrative theory of P3a and P3b. Clinical Neurophysiology, 118(10), 2128–2148. https://doi.org/10.1016/j.clinph.2007.04.019

    Article  PubMed  PubMed Central  Google Scholar 

  51. Quak, M., London, R. E., & Talsma, D. (2015). A multisensory perspective of working memory. Frontiers in Human Neuroscience, 9, Article 197.

    Article  Google Scholar 

  52. Ricker, T. J., AuBuchon, A. M., & Cowan, N. (2010). Working memory. WIREs Cognitive Science, 1(4), 573–585.

    Article  Google Scholar 

  53. Rouder, J. N., Morey, R. D., Verhagen, J., Swagman, A. R., & Wagenmakers, E.-J. (2017). Bayesian analysis of factorial designs. Psychological Methods, 22(2), 304–321. https://doi.org/10.1037/met0000057

    Article  PubMed  Google Scholar 

  54. Ruthruff, E., Remington, R. W., & Johnston, J. C. (2001). Switching between simple cognitive tasks: The interaction of top-down and bottom-up factors. Journal of Experimental Psychology: Human Perception and Performance, 27(6), 1404–1419. https://doi.org/10.1037/0096-1523.27.6.1404

    Article  PubMed  Google Scholar 

  55. Sinnett, S., Juncadella, M., Rafal, R., Azañón, E., & Soto-Faraco, S. (2007). A dissociation between visual and auditory hemi-inattention: Evidence from temporal order judgements. Neuropsychologia, 45(3), 552–560. https://doi.org/10.1016/j.neuropsychologia.2006.03.006

    Article  PubMed  Google Scholar 

  56. Smith, E. E., & Jonides, J. (1997). Working memory: A view from neuroimaging. Cognitive Psychology, 33(1), 5–42. https://doi.org/10.1006/cogp.1997.0658

    Article  PubMed  Google Scholar 

  57. Smyth, M. M., & Scholey, K. A. (1994). Interference in immediate spatial memory. Memory & Cognition, 22(1), 1–13. https://doi.org/10.3758/BF03202756

    Article  Google Scholar 

  58. Spence, C., Nicholls, M. R., & Driver, J. (2001). The cost of expecting events in the wrong sensory modality. Perception & Psychophysics, 63(2), 330–336. https://doi.org/10.3758/BF03194473

    Article  Google Scholar 

  59. Sperling, G. (1960). The information available in brief visual presentations. Psychological Monographs: General and Applied, 74(11), 1–29. https://doi.org/10.1037/h00937593

    Article  Google Scholar 

  60. Suchan, B., Linnewerth, B., Köster, O., Daum, I., & Schmid, G. (2006). Cross-modal processing in auditory and visual working memory. Neuroimage, 29(3), 853–858. https://doi.org/10.1016/j.neuroimage.2005.08.014

    Article  PubMed  Google Scholar 

  61. Tao, Q., Chan, C. C. H., Luo, Y.-j., Li, J.-j., Ting, K.-h., Lu, Z.-l., … Lee, T. M. C. (2017). Prior visual experience modulates learning of sound localization among blind individuals. Brain Topography, 30(3), 364–379. https://doi.org/10.1007/s10548-017-0549-z

    Article  PubMed  PubMed Central  Google Scholar 

  62. Taylor, M., & Creelman, C. D. (1967). PEST: Efficient estimates on probability functions. The Journal of the Acoustical Society of America, 41(4A), 782–787. https://doi.org/10.1121/1.1910407

    Article  Google Scholar 

  63. Tremblay, S., Parmentier, F. B. R., Guérard, K., Nicholls, A. P., & Jones, D. M. (2006). A spatial modality effect in serial memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 32(5), 1208–1215. https://doi.org/10.1037/0278-7393.32.5.1208

    Article  PubMed  Google Scholar 

  64. Vogel, E. K., McCullough, A. W., & Machizawa, M. G. (2005). Neural measures reveal individual differences in controlling access to working memory. Nature, 438(7067), 500–503. https://doi.org/10.1038/nature04171

    Article  PubMed  Google Scholar 

  65. Vuontela, V., Steenari, M.-R., Carlson, S., Koivisto, J., Fjällberg, M., & Aronen, E. T. (2003). Audiospatial and visuospatial working memory in 6-13 year old school children. Learning & Memory, 10(1), 74–81. https://doi.org/10.1101/lm.53503

    Article  Google Scholar 

  66. Wagenmakers, E. J., Love, J., Marsman, M., Jamil, T., Ly, A., Verhagen, J., … Morey, R. D. (2018). Bayesian inference for psychology. Part II: Example applications with JASP. Psychonomic Bulletin & Review, 25(1), 58–76. https://doi.org/10.3758/s13423-017-1323-7

    Article  Google Scholar 

  67. Westfall, J., Kenny, D. A., & Judd, C. M. (2014). Statistical power and optimal design in experiments in which samples of participants respond to samples of stimuli. Journal of Experimental Psychology: General, 143(5), 2020–2045. https://doi.org/10.1037/xge0000014

    Article  Google Scholar 

  68. Wetzels, R., Grasman, R. P., & Wagenmakers, E. J. (2012). A default Bayesian hypothesis test for ANOVA designs. The American Statistician, 66(2), 104–111. https://doi.org/10.1080/00031305.2012.695956

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by Grants-in-Aid for Scientific Research from the Japan Society for the Promotion of Science (17H02648 and 20H01779) to JK and Graduate Grant Program of Graduate School of Letters, Hokkaido University, Japan, and a Grant-in-Aid from the Japan Society for the Promotion of Science Fellows (20J20490) to TM.

Open practices statement

The data and significant program code will be made available after acceptance via the Open Science Framework (https://osf.io/sf742/), and none of the experiments were preregistered.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Tomoki Maezawa.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Maezawa, T., Kawahara, J.I. Commonalities of visual and auditory working memory in a spatial-updating task. Mem Cogn (2021). https://doi.org/10.3758/s13421-021-01151-8

Download citation

Keywords

  • Working memory
  • Audition
  • Spatial updating
  • Imagery processing
  • Location memory