Skip to main content

Benefits and pitfalls of data compression in visual working memory

Abstract

Data compression in memory is a cognitive process allowing participants to cope with complexity to reduce information load. However, previous studies have not yet considered the hypothesis that this process could also lead to over-simplifying information due to haphazard amplification of the compression process itself. For instance, we could expect that the over-regularized features of a visual scene could produce false recognition of patterns, not because of storage capacity limits but because of an errant compression process. To prompt memory compression in our participants, we used multielement visual displays for which the underlying information varied in compressibility. The compressibility of our material could vary depending on the number of common features between the multi-dimensional objects in the displays. We measured both accuracy and response times by probing memory representations with probes that we hypothesized could modify the participants’ representations. We confirm that more compressible information facilitates performance, but a more novel finding is that compression can produce both typical memory errors and lengthened response times. Our findings provide clearer evidence of the forms of compression that participants carry out.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Data availability

The data for all experiments are available in the OSF repository at https://osf.io/ehjrw/?view_only=0373989a406643d0ad0a8548e4829ff3

References

  1. Akaike, H. (1987). Factor analysis and AIC. In Selected papers of hirotugu akaike (p. 371-386). Springer.

  2. Alvarez, G. A., & Cavanagh, P. (2004). The capacity of visual short-term memory is set both by visual information load and by number of objects. Psychological science, 15(2), 106-111.

    PubMed  Article  PubMed Central  Google Scholar 

  3. Awh, E., Barton, B., & Vogel, E. K. (2007). Visual working memory represents a fixed number of items regardless of complexity. Psychological science, 18(7), 622-628.

    PubMed  Article  PubMed Central  Google Scholar 

  4. Bays, P. M., Catalao, R. F., & Husain, M. (2009). The precision of visual working memory is set by allocation of a shared resource. Journal of vision, 9(10), 7-7.

    PubMed  PubMed Central  Article  Google Scholar 

  5. Bays, P. M., & Husain, M. (2008). Dynamic shifts of limited working memory resources in human vision. Science, 321(5890), 851-854.

    PubMed  PubMed Central  Article  Google Scholar 

  6. Bays, P. M., Wu, E. Y., & Husain, M. (2011). Storage and binding of object features in visual working memory. Neuropsychologia, 49(6), 1622-1631.

    PubMed  Article  PubMed Central  Google Scholar 

  7. Bradmetz, J., & Mathy, F. (2008). Response times seen as decompression times in Boolean concept use. Psychological Research, 72(2), 211-234.

    PubMed  Article  PubMed Central  Google Scholar 

  8. Brady, T. F., & Alvarez, G. A. (2011). Hierarchical encoding in visual working memory : Ensemble statistics bias memory for individual items. Psychological science, 22(3), 384-392.

    PubMed  Article  PubMed Central  Google Scholar 

  9. Brady, T. F., & Alvarez, G. A. (2015). No evidence for a fixed object limit in working memory : Spatial ensemble representations inflate estimates of working memory capacity for complex objects. Journal of Experimental Psychology: Learning, Memory, and Cognition, 41(3), 921.

    PubMed  PubMed Central  Google Scholar 

  10. Brady, T. F., Konkle, T., & Alvarez, G. A. (2009). Compression in visual working memory : Using statistical regularities to form more efficient memory representations. Journal of Experimental Psychology: General, 138(4), 487-502. https://doi.org/10.1037/a0016797

    Article  Google Scholar 

  11. Brady, T. F., & Tenenbaum, J. B. (2013). A probabilistic model of visual working memory : Incorporating higher order regularities into working memory capacity estimates. Psychological Review, 120(1), 85.

    PubMed  Article  PubMed Central  Google Scholar 

  12. Brown, V. A. (2020). An approachable introduction to linear mixed effects modeling with implementation in R.

  13. Chekaf, M., Cowan, N., & Mathy, F. (2016). Chunk formation in immediate memory and how it relates to data compression. Cognition, 155, 96-107.

    PubMed  PubMed Central  Article  Google Scholar 

  14. Chekaf, M., Gauvrit, N., Guida, A., & Mathy, F. (2018). Compression in working memory and its relationship with fluid intelligence. Cognitive Science, 42, 904-922.

    PubMed  Article  PubMed Central  Google Scholar 

  15. Christiansen, M. H., & Chater, N. (2016). The now-or-never bottleneck : A fundamental constraint on language. Behavioral and Brain Sciences, 39.

  16. Cowan, N. (2001). The magical number 4 in short-term memory : A reconsideration of mental storage capacity. Behavioral and Brain Sciences, 24(1), 87-114.

    Article  Google Scholar 

  17. Cowan, N., Blume, C. L., & Saults, J. S. (2013). Attention to attributes and objects in working memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 39(3), 731.

    PubMed  PubMed Central  Google Scholar 

  18. Cowan, N., Rouder, J. N., Blume, C. L., & Saults, J. S. (2012). Models of verbal working memory capacity : What does it take to make them work? Psychological review, 119(3), 480.

    PubMed  PubMed Central  Article  Google Scholar 

  19. De Lillo, C. (2004). Imposing structure on a Corsi-type task : Evidence for hierarchical organisation based on spatial proximity in serial-spatial memory. Brain and Cognition, 55(3), 415-426.

    PubMed  Article  PubMed Central  Google Scholar 

  20. Dry, M., Preiss, K., & Wagemans, J. (2012). Clustering, randomness, and regularity : Spatial distributions and human performance on the traveling salesperson problem and minimum spanning tree problem. The Journal of Problem Solving, 4(1), 2.

    Article  Google Scholar 

  21. Feldman, J. (1999). The role of objects in perceptual grouping. Acta Psychologica, 102(2-3), 137-163.

    PubMed  Article  PubMed Central  Google Scholar 

  22. Feldman, J. (2000). Minimization of Boolean complexity in human concept learning. Nature, 407(6804), 630-633.

    PubMed  Article  PubMed Central  Google Scholar 

  23. Feldman, J. (2003). A catalog of Boolean concepts. Journal of Mathematical Psychology, 47(1), 75-89.

    Article  Google Scholar 

  24. Ferrer-i-Cancho, R., Hernández-Fernández, A., Lusseau, D., Agoramoorthy, G., Hsu, M. J., & Semple, S. (2013). Compression as a universal principle of animal behavior. Cognitive Science, 37(8), 1565-1578.

    PubMed  Article  PubMed Central  Google Scholar 

  25. Fific, M., Little, D. R., & Nosofsky, R. M. (2010). Logical-rule models of classification response times : A synthesis of mental-architecture, random-walk, and decision-bound approaches. Psychological Review, 117(2), 309.

    PubMed  PubMed Central  Article  Google Scholar 

  26. Fougnie, D., & Alvarez, G. A. (2011). Object features fail independently in visual working memory : Evidence for a probabilistic feature-store model. Journal of vision, 11(12), 3-3.

    PubMed  Article  PubMed Central  Google Scholar 

  27. Gao, Z., Gao, Q., Tang, N., Shui, R., & Shen, M. (2016). Organization principles in visual working memory : Evidence from sequential stimulus display. Cognition, 146, 277-288.

    PubMed  Article  PubMed Central  Google Scholar 

  28. Gauvrit, N., Singmann, H., Soler-Toscano, F., & Zenil, H. (2016). Algorithmic complexity for psychology : A user-friendly implementation of the coding theorem method. Behavior research methods, 48(1), 314-329.

    PubMed  Article  PubMed Central  Google Scholar 

  29. Haladjian, H. H., & Mathy, F. (2015). A snapshot is all it takes to encode object locations into spatial memory. Vision research, 107, 133-145.

    PubMed  Article  PubMed Central  Google Scholar 

  30. Hardman, K. O., & Cowan, N. (2015). Remembering complex objects in visual working memory : Do capacity limits restrict objects or features? Journal of Experimental Psychology: Learning, Memory, and Cognition, 41(2), 325.

    PubMed  PubMed Central  Google Scholar 

  31. Hutter, M. (2004). Universal artificial intelligence : Sequential decisions based on algorithmic probability. Springer Science & Business Media.

  32. Jiang, Y., Chun, M. M., & Olson, I. R. (2004). Perceptual grouping in change detection. Perception & Psychophysics, 66(3), 446-453.

    Article  Google Scholar 

  33. Jiang, Y., Olson, I. R., & Chun, M. M. (2000). Organization of visual short-term memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 26(3), 683.

    PubMed  PubMed Central  Google Scholar 

  34. Kemp, C. (2012). Exploring the conceptual universe. Psychological Review, 119(4), 685.

    PubMed  Article  PubMed Central  Google Scholar 

  35. Kibbe, M. M., & Kowler, E. (2011). Visual search for category sets : Tradeoffs between exploration and memory. Journal of Vision, 11(3), 14-14.

    PubMed  Article  PubMed Central  Google Scholar 

  36. Kirby, S., Tamariz, M., Cornish, H., & Smith, K. (2015). Compression and communication in the cultural evolution of linguistic structure. Cognition, 141, 87-102.

    PubMed  Article  PubMed Central  Google Scholar 

  37. Korjoukov, I., Jeurissen, D., Kloosterman, N. A., Verhoeven, J. E., Scholte, H. S., & Roelfsema, P. R. (2012). The time course of perceptual grouping in natural scenes. Psychological Science, 23(12), 1482-1489.

    PubMed  Article  PubMed Central  Google Scholar 

  38. Lafond, D., Lacouture, Y., & Mineau, G. (2007). Complexity minimization in rule-based category learning : Revising the catalog of Boolean concepts and evidence for non-minimal rules. Journal of Mathematical Psychology, 51(2), 57-74.

    Article  Google Scholar 

  39. Li, M., & Vitányi, P. (2008). An introduction to Kolmogorov complexity and its applications (Vol. 3). Springer.

  40. Luck, S. J., & Vogel, E. K. (1997). The capacity of visual working memory for features and conjunctions. Nature, 390(6657), 279.

    PubMed  Article  PubMed Central  Google Scholar 

  41. Ma, W. J., Husain, M., & Bays, P. M. (2014). Changing concepts of working memory. Nature Neuroscience, 17(3), 347.

    PubMed  PubMed Central  Article  Google Scholar 

  42. Mathy, F., & Bradmetz, J. (2004). A theory of the graceful complexification of concepts and their learnability. Cahiers de Psychologie Cognitive-Current Psychology of Cognition, 22, 39-80.

    Google Scholar 

  43. Mathy, F., & Feldman, J. (2012). What’s magic about magic numbers? Chunking and data compression in short-term memory. Cognition, 122(3), 346-362.

    PubMed  Article  PubMed Central  Google Scholar 

  44. Mewhort, D. J. K., & Johns, E. E. (2000). The extralist-feature effect : Evidence against item matching in short-term recognition memory. Journal of Experimental Psychology: General, 129(2), 262.

    Article  Google Scholar 

  45. Morey, C. C., Cong, Y., Zheng, Y., Price, M., & Morey, R. D. (2015). The color-sharing bonus : Roles of perceptual organization and attentive processes in visual working memory. Archives of Scientific Psychology, 3(1), 18.

    Article  Google Scholar 

  46. Nassar, M. R., Helmers, J. C., & Frank, M. J. (2018). Chunking as a rational strategy for lossy data compression in visual working memory. Psychological Review, 125(4), 486.

    PubMed  PubMed Central  Article  Google Scholar 

  47. Ngiam, W. X., Brissenden, J. A., & Awh, E. (2019). “Memory compression” effects in visual working memory are contingent on explicit long-term memory. Journal of Experimental Psychology: General, 148(8), 1373.

    Article  Google Scholar 

  48. Norris, D. G., & Kalm, K. (2018). Chunking and redintegration in verbal short-term memory.

  49. Nosofsky, R. M., Palmeri, T. J., & McKinley, S. C. (1994). Rule-plus-exception model of classification learning. Psychological Review, 101(1), 53.

    PubMed  Article  PubMed Central  Google Scholar 

  50. Oberauer, K., & Eichenberger, S. (2013). Visual working memory declines when more features must be remembered for each object. Memory & Cognition, 41(8), 1212-1227.

    Article  Google Scholar 

  51. Oberauer, K., & Lin, H.-Y. (2017). An interference model of visual working memory. Psychological Review, 124(1), 21.

    PubMed  Article  PubMed Central  Google Scholar 

  52. Orbán, G., Fiser, J., Aslin, R. N., & Lengyel, M. (2008). Bayesian learning of visual chunks by human observers. Proceedings of the National Academy of Sciences, 105(7), 2745-2750.

    Article  Google Scholar 

  53. Peirce, J. W. (2007). PsychoPy—Psychophysics software in Python. Journal of Neuroscience Methods, 162(1-2), 8-13.

    PubMed  PubMed Central  Article  Google Scholar 

  54. Peterson, D. J., & Berryhill, M. E. (2013). The Gestalt principle of similarity benefits visual working memory. Psychonomic Bulletin & Review, 20(6), 1282-1289.

    Article  Google Scholar 

  55. Quinlan, P. T., & Cohen, D. J. (2012). Grouping and binding in visual short-term memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 38(5), 1432.

    PubMed  PubMed Central  Google Scholar 

  56. Ramzaoui, H., & Mathy, F. (2021). A compressibility account of the color-sharing bonus in working memory. Attention, Perception, & Psychophysics, 1-16.

  57. Ratcliff, R. (1978). A theory of memory retrieval. Psychological Review, 85(2), 59.

    Article  Google Scholar 

  58. Reder, L. M., Liu, X. L., Keinath, A., & Popov, V. (2016). Building knowledge requires bricks, not sand : The critical role of familiar constituents in learning. Psychonomic Bulletin & Review, 23(1), 271-277.

    Article  Google Scholar 

  59. Rouder, J. N., Morey, R. D., Cowan, N., Zwilling, C. E., Morey, C. C., & Pratte, M. S. (2008). An assessment of fixed-capacity models of visual working memory. Proceedings of the National Academy of Sciences, 105(16), 5975-5979.

    Article  Google Scholar 

  60. Saiki, J. (2019). Robust color-shape binding representations for multiple objects in visual working memory. Journal of Experimental Psychology: General.

  61. Sargent, J., Dopkins, S., Philbeck, J., & Chichka, D. (2010). Chunking in spatial memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 36(3), 576.

    PubMed  PubMed Central  Google Scholar 

  62. Schurgin, M. W., Wixted, J. T., & Brady, T. F. (2020). Psychophysical scaling reveals a unified theory of visual memory strength. Nature human behaviour, 4(11), 1156-1172.

    PubMed  Article  PubMed Central  Google Scholar 

  63. Sims, C. R. (2015). The cost of misremembering : Inferring the loss function in visual working memory. Journal of vision, 15(3), 2-2.

    PubMed  Article  PubMed Central  Google Scholar 

  64. Singmann, H., & Kellen, D. (2019). An introduction to mixed models for experimental psychology. New methods in cognitive psychology, 28, 4-31.

    Article  Google Scholar 

  65. Soler-Toscano, F., Zenil, H., Delahaye, J.-P., & Gauvrit, N. (2014). Calculating Kolmogorov complexity from the output frequency distributions of small Turing machines. PloS one, 9(5), e96223.

    PubMed  PubMed Central  Article  Google Scholar 

  66. Thalmann, M., Souza, A. S., & Oberauer, K. (2019). How does chunking help working memory? Journal of Experimental Psychology: Learning, Memory, and Cognition, 45(1), 37.

    PubMed  PubMed Central  Google Scholar 

  67. Vigo, R. (2006). A note on the complexity of Boolean concepts. Journal of Mathematical Psychology, 50(5), 501-510.

    Article  Google Scholar 

  68. Wagenmakers, E.-J., Van Der Maas, H. L., & Grasman, R. P. (2007). An EZ-diffusion model for response time and accuracy. Psychonomic Bulletin & Review, 14(1), 3-22.

    Article  Google Scholar 

  69. Wheeler, M. E., & Treisman, A. M. (2002). Binding in short-term visual memory. Journal of Experimental Psychology: General, 131(1), 48.

    Article  Google Scholar 

  70. Woodman, G. F., Vecera, S. P., & Luck, S. J. (2003). Perceptual organization influences visual working memory. Psychonomic Bulletin & Review, 10(1), 80-87.

    Article  Google Scholar 

  71. Xu, Y. (2002). Limitations of object-based feature encoding in visual short-term memory. Journal of Experimental Psychology: Human Perception and Performance, 28(2), 458.

    PubMed  PubMed Central  Google Scholar 

  72. Xu, Y., & Chun, M. M. (2006). Dissociable neural mechanisms supporting visual short-term memory for objects. Nature, 440(7080), 91-95.

    PubMed  Article  PubMed Central  Google Scholar 

  73. Zhang, W., & Luck, S. J. (2008). Discrete fixed-resolution representations in visual working memory. Nature, 453(7192), 233.

    PubMed  PubMed Central  Article  Google Scholar 

Download references

Funding

This research was supported in part by a grant from the Agence Nationale de la Recherche (ANR-17-CE28-0013-01) awarded to Fabien Mathy.

Author information

Affiliations

Authors

Contributions

The authors’ contributions are reported following the CRediT taxonomy (see https://casrai.org/credit/). Conceptualization, N.C., L.L., F. L, and F.M.; Methodology, L.L. and C.A.; Formal Analysis, L.L..; Writing, N.C., L.L., and F.M. Funding Acquisition (F.M.).

Corresponding author

Correspondence to Laura Lazartigues.

Ethics declarations

Ethics approval

The experiment was approved by the local ethics committee (CERNI) of the Université Côte d'Azur.

Consent to participate

The experiment was conducted with the informed consent of the participants.

Consent for publication

Participants signed informed consent regarding publishing their data.

Conflicts of interest/Competing interests

The authors have no conflicts of interest to declare that are relevant to the content of this article.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

ESM 1

(DOCX 2308 kb)

Appendix

Appendix

Example of display and probe for each condition of Experiment 1 and Experiment 2

figureafigureafigurea

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Lazartigues, L., Lavigne, F., Aguilar, C. et al. Benefits and pitfalls of data compression in visual working memory. Atten Percept Psychophys (2021). https://doi.org/10.3758/s13414-021-02333-x

Download citation

Keywords

  • Visual
  • Working memory
  • Compression of information
  • Capacity
  • Response times