Skip to main content
SpringerLink
Log in

A computational cognition model of perception, memory, and judgment

  • Research Paper
  • Published:
Science China Information Sciences Aims and scope Submit manuscript

Abstract

The mechanism of human cognition and its computability provide an important theoretical foundation to intelligent computation of visual media. This paper focuses on the intelligent processing of massive data of visual media and its corresponding processes of perception, memory, and judgment in cognition. In particular, both the human cognitive mechanism and cognitive computability of visual media are investigated in this paper at the following three levels: neurophysiology, cognitive psychology, and computational modeling. A computational cognition model of Perception, Memory, and Judgment (PMJ model for short) is proposed, which consists of three stages and three pathways by integrating the cognitive mechanism and computability aspects in a unified framework. Finally, this paper illustrates the applications of the proposed PMJ model in five visual media research areas. As demonstrated by these applications, the PMJ model sheds some light on the intelligent processing of visual media, and it would be innovative for researchers to apply human cognitive mechanism to computer science.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Gallistel C R, King A. Memory and the Computational Brain: Why Cognitive Science Will Transform Neuroscience. New York: Blackwell/Wiley, 2009. iiv–xvi

    Google Scholar 

  2. Hu S M, Chen T, Xu K, et al. Internet visual media processing: a survey with graphics and vision applications. Vis Comput, 2013, 29: 393–405

    Google Scholar 

  3. Hulusic V, Debattista K, Aggarwal V, et al. Maintaining frame rate perception in interactive environments by exploiting audio-visual cross-modal interaction. Vis Comput, 2011, 27: 57–66

    Google Scholar 

  4. Vazquez P-P, Marco J. Using normalized compression distance for image similarity measurement: an experimental study. Vis Comput, 2012, 28: 1063–1084

    Google Scholar 

  5. Eysenck M W, Keane M T. Cognitive Psychology: a Student’s Handbook. 6th ed. New York: Psychology Press, 2010. 1–50

    Google Scholar 

  6. National Institute on Drug Abuse. Computational neuroscience at the NIH. Nat Neurosci, 2000, 3: 1161–1164

    Google Scholar 

  7. Buschman T J, Miller E K. Top-down versus bottom-up control of attention in the prefrontal and posterior parietal cortices. Science, 2007, 315: 1860–1862

    Google Scholar 

  8. Navalpakkam V, Itti L. Search goal tunes visual features optimally. Neuron, 2007, 53: 605–617

    Google Scholar 

  9. Katsuki F, Constantinidis C. Early involvement of prefrontal cortex in visual bottom-up attention. Nat Neurosci, 2012, 15: 1160–1166

    Google Scholar 

  10. Corbetta M, Shulman G L. Control of goal-directed and stimulus-driven attention in the brain. Nat Rev Neurosci, 2002, 3: 201–215

    Google Scholar 

  11. Zanto T P, Rubens M T, Thangavel A, et al. Causal role of the prefrontal cortex in top-down modulation of visual processing and working memory. Nat Neurosci, 2011, 14: 656–661

    Google Scholar 

  12. Tomita H M, Ohbayashi K, Nakahara I, et al. Top-down signal from prefrontal cortex in executive control of memory retrieval. Nature, 1999, 401: 699–703

    Google Scholar 

  13. Itti L, Koch C. Computational modelling of visual attention. Nat Rev Neurosci, 2001, 2: 194–203

    Google Scholar 

  14. Cox D, Meyers E, Sinha P. Contextually evoked object-specific responses in human visual cortex. Science, 2004, 303: 115–117

    Google Scholar 

  15. Kouh M, Poggio T. A canonical neural circuit for cortical nonlinear operations. Neural Comput, 2008, 20: 1427–1451

    MATH  MathSciNet  Google Scholar 

  16. Poggio T, Bizzi E. Generalization in vision and motor control. Nature, 2004, 431: 768–774

    Google Scholar 

  17. Hung C P, Kreiman G, Poggio T, et al. Fast readout of object identity from macaque inferior temporal cortex. Science, 2005, 310: 863–866

    Google Scholar 

  18. Pylyshyn Z W. Computation and Cognition: toward a Foundation for Cognitive Science. Cambridge: The MIT Press, 1984. 1–16

    Google Scholar 

  19. Hasselmo M E, Sarter M. Modes and models of forebrain cholinergic neuromodulation of cognition. Neuropsychopharmacology, 2010, 36: 52–73

    Google Scholar 

  20. Tamietto M, de Gelder B. Neural bases of the non-conscious perception of emotional signals. Nat Rev Neurosci, 2010, 11: 697–709

    Google Scholar 

  21. Fries P, Reynolds J H, Rorie A E, et al. Modulation of oscillatory neuronal synchronization by selective visual attention. Science, 2001, 291: 1560–1563

    Google Scholar 

  22. Roberts M, Delicato L S, Herrero J, et al. Attention alters spatial integration in macaque V1 in an eccentricitydependent manner. Nat Neurosci, 2007, 10: 1483–1491

    Google Scholar 

  23. Qiu F T, Sugihara T, von der Heydt R. Figure-ground mechanisms provide structure for selective attention. Nat Neurosci, 2007, 10: 1492–1499

    Google Scholar 

  24. Hübner R, Steinhauser M, Lehle C. A dual-stage two-phase model of selective attention. Psychol Rev, 2010, 117: 759–784

    Google Scholar 

  25. Gondan M, Blurton S P, Hughes F, et al. Effects of spatial and selective attention on basic multisensory integration. J Exp Phychol-Hum Percep Perf, 2011, 37: 1887–1897

    Google Scholar 

  26. Schafer R J, Moore T. Selective attention from voluntary control of neurons in prefrontal cortex. Science, 2011, 332: 1568–1571

    Google Scholar 

  27. Couperus J W. Perceptual load influences selective attention across development. Develop Psychol, 2011, 47: 1431–1439

    Google Scholar 

  28. Cosman J D, Vecera S P. Object-based attention overrides perceptual load to modulate visual distraction. J Exp Phychol-Hum Percep Perf, 2012, 38: 576–579

    Google Scholar 

  29. Chen C C, Wu J K, Lin H W, et al. Visualizing long-term memory formation in two neurons of the drosophila brain. Science, 2012, 335: 678–685

    Google Scholar 

  30. Fell J, Axmacher N. The role of phase synchronization in memory processes. Nat Rev Neurosci, 2011, 12: 105–118

    Google Scholar 

  31. Fusi S, Abbott L F. Limits on the memory storage capacity of bounded synapses. Nat Neurosci, 2007, 10: 485–493

    Google Scholar 

  32. Eichenbaum H. A cortical-hippocampal system for declarative memory. Nat Rev Neurosci, 2000, 1: 41–50

    Google Scholar 

  33. McGaugh J L. Memory-a century of consolidation. Science, 2000, 287: 248–251

    Google Scholar 

  34. Tronson N C, Taylor J R. Molecular mechanisms of memory reconsolidation. Nat Rev Neurosci, 2007, 8: 262–275

    Google Scholar 

  35. Edelson M, Sharot T, Dolan R J, et al. Following the crowd: brain substrates of long-term memory conformity. Science, 2011, 333: 108–111

    Google Scholar 

  36. Frankland P W, Bontempi B. The organization of recent and remote memories. Nat Rev Neurosci, 2005, 6: 119–130

    Google Scholar 

  37. Nadel L, Hardt O. Update on memory systems and processes. Neuropsychopharmacology, 2011, 36: 251–273

    Google Scholar 

  38. Nader K, Hardt O. A single standard for memory: the case for reconsolidation. Nat Rev Neurosci, 2009, 10: 224–234

    Google Scholar 

  39. Gonzalez C, Dutt V. Instance-based learning: Integrating sampling and repeated decisions from experience. Psychol Rev, 2011, 118: 523–551

    Google Scholar 

  40. Homa D, Hout M C, Milliken L, et al. Bogus concerns about the false prototype enhancement effect. J Exp Psychol-Learn Mem Cogn, 2011, 37: 368–377

    Google Scholar 

  41. Smith J D, Minda J P. Distinguishing prototype-based and exemplar-based processes in dot-pattern category learning. J Exp Psychol-Learn Mem Cogn, 2002, 28: 800–811

    Google Scholar 

  42. Smith J D, Redford J S, Haas S M. Prototype abstraction by monkeys (Macaca mulatta). J Exp Psychol-Gen, 2008, 137: 390–401

    Google Scholar 

  43. Lewandowsky S, Palmeri T J, Waldmann M R. Introduction to the special section on theory and data in categorization: integrating computational, behavioral, and cognitive neuroscience approaches. J Exp Psychol-Learn Mem Cogn, 2012, 38: 803–806

    Google Scholar 

  44. Freedman D J, Assad J A. A proposed common neural mechanism for categorization and perceptual decisions. Nat Neurosci, 2011, 14: 143–146

    Google Scholar 

  45. Gold J I, Shadlen M N. The influence of behavioral context on the representation of a perceptual decision in developing oculomotor commands. J Neurosci, 2003, 23: 632–651

    Google Scholar 

  46. Kable J W, Glimcher P W. The neurobiology of decision: consensus and controversy. Neuron, 2009, 63: 733–745

    Google Scholar 

  47. Freedman D J, Assad J A. Experience-dependent representation of visual categories in parietal cortex. Nature, 2006, 443: 85–88

    Google Scholar 

  48. Freedman D J, Riesenhuber M, Poggio T, et al. Categorical representation of visual stimuli in the primate prefrontal cortex. Science, 2001, 291: 312–316

    Google Scholar 

  49. Freedman D J, Riesenhuber M, Poggio T, et al. A comparison of primate prefrontal and inferior temporal cortices during visual categorization. J Neurosci, 2003, 23: 5235–5246

    Google Scholar 

  50. Williams Z M, Elfar J C, Eskandar E N, et al. Parietal activity and the perceived direction of ambiguous apparent motion. Nat Neurosci, 2003, 6: 616–623

    Google Scholar 

  51. Toth L J, Assad J A. Dynamic coding of behaviourally relevant stimuli in parietal cortex. Nature, 2002, 415: 165–168

    Google Scholar 

  52. Stoet G, Snyder L H. Single neurons in posterior parietal cortex of monkeys encode cognitive set. Neuron, 2004, 42: 1003–1012

    Google Scholar 

  53. Gottlieb J. From thought to action: the parietal cortex as a bridge between perception, action, and cognition. Neuron, 2007, 53: 9–16

    Google Scholar 

  54. Chen Y, Martinez-Conde S, Macknik S L, et al. Task difficulty modulates the activity of specific neuronal populations in primary visual cortex. Nat Neurosci, 2008, 11: 974–982

    Google Scholar 

  55. Asplund C L, Todd J J, Snyder A P, et al. A central role for the lateral prefrontal cortex in goal-directed and stimulus-driven attention. Nat Neurosci, 2010, 13: 507–512

    Google Scholar 

  56. Solway A, Botvinick M M. Goal-directed decision making as probabilistic inference: a computational framework and potential neural correlates. Psychol Rev, 2012, 119: 120–154

    Google Scholar 

  57. Purcell B A, Heitz R P, Cohen J Y, et al. Neurally constrained modeling of perceptual decision making. Psychol Rev, 2010, 117: 1113–1143

    Google Scholar 

  58. Palmeri T J, Gauthier I. Visual object understanding. Nat Rev Neurosci, 2004, 5: 291–304

    Google Scholar 

  59. Peyrin C, Michel C M, Schwartz S, et al. The neural substrates and timing of top-down processes during coarse-to-fine categorization of visual scenes: a combined fMRI and ERP study. J Cognitive Neurosci, 2010, 22: 2768–2780

    Google Scholar 

  60. Gao Z F, Bentin S. Coarse-to-fine encoding of spatial frequency information into visual short-term memory for faces but impartial decay. J Exp Phychol-Hum Percep Perf, 2011, 37: 1051–1064

    Google Scholar 

  61. Goffaux V, Peters J, Haubrechts J, et al. From coarse to fine? Spatial and temporal dynamics of cortical face processing. Cereb Cortex, 2011, 21: 467–476

    Google Scholar 

  62. Griffiths O, Mitchell C J. Selective attention in human associative learning and recognition memory. J Exp Psychol-Gen, 2008, 137: 626–648

    Google Scholar 

  63. Deng W, Aimone J B, Gage F H. New neurons and new memories: how does adult hippocampal neurogenesis affect learning and memory? Nat Rev Neurosci, 2010, 11: 339–350

    Google Scholar 

  64. De Fockert J W, Rees G, Frith C D, et al. The role of working memory in visual selective attention. Science, 2001, 291: 1803–1806

    Google Scholar 

  65. Saalmann Y B, Pigarev I N, Vidyasagar T R. Neural mechanisms of visual attention: how top-down feedback highlights relevant locations. Science, 2007, 316: 1612–1615

    Google Scholar 

  66. Sigala N, Logothetis N K. Visual categorization shapes feature selectivity in the primate temporal cortex. Nature, 2002, 415: 318–320

    Google Scholar 

  67. Kundel H L, Nodine C F. Interpreting chest radiographs without visual search. Radiology, 1975, 116: 527–532

    Google Scholar 

  68. Treisman A M, Gelade G. A feature-integration theory of attention. Cog Psychol, 1980, 12: 97–136

    Google Scholar 

  69. Liu Y J, Fu Q F, Liu Y, et al. 2D-line-drawing-based 3D object recognition. In: Computational Visual Media, Beijing, 2012. 146–153

    Google Scholar 

  70. Liu Y J, Luo X, Joneja A, et al. User-adaptive sketch-based 3D CAD model retrieval. IEE Trans Autom Sci Eng, 2013, 99: 1–13

    Google Scholar 

  71. Wolfe J M. Guided Search 4.0: current progress with a model of visual search. In: Integrated Models of Cognitive Systems. New York: Oxford, 2007. 99–119

    Google Scholar 

  72. Wolfe J M, Cave K R, Franzel S L. Guided search: an alternative to feature integration model for visual search. J Exp Phychol-Hum Percep Perf, 1989, 15: 419–433

    Google Scholar 

  73. Williams C C, Henderson J M, Zacks R T. Incidental visual memory for targets and distractors in visual search. Percept Psychophys, 2005, 67: 816–827

    Google Scholar 

  74. Wolfe J M. Guided search 2.0: a revised model of visual search. Psychonomic Bull Rev, 1994, 1: 202–238

    Google Scholar 

  75. Hao F, Zhang H, Fu X L. Modulation of attention by faces expressing emotion: evidence from visual marking. In: Tao J H, Tan T N, Picard R W, eds. Affective Computing and Intelligent Interaction. Berlin/Heidelberg: Springer-Verlag, 2005. 127–134

    Google Scholar 

  76. Hao F, Fu X L. Visual marking: a mechanism of prioritizing selection. Adv Psychol Sci, 2006, 14: 7–11

    Google Scholar 

  77. Sternberg S. High-speed scanning in human memory. Science, 1966, 153: 652–654

    Google Scholar 

  78. Hawkins J, Blakeslee S. On Intelligence. New York: Times Books, 2004

    Google Scholar 

  79. Bear M F, Connors B W, Paradiso M A. Neuroscience: exploring the brain. 3rd ed. Philadelphia: Lippincott Williams & Wilkins, 2006

    Google Scholar 

  80. Rinkus G J. A cortical sparse distributed coding model linking mini’and macrocolumn-scale functionality. Front Neuroanat, 2010, 4: 17

    Google Scholar 

  81. Mountcastle V. An Organizing Principle for Cerebral Function: the Unit Model and the Distributed System. Cambridge: MIT Press, 1978

    Google Scholar 

  82. Elad M. Sparse and Redundant Representations: from Theory to Applications in Signal and Image Processing. New York: Springer, 2010

    Google Scholar 

  83. Bruckstein A M, Donoho D L, Elad M. From sparse solutions of systems of equations to sparse modeling of signals and images. SIAM Rev, 2009, 51: 34–81

    MATH  MathSciNet  Google Scholar 

  84. Yi Z, Tan K K. Convergence Analysis of Recurrent Neural Networks. Dordrecht: Kluwer Academic Publishers, 2004

    MATH  Google Scholar 

  85. Tang H J, Tan K C, Yi Z. Neural Networks: Computational Models and Applications. Heidelberg: Springer-Verlag, 2007

    Google Scholar 

  86. Seung H S. How the brain keeps the eyes still. Proc Nat Acad Sci USA, 1996, 93: 13339–13344

    Google Scholar 

  87. Wu S, Amari S, Nakahara H. Population coding and decoding in a neural field: a computational study. Neural Comput, 2002, 14: 999–1026

    MATH  Google Scholar 

  88. Zhang K. Representation of spatial orientation by the intrinsic dynamics of head-direction cell ensembles: a theory. J Neurosci, 1996, 16: 2112–2126

    Google Scholar 

  89. Yu J, Yi Z, Zhang L. Representations of continuous attractors of recurrent neural networks. IEEE Trans Neural Netw, 2009, 20: 368–372

    Google Scholar 

  90. Wu C, Liu Y. Queuing network modeling of the psychological refractory period (PRP). Psychol Rev, 2008, 115: 913–954

    Google Scholar 

  91. Johnson J G, Busemeyer J R. Rule-based decision field theory: a dynamic computational model of transitions among decision-making strategies. In: Betsch T, Haberstroh S, eds. The Routines of Decision Making. Mahwah: Lawrence Erlbaum, 2005. 3–19

    Google Scholar 

  92. Zhao G, Wu C, Qiao C. A mathematical model for the prediction of speeding with its validation. IEEE Trans Intell Transp Syst, 2013, 14: 828–836

    Google Scholar 

  93. Wang X H, Jia J, Hu P Y, et al. Understanding the emotional impact of image. In: ACM Multimedia, Nara, 2012. 1369–1370

    Google Scholar 

  94. Jia J, Wu S, Wang X H, et al. Can we understand van Gogh’s mood? Learning to infer affects from images in social networks. In: ACM Multimedia, Nara, 2012. 857–860

    Google Scholar 

  95. Wang X H, Jia J, Cai L H. Affective image adjustment with a single word. Vis Comput, 2013, 29: 1121–1133

    Google Scholar 

  96. Wang X H, Jia J, Liao H Y, et al. Affective image colorization. J Comput Sci Technol, 2012, 27: 1119–1128

    Google Scholar 

  97. Kobayashi S. Art of Color Combinations. Tokyo: Kodansha International, 1995

    Google Scholar 

  98. Zhang Y F, Hu S M, Martin R R. Shrinkability maps for content-aware video resizing. Comput Graph Forum, 2008, 27: 1797–1804

    Google Scholar 

  99. Zhang G X, Cheng M M, Hu S M, et al. A shape-preserving approach to image resizing. Comput Graph Forum, 2009, 28: 1897–1906

    Google Scholar 

  100. Dahan M J, Chen N, Shamir A, et al. Combining color and depth for enhanced image segmentation and retargeting. Vis Comput, 2012, 28: 1181–1193

    Google Scholar 

  101. Liu Y J, Luo X, Xuan Y M, et al. Image retargeting quality assessment. Comput Graph Forum, 2011, 30: 583–592

    Google Scholar 

  102. Chen L. Topological structure in visual perception. Science, 1982, 218: 699–700

    Google Scholar 

  103. Anderson J R, Bothell D, Byrne M D, et al. An integrated theory of the mind. Psychol Rev, 2004, 111: 1036–1060

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. State Key Laboratory of Brain and Cognitive Science, Institute of Psychology, Chinese Academy of Sciences, Beijing, 100101, China

    XiaoLan Fu, Ye Liu & WenFeng Chen

  2. TNLIST, Department of Computer Science and Technology, Tsinghua University, Beijing, 100084, China

    LianHong Cai, Jia Jia & YongJin Liu

  3. College of Computer Science, Sichuan University, Chengdu, 610064, China

    Zhang Yi

  4. Key Laboratory of Behavioral Science, Institute of Psychology, Chinese Academy of Sciences, Beijing, 100101, China

    GuoZhen Zhao & ChangXu Wu

Authors
  1. XiaoLan Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. LianHong Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ye Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jia Jia
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. WenFeng Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zhang Yi
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. GuoZhen Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. YongJin Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. ChangXu Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to XiaoLan Fu.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Fu, X., Cai, L., Liu, Y. et al. A computational cognition model of perception, memory, and judgment. Sci. China Inf. Sci. 57, 1–15 (2014). https://doi.org/10.1007/s11432-013-4911-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11432-013-4911-9

Keywords

Search

Navigation