Abstract
The mechanism of human cognition and its computability provide an important theoretical foundation to intelligent computation of visual media. This paper focuses on the intelligent processing of massive data of visual media and its corresponding processes of perception, memory, and judgment in cognition. In particular, both the human cognitive mechanism and cognitive computability of visual media are investigated in this paper at the following three levels: neurophysiology, cognitive psychology, and computational modeling. A computational cognition model of Perception, Memory, and Judgment (PMJ model for short) is proposed, which consists of three stages and three pathways by integrating the cognitive mechanism and computability aspects in a unified framework. Finally, this paper illustrates the applications of the proposed PMJ model in five visual media research areas. As demonstrated by these applications, the PMJ model sheds some light on the intelligent processing of visual media, and it would be innovative for researchers to apply human cognitive mechanism to computer science.
Similar content being viewed by others
References
Gallistel C R, King A. Memory and the Computational Brain: Why Cognitive Science Will Transform Neuroscience. New York: Blackwell/Wiley, 2009. iiv–xvi
Hu S M, Chen T, Xu K, et al. Internet visual media processing: a survey with graphics and vision applications. Vis Comput, 2013, 29: 393–405
Hulusic V, Debattista K, Aggarwal V, et al. Maintaining frame rate perception in interactive environments by exploiting audio-visual cross-modal interaction. Vis Comput, 2011, 27: 57–66
Vazquez P-P, Marco J. Using normalized compression distance for image similarity measurement: an experimental study. Vis Comput, 2012, 28: 1063–1084
Eysenck M W, Keane M T. Cognitive Psychology: a Student’s Handbook. 6th ed. New York: Psychology Press, 2010. 1–50
National Institute on Drug Abuse. Computational neuroscience at the NIH. Nat Neurosci, 2000, 3: 1161–1164
Buschman T J, Miller E K. Top-down versus bottom-up control of attention in the prefrontal and posterior parietal cortices. Science, 2007, 315: 1860–1862
Navalpakkam V, Itti L. Search goal tunes visual features optimally. Neuron, 2007, 53: 605–617
Katsuki F, Constantinidis C. Early involvement of prefrontal cortex in visual bottom-up attention. Nat Neurosci, 2012, 15: 1160–1166
Corbetta M, Shulman G L. Control of goal-directed and stimulus-driven attention in the brain. Nat Rev Neurosci, 2002, 3: 201–215
Zanto T P, Rubens M T, Thangavel A, et al. Causal role of the prefrontal cortex in top-down modulation of visual processing and working memory. Nat Neurosci, 2011, 14: 656–661
Tomita H M, Ohbayashi K, Nakahara I, et al. Top-down signal from prefrontal cortex in executive control of memory retrieval. Nature, 1999, 401: 699–703
Itti L, Koch C. Computational modelling of visual attention. Nat Rev Neurosci, 2001, 2: 194–203
Cox D, Meyers E, Sinha P. Contextually evoked object-specific responses in human visual cortex. Science, 2004, 303: 115–117
Kouh M, Poggio T. A canonical neural circuit for cortical nonlinear operations. Neural Comput, 2008, 20: 1427–1451
Poggio T, Bizzi E. Generalization in vision and motor control. Nature, 2004, 431: 768–774
Hung C P, Kreiman G, Poggio T, et al. Fast readout of object identity from macaque inferior temporal cortex. Science, 2005, 310: 863–866
Pylyshyn Z W. Computation and Cognition: toward a Foundation for Cognitive Science. Cambridge: The MIT Press, 1984. 1–16
Hasselmo M E, Sarter M. Modes and models of forebrain cholinergic neuromodulation of cognition. Neuropsychopharmacology, 2010, 36: 52–73
Tamietto M, de Gelder B. Neural bases of the non-conscious perception of emotional signals. Nat Rev Neurosci, 2010, 11: 697–709
Fries P, Reynolds J H, Rorie A E, et al. Modulation of oscillatory neuronal synchronization by selective visual attention. Science, 2001, 291: 1560–1563
Roberts M, Delicato L S, Herrero J, et al. Attention alters spatial integration in macaque V1 in an eccentricitydependent manner. Nat Neurosci, 2007, 10: 1483–1491
Qiu F T, Sugihara T, von der Heydt R. Figure-ground mechanisms provide structure for selective attention. Nat Neurosci, 2007, 10: 1492–1499
Hübner R, Steinhauser M, Lehle C. A dual-stage two-phase model of selective attention. Psychol Rev, 2010, 117: 759–784
Gondan M, Blurton S P, Hughes F, et al. Effects of spatial and selective attention on basic multisensory integration. J Exp Phychol-Hum Percep Perf, 2011, 37: 1887–1897
Schafer R J, Moore T. Selective attention from voluntary control of neurons in prefrontal cortex. Science, 2011, 332: 1568–1571
Couperus J W. Perceptual load influences selective attention across development. Develop Psychol, 2011, 47: 1431–1439
Cosman J D, Vecera S P. Object-based attention overrides perceptual load to modulate visual distraction. J Exp Phychol-Hum Percep Perf, 2012, 38: 576–579
Chen C C, Wu J K, Lin H W, et al. Visualizing long-term memory formation in two neurons of the drosophila brain. Science, 2012, 335: 678–685
Fell J, Axmacher N. The role of phase synchronization in memory processes. Nat Rev Neurosci, 2011, 12: 105–118
Fusi S, Abbott L F. Limits on the memory storage capacity of bounded synapses. Nat Neurosci, 2007, 10: 485–493
Eichenbaum H. A cortical-hippocampal system for declarative memory. Nat Rev Neurosci, 2000, 1: 41–50
McGaugh J L. Memory-a century of consolidation. Science, 2000, 287: 248–251
Tronson N C, Taylor J R. Molecular mechanisms of memory reconsolidation. Nat Rev Neurosci, 2007, 8: 262–275
Edelson M, Sharot T, Dolan R J, et al. Following the crowd: brain substrates of long-term memory conformity. Science, 2011, 333: 108–111
Frankland P W, Bontempi B. The organization of recent and remote memories. Nat Rev Neurosci, 2005, 6: 119–130
Nadel L, Hardt O. Update on memory systems and processes. Neuropsychopharmacology, 2011, 36: 251–273
Nader K, Hardt O. A single standard for memory: the case for reconsolidation. Nat Rev Neurosci, 2009, 10: 224–234
Gonzalez C, Dutt V. Instance-based learning: Integrating sampling and repeated decisions from experience. Psychol Rev, 2011, 118: 523–551
Homa D, Hout M C, Milliken L, et al. Bogus concerns about the false prototype enhancement effect. J Exp Psychol-Learn Mem Cogn, 2011, 37: 368–377
Smith J D, Minda J P. Distinguishing prototype-based and exemplar-based processes in dot-pattern category learning. J Exp Psychol-Learn Mem Cogn, 2002, 28: 800–811
Smith J D, Redford J S, Haas S M. Prototype abstraction by monkeys (Macaca mulatta). J Exp Psychol-Gen, 2008, 137: 390–401
Lewandowsky S, Palmeri T J, Waldmann M R. Introduction to the special section on theory and data in categorization: integrating computational, behavioral, and cognitive neuroscience approaches. J Exp Psychol-Learn Mem Cogn, 2012, 38: 803–806
Freedman D J, Assad J A. A proposed common neural mechanism for categorization and perceptual decisions. Nat Neurosci, 2011, 14: 143–146
Gold J I, Shadlen M N. The influence of behavioral context on the representation of a perceptual decision in developing oculomotor commands. J Neurosci, 2003, 23: 632–651
Kable J W, Glimcher P W. The neurobiology of decision: consensus and controversy. Neuron, 2009, 63: 733–745
Freedman D J, Assad J A. Experience-dependent representation of visual categories in parietal cortex. Nature, 2006, 443: 85–88
Freedman D J, Riesenhuber M, Poggio T, et al. Categorical representation of visual stimuli in the primate prefrontal cortex. Science, 2001, 291: 312–316
Freedman D J, Riesenhuber M, Poggio T, et al. A comparison of primate prefrontal and inferior temporal cortices during visual categorization. J Neurosci, 2003, 23: 5235–5246
Williams Z M, Elfar J C, Eskandar E N, et al. Parietal activity and the perceived direction of ambiguous apparent motion. Nat Neurosci, 2003, 6: 616–623
Toth L J, Assad J A. Dynamic coding of behaviourally relevant stimuli in parietal cortex. Nature, 2002, 415: 165–168
Stoet G, Snyder L H. Single neurons in posterior parietal cortex of monkeys encode cognitive set. Neuron, 2004, 42: 1003–1012
Gottlieb J. From thought to action: the parietal cortex as a bridge between perception, action, and cognition. Neuron, 2007, 53: 9–16
Chen Y, Martinez-Conde S, Macknik S L, et al. Task difficulty modulates the activity of specific neuronal populations in primary visual cortex. Nat Neurosci, 2008, 11: 974–982
Asplund C L, Todd J J, Snyder A P, et al. A central role for the lateral prefrontal cortex in goal-directed and stimulus-driven attention. Nat Neurosci, 2010, 13: 507–512
Solway A, Botvinick M M. Goal-directed decision making as probabilistic inference: a computational framework and potential neural correlates. Psychol Rev, 2012, 119: 120–154
Purcell B A, Heitz R P, Cohen J Y, et al. Neurally constrained modeling of perceptual decision making. Psychol Rev, 2010, 117: 1113–1143
Palmeri T J, Gauthier I. Visual object understanding. Nat Rev Neurosci, 2004, 5: 291–304
Peyrin C, Michel C M, Schwartz S, et al. The neural substrates and timing of top-down processes during coarse-to-fine categorization of visual scenes: a combined fMRI and ERP study. J Cognitive Neurosci, 2010, 22: 2768–2780
Gao Z F, Bentin S. Coarse-to-fine encoding of spatial frequency information into visual short-term memory for faces but impartial decay. J Exp Phychol-Hum Percep Perf, 2011, 37: 1051–1064
Goffaux V, Peters J, Haubrechts J, et al. From coarse to fine? Spatial and temporal dynamics of cortical face processing. Cereb Cortex, 2011, 21: 467–476
Griffiths O, Mitchell C J. Selective attention in human associative learning and recognition memory. J Exp Psychol-Gen, 2008, 137: 626–648
Deng W, Aimone J B, Gage F H. New neurons and new memories: how does adult hippocampal neurogenesis affect learning and memory? Nat Rev Neurosci, 2010, 11: 339–350
De Fockert J W, Rees G, Frith C D, et al. The role of working memory in visual selective attention. Science, 2001, 291: 1803–1806
Saalmann Y B, Pigarev I N, Vidyasagar T R. Neural mechanisms of visual attention: how top-down feedback highlights relevant locations. Science, 2007, 316: 1612–1615
Sigala N, Logothetis N K. Visual categorization shapes feature selectivity in the primate temporal cortex. Nature, 2002, 415: 318–320
Kundel H L, Nodine C F. Interpreting chest radiographs without visual search. Radiology, 1975, 116: 527–532
Treisman A M, Gelade G. A feature-integration theory of attention. Cog Psychol, 1980, 12: 97–136
Liu Y J, Fu Q F, Liu Y, et al. 2D-line-drawing-based 3D object recognition. In: Computational Visual Media, Beijing, 2012. 146–153
Liu Y J, Luo X, Joneja A, et al. User-adaptive sketch-based 3D CAD model retrieval. IEE Trans Autom Sci Eng, 2013, 99: 1–13
Wolfe J M. Guided Search 4.0: current progress with a model of visual search. In: Integrated Models of Cognitive Systems. New York: Oxford, 2007. 99–119
Wolfe J M, Cave K R, Franzel S L. Guided search: an alternative to feature integration model for visual search. J Exp Phychol-Hum Percep Perf, 1989, 15: 419–433
Williams C C, Henderson J M, Zacks R T. Incidental visual memory for targets and distractors in visual search. Percept Psychophys, 2005, 67: 816–827
Wolfe J M. Guided search 2.0: a revised model of visual search. Psychonomic Bull Rev, 1994, 1: 202–238
Hao F, Zhang H, Fu X L. Modulation of attention by faces expressing emotion: evidence from visual marking. In: Tao J H, Tan T N, Picard R W, eds. Affective Computing and Intelligent Interaction. Berlin/Heidelberg: Springer-Verlag, 2005. 127–134
Hao F, Fu X L. Visual marking: a mechanism of prioritizing selection. Adv Psychol Sci, 2006, 14: 7–11
Sternberg S. High-speed scanning in human memory. Science, 1966, 153: 652–654
Hawkins J, Blakeslee S. On Intelligence. New York: Times Books, 2004
Bear M F, Connors B W, Paradiso M A. Neuroscience: exploring the brain. 3rd ed. Philadelphia: Lippincott Williams & Wilkins, 2006
Rinkus G J. A cortical sparse distributed coding model linking mini’and macrocolumn-scale functionality. Front Neuroanat, 2010, 4: 17
Mountcastle V. An Organizing Principle for Cerebral Function: the Unit Model and the Distributed System. Cambridge: MIT Press, 1978
Elad M. Sparse and Redundant Representations: from Theory to Applications in Signal and Image Processing. New York: Springer, 2010
Bruckstein A M, Donoho D L, Elad M. From sparse solutions of systems of equations to sparse modeling of signals and images. SIAM Rev, 2009, 51: 34–81
Yi Z, Tan K K. Convergence Analysis of Recurrent Neural Networks. Dordrecht: Kluwer Academic Publishers, 2004
Tang H J, Tan K C, Yi Z. Neural Networks: Computational Models and Applications. Heidelberg: Springer-Verlag, 2007
Seung H S. How the brain keeps the eyes still. Proc Nat Acad Sci USA, 1996, 93: 13339–13344
Wu S, Amari S, Nakahara H. Population coding and decoding in a neural field: a computational study. Neural Comput, 2002, 14: 999–1026
Zhang K. Representation of spatial orientation by the intrinsic dynamics of head-direction cell ensembles: a theory. J Neurosci, 1996, 16: 2112–2126
Yu J, Yi Z, Zhang L. Representations of continuous attractors of recurrent neural networks. IEEE Trans Neural Netw, 2009, 20: 368–372
Wu C, Liu Y. Queuing network modeling of the psychological refractory period (PRP). Psychol Rev, 2008, 115: 913–954
Johnson J G, Busemeyer J R. Rule-based decision field theory: a dynamic computational model of transitions among decision-making strategies. In: Betsch T, Haberstroh S, eds. The Routines of Decision Making. Mahwah: Lawrence Erlbaum, 2005. 3–19
Zhao G, Wu C, Qiao C. A mathematical model for the prediction of speeding with its validation. IEEE Trans Intell Transp Syst, 2013, 14: 828–836
Wang X H, Jia J, Hu P Y, et al. Understanding the emotional impact of image. In: ACM Multimedia, Nara, 2012. 1369–1370
Jia J, Wu S, Wang X H, et al. Can we understand van Gogh’s mood? Learning to infer affects from images in social networks. In: ACM Multimedia, Nara, 2012. 857–860
Wang X H, Jia J, Cai L H. Affective image adjustment with a single word. Vis Comput, 2013, 29: 1121–1133
Wang X H, Jia J, Liao H Y, et al. Affective image colorization. J Comput Sci Technol, 2012, 27: 1119–1128
Kobayashi S. Art of Color Combinations. Tokyo: Kodansha International, 1995
Zhang Y F, Hu S M, Martin R R. Shrinkability maps for content-aware video resizing. Comput Graph Forum, 2008, 27: 1797–1804
Zhang G X, Cheng M M, Hu S M, et al. A shape-preserving approach to image resizing. Comput Graph Forum, 2009, 28: 1897–1906
Dahan M J, Chen N, Shamir A, et al. Combining color and depth for enhanced image segmentation and retargeting. Vis Comput, 2012, 28: 1181–1193
Liu Y J, Luo X, Xuan Y M, et al. Image retargeting quality assessment. Comput Graph Forum, 2011, 30: 583–592
Chen L. Topological structure in visual perception. Science, 1982, 218: 699–700
Anderson J R, Bothell D, Byrne M D, et al. An integrated theory of the mind. Psychol Rev, 2004, 111: 1036–1060
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Fu, X., Cai, L., Liu, Y. et al. A computational cognition model of perception, memory, and judgment. Sci. China Inf. Sci. 57, 1–15 (2014). https://doi.org/10.1007/s11432-013-4911-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11432-013-4911-9