There is a broad consensus about the fundamental role of thehippocampal system (hippocampus and its adjacent areas) in theencoding and retrieval of episodic memories. This paper presents afunctional model of this system. Although memory is not asingle-unit cognitive function, we took the view that the wholesystem of the smooth, interrelated memory processes may have acommon basis. That is why we follow the Ockham's razor principleand minimize the size or complexity of our model assumption set.The fundamental assumption is the requirement of solving the socalled ``homunculus fallacy'', which addresses the issue ofinterpreting the input. Generative autoassociators seem to offer aresolution of the paradox. Learning to represent and to recallinformation, in these generative networks, imply maximization ofinformation transfer, sparse representation and noveltyrecognition. A connectionist architecture, which integrates theseaspects as model constraints, is derived. Numerical studiesdemonstrate the novelty recognition and noise filtering propertiesof the architecture. Finally, we conclude that the derivedconnectionist architecture can be related to the neurobiologicalsubstrate.
This is a preview of subscription content, access via your institution.
Amari, S., Cichocki, A. and Yang, H., 1996: A new learning algorithm for blind signal separation, in Advances in Neural Information Processing Systems, Morgan Kaufmann, San Mateo, CA, pp. 757–763.
Attneave, F., 1954: Some informational aspects of visual perception, Psychological Review 61, 183–193.
Baddeley, R., 1996: An efficient code in v1? Nature 381, 560–561.
Baddeley, R., Abbott, L., Booth, M., Sengpiel, F., Freeman, T., Wakeman, E. and Rolls, E., 1997: Responses of neurons in primary and inferior temporal visual cortices to natural scenes, Proc. Roy. Soc. London B264, 1775–1783.
Barlow, H., 1961: Sensory Communication, MIT Press, Cambridge, MA,w.a. rosenblith edition, pp. 217–234.
Barlow, H., 1987: Learning receptive fields, volume IV of Proceedings of the IEEE 1st Annual Conf on Neural Networks, IEEE Press, U.S.A., pp. 115–121.
Barlow, H., 1999: Cerebral cortex, in C. Koch and J. Davis (eds), The MIT Encyclopedia of the Cognitive Sciences, MIT Press, Cambridge, MA, pp. 111–113.
Bell, A. and Sejnowski, T., 1995: An information–maximization approach to blind separation and blind deconvolution, Neural Computation 7, 1129–1159.
Brand, M., 1999: Voice puppetry, in Proceedings of Siggraph 99, ACM Press, New York, pp. 21–28.
Brand, M., Oliver, N. and Pentland, A., 1997: Coupled hidden Markov models for complex action recognition, in Proceedings of IEEE CVPR97, IEEE Press, pp. 994–999.
Burgess, N. and O'Keefe, J., 1996: Neuronal computation underlying the firing of place cells and their role in navigation, Hippocampus 7, 1–15.
Buzsáki, G., 1984: Feed–forward inhibition in the hippocampal formation, Prog. Neurobiol. 22, 131–153.
Cardoso, J. and Laheld, B., 1996: Equivalent adaptive source separation, IEEE Trans. on Signal Proc. 44, 3017–3030.
Charles, D. and Fyfe, C., 1998: Modelling multiple cause structure using rectification constraints, Network: Computations in Neural Systems 9, 167–182.
Chrobak, J., Lörincz, A. and Buzsáki, G., 2000: Physiological patterns in the hippocampo–entorhinal cortex system, Hippocampus 10, 457–465.
Clark, R. and Squire, L., 1998: Classical conditioning and brain systems: The role of awareness, Science 280, 77–81.
Cohen, M. and Grossberg, S., 1983: Absolute stability of global pattern formation and parallel memory storage by competitive neural networks, IEEE Transactions on Systems, Man, and Cybernetics SMC–13, 815–826.
Cohen, N. and Eichenbaum, H., 1993: Memory, Amnesia and the Hippocampal System, MIT Press, Cambridge, MA.
Cohen, N. and Squire, L., 1980: Preserved learning and retrention of pattern–analyzing skill in amnesia: Dissociation of knowing how and knowing that, Science 210, 207–210.
Comon, P., 1994: Independent component analysis – A new concept? Signal Processing 36, 287–314.
Cover, T., 1974: Universal gambling schemes and the complexity measures of Kolmogorov and Chaitin, Technical Report 12, Department of Statistics, Stanford University, Stanford, CA.
Cover, T. and Thomas, J., 1991: Elements of Information Theory, John Wiley and Sons, New York.
Csató, L., Kovács, G., Harnad, S., Pevtzow, R. and Lörincz, A., 2000: Category learning, categorization difficulty, and categorical perception: Computational and behavioral evidence, preprint.
Dayan, P. and Zemel, R., 1995: Competition and multiple cause models, Neural Computation 7, 565–579.
Dennett, D., 1991: Consciousness Explained, Little Brown, Boston, MA.
Diamond, I., 1979: The subdivision of the neocortex: A proposal to revise the traditional view of sensory, motor, and association areas, in J. Sprague and A. Epstein (eds), Progress in Psychobiology and Physiological Psychology, volume 8, Academic Press, New York, pp. 1–43.
Dong, D. and Atick, J., 1995: Temporal decorrelation: A theory of lagged and nonlagged responses in the lateral geniculate nucleus, Network 6, 159–178.
Eichenbaum, H., 2000: A cortical–hippocampal system for declarative memory, Nature Reviews, Neuroscience 1, 41–50.
Eichenbaum, H., Otto, T. and Cohen, N., 1994: Two functional roles of the hippocampal memory system, Behavioral and Brain Sciences 17, 449–518.
Field, D., 1987: Relations between the statistics of natural images and the response properties of cortical cells, Journal of the Optical Society of America A4, 2379–2394.
Földiák, P., 1990: Forming sparse representation by local anti–hebbian learning, Biological Cybernetics 64, 165–170.
Földiák, P. and Young, M., 1995: Sparse coding in the primate cortex, in M. Arbib (ed.), The Handbook of Brain Theory and Neural Networks, MIT Press, Cambridge, MA, pp. 895–898.
Ghahramani, Z. and Hinton, G., 1996: Parameter estimation for linear dynamical systems, Technical Report CRG–TR–96–2, University of Toronto, Toronto, http://www.gatsby.ucl.ac.uk/zoubin/papers.html. Grossberg, S., 1988: Competitive learning: From interactive activation to adaptive resonance, in S. Grossberg (ed.), Neural Networks and Natural Intelligence, MIT Press, Cambridge, MA.
Grossberg, S. and Carpenter, G., 1993: Normal and amnesic learning, recognition, and memory by a neural model of cortico–hippocampal interactions, Trends in Neurosciences 16, 131–137.
Harnad, S., 1987: Psychophysical and Cognitive Aspects of Categorical Perception: A Critical Overview, chapter 1, Cambridge University Press, New York.
Hateren, J. and Ruderman, D., 1998: Independent component analysis of natural image sequences yields spatio–temporal filters similar to simple cells in primary visual cortex. Proc. R. Soc. London B 265, 2315–2320.
Haykin, S., 1999: Neural Networks: A Comprehensive Foundation, Prentice Hall, New Jersey.
Henson, R., Rugg, M., Shallice, T., Josephs, O. and Dolan, R., 1999: Recollection and familiarity in recognition memory: An event–related functional magnetic resonance imaging study, Journal of Neuroscience 19, 3962–3972.
Henze, D., WE, W.C. and Barrionuevo, G., 1996: Dendritic morphology and its effects on the amplitude and rise–time of synaptic signals in hippocampal ca3 pyramidal cells, J. Comp. Neurology 369, 331–344.
Hinton, G. and Ghahramani, Z., 1997: Generative models for discovering sparse distributed representations, Philosophical Transactions of the Royal Society B 352, 1177–1190.
Hinton, G. and Sejnowski, T., 1983: Optimal perceptual inference, in Proc. of the IEEE Computer Society Conf. on Vision and Pattern Recognition, IEEE Computer Society, New York, pp. 448–453.
Hinton, G. and Zemel, R., 1994: Autoencoders, minimum description length and Helmholtz free energy, in J. Cowan, G. Tesauro and J. Alspector (eds), Advances in Neural Processing Systems, volume 6, Morgan Kaufmann, San Mateo, CA, pp. 3–10.
Hochreiter, S. and Schmidhuber, J., 1999: Lococode performs nonlinear ica without knowing the number of sources, in Proceedings of the ICA'99, Aussois, France, pp. 149–154.
Horn, B., 1977: Understanding image intensities, Artificial Intelligence 8, 201–231.
Hyvärinen, A., 1999: Survey on independent component analysis, Neural Computing Surveys 2, 94–128.
Hyvärinen, A., Hoyer, P. and Oja, E., 1999: Sparse code shrinkage: Denoising by nonlinear maximum likelihood estimation, in Advances in Neural Information Processing Systems 11 (NIPS*98), MIT Press, pp. 1739–1768.
Hyvärinen, A. and Oja, E., 1997: A fast fixed–point algorithm for independent component analysis, Neural Computation 9, 1483–1492.
Jaffe, D.B. and Carnevale, N.T., 1999: Passive normalization of synaptic integration influenced by dendritic architecture, J. Neurophysiol 82, 3268–3285.
Jutten, C. and Herault, J., 1991: Blind separation of sources, Part I: An adaptive algorithm based onneuromimetic architecture, Signal Processing 24, 1–10.
Karhunen, J., Oja, E., Wang, L., Vigario, R. and Joutsensalo, J., 1997: A class of neural networks for independent component analysis, IEEE Trans. on Neural Networks 8, 487–504.
Karhunen, J., Wang, L. and Joutsensalo, J., 1995: Neural estimation of basis vectors in independent component analysis, in Proceedings of the 1995 IEEE International Conference on Neural Networks, Perth, Australia, pp. 995–1000
Koch, C. and Poggio, T., 1999: Predicting the visual world: Silence is golden, Nature Neuroscience 2, 9–10.
Laheld, B. and Cardoso, J., 1994: Adaptive source separation with uniform performance, in Signal Processing VII: Theories and applications. Proceedings of EUSIPCO–94, Edinburgh, UK (September), volume 2, 183–186.
Lee, D. and Seung, H., 1999: Learning the parts of objects by non–negative matrix factorization, Nature 401, 788–791.
Lee, D. and Seung, H., 2001: Algorithms for non–negative matrix factorization, in Advances in Neural Processing Systems, volume 13, Morgan Kaufmann, San Mateo, CA, pp. 556–562.
Levy, W., 1996: A sequence predicting CA3 is a flexible associator that learns and uses context to solve hippocampal–like tasks, Hippocampus 6, 579–590.
Li, Z., 1995: A theory of visual motion coding in the primary visual cortex, Neural Computation 7,705–730.
Lisman, J., 1999: Relating hippocampal circuitry to function: Recall of memory sequences by reciprocal dentate–ca3 interactions, Neuron 22, 233–242.
Lisman, J. and Idiart, M., 1995: A mechanism for storing 7ą2 short–term memories in oscillatory subcycles, Science 267, 1512–1514.
Livingston, K. andrews, J. and Harnad, S., 1998: Categorical perception effects induced by category learning, Journal of Experimental Psychology: Learning, Memory, and Cognition 24, 732–753.
Lörincz, A., 1997: Towards a unified model of cortical computation II: From control architecture to a model of consiousness, Neural Network World 7, 137–152.
Lörincz, A., 1998: Forming independent components via temporal locking of reconstruction architectures: A functional model of the hippocampus, Biological Cybernetics 79, 263–275.
Lörincz, A. and Buzsáki, G., 1999: Computational model of the entorhinal–hippocampal region derived from a single principle, in Proceedings of IJCNN (July 9–16), IEEE Catalog Number: 99CH36339C, ISBN: 0–7803–5532–6, Washington.
Lörincz, A. and Buzsáki, G., 2000: Two–phase computational model training long–term memories in the entorhinal–hippocampal region, in H. Scharfman, M. Witter and R. Schwarz (eds), The Parahippocampal Region: Implications for Neurological and Psychiatric Dieseases, volume 911 of Annals of the New York Academy of Sciences, New York Academy of Sciences, New York, pp. 83–111.
Lörincz, A., Szatmáry, B. and Kabán, A., 2001a: Sign–changing filters similar to cells in primary visual cortex emerge by independent component analysis of temporally convolved natural image sequences, Neurocomputing 38–40, 1437–1442.
Lörincz, A., Szatmáry, B., Szirtes, G. and Takács, B., 2001b: Recognition of novelty made easy: Constraints of channel capacity on generative networks, in R. French (ed.), Connectionist Models of Learning, Development and Evolution, Springer–Verlag, London, pp. 73–82.
Mallat, S., 1998: A Wavelet Tour of Signal Processing, Academic Press, San Diego, CA.
McCarthy, R. and Warrington, E., 1990: Cognitive Neuropsychology, Academic Press, San Diego.
McClelland, J., 1981: Retrieving general and specific information from stored knowledge of specifics, in Proceedings of the Third Annual Meeting of the Cognitive Science Society, pp. 170–172.
McClelland, J. and Rumelhart, D., 1981: An interactive activation model of context effects in letter perception: Part 1 an account of basic findings, Psychological Review 88, 375–407.
McClelland, J. and Rumelhart, D., 1982: An interactive activation model of context effects in letter perception: Part 2 the contextual enhancement effect and some tests and extensions of the model, Psychological Review 89, 60–94.
Merhav, N. and Feder, M., 1998: Universal prediction, IEEE Trans. Inform. Theory. IT–44, 2124–2147.
Mishkin, M. and Murray, E., 1994: Stimulus recognition, Current Opinion in Neurobiology 4, 200–206.
Mozer, M., 1991: Discovering discrete distributed representations with iterative competitive learning, in R. Lippmann, J. Moody and D. Touretzky (eds), Advances in Neural Processing Systems, volume 3, Morgan Kaufmann, San Mateo, CA, pp. 627–634.
O'Keefe, J. and Nadel, L., 1978: The Hippocampus as a Cognitive Map, Clarendon Press, Oxford.
Olshausen, B., 1996: Learning linear, sparse factorial codes, A.I. Memo 1580, MIT AI Lab. C.B.C.L.
Olshausen, B. and Field, D., 1996: Emergence of simple–cell receptive field properties by learning a sparse code for natural images, Nature 381, 607–609.
Olshausen, B. and Field, D., 1997: Sparse coding with an overcomplete basis set: A strategy employed by v1? Vision Research 37, 3311–3325.
Pajunen, P., 1998: Blind source separation using algorithmic information theory, in C. Fyfe (ed.), Proceedings of Independence and Artificial Neural Networks, ISCS Academic Press, pp. 26–31.
Palm, G., 1992: On the information storage capacity of local learning rules, Neural Computation 4, 703–711.
Papoulis, A., 1984: Probability, Random Variables and Stochastic Processes, 2nd edition, McGraww–Hill, New York.
Parra, L., Deco, G. and Miesbach, S., 1995: Statistical independence and novelty detection with information preserving nonlinear maps, Neural Computation 8(2), 260–269.
Rao, R. and Ballard, D., 1997: Dynamic model of visual recognition predicts neural response properties in the visual cortex, Neural Computation 9, 721–763.
Rao, R. and Ballard, D., 1999: Predictive coding in the visual cortex: A functional interpretation of some extra–classical receptive–field effects, Nature Neuroscience 2, 79–87.
Rezek, I., Sykacek, P. and Roberts, S., 2000: Coupled hiddenMarkov models for biosignal interaction modelling, Technical report PARG–00–5, Oxford University, Oxford, UK to appear in 2000 IEE Special Issue Proceedings on Advances in Medical Signal and Information Processing.
Riedel, G., Micheau, J., Lam, A., Roloff, E., Martin, S., Bridge, H., Hoz, L., Poeschel, B., McVulloch, J. and Morris, R., 1999: Reversible neural inactivation reveals hippocampal participation in several memory processes, Nature Neuroscience 2, 898–905.
Rissanen, J., 1978: Modeling by shortest data description, Automatica 14, 465–471.
Rissanen, J., 1984: Universal coding, information, prediction and estimation, IEEE Trans. Inform. Theory IT–30, 629–636.
Roberts, S., 2000: Novelty Detection Using Extreme Value Statistics, draft, Oxford University, Oxford, UK, http://www.robots.ox.ac.ukparg.
Roweis, A. and Ghahramani, J., 1999: A unifying review of linear gaussian models, Neural Computation 11, 305–345.
Sáry, G., Vogels, R. and Orban, G., 1994: Orientation discrimination of motion–defined gratings, Vision Res. 34, 1331–1334.
Schachter, D., 1987: Implicit memory: History and current status, Journal of Experimental Psychology: Learning, Memory, and Cognition 13, 501–518.
Schlitz, C., Bodart, J., Dubois, S., Dejardin, S., Michel, C., Roucoux, A., Crommelinck, M. and Orban, G., 1999: Neuronal mechanisms of perceptual learning: Changes in human brain activity with training in orientation discrimination, NeuroImage 9, 46–62.
Schölkopf, B., Platt, J., Shawe–Taylor, J., Smola, A. and Williamson, R., 1999: Estimating the support of a high–dimensional distribution, Technical Report 99–87, Microsoft Research.
Scoville, W. and Milner, B., 1957: Loss of recent memory after bilateral hippocampal lesions, Journal of Neurol. Neurosurg. Psychiatry 20, 11–21.
Searle, J., 1992: The Rediscovery of Mind, Bradford Books, MIT Press, Cambridge, MA.
Shallice, T., 1988: From Neuropsychiology to Mental Structure, Cambridge Univ. Press, New York.
Shannon, C., 1948: A mathematical theory of communication, Bell Sys. Tech. Journal 27, 379–423 and 623–656.
Solomonoff, R., 1964: A formal theory of inductive inference, part i, Information and Control 7, 1–22.
Squire, L., 1992a: Declarative and nondeclarative memory: Multiple brain systems supporting learning and memory, J. Cog. Neurosci. 4, 232–243.
Squire, L., 1992b: Memory and the hippocampus: A synthesis of findings with rats, monkeys, and humans, Psychol. Rev. 99, 195–231.
Squire, L. and Kandel, E., 1999: Memory: From Mind to Molecules, Scientific American Press, New York.
Szatmáry, B. and Lörincz, A., 2001: Independent component analysis of temporal sequences subject to constraints by lgn inputs yields all the three major cell types of the primary visual cortex, J. of Comp. Neurosci. 11, 241–248.
Thorpe, S., Fize, D. and Marlot, C., 1996: Speed of processing in the human visual system, Nature 381, 520–522.
Tornay, S., 1938: Ockham: Studies and selections.
Treves, A., Panzeri, S., Rolls, E., Booth, M. and Wakeman, E., 1999: Firing rate distributions and effi–ciency of information transmission of inferior temporal cortex neurons to natural visual stimuli, Neural Computation 11, 601–631.
Tulving, E., 1983: Elements of Episodic Memory, Clarendon Press, Oxford.
Vovk, V. and Gammerman, A., 1999: Complexity approximation principle, Computer Journal 42, 318–322.
Wallace, C. and Boulton, D., 1968: An information theoretic measure for classification, Computer Journal 11, 185–194.
Wan, H., Aggleton, J. and Brown, M., 1999: Different contributions of the hippocampus and perirhinal cortex to recognition memory, J. Neurosci. 19, 1142–1148.
Wimbauer, S., Wenish, O., Miller, K. and van Hemmen, J., 1997: Development of spatio–temporal receptive fields of simple cells: I. Model formulation, Biological Cybernetics 77, 456–461.
Zemel, R. and Hinton, G., 1994: Developing population codes by minimizing description length, in J. Cowan, G. Tesauro and J. Alspector (eds), Advances in Neural Processing Systems, volume 6, Morgan Kaufmann, San Mateo, CA, pp. 11–18.
About this article
Cite this article
Lörincz, A., Póczos, B., Szirtes, G. et al. Ockham's Razor at Work: Modeling of the ``Homunculus''. Brain and Mind 3, 187–220 (2002). https://doi.org/10.1023/A:1019996320835
- functional modeling
- generative networks
- homunculus fallacy