Summary
In the present paper connectionist approaches to the problem of internal representation and the nature of concepts are discussed. In the first part the concept of representation that underlies connectionist modeling is made explicit. It is argued that the connectionist view of representation relies on a correlational theory of semantic content- i.e., the covariation between internal and external states is taken as the basis for ascribing meaning to internal states. The problems and virtues of such a correlational approach to internal representation are addressed. The second part of the paper is concerned with whether connectionism is capable of accounting for the apparent productivity and systematicity of language and thought. There is an evaluation of the recent arguments of Fodor and Pylyshyn, who claim that systematicity can only be explained if one conceives of mental representations as structured symbols composed of context-free constituents. There is a review of empirical evidence that strongly suggests that concepts are not fixed memory structures and that the meaning of constituent symbols varies, depending on the context in which they are embedded. On the basis of this review it is concluded that the meaning of a complex expression is not computed from the context-free meanings of the constituents, and that strong compositionality, as endorsed by Fodor and Pylyshyn (1988), seems implausible as a process theory for the comprehension of complex concepts. Instead, the hypothesis is endorsed that constraint satisfaction in distributed connectionist networks may allow for an alternative account of weak compositionality compatible with the context sensitivity of meaning. In the final section, it is argued that neither mere implementation of a “language of thought” in connectionist networks nor radical elimination of symbol systems seems to be a fruitful research strategy, but that it might be more useful to discuss how connectionist systems can develop the capacity to use external symbol systems like language or logic without instantiating symbol systems themselves.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Barclay, J. R., Bransford, J. D., Franks, J. J., McCarrell, N. S., & Nitsch, K. (1974). Comprehension and semantic flexibility.Journal of Verbal Learning and Verbal Behavior, 13, 471–481.
Barsalou, L. W. (1982). Context-independent and context-dependent information in concepts.Memory & Cognition, 10, 82–93.
Barsalou, L. W. (1987). The instability of graded structure: Implications for the nature of concepts. In U. Neisser (Ed.),Concepts and conceptual development: Ecological and intellectual factors in categorization. Cambridge: Cambridge University Press.
Barsalou, L. W. (1989). Intraconcept similarity and its implications for interconcept similarity. In S. Vosniadou & A. Ortony, (Eds.),Similarity and analogical reasoning, (pp. 76–121). Cambridge: Cambridge University Press.
Bechtel, W. (1990). Multiple levels of inquiry in cognitive science.Psychological Research, 52, 271–281.
Bechtel, W., & Abrahamsen, A. A. (1990).Connectionism and the mind: An introduction to parallel processing in networks. Oxford: Basil Blackwell.
Block, N. (1986). Advertisment for a semantics for psychology.Midwest Studies in Philosophy, 10, 615–678.
Brousse, O., & Smolensky, P. (1990). Connectionist generalization and incremental learning in combinatorial domains. In H. Haken & M. Stadler (Eds.),Synergetics of cognition. Berlin, Heidelberg, New York: Springer.
Churchland, P. M. (1989).A neurocomputational perspective: The nature of mind and the structure of science. Cambridge, MA: MIT Press.
Churchland, P. M., & Churchland, P. S. (1983). Content: Semantic and information-theoretic.Behavioral and Brain Sciences, 6, 67.
Clark, A. (1989).Microcognition: Philosophy, cognitive science, and parallel distributed processing. Cambridge, MA: MIT Press.
Clark, H. H. (1969). Linguistic processes in deductive reasoning.Psychological Review, 76, 387–404.
Clark, H. H. (1983). Making sense of nonce sense. In G. Flores D'Arcais & R. J. Jarvella (Eds.),The process of language understanding. Chichester: Wiley.
Collins, A. M., & Quillian, M. R. (1969). Retrieval time from semantic memory.Journal of Verbal Learning and Verbal Behavior, 8, 240–247.
Cummins, R. (1989).Meaning and mental representation. Cambridge, MA: MIT Press.
Dennett, D. C. (1987).The intentional stance. Cambridge, MA: MIT Press.
Dretske, F. I. (1981).Knowledge and the flow of information. Cambridge, MA: MIT Press.
Dretske, F. I. (1983). Précis of knowledge and the flow of information.Behavioral and Brain Sciences, 6, 55–90.
Dretske, F. I. (1988).Explaning behavior. Reasons in a world of causes. Cambridge, MA: MIT Press.
Elman, J. L. (1989).Representation and structure in connectionist models. CRL Technical Report 8903, Center for Research in Language, University of California, San Diego.
Elman, J. L., & Zipser, D. (1987).Learning the hidden structure of speech. Technical Report 8701, Institute for Cognitive Science, University of California, San Diego.
Evans, J. St. B. T. (1982).The psychology of deductive reasoning. London: Routledge and Kegan Paul.
Feldman, J. A., & Ballard, D. H. (1982). Connectionist models and their properties.Cognitive Science, 6, 205–254.
Fodor J. A. (1983).The modularity of mind. Cambridge, MA: MIT Press.
Fodor J. A. (1984). Semantics, Wisconsin style.Synthese, 59, 231 -250.
Fodor J. A. (1987).Psyehosemantics. The problem of meaning in the philosophy of mind. Cambridge, MA: MIT Press.
Fodor J. A., & McLaughlin, B. (1990). Connectionism and the problem of systematicity: Why Smolensky's solution doesn't work.Cognition, 35, 183–204.
Fodor J. A., & Pylyshyn, Z. W. (1988). Connectionism and cognitive architecture: A critical analysis.Cognition, 28, 3–71.
Frauenfelder, U. H., & Tyler, L. K. (1987). The process of spoken word recognition: An introduction.Cognition, 25, 1–20.
Fuchs, A., Goschke, T., & Gude, D. (1988). On the role of imagery in linear syllogistic reasoning.Psychological Research, 50, 43–49.
Galotti, K. M. (1989). Approaches to studying formal and everyday reasoning.Psychological Bulletin, 105, 331–351.
Gelder, T. J. van (1988).Connectionist representations: Combinatorial, compostional, or neither? Department of Philosophy, University of Pittsburgh, Pittsburgh.
Goschke, T. (1989). Implikationen des Neuen Konnektionismus für den Begriff der mentalen Repräsentation. In B. Becker (Ed.),Zur Terminologie der Kognitionsforschung. Sankt Augustin: Gesellschaft für Mathematik und Datenverarbeitung.
Goschke, T., & Koppelberg, D. (1990). The concept of representation and the representation of concepts in connectionist models. In W. Ramsey, D. E. Rumelhart, & S. Stich (Eds.),Philosophy and connectionist theory. Hillsdale, NJ: Erlbaum.
Greenspan, S. L. (1986). Semantic flexibility and referential specificity of concrete nouns.Journal of Memory and Language, 25, 539–557.
Grice, P (1957). Meaning.Philosophical Review, 66, 377–388.
Hampton, J. A. (1987). Inheritance of attributes in natural concept conjunctions.Memory & Cognition, 15, 55–71.
Hampton, J. A. (1988). Overextension of conjunctive concepts: Evidence for a unitary model of concept typicality and class inclusion.Journal of Experimental Psychology: Learning, Memory, and Cognition, 14, 12–32.
Hebb, D. O. (1949).The organization of behavior. New York: Wiley.
Hinton, G. E. (1986). Learning distributed representations of concepts.Proceedings of the 8th Annual Conference of the Cognitive Science Society. Hillsdale, NJ: Erlbaum.
Hinton, G. E. (1987).Connectionist learning procedures. Technical Report CMU-CS-87-115. Computer Science Department, CarnegieMellon University, Pittsburgh, PA.
Hinton, G. E., & Anderson, J. A. (1981) (Eds.).Parallel Models of Associative Memory. Hillsdale, NJ: Erlbaum.
Hinton, G. E., McClelland, J. L., & Rumelhart, D. E. (1986). Distributed Representations. In D. E. Rumelhart & J. L. McClelland (Eds.),Parallel Distributed Processing. Explorations in the Microstructure of Cognition (Vol. 1, pp. 77–109). Cambridge, MA: MIT Press.
Hintzman, D. L. (1990). Human learning and memory: Connections and dissociations.Annual Review of Psychology, 41, 109–139.
Hörmann, H. (1976).Meinen und Verstehen. Frankfurt a.M.: Suhrkamp.
Hörmann, H. (1983).Was tun die Wörter miteinander im Satz? oder Wieviele sind einige, mehrere und ein paar? Göttingen: Hogrefe.
Hopfield, J. J. (1982). Neural networks and physical systems with emergent collective computational abilities.Proceedings of the National Academy of Sciences, USA, 79, 2554–2558.
Huttenlocher, J. (1968). Constructing spatial images: A strategy in reasoning.Psychological Review, 75, 550–560.
Jordan, M. I. (1986). Attractor dynamics and parallelism in a connectionist sequential machine.Proceedings of the 8th Annual Conference of the Cognitive Science Society. Hillsdale, NJ: Erlbaum.
Kahneman, D., & Miller, D. T. (1986). Norm theory: Comparing reality to its alternatives.Psychological Review, 93, 136–153.
Klix, F. (1983). An evolutionary approach to cognitive processes and creativity in human being. In R. Groner, M. Groner, & W. F. Bischof (Eds.),Methods of heuristics. Hillsdale, NJ: Erlbaum.
Lachter, J., & Bever, T. G. (1988). The relation between linguistic structure and associative theories of language learning — A constructive critique of some connectionist learning models.Cognition, 28, 195–247.
Lahav, R. (1989). Against compositionality: The case of adjectives.Philosophical Studies, 57, 261–279.
Levelt, W. J. M. (1990). Are multilayer feedforward networks effectively turing machines?Psychological Research, 52, 153–157.
Lloyd, D. (1989).Simple minds. Cambridge, MA: MIT Press.
Margolis, H. (1987).Patterns, thinking, and cognition. Chicago: University of Chicago Press.
Marslen-Wilson, W. (1987). Functional parallelism in spoken word-recognition.Cognition, 25, 71–102.
Marslen-Wilson, W., & Tyler, L. K. (1980). The temporal structure of spoken language understanding.Cognition, 8, 1–71.
Marslen-Wilson, W., & Tyler, L. K. (1987). Against modularity. In J. L. Garfield (Ed.),Modularity in knowledge representation and language comprehension. Cambridge, MA: MIT Press.
Maybery, M. T., Bain, J. D., & Halford, G. S. (1986). Information-processing demands of transitive inference.Journal of Experimental Psychology: Learning, Memory, and Cognition, 12, 600–613.
McClelland, J. L., & Kawamoto, A. H. (1986). Mechanisms of sentence processing: Assigning roles to constituents. In J. L. McClelland, D. E. Rumelhart, & The PDP Research Group (Eds.),Parallel Distributed Processing. Explorations in the Microstructure of Cognition (Vol. 2, pp. 272–326). Cambridge, MA: MIT Press.
McClelland, J. L., & Rumelhart, D. E. (1981). An interactive activation model of context effects in letter perception: Part I. An account of basis findings.Psychological Review, 88, 375–407.
McClelland, J. L., & Rumelhart, D. E., & The PDP Research Group (1986).Parallel Distributed Processing. Explorations in the Microstructure of Cognition (Vol. 2). Cambridge, MA: MIT Press.
McCloskey, M., & Glucksberg, S. (1978). Natural categories: Welldefined or fuzzy sets?Memory & Cognition, 6, 462–472.
McGinn, C. (1982). The structure of content. In A. Woodfield (Ed.),Thought and object. Oxford: Oxford University Press.
Medin, D. L., & Ortony, A. (1989). Psychological essenfalism. In S. Vosniadou & A. Ortony (Eds.),Similarity and analogical reasoning (pp. 179–196). Cambridge: Cambridge University Press.
Medin, D. L., & Shoben, E. J. (1988). Context and structure in conceptual combination.Cognitive Psychology, 20, 158–190.
Millikan, R. G. (1986). Thoughts without laws, cognitive science with content.Philosophical Review, 95, 47–81.
Minsky, M., & Papert, S. (1969).Perceptrons. Cambridge, MA: MIT Press.
Murphy, G. L. (1988). Comprehending complex concepts.Cognitive Science, 12, 529–562.
Osherson, D. N., & Smith, E. E. (1981). On the adequacy of prototype theory as a theory of concepts.Cognition, 9, 35–58.
Partee, B. H. (1984). Compositionality. In F. Landman & F. Veltman (Eds.),Varieties of formal semantics. Dordrecht: Foris.
Ramsey, W., Stich, S., & Garon, J. (1990). Connectionism, eliminativism, and the future of folk psychology. In W. Ramsey, D. E. Rumelhart, & S. Stich (Eds.),Philosophy and connectionist theory. Hillsdale, NJ: Erlbaum.
Rey, G. (1983). Concepts and stereotypes.Cognition, 15, 237–262.
Richardson-Klavehn, A., & Bjork, R. A. (1988). Measures of memory.Annual Review of Psychology, 39, 475–543.
Rips, L. J., Shoben, E. J., & Smith, E. E. (1973). Semantic distance and the verification of semantic relations.Journal of Verbal Learning and Verbal Behavior, 12, 1–20.
Rosch, E. (1973). On the internal structure of perceptual and semantic categories. In T. E. Moore (Ed.),Cognitive development and the acquisition of language. New York: Academic Press.
Rosch, E. (1975). Cognitive representation of semantic categories.Journal of Experimental Psychology: General, 104, 192–233.
Rosenberg, J. F. (1989). Connectionism and cognition. Report 1/1989. Research Group on "Mind and Brain," Center of Interdisciplinary Research (ZIF), University of Bielefeld, Bielefeld.
Rosenblatt, F. (1962).Principles of neurodynamics. New York: Spartan Books.
Roth, E. M., & Shoben, E. J. (1983). The effect of context on the structure of categories.Cognitive Psychology, 15, 346–378.
Rumelhart, D. E. (1979). Some problems with the notion of literal meaning. In A. Ortony (Ed.),Metaphor and thought. Cambridge: Cambridge University Press.
Rumelhart, D. E., & Zipser, D. (1986). Feature discovery by competitive learning. In D. E. Rumelhart, J. L. McClelland, & The PDP Research Group (1986).Parallel distributed processing. Explorations in the microstructure of cognition (Vol. 1). Cambridge, MA: MIT Press.
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning internal representations by back-propagating errors.Nature, 323, 533–536.
Rumelhart, D. E., McClelland, J. L., & The PDP Research Group (1986).Parallel distributed processing. Explorations in the microstructure of cognition (Vol. 1). Cambridge, MA: MIT Press.
Rumelhart, D. E., Smolensky, P., McClelland, J. L., & Hinton, G. E. (1986). Schemata and sequential thought processes in PDP models. In J. L. McClelland, D. E. Rumelhart, & The PDP Research Group (1986).Parallel distributed processing. Explorations in the microstructure of cognition (Vol. 2). Cambridge, MA: MIT Press.
Ryle, G. (1949).The concept of mind. Harmondsworth: Penguin Books.
Sanger, D. (1989). Contribution analysis: A technique for assigning responsibilities to hidden units in connectionist networks. Technical Report CU-CS-435-89, Department of Computer Science, University of Colorado, Boulder.
Schacter, D. L. (1987). Implicit memory: History and current status.Journal of Experimental Psychology: Learning, Memory, and Cognition, 13, 501–518.
Scheerer, E. (1989). Symbolverarbeitung und Konnektionismus: Eine Kontroverse ohne Ende? In B. Becker (Ed.), Zur Terminologie der Kognitionsforschung. Arbeitsberichte der GMD. Birlinghoven: Gesellschaft füir Mathematik und Datenverarbeitung.
Schiffer, S. (1987).Remnants of meaning. Cambridge, MA: MIT Press.
Seidenberg, M. S. (1985). Constraining models of word recognition.Cognition, 14, 169–190.
Seidenberg, M. S., & Tanenhaus, M. K., Leiman, J. M., & Bienkowski, M. (1982). Automatic access of the meanings of ambiguous words in context: Some limitations of knowledge-based processing.Cognitive Psychology, 14, 489–537.
Sellars, W. (1981). Mental events.Philosophical Studies, 39, 325–345.
Servan-Schreiber, D., Cleeremans, A., & McClelland, J. L. (1988).Encoding sequential structure in simple recurrent networks. Technical Report CMU-CS-88-183. School of Computer Science, Carnegie Mellon University, Pittsburgh.
Sejnowski, T. J., & Rosenberg, C. R. (1987). Parallel networks that learn to pronounce English text.Complex Systems, 1, 145–168.
Shastri, L., & Feldman, J. A. (1986). Neural nets, routines and semantic networks. In N. E. Sharkey (Ed.),Advances in cognitive science (Vol. 1). Chichester: Ellis Norwood.
Shaver, P., Pierson, L., & Lang, S. (1974-5). Converging evidence for the functional significance of imagery in problem solving.Cognition, 3, 359–375.
Sherry, D. E., & Schacter, D. L. (1987). The evolution of multiple memory systems.Psychological Review, 94, 439–454.
Smith, E. E. (1987). Concepts and thought. In R. J. Sternberg & E. E. Smith (Eds.),The psychology of human thought. Cambridge, MA: Cambridge University Press.
Smith, E. E., & Medin, D. (1981).Categories and concepts. Cambridge, MA: Harvard Univeristy Press.
Smith, E. E., & Osherson, D. N. (1984). Conceptual combination with prototype concepts.Cognitive Science, 8, 337–361.
Smith, E. E., Shoben, E. J., & Rips, L. J. (1974). Structure and process in semantic memory: A featural model for semantic decisions.Psychological Review, 81, 214–241.
Smith, E. E., & Osherson, D. N., Rips, L. J., & Keane, M. (1988). Combining concepts: A selective modification model.Cognitive Science, 12, 485–527.
Smolensky, P. (1986). Formal modeling of subsymbolic processes: An introduction to harmony theory. In N. E. Sharkey (Ed.),Advances in cognitive science (Vol. 1). Chichester: Ellis Norwood.
Smolensky, P. (1987).A method for connectionist variable binding. Technical Report CU-CS-356-87. Department of Computer Science, University of Colorado, Boulder.
Smolensky, P. (1988). On the proper treatment of connectionism.Behavioral and Brain Sciences, 11, 1–74.
Stich, S. P. (1988). From connectionism to eliminativism.Behavioral and Brain Sciences, 11, 53.
St. John, M., & McClelland, J. L. (1988). Applying contextual constraints in sentence comprehension.Proceedings of the 10th Annual Conference of the Cognitive Science Society (pp. 26–32). Hillsdale: Erlbaum.
Strube, G. (1984).Assoziation. Berlin, Heidelberg, New York: Springer.
Strube, G. (1990). Neokonnektionismus: Fine neue Basis für die Theorie und Modellierung menschlicher Kognition?Psychologische Rundschau, 41, 129–143.
Strube, G., Hemforth, B., & Wrobel, H. (1989). Auf dem Weg zu psychologisch fundierten Modellen menschlicher Sprachverarbeitung: Echtzeitanalysen des Satzverstehens. Manuscript, Ruhr-Universität Bochum.
Swinney, D. A. (1979). Lexical access during sentence comprehension: (Re)consideration of context effects.Journal of Verbal Learning and Verbal Behavior, 6, 645–659.
Tabossi, P. (1988). Effects of context on the immediate interpretation of unambiguous nouns.Journal of Experimental Psychology: Learning, Memory, and Cognition, 14, 153–162.
Tabossi, P. (1989). What's in a context? In D. S. Gorfein (Ed.),Resolving semantic ambiguity (pp. 25–39). New York, Berlin, Heidelberg: Springer.
Tanenhaus, M. K., & Lucas, M. M. (1987). Context effects in lexical processing.Cognition, 25, 213–234.
Thagard, P. (1989). Explanatory coherence.Behavioral and Brain Sciences, 12, 435–502.
Touretzky, D. S., & Hinton, G. E. (1988). A distributed connectionist production system.Cognitive Science, 12, 423–466.
Vygotski, L. S. (1962).Thought and language. Cambridge, MA: MIT Press.
Walker, W. H., & Kintsch, W. (1985). Automatic and strategic aspects of knowledge retrieval.Cognitive Science, 9, 261–283.
Waltz, D. L., & Pollack, J. B. (1985). Massively parallel parsing: A strongly interactive model of natural language interpretation.Cognitive Science, 9, 51–74.
Whitney, P., McKay, T., & Kellas, G. (1985). Semantic activation of noun concepts in context.Journal of Experimental Psychology: Learning, Memory, and Cognition, 11, 126–135.
Ziff, P. (1960).Semantic Analysis. Ithaca: Cornell University Press.
Zwitserlood, P. (1989). The locus of the effects of sentential-semantic context in spoken-word processing.Cognition, 32, 25–64.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Goschke, T., Koppelberg, D. Connectionist representation, semantic compositionality, and the instability of concept structure. Psychol. Res 52, 253–270 (1990). https://doi.org/10.1007/BF00877534
Issue Date:
DOI: https://doi.org/10.1007/BF00877534