This is a preview of subscription content, log in via an institution to check access.
Notes
For information about Paul Smolenky refer to the website http://www.cog.jhu.edu/faculty/smolensky/.
PDP stands for “Parallel Distributed Processing”.
The term “subsymbolic approach” or “subsymbolic paradigm” can be found in Smolensky (1988a), p. 139, in Smolensky (1988b), p. 3, in Smolensky (1988c), p. 63 and in Smolensky (1997), p. 235. A general introduction to the subsymbolic paradigma is presented in Clark (1993), pp. 18, 24–26, 29, 47–48, 61–62, 82, 99, 102, 118–19, 154, 168 and Maurer (2006, 2009).
Smolensky (1988b, c).
For an introduction to this debate see for example Eliasmith and Bechtel (2003) or Maurer (2006, 2009, 2008b).
In Fodor and Pylyshyn (1988) the authors argue that the classical symbolic model is characterized by (1) a combinatorial syntax and semantics for mental representations, and (2) mental operations in the form of structure sensitive processes in such a manner that several fundamental and necessary properties for an adequate theory of cognition can be explained. These properties of a cognitive system are its productivity, systematicity, compositionality and its inferential coherence.
Thus, the core of the symbolism versus connectionism debate consists in the following dispute, designated as the “Fodor-Pylyshyn-challenge”, best described in Fodor and McLaughlin (1990), pp. 183–184: “[…] Paul Smolensky responds to a challenge Jerry Fodor and Zenon Pylyshyn […] have posed for connectionist theories of cognition: to explain the existence of systematic relations among cognitive capacities without assuming that cognitive processes are causally sensitive to the constituent structure of mental representations. This challenge implies a dilemma: if connectionism can't account for systematicity, it thereby fails to provide an adequate basis for a theory of cognition; but if its account of systematicity requires mental processes that are sensitive to the constituent structure of mental representations, then the theory of cognition it offers will be, at best, an implementation architecture for a ‘classical’ (language of thought) model.”
Further characterizations of the Fodor-Pylyshyn-challenge can be found in the book reviewed on p. 513 in volume 2, and in Maurer (2006), p. 35.
An introduction can be found in McLaughlin (1998).
The extended rough formulation can be found in Chap. 1 p. 46: “At a lower level of analysis, the mind/brain is a computer with a connectionist architecture. Parts of this architecture are organized so that they give rise, at a higher level of analysis, to a virtual machine with a (perhaps novel type of) symbolic architecture. In higher cognitive domains where symbolic theory has been successful, this symbolic architecture governs central aspects of the phenomena.”.
In Sect. 2.8, the “big picture” of the ICS research program (p. 95) is presented, with significant technical detail, by illustrating how these principles apply to a cognitive function, namely sentence comprehension. This section constitutes an ICS map, which relates the work discussed in all the chapters in parts II-IV.
Here, the activation vectors which realize the atomic symbols are linearly independent (p. 164).
Such a symbolic structure s is defined by a set of structural roles {\( r_{i} \)} as variables, which, for each single instance of the structure, may be individually occupied by single fillers {\( f_{i} \)} as values, which therefore individuate the structure. Thus, the symbolic structure s consists in a set of symbolic constituents, each of which corresponds to a filler/role binding \( f_{i} /r_{i} \), respectively. This filler/role binding \( f/r \) is now realized by a binding vector \( b = f/r \) consisting in the tensor product of the filler vector f, which realizes a filler f, and the role vector r, which realizes a role r, so that \( {\bf b} = {\bf f}/{\bf r} = {\bf f} \otimes {\bf r}. \) Thus, the connectionist realization of a symbolic structure s corresponds to an activation vector
\({\bf s} = \sum\limits_{i} {{\bf f_{i}} } \otimes {\bf r_{i}} \) consisting in the vector sum of the binding vectors, insofar as one identifies the structure s with the conjunction of the fillers/roles bindings \( f_{i} /r_{i} . \)
The tensor product representation is first described in Smolensky (1990). A brief introduction can be found in Petitot (1994), pp. 214–216.
Referring to Smolensky (1991), pp. 225–226, fn. 7 the tensor product representation scheme can be considered by analogy to Gödel number encodings.
Smolensky (1995), pp. 236–239 offers an example: The proposition p “Sandy (S) loves (L) Kim (K)”, following the LISP-convention, can be described by the symbolic representation p = [L, [S, K]], which mirrors in the following connectionist composite vector
\( {\bf p} = {\bf r_{0}} \otimes {\bf L }+ {\bf r_{1}} \otimes \left[ {{\bf r_{0}} \otimes {\bf S} + {\bf r_{1}} \otimes {\bf K}} \right] \) where the two—linear independent—role vectors r 0 and r 1 denote the left or right branch of a binary-branching tree.
Plate (2003).
The formal structure of the informational processing in a PDP connectionist network purely appears in a so-called “linear associator” (p. 193), where the output vector o results from the product of the matrix of connection weights W and the input vector i
o = W · i, where “·” denotes the matrix multiplication.
Thus, one may generally say that the activation spread is a process of a so-called “parallel soft constraint satisfaction”, where the constraints are ‘soft’ in the sense that each may be overruled or violated by other constraints. The total Harmony of a pattern a in a network with a connection weight matrix W is just the sum of the Harmony values of a applied to the complete set of all the individual connections in a network. Hence, one receives the following formal definition of well-formedness or total Harmony of a total activation vector a in a PDP network:
\( H (a )= \sum\nolimits_{\beta \alpha } {H_{\beta \alpha } } = \sum\nolimits_{\beta \alpha } {a_{\beta } } W_{\beta \alpha } a_{\alpha } = a^{T} \cdot W \cdot a, \) where here ∑βα means sum over all pairs of network units β, α. The last term reflects this sum more compact using the matrix multiplication and matrix transpose operation.
Building on this Harmony maximization process, P. Smolensky states the Harmony Maximization Theorem, which says, that in so-called “Harmonic networks” (p. 217) the Harmony of the total activation vector increases or stays the same at each moment of processing. In other words, cognitive processes are spreading activation algorithms that complete input activation vectors to final activation vectors that maximizes H (a).
An example of Harmony maximization is offered by P. Smolensky on page 213–15.
The Harmony maximization is first described in Smolensky (1995), 250–252.
Concerning the formal structure of a grammatical theory within ICS, P. Smolensky now shows how the ICS Principle P3 together with the principles P1 and P2 leads to a general grammar formalism based on numerical optimization, namely Harmonic Grammar, which is summed up in the Harmonic Grammar Soft-Constraint Theorem (p. 219–220). This theorem describes how the Harmony of a symbolic structure is to be computed: the Harmony of a structure as a whole is the sum of the amounts of Harmonies contributed by all its symbolic constituents, from which a simple kind of compositionality results.
See fn. 6.
P. Smolensky (p. 102) uses this expression to denote the architecture of the Classical symbolic theory, as presented by J. A. Fodor, Z. W. Pylyshyn and Br. P. McLaughlin.
An isomorphic relation only exists concerning the transformation from a symbolic representation into a vectorial, connectionist representation via the tensor product representation.
An introduction to the typology of local and distributed representational forms can be found in van Gelder (1999).
I.e., in their criticism, J. A. Fodor, Z. W. Pylyshyn and Br. P. McLaughlin argue that in a tensor product representation the constituent vectors of complex activity vectors aren't physically present when the complex vector itself is tokened, so that they cannot have any causal consequences referring to the constituent structure (Fodor and McLaughlin (1990), 186–188, 196–203 and McLaughlin (1993), 167–71, 178–180). Further, Br. P. McLaughlin (1997), 342 claims that the ICS architecture may be interpreted as an (mere) implementation architecture for Classical symbolic architectures. However, it would be a matter of a merely implementational connectionism if one presumed that an ultra or strictly local representational form (see fn. 21) were used, because in this case one would have constructed an architecture with a Classical constituent structure, whereas P. Smolensky stresses that connectionist representations are (fully) distributed (see fn. 21) according to the superposition principle. Finally, if J. A. Fodor and Z. W. Pylyshyn (1988) argue that the combinatorial properties, i.e., systematicity etc., are necessary properties of a cognitive system and are explained by the classical but not by the connectionist architecture, it is assumed by the authors that “structural necessarily” means ipso facto “formal-symbolic”, whereas J. Petitot (1994), 203, 210, 212–213 stresses, that this “dogma of logical form” is the “Achille's heel” of all their arguments, so that the Fodor-Pylyshyn-challenge does not take into consideration other versions of structural mathematical modelling, such as the connectionist model.
A brief introduction in the symbolism versus connectionism debate can be found in the book to be reviewed on p. 513–14 (vol. 2) and in Petitot (1994), pp. 217–220. An extended analysis can be found in Maurer (2006, 2009).
References
Clark, A. (1993). Associative engines: Connectionism, concepts, and representational change. Cambridge, MA: A Bradford Book, The MIT Press.
Eliasmith, C., & Bechtel, W. (2003). Symbolic versus subsymbolic. In L. Nadel (Ed.), Encyclopedia of cognitive science (Vol. 4, pp. 288–295). London: Nature Publishing Group.
Fodor, J. A., & McLaughlin, Br. P. (1990). Connectionism and the problem of systematicity: Why Smolensky’s solution doesn’t work. Cognition, 35, 183–204.
Fodor, J. A., & Pylyshyn, Z. W. (1988). Connectionism and cognitive architecture: A critical analysis. Cognition, 28, 4–50.
Maurer, H. (2006/2009). Das Subsymbolische Paradigma Paul Smolensky’s vor dem Hintergrund der Symbolismus vs. Konnektionismus Debatte. Master Thesis. University of Tuebingen. Norderstedt: BoD-Verlag.
Maurer, H. (2008a). Die ICS-Architektur P. Smolensky’s. Unpublished manuscript.
Maurer, H. (2008b). Die Konstruktion einer integrativen Theorie der Kognition vor dem Hintergrund der Symbolismus vs. Konnektionismus Debatte. Unpublished manuscript. Submitted to the journal Philosophia Naturalis.
McLaughlin, Br. P. (1993). The connectionism/classicism battle to win souls. Philosophical Studies, 71, 163–190.
McLaughlin, Br. P. (1997). Classical constituents in Smolensky’s ICS architecture. In M. L. Dalla Chiara, et al. (Eds.), Structures and norms in science. Tenth international congress of logic, methodology and philosophy of science (Vol. 2, pp. 331–343). Florence, Dordrecht, NL: Kluwer Academic Publishers. August 1995.
McLaughlin, Br. P. (1998). Connectionism. In E. Craig (Ed.), Concise Routledge encyclopedia of philosophy (pp. 166–167). London: Routledge.
Petitot, J. (1994). Dynamical constituency: An epistemological analysis. Sémiotiques, 6, 214–216.
Plate, T. A. (2003). Holographic reduced representations: Distributed representation for cognitive structures. Center for the Study of Language and Information, Leland Stanford Junior University. Stanford, CA: CSLI Publications.
Smolensky, P. (1988a). The constituent structure of connectionist mental states: A reply to Fodor and Pylyshyn. In T. Horgan & J. Tienson (Eds.), Spindel conference 1987: Connectionism and the philosophy of mind. The Southern Journal of Philosophy. Special Issue on Connectionism and the Foundations of Cognitive Science, Vol. 26, Supplement, pp. 137–161.
Smolensky, P. (1988b). On the proper treatment of connectionism. Behavioral and Brain Sciences, 11, 1–23.
Smolensky, P. (1988c). Putting together connectionism – again. Behavioral and Brain Sciences, 11, 59–74.
Smolensky, P. (1990). Tensor product variable binding and the representation of symbolic structures in connectionist systems. Artificial Intelligence, 46, 159–216.
Smolensky, P. (1991). Connectionism, constituency, and the language of thought. In B. Loewer & G. Rey (Eds.), Meaning in mind: Fodor and his critics (pp. 201–227). Cambridge, MA: Blackwell.
Smolensky, P. (1994a, 1995). Reply: Constituent structure and explanation in an integrated connectionist/symbolic cognitive architecture. In C. MacDonald & G. MacDonald (Eds.), Connectionism: Debates on psychological explanation (Vol. 1, 1994; Vol. 2, 1995, pp. 223–290). Oxford, UK: Blackwell Publishers.
Smolensky, P. (1997). Connectionist modeling: Neural computation/mental connections. In J. Haugeland (Ed.), Mind design II: Philosophy – psychology – artificial intelligence (pp. 233–250). Cambridge, MA: A Bradford Book, The MIT Press.
Smolensky, P., Legendre, G., & Miyata, Y. (1992). Principles for an integrated connectionist/symbolic theory of higher cognition (pp. 1–75). Technical Report CU-CS-600-92. Department of Computer Science.
Smolensky, P., Legendre, G., & Miyata, Y. (1994b). Integrating connectionist and symbolic computation for the theory of language. In V. Honavar & L. Uhr (Eds.), Artificial intelligence and neural networks: Steps toward principled integration (pp. 509–530). San Diego, CA: Academic Press.
van Gelder, T. (1999). Distributed vs. local representation. In R. A. Wilson & Fr. C. Keil (Eds.), The MIT encyclopedia of the cognitive sciences (pp. 236–238). Cambridge, MA: The MIT Press.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Maurer, H. Paul Smolensky, Géraldine Legendre: The Harmonic Mind. From Neural Computation to Optimality-Theoretic Grammar. Vol. 1: Cognitive Architecture. Vol. 2: Linguistic and Philosophical Implications. J Gen Philos Sci 40, 141–147 (2009). https://doi.org/10.1007/s10838-009-9089-x
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10838-009-9089-x