Advertisement

Artificial Intelligence Review

, Volume 3, Issue 1, pp 3–20 | Cite as

Connectionism: past, present, and future

  • J. B. Pollack
Article

Abstract

Research efforts to study computation and cognitive modeling on neurally-inspired mechanisms have come to be called Connectionism. Rather than being brand new, it is actually the rebirth of a research programme which thrived from the 40s through the 60s and then was severely retrenched in the 70s. Connectionism is often posed as a paradigmatic competitor to the Symbolic Processing tradition of Artificial Intelligence (Dreyfus & Dreyfus, 1988), and, indeed, the counterpoint in the timing of their intellectual and commercial fortunes may lead one to believe that research in cognition is merely a zero-sum game. This paper surveys the history of the field, often in relation to AI, discusses its current successes and failures, and makes some predictions for where it might lead in the future.

Keywords

Neural Network Artificial Intelligence Complex System Research Programme Nonlinear Dynamics 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Ackley, D.H., Hinton, G.E. & Sejnowski, T.J. (1985) A learning algorithm for Boltzmann Machines. Cognitive Science, 9, 147–169.Google Scholar
  2. Allen, R. (1987). Several studies on natural language and back propagation. In Institute of Electrical and Electronics Engineers First International Conference on Neural Networks. San Diego, Vol. II, 335–342.Google Scholar
  3. Anderson, J.A., Silverstein, J.W., Ritz, S.A. & Jones, R.S. (1977) Distinctive Features, Categorical Perception, and Probability Learning: Some Applications of a Neural Model. Psychological Review, 84, 413–451.Google Scholar
  4. Ashby, W.R. (1960). Design for a Brain: The origin of adaptive behaviour. (2nd edn). John Wiley & Sons, New York.Google Scholar
  5. Barto, A.G., Anderson, C.W. & Sutton, R.S. (1982) Synthesis of Nonlinear Control Surfaces by a layered Associative Search Network. Biological Cybernetics, 43, 175–185.Google Scholar
  6. Barto, A.G. (1985) Learning by statistical cooperation of self-interested neuron-like computing elements. Human Neurobiology, 4, 229–256.Google Scholar
  7. Cater, J.P. (1987). Successfully using peak learning rates of 10 (and greater) in back-propagation networks with the heuristic learning algorithm. In Institute of Electrical and Electronics Engineers First International Conference on Neural Networks. San Diego, II-645-652.Google Scholar
  8. Cottrell, G.W. (1985a) Connectionist Parsing. In Proceedings of the Seventh Annual Conference of the Cognitive Science Society. Irvine, CA, USA.Google Scholar
  9. Cottrell, G.W. (1985b) A Connectionist Approach to Word-Sense Disambiguation. TR154 Computer Science Department, University of Rochester, Rochester, USA.Google Scholar
  10. Crutchfield, J.P., Farmer, J.D., Packard, N.H. & Shaw, R.S. (1986) Chaos. Scientific American, 255, 46–57.Google Scholar
  11. Cun, Y. Le (1985) A Learning Scheme for Asymmetric Threshold Networks. In Proceedings of Cognitiva 85. Paris, 599–604.Google Scholar
  12. Dahl, E.D. (1987) Accelerated learning using the generalized delta rule. In Institute of Electrical and Electronics Engineers First International Conference on Neural Networks. San Diego, II-523-530.Google Scholar
  13. Dreyfus, H.L. & Dreyfus, S.E. (1988) Making a Mind versus Modeling the Brain: Artificial Intelligence Again at the Crossroads. Daedalus, 117, 15–43.Google Scholar
  14. Dyer, M.G.,(1983). In Depth Understanding. MIT Press, Cambridge, USA.Google Scholar
  15. Dyer, M.G., Flowers, M. & Wang, Y.A. (1988) Weight Matrix = Pattern of Activation: Encoding Semantic Networks as Distributed Representations in DUAL, a PDP architecture. UCLA-AI-88–5, Los Angeles: Artificial Intelligence Laboratory, UCLA.Google Scholar
  16. Elman, J. & Stork, D. (1987) Session on Speech Recognition and Synthesis. In Institute of Electrical and Electronics Engineers First International Conference on Neural Networks. San Diego, IV-381-504.Google Scholar
  17. Elman, J.L. (1988) Finding Structure in Time. Report 8801, San Diego: Center for Research in Language, UCSD.Google Scholar
  18. Fanty, M. (1985) Context-free parsing in Connectionist Networks. TR174, Rochester, N.Y.: Computer Science Department, University of Rochester, USA.Google Scholar
  19. Feldman, J.A. & Ballard, D.H. (1982) Connectionist models and their properties. Cognitive Science, 6, 205–254.Google Scholar
  20. Fodor, J. & Pylyshyn, A. (1988) Connectionism and Cognitive Architecture: A Critical Analysis. Cognition, 28, 3–71.Google Scholar
  21. Gasser, M. & Dyer, M.G. (1988) Sequencing in a Connectionist Model of Language Processing. In Proceedings of the 12th International Conference on Computational Linguistics. Budapest.Google Scholar
  22. Gleick, J. (1987) Chaos: Making a new science. Viking, New York.Google Scholar
  23. Gorman, R.P. & Sejnowski, T.J. (1988) Analysis of hidden units in a layered network trained to classify sonar targets. Neural Networks, 1, 75–90.Google Scholar
  24. Grebogi, C., Ott, E. & Yorke, J.A. (1987) Chaos, Strange Attractors, and Fractal Basin Boundaries in Nonlinear Dynamics, Science, 238, 632–638.Google Scholar
  25. Grossberg, S. (1987) Competitive Learning: From Interactive Activation to Adaptive Resonance. Cognitive Science, 11, 23–63.Google Scholar
  26. Hanson, S.J. & Kegl, J. (1987) PARSNIP: A connectionist network that learns natural language grammar from exposure to natural language sentences. In Proceedings of the Ninth Conference of the Cognitive Science Society. Seattle, 106–119.Google Scholar
  27. Hebb, D.O. (1949) The Organization of Behavior: A Neuropsychological Theory. John Wiley & Sons, New York.Google Scholar
  28. Hinton, G.E. (1986) Learning Distributed Representations of Concepts. In Proceedings of the Eighth Annual Conference of the Cognitive Science Society. Amherst, MA, 1–12.Google Scholar
  29. Hinton, G.E. (1988) Representing part-whole hierarchies in connectionist networks. In Proceedings of the Tenth Annual Conference of the Cognitive Science Society. Montreal, p. 48–54.Google Scholar
  30. Hopfield, J.J. (1982) Neural Networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences USA, 79, 2554–2558.Google Scholar
  31. Hopfield, J.J. & Tank, D.W. (1985) ‘Neural’ computation of decisions in optimization problems. Biological Cybernetics, 52, 141–152.Google Scholar
  32. Huberman, B.A. & Hogg, T. (1987) Phase Transitions in Artificial Intelligence Systems. Artificial Intelligence, 33, 155–172.Google Scholar
  33. Jordan, M.I. (1986) Serial Order: A Parallel Distributed Processing Appraoch. ICS Report 8608, La Jolla: Institute for Cognitive Science, UCSD.Google Scholar
  34. Kawamoto, A.H. (1985) Dynamic Processes in the (Re)Solution of Lexical Ambiguity. Doctoral Dissertation, Providence: Brown University, Department of Psychology, USA.Google Scholar
  35. Kirkpatrick, S., Gelatt, C.D. & Vecchi, M.P. (1983) Optimization by simulated annealing, Science, 220, 671–680.Google Scholar
  36. Klopf, A.H. (1982) The Hedonistic Neuron. Hemisphere Publishing Corporation, Washington, DC.Google Scholar
  37. Kohonen, T. (1977) Associative Memory: A Systems-Theoretical Approach. Springer-Verlag, Berlin.Google Scholar
  38. Kohonen, T., Oja, E. & Lehtio, P. (1981) Storage and Processing of Information in Distributed Associative Memory Systems. In G.E.Hinton & J.A.Anderson, (Eds.), Parallel models of associative memory. Lawrence Erlbaum Associates, Hillsdale, USA.Google Scholar
  39. Kurten, K.E. (1987) Phase transitions in quasirandom neural networks. In Institute of Electrical and Electronics Engineers First International Conference on Neural Networks. San Diego, II-197-204.Google Scholar
  40. Lapedes, A.S. & Farber, R.M. Nonlinear signal processing using several networks: prediction and system modeling. Biological Cybernetics (in press).Google Scholar
  41. Lenat, D.B. (1977) The Ubiquity of Discovery. In Proceedings of the Fifth International Joint Conference on Artifical Intelligence. Cambridge, MA, 1093–1105.Google Scholar
  42. Mandelbrot, B. (1982) The Fractal Geometry of Nature. Freeman, San Francisco.Google Scholar
  43. McClelland, J.L. & Rumelhart, D.E. (1981) An interactive activation model of the effect of context in perception: Part 1. An account of basic findings. Psychology Review, 88, 375–407.Google Scholar
  44. McClelland, J. & Kawamoto, A. (1986) Mechanisms of Sentence Processing: Assigning Roles to Constituents. In J.L.McClelland, D.E.Rumelhart & the PDP research Group, (Eds.), Parallel Distributed Processing: Experiments in the Microstructure of Cognition, Vol. 2. MIT Press, Cambridge, USA.Google Scholar
  45. McCorduck, P. (1979) Machines Who Think. Freeman, San Francisco, CA, USA.Google Scholar
  46. McCulloch, W.S. & Pitts, W. (1943) A logical calculus of the ideas immanent in nervious activity. Bulletin of Mathematical Biophysics, 5, 115–133.Google Scholar
  47. Minsky, M. & Papert, S. (1969) Perceptrons. MIT Press, Cambridge, MA., USA.Google Scholar
  48. Minsky, M. (1986) The Society of Mind. Simon & Schuster, New York.Google Scholar
  49. Norman, D.A. & Rumelhart, D.E. (1975) Explorations in Cognition. W.H. Freeman & Co., San Francisco.Google Scholar
  50. Parker, D.B. (1985) Learning Logic. Technical Report-47, MIT Center for Computational Research in Economics and Management Science, Cambridge, MA, USA.Google Scholar
  51. Pollack, J.B. (1987) Cascaded Back Propagation on Dynamic Connectionist Networks. In Proceedings of the Ninth Conference of the Cognitive Science Society. Seattle, 391–404Google Scholar
  52. Pollack, J.B. (1988) Recursive Auto-Associative Memory: Devising Compositional Distributed Representations. Proceedings of the Tenth Conference of The Cognitive Science Society, Montreal, pp. 33–39.Google Scholar
  53. Rosenblatt, F. (1959) Two theorems of statistical separability in the perceptron. In Mechanization of Thought Processes, Vol. 1. H.M.S.O., London.Google Scholar
  54. Rosenblatt, F. (1962) Principles of Neurodynamics, Spartan, New York.Google Scholar
  55. Rumelhart, W.E., Hinton, G.E. & Williams, R.J. (1986) Learning internal representations by error propagation. In Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Vol. 1 (eds J.L.McClelland, D.E.Rumelhart & the PDP research Group). MIT Press, Cambridge.Google Scholar
  56. Rumelhart, D.E. & McClelland, J.L. (1982) An interactive activation model of the effect of context in perception: Part 2 The contextual enhancement effect and some tests and extensions of the model. Psychology Review, 89, 60–94.Google Scholar
  57. Rumelhart, D.E. & Zipser, D. (1986) Feature Discovery by Competitive Learning. In D.E.Rumelhart, J.L.McClelland & the PDP research Group, (eds.), Parallel Distributed Processing: Experiments in the Microstructure of Cognition, Vol. 1. MIT Press, Cambridge, USA.Google Scholar
  58. Rumelhart, D.E. & McClelland, J.L. (1986) On Learning the Past Tenses of English Verbs. In Parallel Distributed Processing: Experiments in the Microstructure of Cognition, Vol. 2. J.L.McClelland, D.E.Rumelhart & the PDP research Group, (eds.), MIT Press, Cambridge, USA.Google Scholar
  59. Sabbah, D. (1982) A Connectionist Approach to Visual Recognition. TR107: Computer Science Department, University of Rochester, USA.Google Scholar
  60. Sampson, G. (1986) A stochastic approach to parsing. In Proceedings of the 11th International Conference on Computational Linguistics. pp. 151–155. Bonn.Google Scholar
  61. Sejnowski, T.J. & Rosenberg, C.R. (1986) NETtalk: A parallel network that learns to read aloud. JHU/EECS-86/01: The Johns Hopkins University, Electrical Engineering and Computer Science Department.Google Scholar
  62. Selman, B. (1985) Rule-Based Processing in a Connectionist System for Natural Language Understanding. CSRI-168, Toronto, Computer Systems Research Institute, University of Toronto, Canada.Google Scholar
  63. Shastri, L. (1985) Evidential reasoning in semantic networks: A formal theory and its parallel implementation. TR166, Rochester: Computer Science Department, University of Rochester.Google Scholar
  64. Smolensky, P. (1986) Information Processing in Dynamical Systems: Foundations of Harmony Theory. In Parallel Distributed Processing: Experiments in the Microstructure of Cognition, Vol. 1. D.E.Rumelhart, J.L.McClelland & the PDP research Group, (eds.), MIT Press, Cambridge, USA.Google Scholar
  65. Smolensky, P. (1988) On the proper treatment of Connectionism. Behavioural and Brain Sciences, II, no. 1, 1–23.Google Scholar
  66. Stornetta, W.S. & Huberman, B.A. (1987) An Improved three-layer back propagation algorithm. In Institute of Electrical and Electronics Engineers First International Conference on Neural Networks. San Diego, II-637-644.Google Scholar
  67. Touretzky, D.S. & Hinton, G.E. (1985) Symbols among the neurons: details of a connectionist inference architecture. In Proceedings of the Ninth International Joint Conference on Artificial Intelligence. Los Angeles, CA, USA.Google Scholar
  68. Willshaw, D.J. (1981) Holography, Associative Memory, and Inductive Generalization. In G.E.Hinton & J.A.Anderson, (eds.), Parallel models of associative memory. Lawrence Erlbaum Associates, Hillsdale, USA.Google Scholar

Copyright information

© Intellect Ltd. 1989

Authors and Affiliations

  • J. B. Pollack
    • 1
  1. 1.Department of Computer and Information ScienceThe Ohio State UniversityUSA

Personalised recommendations