Turing’s Connectionism

  • Christof Teuscher


In a “little known” paper entitled “Intelligent Machinery,” Turing had already investigated connectionist models as early as 1948. Unfortunately, his work was dismissed by his employer and went unpublished until 1968, 14 years after his death.

This chapter provides an overview on all aspects of Turing’s unorganized machines and on the extensions proposed by the author. Amazingly, Turing also proposed a sort of “genetical search” to organize the networks, an idea that will be illustrated using a toy pattern classification task.


Artificial Neural Network Boolean Function Turing Machine Boolean Network Artificial Life 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    NPL Executive Committee Minutes, 28 September 1948, page 4.Google Scholar
  2. 2.
    I. Aleksander. Random logic nets: Stability and adaptation. International Journal of Man-Machine Studies, 5:115–131, 1973.CrossRefGoogle Scholar
  3. 3.
    I. Aleksander. From Wisard to Magnus: A family of weightless virtual neural machines. In J. Austin, editor, RAM-Based Neural Networks, volume 9 of Progress in Neural Processing. World Scientific, February 1998.Google Scholar
  4. 4.
    I. Aleksander, W. V. Thomas, and P. A. Bowden. WISARD: A radical step forward in image recognition. Sensor Review, 4:120–124, July 1984.CrossRefGoogle Scholar
  5. 5.
    J. T. Allanson. Some properties of randomly connected neural nets. In C. Cherry, editor, Proceedings of the 3 rd London Symposium on Information Theory, pages 303–313, Butterworths, London, 1956.Google Scholar
  6. 6.
    S. I. Amari. Characteristics of randomly connected threshold-element networks and network systems. Proceedings of the IEEE, 59(l):35–47, January 1971.MathSciNetCrossRefGoogle Scholar
  7. 7.
    S. I. Amari. Learning patterns and pattern sequences by self-organizing nets of threshold elements. IEEE Transactions on Computers, C-21(11):1197–1206, November 1972.MathSciNetCrossRefGoogle Scholar
  8. 8.
    W. R. Ashby. An Introduction to Cybernetics. Chapman and Hall, London, 1956.zbMATHGoogle Scholar
  9. 9.
    W. R. Ashby, H. von Forster, and C. C. Walker. Instability of pulse activity in a net with threshold. Nature, 196:561, 1966.CrossRefGoogle Scholar
  10. 10.
    W. R. Ashby. Design for a Brain. Wiley, New York, 1956.Google Scholar
  11. 11.
    W. Bechtel and A. Abrahamsen. Connectionism and the Mind. Parallel Processing, Dynamics, and Evolution of Networks. Blackwell Publishers, Maiden, MA, 1st edition, 1991.Google Scholar
  12. 12.
    E. R. Berlekamp, J. H. Conway, and R. K. Guy. Winning Ways for Your Mathematical Plays. Volume 2: Games in Particular. Academic Press, London, 1982.zbMATHGoogle Scholar
  13. 13.
    R. Brooks. The relationship between matter and life. Nature, 409:409–411, January 18, 2001.CrossRefGoogle Scholar
  14. 14.
    W. H. Calvin. The emergence of intelligence. Scientific American: Exploring Intelligence, 9(4):44–50, 1998.MathSciNetGoogle Scholar
  15. 15.
    P. S. Churchland and T. J. Sejnowski. The Computational Brain. MIT Press, Cambridge, MA, 1992.Google Scholar
  16. 16.
    W. A. Clark and B. G. Farley. Generalisation of pattern recognition in a self-organising system. In Proceedings of the Western Joint Computer Conference, pages 86–91, 1955.Google Scholar
  17. 17.
    B. J. Copeland and D. Proudfoot. On Alan Turing’s anticipation of connec-tionism. Synthese: An International Journal for Epistemology, Methodology and Philosophy of Science, 108:361–377, 1996.MathSciNetzbMATHCrossRefGoogle Scholar
  18. 18.
    B. J. Copeland and D. Proudfoot. Alan Turing’s forgotten ideas in computer science. Scientific American, 280(4):76–81, April 1999.CrossRefGoogle Scholar
  19. 19.
    J. L. Elman, E. A. Bates, M. H. Johnson, A. Karmiloff-Smith, D. Parisi, and K. Plunkett. Rethinking Innateness. A Connectionist Perspective on Development. A Bradford Book, MIT Press, Cambridge, MA; London, UK, 1996.Google Scholar
  20. 20.
    C. R. Evans and A. D. J. Robertson, editors. Cybernetics: Key Papers. University Park Press, Baltimore MD and Manchester, London, 1968.Google Scholar
  21. 21.
    B. G. Farley and W. A. Clark. Simulation of self-organising systems by digital computer. Institute of Radio Engineers Transactions on Information Theory, 4:76–84, 1954.MathSciNetGoogle Scholar
  22. 22.
    R. Gandy, 1995. In interview with B. J. Copeland.Google Scholar
  23. 23.
    R. Gandy. Human versus mechanical intelligence. In Millican and Clark [43], Chap. 7, pages 125–136.Google Scholar
  24. 24.
    H. Gardner. A multiplicity of intelligences. Scientific American: Exploring Intelligence, 9(4): 19–23, 1998.Google Scholar
  25. 25.
    C. Gershenson. Classification of random Boolean networks. In R. K. Standish, M. A. Bedau, and H. A. Abbass, editors, ArtiGcial Life VIII. Proceedings of the Eighth International Conference on ArtiGcial Life, Complex Adaptive Systems Series, pages 1–8. A Bradford Book, MIT Press, Cambridge, MA, 2003.Google Scholar
  26. 26.
    R. O. Grondin, W. Porod, C. M. Loeffler, and D. K. Ferry. Synchronous and asynchronous systems of threshold elements. Biological Cybernetics, 49:1–7, 1983.zbMATHCrossRefGoogle Scholar
  27. 27.
    I. Harvey and T. Bossomaier. Time out of joint: Attractors in asynchronous random Boolean networks. In P. Husbands and I. Harvey, editors, Proceedings of the Fourth European Conference on ArtiGcial Life, pages 67–75. MIT Press, Cambridge, MA, 1997.Google Scholar
  28. 28.
    S. Haykin. Neural Networks: A Comprehensive Foundation. Prentice Hall, New Jersey, 2nd edition, 1999.zbMATHGoogle Scholar
  29. 29.
    D. Hebb. The Organization of Behavior. John Wiley, New York, 1949.Google Scholar
  30. 30.
    A. Hodges. Alan Turing: The Enigma. Walker & Company, New York, 2000.Google Scholar
  31. 31.
    D. C. Ince, editor. Collected Works of A. M. Turing: Mechanical Intelligence. North-Holland, Amsterdam, 1992.Google Scholar
  32. 32.
    S. A. Kauffman. Metabolic stability and epigenesis in randomly connected genetic nets. Journal of Theoretical Biology, 22:437–467, 1968.CrossRefGoogle Scholar
  33. 33.
    S. A. Kauffman. The Origins of Order: Self-Organization and Selection in Evolution. Oxford University Press, New York; Oxford, 1993.Google Scholar
  34. 34.
    S. A. Kauffman. At Home in the Universe. Oxford University Press, New York; Oxford, 1995.Google Scholar
  35. 35.
    S. A. Kauffman. Investigations. Oxford University Press, New York; Oxford, 2000.Google Scholar
  36. 36.
    T. S. Kuhn. The Structure of Scientific Revolutions. University of Chicago Press, Chicago, 1962.Google Scholar
  37. 37.
    D. Martland. Auto-associative pattern storage using synchronous Boolean networks. In Proceedings of the First IEEE International Conference on Neural Networks, volume III, pages 355–366, San Diego, CA, 1987.Google Scholar
  38. 38.
    D. Martland. Behaviour of autonomous, (synchronous) Boolean networks. In Proceedings of the First IEEE International Conference on Neural Networks, volume II, pages 243–250, San Diego, CA, 1987.Google Scholar
  39. 39.
    D. Martland. Configurable Boolean networks. In L. Personnaz and G. Dreyfus, editors, Neural Networks from Models to Applications. Proceedings of the First European Conference on Neural Networks, nEuro’88, volume III, pages 355–366, IDSET, Paris, June 1988.Google Scholar
  40. 40.
    D. Martland. Dynamic behavior of Boolean networks. In I. Aleksander, editor, Neural Computing Architectures: The Design of Brain-Like Machines, Chap. 11, pages 217–235. North Oxford Academic, London, 1989.Google Scholar
  41. 41.
    W. S. McCulloch and W. H. Pitts. A logical calculus of the ideas immanent in neural nets. Bulletin of Mathematical Biophysics, 5:115–133, 1943.MathSciNetzbMATHCrossRefGoogle Scholar
  42. 42.
    D. F. Miller, P. M. Todd, and S. U. Hegde. Designing neural networks using genetic algorithms. In J. D. Schaffer, editor, Third International Conference on Genetic Algorithms, pages 379–384. Morgan Kaufmann, 1989.Google Scholar
  43. 43.
    P. Millican and A. Clark, editors. The Legacy of Alan Turing: Machines and Thought, volume 1. Oxford University Press, New York, 1996.Google Scholar
  44. 44.
    M. L. Minsky. Computation: Finite and Infinite Machines. Prentice-Hall, Englewood Cliffs, NJ, 1967.zbMATHGoogle Scholar
  45. 45.
    M. L. Minsky and S. Papert. Perceptron: An Introduction to Computational Geometry. MIT Press, Cambridge, MA, 1972.Google Scholar
  46. 46.
    S. Nolfl and D. Floreano. Learning and evolution. Autonomous Robots, 7(1):89–113, 1999.CrossRefGoogle Scholar
  47. 47.
    S. Nolfi and D. Floreano. Evolutionary Robotics: The Biology, Intelligence, and Technology of Self-Organizing Machines. MIT Press, Cambridge, MA, 2000.Google Scholar
  48. 48.
    R. C. O’Reilly and Y. Munakata. Computational Explorations in Cognitive Neuroscience. A Bradford Book, MIT Press, Cambridge, MA, 2000.Google Scholar
  49. 49.
    E. A. Di Paolo. Searching for rhythms in asynchronous Boolean networks. In M. A. Bedau, J. S. McCaskill, N. H. Packard, and S. Rassmussen, editors, Proceedings of the Seventh International Conference on Artificial Life, Reed College, Portland, OR, August 1–6, 2000. A Bradford Book, MIT Press, Cambridge, MA; London, UK.Google Scholar
  50. 50.
    E. A. Di Paolo. Rhythmic and non-rhythmic attractors in asynchronous random Boolean networks. Biosystems, 59(3): 185–195, 2001.CrossRefGoogle Scholar
  51. 51.
    P. Rendell. Turing universality of the Game of Life. In A. Adamatzky, editor, Collision-Based Computing, pages 513–539. Springer-Verlag, London, 2002.CrossRefGoogle Scholar
  52. 52.
    J.-P. Rennard. Implementation of logical functions in the Game of Life. In A. Adamatzky, editor, Collision-Based Computing, pages 491–512. Springer-Verlag, London, 2002.CrossRefGoogle Scholar
  53. 53.
    M. C. Roco and W. S. Bainbridge, editors. Converging Technologies for Improving Human Performance: Nanotechnology, Biotechnology, Information Technology and Cognitive Science. World Technology Evaluation Center (WTEC), Arlington, Virginia, June 2002. NSF/DOC-sponsored report.Google Scholar
  54. 54.
    R. Rojas. Neural Networks: A Systematic Introduction. Springer-Verlag, Berlin, 1996.Google Scholar
  55. 55.
    F. Rosenblatt. The perceptron: A probabilistic model for information storage and organization in the brain. Psychological Review, 65:386–408, 1958.MathSciNetCrossRefGoogle Scholar
  56. 56.
    F. Rosenblatt. Principles of Neurodynamics. Spartan, Washington, DC, 1961.Google Scholar
  57. 57.
    L. I. Rozonoer. Random logical nets I. Automation and Remote Control, 5:773–781, 1969. Translation of Avtomatika i Telemekhanika.MathSciNetGoogle Scholar
  58. 58.
    H. A. Simon. Machine as mind. In Millican and Clark [43], Chap. 5, pages 81–102.Google Scholar
  59. 59.
    D. R. Smith and C. H. Davidson. Maintained activity in neural nets. Journal of the ACM, 9:268–279, 1962.MathSciNetzbMATHCrossRefGoogle Scholar
  60. 60.
    M. Stannett. Hypercomputational models. In this volume, 2003.Google Scholar
  61. 61.
    A. H. Taub, editor. Collected Works of John von Neumann, volume 5. Pergamon Press, Oxford, 1961.Google Scholar
  62. 62.
    C. Teuscher. Turing’s Connectionism. An Investigation of Neural Network Architectures: Springer-Verlag, London, 2002.Google Scholar
  63. 63.
    C. Teuscher and M. Sipper. Hypercomputation: Hype or computation? Communications of the ACM, 45(8):23–24, August 2002.CrossRefGoogle Scholar
  64. 64.
    A. M. Turing. On computable numbers, with an application to the Entscheidungsproblem. Proceedings of the London Mathematical Society, series 2, 42:230–265, 1936–37. Corrections in Proceedings of the London Mathematical Society, 43:544–546, 1937.Google Scholar
  65. 65.
    A. M. Turing. Computing machinery and intelligence. Mind, 59(236):433–460, 1950.MathSciNetCrossRefGoogle Scholar
  66. 66.
    A. M. Turing. Intelligent machinery. In B. Meltzer and D. Michie, editors, Machine Intelligence, volume 5, pages 3–23. Edinburgh University Press, EdinburghS, 1969.Google Scholar
  67. 67.
    A. M. Turing. Intelligent machinery. In Ince [31], pages 107–127.Google Scholar
  68. 68.
    J. von Neumann. Probabilistic logic and the synthesis of reliable organisms from unreliable components. In Automata Studies, pages 43–98. Princeton University Press, Princeton, NJ, 1956.Google Scholar
  69. 69.
    J. von Neumann. The NORC and problems in high speed computing. In Taub [61], pages 238–247.Google Scholar
  70. 70.
    J. F. Wakerly. Digital Design: Principles Sz Practices. Prentice Hall International, New Jersey, 3rd edition, 2000.Google Scholar
  71. 71.
    C. C. Walker. Behavior of a class of complex systems: the effect of system size on properties of terminal cycles. Journal of Cybernetics, 1(4) :55–67, 1971.CrossRefGoogle Scholar
  72. 72.
    C. C. Walker. At tract or dominance patterns in sparsely connected Boolean nets. Physica D, 45:441–451, 1990.zbMATHCrossRefGoogle Scholar
  73. 73.
    G. Weisbuch. Dynamique des systemes complexes: Une introduction aux reseaux d’automates. Inter Editions, France, 1989.Google Scholar
  74. 74.
    G. Weisbuch. Complex Systems Dynamics: An Introduction to Automata Networks, volume 2 of Lecture Notes, Santa Fe Institute, Studies in the Sciences of Complexity. Addison-Wesley, Redwood City, CA, 1991.Google Scholar
  75. 75.
    S. Wermter, J. Austin, and D. Willshaw, editors. Emergent Neural Computation Architectures Based on Neuroscience. Towards Neuroscience-Inspired Computing. Number 2036 in Lecture Notes in Artificial Intelligence (LNAI). Springer-Verlag, Berlin, Heidelberg, 2001.Google Scholar
  76. 76.
    S. Wermter, S. Austin, D. Willshaw, and M. Elshaw. Towards novel neuroscience-inspired computing. In Wermter et al. [75], pages 1–19.Google Scholar
  77. 77.
    N. Wiener. Cybernetics or Control and Communication in the Animal and Machine. MIT Press, Cambridge, MA, 1948.Google Scholar
  78. 78.
    X. Yao. Evolving artificial neural networks. Proceedings of the IEEE, 87(9): 1423–1447, September 1999.CrossRefGoogle Scholar
  79. 79.
    M. J. Zigmond, F. E. Bloom, S. C. Landis, J. L. Roberts, and L. R. Squire, editors. Fundamental Neuroscience. Academic Press, San Diego, CA, 1999.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Christof Teuscher
    • 1
  1. 1.Lausanne, Logic Systems LaboratorySwiss Federal Institute of TechnologySwitzerland

Personalised recommendations