Advertisement

Stochastic Computing Systems

Chapter
Part of the Advances in Information Systems Science book series (AISS)

Abstract

The invention of the steam engine in the late eighteenth century made it possible to replace the muscle-power of men and animals by the motive power of machines. The invention of the stored-program digital computer during the second world war made it possible to replace the lower-level mental processes of man, such as arithmetic computation and information storage, by electronic data-processing in machines. We are now coming to the stage where it is reasonable to contemplate replacing some of the higher mental processes of man, such as the ability to recognize patterns and to learn, with similar capabilities in machines. However, we lack the “steam engine” or “digital computer” which will provide the necessary technology for learning and pattern recognition by machines.

Keywords

Logic Level Computing Element Clock Pulse Input Line Analog Multiplier 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in to check access.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    W. J. Poppelbaum and C. Afuso, Noise Computer, University of Illinois, Dept. Computer Science, Quarterly Technical Progress Reports (April 1965-January 1966).Google Scholar
  2. 2.
    W. J. Poppelbaum, C. Afuso, and J. W. Esch, Stochastic Computing Elements and Systems, in “Proc. American Federation of Information Processing Societies, Fall Joint Computer Conference,” Vol. 31, pp. 635–644, Books, Inc., New York (1967).Google Scholar
  3. 3.
    S. T. Ribeiro, Random Pulse Machines, IEEE Trans. Electronic Computers EC-16 261–276 (June 1967).Google Scholar
  4. 4.
    B. R. Gaines, Stochastic Computing, in “Proc. American Federation of Information Processing Societies, Spring Joint Computer Conference,” Vol. 30, pp. 149–156, Books, Inc., New York (1967).Google Scholar
  5. 5.
    B. R. Gaines, Techniques of Identification with the Stochastic Computer, in “Proc. International Federation of Automatic Control Symposium on Identification, Progue, June 1967.”Google Scholar
  6. 6.
    B. R. Gaines, Stochastic Computer Thrives on Noise, Electronics 40(14), 72–79 (July 10 1967).Google Scholar
  7. 7.
    B. R. Gaines, Stochastic Computing, in “Encyclopaedia of Information, Linguistics and Control,” pp. 766–781, Pergamon Press, New York and London (1968).Google Scholar
  8. 7a.
    J. T. Tou, Engineering Principles of Pattern Recognition, in “Advances in Information Systems Science,” Vol. 1, J. T. Tou, ed., Plenum Press, New York (1969).CrossRefGoogle Scholar
  9. 7b.
    K. S. Fu, Learning Control Systems, in “Advances in Information Systems Science,” Vol. 1, J. T. Tou, ed., Plenum Press, New York (1969).Google Scholar
  10. 8.
    J. H. Andreae, Learning Machines—A Unified View, in “Encyclopaedia of Information, Linguistics and Control,” Pergamon Press, New York and London (1968).Google Scholar
  11. 9.
    E. Feigenbaum and J. Feldman, “Computers and Thought,” McGraw-Hill Book Co., New York (1964).Google Scholar
  12. 10.
    L. Uhr, “Pattern Recognition,” John Wiley and Sons, New York (1966).Google Scholar
  13. 11.
    J. T. Tou and R. H. Wilcox, “Computer and Information Sciences,” Spartan Books, Washington, D.C. (1964).Google Scholar
  14. 12.
    W. J. Karplus and J. A. Howard, A Transfluxor Analog Memory Using Frequency Modulation, in “American Federation of Information Processing Societies,” Vol. 26, Part 1, pp. 673–683, Books, Inc., New York.Google Scholar
  15. 13.
    W. J. Karplus, A Hybrid Computer Technique for Treating Nonlinear Partial Differential Equations, IEEE Trans. Electronic Computers EC-13 (5) 597–605 (1964).Google Scholar
  16. 14.
    B. R. Gaines and J. H. Andreae, A Learning Machine in the Context of the General Control Problem, in “Proc. 3rd International Congress International Federation of Automatic Control, London, 1966,” Institution of Mechanical Engineers, London (1967).Google Scholar
  17. 15.
    W. J. Poppelbaum, Hybrid Graphical Processors, in “Computer Technology,” IEEE Conf. Publ., Vol. 32 (July 1967).Google Scholar
  18. 16.
    T. J. Williams, Process Dynamics, in “Proc. 2nd International Congress International Federation of Automatic Control, Basle, 1963.”Google Scholar
  19. 17.
    G. A. Korn and T. M. Korn, “Electronic Analog and Hybrid Computers,” McGraw-Hill Book Co., New York (1964).Google Scholar
  20. 18.
    J. R. Smith and C. O. Harbourt, An Adaptive Threshold Logic Gate Using Capacitive Analog Weights, IEEE Trans. Electronic Computers EC-17 (1), 78–81 (1968).CrossRefGoogle Scholar
  21. 19.
    H. Schmid, Sequential Analog-Digital Computer (SADC), in “American Federation of Information Processing Societies, Joint Computer Conference,” Vol. 27, Part 1, pp. 915–928, Books, Inc., New York (1965).Google Scholar
  22. 20.
    D. L. Greer, Characterization of the Magnetic Second-Harmonic Memory, IEEE Trans. Electronic Computers EC-17 (6), 551–558 (1968).CrossRefGoogle Scholar
  23. 21.
    B. Widrow, An Adaptive ADALINE Neuron Using Chemical Memistors, ERL Tech. Rep. No. 1553–2, Stanford University (1960).Google Scholar
  24. 22.
    G. Nagy, A Survey of Analog Memory Devices, IEEE Trans. Electronic Computers EC-12 388–393 (August 1963).CrossRefGoogle Scholar
  25. 23.
    S. Larach, “Photoelectronic Materials and Devices,” D. van Nostrand, Englewood Cliffs, New Jersey (1965).Google Scholar
  26. 24.
    D. R. Bosomworth and H. J. Gerritsen, Thick Holograms in Photocromic Materials, Appl. Optics 7(1), 95–98(1968).CrossRefGoogle Scholar
  27. 25.
    G. U. Kaiman, Holographic Graphical Storage in Thick Alkali-Halide Crystals, IEEE Int. Conv. Digest, p. 35(1968).Google Scholar
  28. 26.
    J. K. Hawkins and C. J. Munsey, Parallel Computer Organizations and Mechanizations, IEEE Trans. Electronic Computers EC-12 (3), 251–262 (1963).CrossRefGoogle Scholar
  29. 27.
    D. K. Pollock, C. J. Koester, and J. T. Tippett, “Optical Processing of Information,” Spartan Books, Washington, D.C. (1963).Google Scholar
  30. 28.
    S. J. Mathis, R. E. Wiley, and L. M. Spandorfer, “Microelectronics and Large Systems,” Spartan Books, Washington, D.C. (1965).Google Scholar
  31. 29.
    R. L. Petritz, Technological Foundations and Future Directions of Large-Scale Integrated Electronics, in “American Federation of Information Processing Societies, Fall Joint Computer Conference,” Vol. 29, pp. 65–87, Books, Inc., New York (1966).Google Scholar
  32. 30.
    D. L. Slotnick, W. C. Borck, and R. C. McReynolds, The Solomon Computer— A Preliminary Report, in “Computer Organization,” pp. 66–92, Spartan Books, Washington, DC, (1963).Google Scholar
  33. 31.
    W. A. Clark, S. M. Orstein, M. J. Stuki, A. S. Blum, T. J. Chaney, R. E. Olsen, R. A. Dammhoehler, W. E. Ball, C. E. Moinar, A. Antharvedi, Macromodular Computer Systems, in “American Federation of Information Processing Societies, Spring Joint Computer Conference,” Vol. 30, pp. 355–401, Books, Inc., New York (1967).Google Scholar
  34. 32.
    R. H. Fuller and R. M. Bird, An Associative Parallel Processor with Application to Picture Processing, in “American Federation of Information Processing Societies, Fall Joint Computer Conference,” Vol. 27, Part 1, pp. 105–116, Books, Inc., New York (1965).Google Scholar
  35. 33.
    J. C. Murtha, Highly Parallel Information Processing Systems, in “Advances in Computers,” Vol. 7, pp. 1–116 Academic Press, New York (1966).Google Scholar
  36. 34.
    E. L. Braun, “Digital Computer Design,” Chapter 8, Academic Press, New York (1963).Google Scholar
  37. 35.
    F. V. Mayorov and Y. Chu, “Digital Differential Analysers,” Iliffe Books (1964).Google Scholar
  38. 36.
    W. J. Karplus, Analog and Digital Techniques Combined, in “Computer Control Systems Technology,” pp. 148–155 McGraw-Hill Book Co., New York (1961).Google Scholar
  39. 37.
    Digital Operational Techniques, Computer Design 1963 (November), 12.Google Scholar
  40. 38.
    B. R. Gaines and P. L. Joyce, Phase Computers, in “Proc. 5th International Congress AICA, Lausanne, 1967.”Google Scholar
  41. 39.
    B. R. Gaines, A Modular Programmed DDA for Real-Time Computation, in “Proc. IFIP 68, Edinburgh, 1968.”Google Scholar
  42. 40.
    W. R. Schumann, Method and Apparatus for Averaging a Series of Transients, United States Patent 3, 182, 181 (May 5, 1965).Google Scholar
  43. 41.
    B. P. Th. Veltman and A. van den Bos, The Applicability of the Relay Correlator and Polarity Coincidence Correlator in Automatic Control, in “Proc. 2nd International Congress International Federation of Automatic Control,” Basle, 1963.Google Scholar
  44. 42.
    W. W. Peterson, “Error Correcting Codes,” John Wiley and Sons, New York (1961).Google Scholar
  45. 43.
    W. H. Kautz, “Linear Sequential Switching Circuits,” Holden-Day, San Francisco (1965).Google Scholar
  46. 44.
    S. W. Golomb, “Shift Register Sequences,” Holden-Day, San Francisco (1967).Google Scholar
  47. 45.
    A. Gill, “Finite-State Machines,” McGraw-Hill Book Co., New York (1962).Google Scholar
  48. 46.
    A. Gill, “Linear Sequential Circuits,” McGraw-Hill Book Co., New York (1967).Google Scholar
  49. 47.
    G. A. Korn, “Random-Process Simulation and Measurements,” McGraw-Hill Book Co., New York (1966).Google Scholar
  50. 48.
    P. A. N. Briggs, P. H. Hammond, M. T. G. Hughes, and G. O. Plumb, Correlation Analysis of Process Dynamics Using Pseudo-Random Binary Test Perturbations, in “Advances in Automatic Control,” pp. 37–51, Institution of Mechanical Engineers, London (1965).Google Scholar
  51. 49.
    R. P. Gilson, Some Results of Amplitude Distribution Experiments on Shift-Register Generated Pseudo-Random Noise, IEEE Trans. Electronic Computers EC-15 (6), 926–927 (1966).Google Scholar
  52. 50.
    R. C. White, Experiments with Digital Computer Simulations of Pseudo-Random Noise Generators, IEEE Trans. Electronic Computers EC-16 (3), 355–357 (1967).CrossRefGoogle Scholar
  53. 51.
    R. E. Kaiman, Nonlinear Aspects of Sampled-Data Control Systems, in “Proc. Symp. Nonlinear Circuit Analysis,” pp. 273–313, Polytechnic Institute of Brooklyn, New York (1957).Google Scholar
  54. 52.
    M. Arbib, Tolerance Automata, in “Kybernetika Cislo 3,” pp. 223–233 (1967).Google Scholar
  55. 53.
    W. De Backer and L. Verbeek, Study of Analog, Digital, and Hybrid Computers Using Automata Theory, ICC Bulletin 5, 215–244 (1966).Google Scholar
  56. 54.
    M. Arbib, “Algebraic Theory of Machines, Languages and Semigroups,” Academic Press, New York (1968).Google Scholar
  57. 55.
    K. H. Hofman and P. S. Mostert, “Elements of Compact Semigroups,” Merrill Books, Columbus, Ohio (1966).Google Scholar
  58. 55a.
    J. T. Tou, ed., “Applied Automata Theory,” Academic Press, New York (1968).Google Scholar
  59. 56.
    M. O. Rabin, Probabilistic Automata, Information and Control 6, 230–245 (1963).CrossRefGoogle Scholar
  60. 57.
    A. Paz, Some Aspects of Probabilistic Automata, Information and Control 9, 26–60 (1966).CrossRefGoogle Scholar
  61. 58.
    W. Feller, “An Introduction to Probability Theory and Its Applications,” John Wiley and Sons, New York (1957).Google Scholar
  62. 59.
    L. Takacs, “Stochastic Processes,” Methuen, London (1960).Google Scholar
  63. 60.
    R. McNaughton, The Theory of Automata—A Survey, in “Advances in Computers,” Vol. 2, pp. 379–421, Academic Press, New York (1961).Google Scholar
  64. 61.
    J. Fox, “Mathematical Theory of Automata,” Polytechnic Institute of Brooklyn Press, New York (1963).Google Scholar
  65. 62.
    T. L. Booth, “Sequential Machines and Automata Theory and Sons, New York” John Wiley (1967).Google Scholar
  66. 63.
    G. C. Bacon, Minimal-State Stochastic Finite State Systems, IEEE Trans. CT-11, 307–308 (1964).Google Scholar
  67. 64.
    G. H. Ott, Reconsider the Minimization Problem for Stochastic Finite State Systems, in “Proc. IEEE 7th Symp. Switching and Automata Theory,” pp. 251–261 (1966).Google Scholar
  68. 65.
    G. C. Bacon, The Decomposition of Stochastic Automata, Information and Control 7, 320–339 (1964).CrossRefGoogle Scholar
  69. 66.
    C. V. Page, Equivalences between Probabilistic Machines, Tech. Rep. 03105–41-T, University of Michigan (1965).Google Scholar
  70. 67.
    J. W. Carlyle, State-Calculable Stochastic Sequential Machines, Equivalences and Events, in “Proc. IEEE Symp. Switching Circuit Theory and Logical Design,” pp. 258–263 (1965).Google Scholar
  71. 68.
    J. von Neumann, Probabilistic Logics and the Synthesis of Reliable Organisms from Unreliable Components, in “Automata Studies,” Princeton University Press, Princeton, New Jersey (1956).Google Scholar
  72. 69.
    S. Winograd and J. D. Cowan, “Reliable Computation in the Presence of Noise,” MIT Press, Cambridge, Mass. (1963).Google Scholar
  73. 70.
    J. D. Cowan, The Problem of Organismic Reliability, in “Cybernetics of the Nervous System,” Elsevier Amsterdam (1965).Google Scholar
  74. 71.
    F. Rosenblatt, A Model for Experiental Storage in Neural Networks, in “Computer and Information Sciences,” (J. T. Tou and R. H. Wilcox, eds.), pp. 16–66, Spartan Books Washington, D.C. (1964).Google Scholar
  75. 72.
    B. Widrow and F. W. Smith, Pattern-Recognizing Control Systems, in “Computer and Information Sciences,” (J. T. Tou and R. H. Wilcox, eds.), pp. 288–317, Spartan Books Washington, D.C. (1964).Google Scholar
  76. 73.
    K. Steinbuch and U. A. W. Piske, Learning Matrices and Their Applications, IEEE Trans. Electronic Computers EC-12 (5), 846–862 (1963).CrossRefGoogle Scholar
  77. 74.
    N. J. Nilsson, “Learning Machines,” McGraw-Hill Book Co., New York (1965).Google Scholar
  78. 75.
    A. Novikoff, On Convergence Proofs for Perceptrons, in “Automata Theory,” pp. 615–622, Polytechnic Institute of Brooklyn Press, New York (1963).Google Scholar
  79. 76.
    G. L. Clapper, Machine Looks, Listens, Learns, Electronics 1967 (October 30), 91–102.Google Scholar
  80. 77.
    W. C. Ridgeway, An Adaptive Logic System with Generalizing Properties, Report SEL-62–040, Stanford Electronics Laboratories (April 1962).Google Scholar
  81. 78.
    I. Aleksander and R. C. Albrow, Adaptive Logic Circuits, Computer J. 11 (1), 65–71 (1968).CrossRefGoogle Scholar
  82. 79.
    S. Muroga, Lower Bounds on the Number of Threshold Functions and a Maximum Weight, IEEE Trans. Electronic Computers EC-14 136–148 (1965).CrossRefGoogle Scholar
  83. 80.
    Sze-Tsen Hu, “Threshold Logic,” University of California Press, Berkeley, Calif. (1965).Google Scholar
  84. 81.
    M. Cuenod and A. P. Sage, Comparison of Some Methods Used for Process Identification, in “Proc. International Federation of Automatic Control Symposium on Identification in Automatic Control Systems, Prague, June 1967.”Google Scholar
  85. 82.
    P. Eykhoff, Process Parameter and State Estimation, in “Proc. International Federation of Automatic Control Symposium on Identification in Automatic Control Systems, Prague, June 1967.”Google Scholar
  86. 83.
    M. J. Levin, Optimum Estimation of Impulse Response in the Presence of Noise, IRE Natl. Com. Record 4, 147–182 (1959).Google Scholar
  87. 84.
    W. W. Lichtenberger, A Technique of Linear System Identification Using Correlating Filters, IRE Trans. Automatic Control AC-6 (2), 183–199 (1961).CrossRefGoogle Scholar
  88. 85.
    Y. Kaya and S. Yamamura, A Self-Adaptive System with a Variable Parameter PID Control, AIEE Trans. Appln. Ind. 58, 378–386 (1962).Google Scholar
  89. 86.
    M. Margolis and C. T. Leondes, A Model-Referenced Parameter Tracking Technique for Adaptive Control Systems, IEEE Trans. Appln. Ind. 68, 241–261 (1963).Google Scholar
  90. 87.
    B. G. Madden, Simultaneous Determination of System Parameters from Transient Response, IEEE Trans. Appln. Ind. 69, 327–331 (1963).CrossRefGoogle Scholar
  91. 88.
    P. C. Young, The Determination of the Parameters of a Dynamic Process, Radio and Electronic Engineer 29 (6), 345–361 (1965).CrossRefGoogle Scholar
  92. 89.
    D. D. Donalson and F. H. Kishi, Review of Adaptive Control System Theories and Techniques, in “Modern Control Systems Theory,” McGraw-Hill Book Co., New York (1965).Google Scholar
  93. 90.
    P. E. K. Donaldson, Error Decorrelation Studies on a Human Operator Performing a Balancing Task, Med. Electron. Biol. Eng. 2, 393–410 (1964).CrossRefGoogle Scholar
  94. 91.
    T. F. Potts, G. N. Ornstein, and A. B. Clymer, The Automatic Determination of Human and Other System Parameters, in “Proc. Western Joint Computer Conference, Los Angeles, pp. 645–660 (1961).Google Scholar
  95. 92.
    C. L. Becker and J. V. Wait, Two-Level Correlation on an Analog Computer, IRE Trans. Electronic Computers EC-10 (4), 752–758 (1961).CrossRefGoogle Scholar
  96. 93.
    Y. Lundh, A Digital Integrator for On-line Signal Processing, IEEE Trans. Electronic Computers EC-12 (1), 26–28 (1963).CrossRefGoogle Scholar
  97. 94.
    P. Jespers, P. T. Chu, and A. Fettweis, A New Method to Compute Correlation Functions, in “International Symp. Information Theory, Brussels, 1962.”Google Scholar
  98. 95.
    B. Widrow, Statistical Analysis of Amplitude-Quantized Sampled Data Systems, IRE Trans. CT-3 (1956).Google Scholar
  99. 96.
    D. G. Watts, “A General Theory of Amplitude Quantization with Applications to Correlation Determination,” IEE Monograph No. 481 M (November 1961).Google Scholar
  100. 97.
    A. K. Nath and A. K. Mathalanabis, Method of Statistical Linearization, Proc. IEE 113 (12), 2081–2086 (1966).Google Scholar
  101. 98.
    A. A. Pervozanskii, “Random Processes in Nonlinear Control,” Academic Press, New York (1965).Google Scholar
  102. 99.
    O. I. Elgard, High-Frequency Signal Injection : A Means of Changing the Transfer Characteristics of Nonlinear Elements, WESCON (1962).Google Scholar
  103. 100.
    G. R. Cooper, R. L. Gassner, and C. D. McGillem, in “Proc. 21st National Electronics Conference,” pp. 656–661, National Electronics Conference, Chicago, Illinois (1965).Google Scholar
  104. 101.
    L. J. Savage, “The Foundations of Statistical Inference,” Methuen, London (1962).Google Scholar
  105. 102.
    S. Watanabe, Information-Theoretical Aspects of Inductive and Deductive Inference, IBM J. Res. Dev. 14 (2), 208–231 (1960).CrossRefGoogle Scholar
  106. 103.
    D. Middleton, “Topics in Communication Theory,” McGraw-Hill Book Co., New York (1965).Google Scholar
  107. 104.
    M. E. Maron, Design Principles for an Intelligent Machine, IRE Trans. Information Theory IT-8 (5), 179–185 (1962).CrossRefGoogle Scholar
  108. 105.
    M. Minsky and O. G. Selfridge, Learning in Random Nets, in “Information Theory,” pp. 335–347, Butterworths, London (1961).Google Scholar
  109. 106.
    K. S. Fu, A Learning Control System Using Statistical Decision Processes, in “Proc. International Federation of Automatic Control Symp. Self-Adaptive Control Systems, Teddington, England, 1965,” Plenum Press, New York (1965).Google Scholar
  110. 107.
    D. R. Hill, An ESOTerIC Approach to Automatic Speech Recognition, Int. J. Man-Machine Studies 1(1) (January 1969).Google Scholar
  111. 108.
    S. Kullback, “Information Theory and Statistics,” John Wiley and Sons, New York (1959).Google Scholar
  112. 109.
    B. G. Farley and W. A. Clark, Simulation of Self-Organizing Systems by Digital Computer, IRE Trans. Information Theory IT-4, 76–84 (September 1954).Google Scholar
  113. 110.
    R. L. Beurle, Properties of a Mass of Cells Capable of Regenerating Pulses, Trans. Roy. Soc. (London) B240, 55–94 (August 1956).CrossRefGoogle Scholar
  114. 111.
    L. D. Harmon, W. A. Bergeijk, and J. Levinson, Studies with Artificial Neurons, Kybernetik 1, 89–117 (December 1961).CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 1969

Authors and Affiliations

  1. 1.Department of Electrical Engineering ScienceUniversity of EssexColchester, EssexUK

Personalised recommendations

Cite chapter

Buy options