Skip to main content
SpringerLink
Log in
Book cover

Neural Networks and Statistical Learning pp 1–19Cite as

Introduction

  • Chapter
  • First Online:

  • 4456 Accesses

Abstract

This chapter gives a brief introduction to the history of neural networks and machine learning. The concepts related to neurons, neural networks, and neural network processors are also described. This chapter concludes with an outline of the book.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   159.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Ackley, D. H., Hinton, G. E., & Sejnowski, T. J. (1985). A learning algorithm for Boltzmann machines. Cognitive Science, 9, 147–169.

    Article  Google Scholar 

  2. Adhikari, S. P., Yang, C., Kim, H., & Chua, L. O. (2012). Memristor bridge synapse-based neural network and its learning. IEEE Transactions on Neural Networks and Learning Systems, 23(9), 1426–1435.

    Article  Google Scholar 

  3. Back, A. D., & Trappenberg, T. P. (2001). Selecting inputs for modeling using normalized higher order statistics and independent component analysis. IEEE Transactions on Neural Networks, 12(3), 612–617.

    Article  Google Scholar 

  4. Battiti, R. (1994). Using mutual information for selecting features in supervised neural net learning. IEEE Transactions on Neural Networks, 5(4), 537–550.

    Article  Google Scholar 

  5. Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007). Greedy layer-wise training of deep networks. In B. Schlkopf, J. Platt, & T. Hofmann (Eds.), Advances in neural information processing systems (Vol. 19, pp. 153–160). Cambridge, MA: MIT Press.

    Google Scholar 

  6. Bishop, C. M. (1995). Neural networks for pattern recognition. New York: Oxford Press.

    MATH  Google Scholar 

  7. Broomhead, D. S., & Lowe, D. (1988). Multivariable functional interpolation and adaptive networks. Complex Systems, 2, 321–355.

    MathSciNet  MATH  Google Scholar 

  8. Candes, E. J., & Recht, B. (2009). Exact matrix completion via convex optimization. Foundations of Computational Mathematics, 9(6), 717–772.

    Article  MathSciNet  Google Scholar 

  9. Candes, E. J., Romberg, J. K., & Tao, T. (2006). Stable signal recovery from incomplete and inaccurate measurements. Communications on Pure and Applied Mathematics, 59(8), 1207–1223.

    Article  MathSciNet  Google Scholar 

  10. Carpenter, G. A., & Grossberg, S. (1987). A massively parallel architecture for a self-organizing neural pattern recognition machine. Computer Vision, Graphics, and Image Processing, 37, 54–115.

    Article  Google Scholar 

  11. Chua, L. O., & Yang, L. (1988). Cellular neural network: I. Theory; II. Applications. IEEE Transactions on Circuits and Systems, 35, 1257–1290.

    Article  MathSciNet  Google Scholar 

  12. Comon, P. (1994). Independent component analysis—A new concept? Signal Processing, 36(3), 287–314.

    Article  Google Scholar 

  13. Donoho, D. L. (2006). Compressed sensing. IEEE Transactions on Information Theory, 52(4), 1289–1306.

    Article  MathSciNet  Google Scholar 

  14. Du, K.-L., & Swamy, M. N. S. (2006). Neural networks in a softcomputing framework. London: Springer.

    MATH  Google Scholar 

  15. Eccles, J. (1976). From electrical to chemical transmission in the central nervous system. Notes and Records of the Royal Society of London, 30(2), 219–230.

    Article  Google Scholar 

  16. Estevez, P. A., Tesmer, M., Perez, C. A., & Zurada, J. M. (2009). Normalized mutual information feature selection. IEEE Transactions on Neural Networks, 20(2), 189–201.

    Article  Google Scholar 

  17. FitzHugh, R. (1961). Impulses and physiological states in theoretical models of nerve membrane. Biophysical Journal, 1, 445–466.

    Article  Google Scholar 

  18. Friedman, J. H., & Tukey, J. W. (1974). A projection pursuit algorithm for exploratory data analysis. IEEE Transactions on Computers, 23(9), 881–889.

    Article  Google Scholar 

  19. Fukushima, K. (1975). Cognition: A self-organizing multulayered neural network. Biological Cybernetics, 20, 121–136.

    Article  Google Scholar 

  20. Fukushima, K. (1980). Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biological Cybernetics, 36, 193–202.

    Article  Google Scholar 

  21. Fukushima, K. (2011). Increasing robustness against background noise: Visual pattern recognition by a neocognitron. Neural Networks, 24(7), 767–778.

    Article  Google Scholar 

  22. Grossberg, S. (1972). Neural expectation: Cerebellar and retinal analogues of cells fired by unlearnable and learnable pattern classes. Kybernetik, 10, 49–57.

    Article  Google Scholar 

  23. Grossberg, S. (1976). Adaptive pattern classification and universal recording: I. Parallel development and coding of neural feature detectors; II. Feedback, expectation, olfaction, and illusions. Biological Cybernetics, 23, 121–134 & 187–202.

    Google Scholar 

  24. Hebb, D. O. (1949). The organization of behavior. New York: Wiley.

    Google Scholar 

  25. Hinton, G. E., & Salakhutdinov, R. R. (2006). Reducing the dimensionality of data with neural networks. Science, 313(5786), 504–507.

    Article  MathSciNet  Google Scholar 

  26. Hodgkin, A. L., & Huxley, A. F. (1952). A quantitative description of ion currents and its applications to conductance and excitation in nerve membranes. Journal of Physiology, 117, 500–544.

    Article  Google Scholar 

  27. Holland, J. (1975). Adaptation in natural and artificial systems. Ann Arbor, MI: University of Michigan Press.

    Google Scholar 

  28. Hopfield, J. J. (1982). Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences of the United States of America, 79, 2554–2558.

    Article  MathSciNet  Google Scholar 

  29. Izhikevich, E. M. (2003). Simple model of spiking neurons. IEEE Transactions on Neural Networks, 14(6), 1569–1572.

    Article  MathSciNet  Google Scholar 

  30. Kasabov, N. (2009). Integrative connectionist learning systems inspired by nature: Current models, future trends and challenges. Natural Computing, 8, 199–218.

    Article  MathSciNet  Google Scholar 

  31. Kirkpatrick, S., Gelatt, C. D, Jr., & Vecchi, M. P. (1983). Optimization by simulated annealing. Science, 220, 671–680.

    Article  MathSciNet  Google Scholar 

  32. Kohavi, R., & John, G. H. (1997). Wrappers for feature subset selection. Artificial Intelligence, 97, 273–324.

    Article  Google Scholar 

  33. Kohonen, T. (1972). Correlation matrix memories. IEEE Transactions on Computers, 21, 353–359.

    Article  Google Scholar 

  34. Kohonen, T. (1982). Self-organized formation of topologically correct feature maps. Biological Cybernetics, 43, 59–69.

    Article  MathSciNet  Google Scholar 

  35. Kosko, B. (1987). Adaptive bidirectional associative memories. Applied Optics, 26, 4947–4960.

    Article  Google Scholar 

  36. Leiva-Murillo, J. M., & Artes-Rodriguez, A. (2007). Maximization of mutual information for supervised linear feature extraction. IEEE Transactions on Neural Networks, 18(5), 1433–1441.

    Article  Google Scholar 

  37. Lippman, R. P. (1987). An introduction to computing with neural nets. IEEE ASSP Magazine, 4(2), 4–22.

    Article  MathSciNet  Google Scholar 

  38. Markram, H., Lubke, J., Frotscher, M., & Sakmann, B. (1997). Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs. Science, 275(5297), 213–215.

    Article  Google Scholar 

  39. McCulloch, W. S., & Pitts, W. (1943). A logical calculus of the ideas immanent in nervous activity. Bulletin of Mathematical Biology, 5, 115–133.

    MathSciNet  MATH  Google Scholar 

  40. Minsky, M. L., & Papert, S. (1969). Perceptrons. Cambridge, MA: MIT Press.

    MATH  Google Scholar 

  41. Oja, E. (1982). A simplified neuron model as a principal component analyzer. Journal of Mathematical Biology, 15, 267–273.

    Article  MathSciNet  Google Scholar 

  42. Parisien, C., Anderson, C. H., & Eliasmith, C. (2008). Solving the problem of negative synaptic weights in cortical models. Neural Computation, 20(6), 1473–1494.

    Article  MathSciNet  Google Scholar 

  43. Pearl, J. (1988). Probabilistic reasoning in intelligent systems: Networks of plausible inference. San Mateo, CA: Morgan Kaufmann.

    MATH  Google Scholar 

  44. Picone, J. (1993). Signal modeling techniques in speech recognition. Proceedings of the IEEE, 81(9), 1215–1247.

    Article  Google Scholar 

  45. Rosenblatt, R. (1962). Principles of neurodynamics. New York: Spartan Books.

    MATH  Google Scholar 

  46. Rubner, J., & Tavan, P. (1989). A self-organizing network for principal-component analysis. Europhysics Letters, 10, 693–698.

    Article  Google Scholar 

  47. Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning internal representations by error propagation. In D. E. Rumelhart & J. L. McClelland (Eds.), Parallel distributed processing: Explorations in the microstructure of cognition, 1: Foundation (pp. 318–362). Cambridge, MA: MIT Press.

    Google Scholar 

  48. Schwefel, H. P. (1981). Numerical optimization of computer models. Chichester: Wiley.

    MATH  Google Scholar 

  49. Tripp, B., & Eliasmith, C. (2016). Function approximation in inhibitory networks. Neural Networks, 77, 95–106.

    Article  Google Scholar 

  50. Tuckwell, H. C. (1988). Introduction to theoretical neurobiology. Cambridge, UK: Cambridge University Press.

    Book  Google Scholar 

  51. Vapnik, V. N. (1998). Statistical learning theory. New York: Wiley.

    MATH  Google Scholar 

  52. von der Malsburg, C. (1973). Self-organizing of orientation sensitive cells in the striata cortex. Kybernetik, 14, 85–100.

    Article  Google Scholar 

  53. Werbos, P. J. (1974). Beyond regressions: New tools for prediction and analysis in the behavioral sciences. Ph.D. thesis, Harvard University, Cambridge, MA.

    Google Scholar 

  54. Widrow, B., & Hoff, M. E. (1960). Adaptive switching circuits. Convention Record of IRE Eastern Electronic Show & Convention (WESCON1960), 4, 96–104.

    Google Scholar 

  55. Zadeh, L. A. (1965). Fuzzy sets. Information and Control, 8, 338–353.

    Article  MathSciNet  Google Scholar 

  56. Zilles, K. (1990). Cortex. In G. Pixinos (Ed.), The human nervous system. New York: Academic Press.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, Concordia University, Montreal, QC, Canada

    Ke-Lin Du & M. N. S. Swamy

  2. Xonlink Inc., Hangzhou, China

    Ke-Lin Du

Authors
  1. Ke-Lin Du
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. N. S. Swamy
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ke-Lin Du .

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer-Verlag London Ltd., part of Springer Nature

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Du, KL., Swamy, M.N.S. (2019). Introduction. In: Neural Networks and Statistical Learning. Springer, London. https://doi.org/10.1007/978-1-4471-7452-3_1

Download citation

Publish with us

Policies and ethics

Search

Navigation