Skip to main content

Classical Neural Networks

  • Chapter
  • First Online:

Abstract

Dring the last few decades, neural networks have moved from theory to offering solutions for industrial and commercial problems. Many people are interested in neural networks from many different perspectives. Engineers use them to build practical systems to solve industrial problems. For example, neural networks can be used for the control of industrial processes. There are many publications that relate to the neural network theme. Every year, tens or even hundreds of international conferences, symposiums, congresses, and seminars take place in the world. As an introduction to this theme we can recommend the books of Robert Hecht-Nielsen [1], Teuvo Kohonen [2], and Philip Wasserman [3], and a more advanced book that is oriented on the applications of neural networks and is edited by A. Browne [4]. In this book it is assumed that the reader has some previous knowledge of neural networks and an understanding of their basic mechanisms. In this section we want to present a very short introduction to neural networks and to highlight the most important moments in neural network development.

This is a preview of subscription content, log in via an institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Hecht-Nielsen, R., Neurocomputing. Addison-Wesley, 1991, pp. 433.

    Google Scholar 

  2. Kohonen, T., Self-Organization and Associative Memory, Springer-Verlag, Berlin, 1984.

    MATH  Google Scholar 

  3. Wasserman, Philip D., Neural Computing: Theory and practice, Van Nostrand Reinhold, Inc., 1989.

    Google Scholar 

  4. Ed., Browne, A., Neural Network Analysis, Architectures and Applications. Institute of Physics Publishing, 1997, pp. 264.

    MATH  Google Scholar 

  5. McCulloch, W. S., and Pitts W. A., Logical Calculus of the Ideas Immanent in Nervous Activity. Bulletin of Math. Biophysics, 5, 1943, pp. 115–133.

    MathSciNet  MATH  Google Scholar 

  6. Hebb, D. O., The Organization of Behavior. A Neuropsychological Theory. Wiley, New York, 1949, pp. 335.

    Google Scholar 

  7. Rosenblatt, F., “The perceptron: A Probabilistic Model for Information Storage and Organization in the Brain,” Psychol. Rev., 65, pp. 386–408, 1958.

    Article  MathSciNet  Google Scholar 

  8. Rosenblatt, F., “Principles of Neurodynamics,” Spartan Books, Washington, D.C., 1962, pp. 616.

    Google Scholar 

  9. Diggavi, S. N., Shynk, J. J., Engel, I., Bershad, N. J., “On the Convergence Behavior of Rosenblatt's Perceptron Learning Algorithm”, 1992 Conference Record of the 26 Asilomar Conference on Signals, Systems and Computers, Vol. 2, 1992, pp. 852–856.

    Article  Google Scholar 

  10. Diggavi, S. N., Shynk, J. J., Bershad, N. J., Convergence Models for Rosenblatt's Perceptron Learning Algorithm, IEEE Transactions on Signal Processing, Vol. 43, Issue 7, July 1995, pp. 1696–1702.

    Article  Google Scholar 

  11. Nagy, G., Neural Networks – Then and Now, IEEE Transactions on Neural Networks, Vol. 2, Issue 2, March 1991, pp. 316–318.

    Article  Google Scholar 

  12. Kussul, E., Baidyk, T., Kasatkina, L., Lukovich, V., Rosenblatt Perceptrons for Handwritten Digit Recognition. Proceedings of International Joint Conference on Neural Networks “IJCNN'01”, Washington, D.C., USA, July 15–19, 2001, pp. 1516–1521.

    Google Scholar 

  13. Kussul, E. M., Baidyk, T. N., Improved Method of Handwritten Digit Recognition Tested on MNIST Database, 15th International Conference on Vision Interface VI'2002, Calgary, Canada, 2002, pp. 192–197.

    Google Scholar 

  14. Baidyk, T. N., Kussul, E. M., Application of Neural Classifier for Flat Image Recognition in the Process of Microdevice Assembly, International Joint Conference on Neural Networks, IEEE, USA, Hawaii, May 12–17, 2002 Vol. 1, pp.160-164.

    Google Scholar 

  15. Makeyev, O., “Neural interpolator for image recognition in the process of microdevice assembly”. In IEEE IJCNN'2003, Portland, Oregon, USA, July 20–24, 2003, Vol. 3, pp. 2222–2226.

    Google Scholar 

  16. Baidyk, T., Kussul, E., Makeyev, O., “Image recognition system for microdevice assembly”, Twenty-First IASTED International Multi-Conference on APPLIED INFORMATICS AI2003. Innsbruck, Austria, pp. 243–248.

    Google Scholar 

  17. Baidyk, T., Kussul, E., Makeyev, O., Caballero, A., Ruiz, L., Carrera, G., Velasco, G., Flat image recognition in the process of microdevice assembly. Pattern Recognition Letters, 2004, Vol. 25/1, pp. 107–118.

    Article  Google Scholar 

  18. Toledo, G. K., Kussul, E., Baidyk, T., Neural classifier LIRA for recognition of micro work pieces and their positions in the processes of microassembly and micromanufacturing. Proceedings of the 7 All-Ukrainian International Conference, 11–15 October 2004, Kiev, Ukraine, p. 17–20.

    Google Scholar 

  19. Kussul, E., Baidyk, T., Improved Method of Handwritten Digit Recognition Tested on MNIST Database. Image and Vision Computing, 2004, Vol. 22/12, pp. 971–981,

    Article  Google Scholar 

  20. Bottou, L., Cortes, C., Denker, J.S., Drucker, H., Guyon, L., Jackel, L.D., LeCun, J., Muller, U.A., Sackinger, E., Simard, P., Vapnik, V., “Comparison of Classifier Methods: A Case Study in Handwritten Digit Recognition”, Proceedings of 12th IAPR International Conference on Pattern Recognition, 1994, Vol. 2, pp. 77–82.

    Google Scholar 

  21. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P., “Gradient-based Learning Applied to Document Recognition”, Proceedings of the IEEE, Vol. 86, No. 11, November 1998, pp. 2278–2344.

    Article  Google Scholar 

  22. Widrow, B., Hoff, M., Adaptive switching circuits, 1960 IRE WESCON Convention Record, New York, 1960, 96–104.

    Google Scholar 

  23. Widrow, B., Generalization and information storage in networks of ADALINE neurons. In: Self-Organizing Systems, Ed. Yovitts G., Spartan Books, Washington DC, 1962.

    Google Scholar 

  24. Minsky, M., and Papert, S., Perceptrons. An Introduction to Computational Geometry. The MIT Press. 1969.

    MATH  Google Scholar 

  25. Amosov, N., Modeling of thinking and the mind. New York: Spartan Books, 1967, pp. 192.

    Google Scholar 

  26. Amosov, N., Kussul, E., Fomenko, V., Transport Robot with Network Control System. The 4th International Joint Conference on Artificial Intelligence. Tbilisi, Georgia, 1975, pp. 727–730.

    Google Scholar 

  27. Amosov, N., Kasatkin, A., Kasatkina, L., Active Semantic Networks in Robots with Independent Control. The 4th International Joint Conference on Artificial Intelligence. Tbilisi, Georgia, 1975, pp. 722–726.

    Google Scholar 

  28. Ivahnenko, A. G., Self-organizing systems with positive feedback loops. IEEE Transactions on Automatic Control, Vol. 8, Issue 3, July 1963, pp. 247–254.

    Article  MathSciNet  Google Scholar 

  29. Kohonen, T., Correlation matrix memories. IEEE Trans. Computers, C-21 (4), 1972, pp. 353–359.

    MATH  Google Scholar 

  30. Fukushima, K., Visual feature extraction by a multilayered network of analog threshold elements. IEEE Trans. Systems. Sci.&Cyber., SSC-5 (4), 1969, pp. 322-333.

    Article  MathSciNet  Google Scholar 

  31. Hopfield, J., Neural networks and physical systems with emergent collective computational abilities. Proc. Nat. Acad. Sci., USA, Vol. 79, 1982, pp. 2554–2558.

    Article  MathSciNet  Google Scholar 

  32. Kussul, E. M., Baidyk, T. N., Design of a Neural-Like Network Architecture for Recognition of Object Shapes in Images. Soviet Journal of Automation and Information Sciences (formerly Soviet Automatic Control), Vol. 23, No. 5, 1990, pp. 53–59.

    MATH  Google Scholar 

  33. Artykutsa, S. Y., Baidyk, T. N., Kussul, E. M., Rachkovskij, D. A., Textura Recognition with Neurocomputers. Preprint 91-8, Institute of Cibernetics of Academy of Science of Ukraine, 1991, pp. 20 (in Russian).

    Google Scholar 

  34. Kussul, E. M., Rachkovskij, D. A., Baidyk, T. N., Associative-Projective Neural Networks: Architecture, Implementation, Applications. Proc. of the Fourth Intern. Conf. "Neural Networks & their Applications", EC2 Publishing, Nimes, France, Nov. 4–8, 1991, pp. 463–476.

    Google Scholar 

  35. Kussul, E. M., Rachkovskij, D. A., Baidyk, T. N., On image texture recognition by associative-projective neurocomputer. Proc. of the ANNIE'91 conference. "Intelligent engineering systems through artificial neural networks", Ed. by C.H.Dagli, S. Kumara and Y.C.Shin, ASME Press, 1991, pp. 453–458.

    Google Scholar 

  36. Amosov, N., Baidyk, T., Goltsev, A., Kasatkin, A., Kasatkina, L., Kussul, E., Rachkovskij, D., Neurocomputers and Intelligent Robots. Ed. by N. Amosov, Kiev, Naukova Dumka, 1991, pp. 272 (in Russian).

    Google Scholar 

  37. Baidyk, T. N., and Kussul, E. M., Application of Genetic Algorithms to Optimization of Neuronet Recognition Devices, Cybernetics and System Analysis. Vol. 35, No. 5, 1999, pp. 700–707.

    Article  MATH  Google Scholar 

  38. Hopfield, J., Neural Networks and Physical Systems with Emergent Collective Computational Abilities. Proc. Natl. Acad. Sci., 79, 1982, pp. 2554–2558.

    Article  MathSciNet  Google Scholar 

  39. Hopfield, J., Neurons with Grated Response Have Collective Computational Properties Like Those of Two-State Neurons. Proc. Natl. Acad. Sci., 81, 1984, pp. 3088–3092.

    Article  Google Scholar 

  40. Grossberg, S., Studies of Mind and Brain: Neural Principles of Learning, Perception, Development, Cognition, and Motor Control. Reidel Press, Boston, 1982.

    MATH  Google Scholar 

  41. Carpenter, G., Grossberg, S., ART2: Self-organization of Stable Category Recognition Codes for Analog Input Patterns. Applied Optics, 26 (23), December 1987, pp. 4919–4930.

    Article  Google Scholar 

  42. Kohonen, T., Self-organization and Associative Memory. Springer-Verlag, Berlin, 1984.

    MATH  Google Scholar 

  43. Fukushima, K., Neocognitron: A Hierarchical Neural Network Capable of Visual Pattern Recognition. Neural Networks, 1, 1988, pp. 119–130.

    Article  Google Scholar 

  44. Rumelhart, D. E., Hinton, G. E., Williams, R., Learning Representations by Backpropagating Errors. Nature, 1986, 323, pp. 533–536.

    Article  Google Scholar 

  45. Rumelhart, D., McClelland J. Parallel Distributed Processing: Explorations in the Microstructure of Cognition, I & II, MIT Press, Cambridge MA, 1986.

    Google Scholar 

  46. Werbos, P., Beyond regression: New tools for prediction and analysis in the behavioral sciences. Doctoral Dissertation, Appl. Math., Harvard University, November 1974.

    Google Scholar 

  47. Werbos, P., Approximate Dynamic Programming for Real-Time Control and Neural Modeling. Chapter 13 in Handbook of Intelligent Control: Neural, Fuzzy and Adaptive Approaches (White & Sofge, eds.), Van Nostrand Reinhold, New York, NY, 1992.

    Google Scholar 

  48. Werbos, P., New Directions in ACDs: Key to Intelligent Control an Understanding the brain. Proceedings of the International Joint Conference on Neural Networks, Vol. 3, 2000, pp. 61–66.

    Google Scholar 

  49. Salcedo-Sanz, S., Santiago-Mozos, R., Bousoño-Calzón, C., A Hybrid Hopfield Network-Simulated Annealing Approach for Frequency Assignment in Satellite Communications Systems. IEEE Transactions on Systems, Man, and Cybernetics. Part B, Vol. 34, No. 2, April 2004, pp. 1108–1116.

    Article  Google Scholar 

  50. Pu Han, Peng Guo, Dong-Feng Wang. The Research of GPS Based on Hopfield Network and its Application in Unit Load System. Proceedings of the Second Conference on Machine Learning and Cybernetics, Xi'an, 2–5 November 2003, pp. 1226–1230.

    Google Scholar 

  51. Xiaowei Sheng, Minghu Jiang. A Information Retrieval System Based on Automatic Query Expansion and Hopfield Network. Proceedings of the IEEE International Neural Networks and Signal Processing, Nanjing, China, December 14–17, 2003, pp. 1624–1627.

    Google Scholar 

  52. Carpenter, G. A., and Grossberg, S., A massively parallel architecture for a self-organizing neural pattern recognition machine. Computer Vision, Graphics and Image Processing, 37, 1987, pp. 54–115.

    Article  Google Scholar 

  53. Chiung-Yao Fang, Sei-Wang Chen, Chiou-Shann Fuh. Automatic Change Detection of Driving Environments in a Vision-Based Driver Assistance System. IEEE Transactions on Neural Networks, Vol. 14, No. 3, May 2003, pp. 646–657.

    Article  Google Scholar 

  54. Mulder, S. A., Wunsch, D. C., Using Adaptive Resonance Theory and Local Optimization to Divide and Conquer Large Scale Traveling Salesman Problems. Proceedings of the International Joint Conference on Neural Networks, IJCNN?03, 20–24 July 2003, Vol. 2, pp. 1408–1411.

    Google Scholar 

  55. Yu-Hu Cheng, Jian-Qiang Yi, Dong-Bin Zhao. Sensor'Based Navigation System Using a Modified AER-2 Network and Fuzzy Controller for an Indoor Mobile Robot. Proceedings of the Second International Conference on Machine Learning and Cybernetics, Xi'an, 2–5 November 2003, pp. 1374–1379.

    Google Scholar 

  56. Sugisaka, M., Dai, F., The Application of ART Neural Network to Image Processing for Controlling a Mobile Vehicle. Proceedings of the IEEE International Symposium on Industrial Electronics, ISIE 2001, 12–16 June 2001, Vol. 3, pp. 1951–1955.

    Google Scholar 

  57. Siew Chan Woo, Chee Peng Lim, R Osman. Development of a Speaker Recognition System using Wavelets and Artificial Neural Networks. Proceedings of the International Symposium on Intelligent Multimedia, Video and Speech Processing, Hong Kong, 2–4 May 2001, pp. 413–416.

    Google Scholar 

  58. Morra, L., Lamberti, F., Demartini, C., A Neural Network Approach to Unsupervised Segmentation of Single-Channel MR Images. Proceedings of the 1st International IEEE EMBS Conference on Neural Engineering, Capri Island, Italy, March 20–22, 2003, pp. 515–518.

    Google Scholar 

  59. Postolache, O., Girao, P., Pereira, S., Ramos, H., Wireless Water Quality Monitoring System Based on Field Point Technology and Kohonen Maps. Proceedings of IEEE Canadian Conference on Electrical and Computer Engineering, Montreal, May 4–7, Vol. 3, 2003, pp. 1872–1876.

    Google Scholar 

  60. Li Pan, Hong Zheng, Saeid Nahavandi. The Application of Rough Set and Kohonen Network to Feature Selection for Object Extraction. Proceedings of the Second In ternational Conference on Machine Learning and Cybernetics, Xi'an, 2–5 November 2003, pp. 1185–1189.

    Google Scholar 

  61. Sudha, N., An ASIC Implementation of Kohonen's Map based Color Image Compression. Proceedings of the 17th IEEE International Conference on VLSI Design (VLSID'04), 2004, pp. 4.

    Google Scholar 

  62. Hulsen, H., Garnica, St., Fatikow, S., Extended Kohonen Networks for the Pose Control of Microrobots in a Nanohandling Station. Proceedings of the 2003 IEEE International Symposium on Intelligent Control, Houston, Texas, Oct. 5-8, 2003, pp. 116–121.

    Google Scholar 

  63. Hubel, D. H., and Wiesel, T. N., Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. J. Physiology (Lond.), 1962, Vol. 106, No. 1, pp. 106–154.

    Google Scholar 

  64. Hubel, D. H., and Wiesel, T. N., Receptive fields and functional architecture in two nonstriate visual areas (18 and 19) of the cat. J. Neurophysiology, 1965, Vol. 28, No. 2, pp. 229–289.

    Google Scholar 

  65. Watanabe, A., Andoh, M., Chujo, N., Harata, Y., Neocognitron Capableof Position Detection and Vehicle Recognition. Proceedings of the International Joint Conference on Neural Networks, IJCNN?99, 10–16 July 1999, Vol. 5, pp. 3170–3173.

    Google Scholar 

  66. Fukushima, K., Neocognitron of a new version: handwritten digit recognition. Proceedings of the International Joint Conference on Neural Networks, IJCNN?01, 15–19 July 2001, Vol. 2, pp. 1498–1503.

    Google Scholar 

  67. San Kan Lee, Pau-choo Chung, Chein-I Chang, Chien-Shun Lo, Tain Lee, Giu-Cheng Hsu, Chin-Wen Yang. A Shape Cognitron Neural Network for Breast Cancer Detection. Proceedings of the International Joint Conference on Neural Networks, IJCNN?02, 12–17 May 2002, Vol. 1 pp. 822–827.

    Google Scholar 

  68. Fukushima, K., Neural Network Models for Vision. Proceedings of the International Joint Conference on Neural Networks, IJCNN?03, 20–24 July 2003, pp. 2625–2630.

    Google Scholar 

  69. Bintu G. Vasudevan, Bhawani S. Gohil, Vijay K. Agarwal, Backpropagation Neural-Network-Based Retrieval of Atmospheric Water Vapor and Cloud Liquid Water From IRS-P4 MSMR, IEEE Transactions on Geoscience and Remote Sensing, Vol. 42, N 5, May 2004, pp. 985–990.

    Article  Google Scholar 

  70. Hu Y. H. and Hwang J.-N., Eds., Handbook of Neural Network Signal Processing, Boca Raton, FL: CRC, 2002.

    Google Scholar 

  71. Betker, A. L., Moussavi, Z., Szturm, T., On Classification of Simulated Sensory Deficiency, Canadian Conference on Electrical and Computer Engineering, Vol. 2, 2–5 May 2004, pp. 1211–1214.

    Google Scholar 

  72. Helena Grip, Fredrik Ohberg, Urban Wiklund, Ylva Sterner, J. Stefan Karlsson, Bjorn Gerdle, Classification of Neck Movement Patterns Related to Whiplash-Associated Disorders Using Neural Networks, IEEE Transactions on Information Technology in Biomedicine, Vol. 7, No. 4, December 2003, pp. 412–418.

    Article  Google Scholar 

  73. Jiebo Luo, Stephen P. Etz, A Physical Model-Based Approach to Detecting Sky in Photographic Images, IEEE Transactions on Image Processing, Vol. 11, No. 3, March 2002, pp. 201–212.

    Article  Google Scholar 

  74. Dennis S.S. Lee, Brian J. Lightgow, Rob E.Morrison, New Fault Diagnosis of Circuit Breakers, IEEE Transactions on Power Delivery, Vol. 18, No. 2, April 2003, pp. 454–459.

    Article  Google Scholar 

  75. Werbos, P., Neurocontrol and Supervised Learning: An Overview and Evaluation, Ed. by David A. White, Donald A. Sofge, In: Handbook on Intelligent Control: Neural, Fuzzy, and Adaptive Approaches, Part 3, 1992, Van Nostrand Reinhold, pp. 65–89.

    Google Scholar 

  76. Gaurav Saini, S.N. Balakrishnan, Adaptive Critic Based Neurocontroller for Autolanding of Aircrafts, Proceedings of the American Control Conference, Albuquerque, New Mexico, USA, June 1997, pp. 1081–1085.

    Google Scholar 

  77. Dongchen Han, S.N. Balakrishnan, State-Constrained Agile Missile Control with Adaptive-Critic-Based Neural networks, IEEE Transactions on Control Systems Technology, Vol. 10, No. 4, July 2002, pp. 481–489.

    Article  Google Scholar 

  78. Venayagamoorthy, G. K., Harley, R. G., Wunsch, D. C., Implementation of Adaptive Critic-Based Neurocontrollers for Turbogenerators in a Multimachine Power System, IEEE Transactions on Neural Networks, Vol. 14, Issue 5, Sept. 2003, pp. 1047– 1064.

    Article  Google Scholar 

  79. Xiaoqun Liao, Ernst L. Hall, Beyond Adaptive Critic - Creative Learning for Intelligent Autonomous Mobile Robots, Proceedings of ANNIE 2002, St. Louis, Missouri, USA, November 2002, pp. 1–6.

    Google Scholar 

  80. Kussul, E. M., Baidyk, T. N., Lukovitch, V. V., Rachkovskij, D. A., Adaptive high performance classifier based on random threshold neurons. Proc. of Twelfth European Meeting on Cybernetics and Systems Research (EMCSR-94), Austria, Vienna, April 5–8, 1994. In: R. Trappl, ed., Cybernetics and Systems 94, World Scientific Publishing Co. Pte. Ltd, Singapore, pp. 1687–1695.

    Google Scholar 

  81. Kussul, E. M., Baidyk, T. N., Neural Random Threshold Classifier in OCR Application, Proceedings of the Second All-Ukrainian Intern. Conf. UkrOBRAZ'94, Kyjiv, Ukraine, December 20–24, 1994, pp. 154–157.

    Google Scholar 

  82. Baidyk, T., Kussul, E., Makeyev, O., Texture Recognition with Random Subspace Neural Classifier, WSEAS Transactions on Circuits and Sysytems, Issue 4, Vol. 4, April 2005, pp. 319–325.

    Google Scholar 

  83. Baidyk, T., and Kussul, E., Neural Network Based Vision System for Micro Workpieces Manufacturing. WSEAS Transactions on Systems, Issue 2, Volume 3, April 2004, pp. 483–488.

    Google Scholar 

  84. Kussul, E., Baidyk, T., Wunsch, D., Makeyev, O., Martín, A., Permutation coding technique for image recognition systems. IEEE Transactions on Neural Networks, Vol. 17/6, November 2006, pp. 1566–1579.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Ernst Kussul .

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Kussul, E., Baidyk, T., Wunsch, D.C. (2010). Classical Neural Networks. In: Neural Networks and Micromechanics. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-02535-8_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-02535-8_2

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-02534-1

  • Online ISBN: 978-3-642-02535-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics