Abstract
Dring the last few decades, neural networks have moved from theory to offering solutions for industrial and commercial problems. Many people are interested in neural networks from many different perspectives. Engineers use them to build practical systems to solve industrial problems. For example, neural networks can be used for the control of industrial processes. There are many publications that relate to the neural network theme. Every year, tens or even hundreds of international conferences, symposiums, congresses, and seminars take place in the world. As an introduction to this theme we can recommend the books of Robert Hecht-Nielsen [1], Teuvo Kohonen [2], and Philip Wasserman [3], and a more advanced book that is oriented on the applications of neural networks and is edited by A. Browne [4]. In this book it is assumed that the reader has some previous knowledge of neural networks and an understanding of their basic mechanisms. In this section we want to present a very short introduction to neural networks and to highlight the most important moments in neural network development.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Hecht-Nielsen, R., Neurocomputing. Addison-Wesley, 1991, pp. 433.
Kohonen, T., Self-Organization and Associative Memory, Springer-Verlag, Berlin, 1984.
Wasserman, Philip D., Neural Computing: Theory and practice, Van Nostrand Reinhold, Inc., 1989.
Ed., Browne, A., Neural Network Analysis, Architectures and Applications. Institute of Physics Publishing, 1997, pp. 264.
McCulloch, W. S., and Pitts W. A., Logical Calculus of the Ideas Immanent in Nervous Activity. Bulletin of Math. Biophysics, 5, 1943, pp. 115–133.
Hebb, D. O., The Organization of Behavior. A Neuropsychological Theory. Wiley, New York, 1949, pp. 335.
Rosenblatt, F., “The perceptron: A Probabilistic Model for Information Storage and Organization in the Brain,” Psychol. Rev., 65, pp. 386–408, 1958.
Rosenblatt, F., “Principles of Neurodynamics,” Spartan Books, Washington, D.C., 1962, pp. 616.
Diggavi, S. N., Shynk, J. J., Engel, I., Bershad, N. J., “On the Convergence Behavior of Rosenblatt's Perceptron Learning Algorithm”, 1992 Conference Record of the 26 Asilomar Conference on Signals, Systems and Computers, Vol. 2, 1992, pp. 852–856.
Diggavi, S. N., Shynk, J. J., Bershad, N. J., Convergence Models for Rosenblatt's Perceptron Learning Algorithm, IEEE Transactions on Signal Processing, Vol. 43, Issue 7, July 1995, pp. 1696–1702.
Nagy, G., Neural Networks – Then and Now, IEEE Transactions on Neural Networks, Vol. 2, Issue 2, March 1991, pp. 316–318.
Kussul, E., Baidyk, T., Kasatkina, L., Lukovich, V., Rosenblatt Perceptrons for Handwritten Digit Recognition. Proceedings of International Joint Conference on Neural Networks “IJCNN'01”, Washington, D.C., USA, July 15–19, 2001, pp. 1516–1521.
Kussul, E. M., Baidyk, T. N., Improved Method of Handwritten Digit Recognition Tested on MNIST Database, 15th International Conference on Vision Interface VI'2002, Calgary, Canada, 2002, pp. 192–197.
Baidyk, T. N., Kussul, E. M., Application of Neural Classifier for Flat Image Recognition in the Process of Microdevice Assembly, International Joint Conference on Neural Networks, IEEE, USA, Hawaii, May 12–17, 2002 Vol. 1, pp.160-164.
Makeyev, O., “Neural interpolator for image recognition in the process of microdevice assembly”. In IEEE IJCNN'2003, Portland, Oregon, USA, July 20–24, 2003, Vol. 3, pp. 2222–2226.
Baidyk, T., Kussul, E., Makeyev, O., “Image recognition system for microdevice assembly”, Twenty-First IASTED International Multi-Conference on APPLIED INFORMATICS AI2003. Innsbruck, Austria, pp. 243–248.
Baidyk, T., Kussul, E., Makeyev, O., Caballero, A., Ruiz, L., Carrera, G., Velasco, G., Flat image recognition in the process of microdevice assembly. Pattern Recognition Letters, 2004, Vol. 25/1, pp. 107–118.
Toledo, G. K., Kussul, E., Baidyk, T., Neural classifier LIRA for recognition of micro work pieces and their positions in the processes of microassembly and micromanufacturing. Proceedings of the 7 All-Ukrainian International Conference, 11–15 October 2004, Kiev, Ukraine, p. 17–20.
Kussul, E., Baidyk, T., Improved Method of Handwritten Digit Recognition Tested on MNIST Database. Image and Vision Computing, 2004, Vol. 22/12, pp. 971–981,
Bottou, L., Cortes, C., Denker, J.S., Drucker, H., Guyon, L., Jackel, L.D., LeCun, J., Muller, U.A., Sackinger, E., Simard, P., Vapnik, V., “Comparison of Classifier Methods: A Case Study in Handwritten Digit Recognition”, Proceedings of 12th IAPR International Conference on Pattern Recognition, 1994, Vol. 2, pp. 77–82.
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P., “Gradient-based Learning Applied to Document Recognition”, Proceedings of the IEEE, Vol. 86, No. 11, November 1998, pp. 2278–2344.
Widrow, B., Hoff, M., Adaptive switching circuits, 1960 IRE WESCON Convention Record, New York, 1960, 96–104.
Widrow, B., Generalization and information storage in networks of ADALINE neurons. In: Self-Organizing Systems, Ed. Yovitts G., Spartan Books, Washington DC, 1962.
Minsky, M., and Papert, S., Perceptrons. An Introduction to Computational Geometry. The MIT Press. 1969.
Amosov, N., Modeling of thinking and the mind. New York: Spartan Books, 1967, pp. 192.
Amosov, N., Kussul, E., Fomenko, V., Transport Robot with Network Control System. The 4th International Joint Conference on Artificial Intelligence. Tbilisi, Georgia, 1975, pp. 727–730.
Amosov, N., Kasatkin, A., Kasatkina, L., Active Semantic Networks in Robots with Independent Control. The 4th International Joint Conference on Artificial Intelligence. Tbilisi, Georgia, 1975, pp. 722–726.
Ivahnenko, A. G., Self-organizing systems with positive feedback loops. IEEE Transactions on Automatic Control, Vol. 8, Issue 3, July 1963, pp. 247–254.
Kohonen, T., Correlation matrix memories. IEEE Trans. Computers, C-21 (4), 1972, pp. 353–359.
Fukushima, K., Visual feature extraction by a multilayered network of analog threshold elements. IEEE Trans. Systems. Sci.&Cyber., SSC-5 (4), 1969, pp. 322-333.
Hopfield, J., Neural networks and physical systems with emergent collective computational abilities. Proc. Nat. Acad. Sci., USA, Vol. 79, 1982, pp. 2554–2558.
Kussul, E. M., Baidyk, T. N., Design of a Neural-Like Network Architecture for Recognition of Object Shapes in Images. Soviet Journal of Automation and Information Sciences (formerly Soviet Automatic Control), Vol. 23, No. 5, 1990, pp. 53–59.
Artykutsa, S. Y., Baidyk, T. N., Kussul, E. M., Rachkovskij, D. A., Textura Recognition with Neurocomputers. Preprint 91-8, Institute of Cibernetics of Academy of Science of Ukraine, 1991, pp. 20 (in Russian).
Kussul, E. M., Rachkovskij, D. A., Baidyk, T. N., Associative-Projective Neural Networks: Architecture, Implementation, Applications. Proc. of the Fourth Intern. Conf. "Neural Networks & their Applications", EC2 Publishing, Nimes, France, Nov. 4–8, 1991, pp. 463–476.
Kussul, E. M., Rachkovskij, D. A., Baidyk, T. N., On image texture recognition by associative-projective neurocomputer. Proc. of the ANNIE'91 conference. "Intelligent engineering systems through artificial neural networks", Ed. by C.H.Dagli, S. Kumara and Y.C.Shin, ASME Press, 1991, pp. 453–458.
Amosov, N., Baidyk, T., Goltsev, A., Kasatkin, A., Kasatkina, L., Kussul, E., Rachkovskij, D., Neurocomputers and Intelligent Robots. Ed. by N. Amosov, Kiev, Naukova Dumka, 1991, pp. 272 (in Russian).
Baidyk, T. N., and Kussul, E. M., Application of Genetic Algorithms to Optimization of Neuronet Recognition Devices, Cybernetics and System Analysis. Vol. 35, No. 5, 1999, pp. 700–707.
Hopfield, J., Neural Networks and Physical Systems with Emergent Collective Computational Abilities. Proc. Natl. Acad. Sci., 79, 1982, pp. 2554–2558.
Hopfield, J., Neurons with Grated Response Have Collective Computational Properties Like Those of Two-State Neurons. Proc. Natl. Acad. Sci., 81, 1984, pp. 3088–3092.
Grossberg, S., Studies of Mind and Brain: Neural Principles of Learning, Perception, Development, Cognition, and Motor Control. Reidel Press, Boston, 1982.
Carpenter, G., Grossberg, S., ART2: Self-organization of Stable Category Recognition Codes for Analog Input Patterns. Applied Optics, 26 (23), December 1987, pp. 4919–4930.
Kohonen, T., Self-organization and Associative Memory. Springer-Verlag, Berlin, 1984.
Fukushima, K., Neocognitron: A Hierarchical Neural Network Capable of Visual Pattern Recognition. Neural Networks, 1, 1988, pp. 119–130.
Rumelhart, D. E., Hinton, G. E., Williams, R., Learning Representations by Backpropagating Errors. Nature, 1986, 323, pp. 533–536.
Rumelhart, D., McClelland J. Parallel Distributed Processing: Explorations in the Microstructure of Cognition, I & II, MIT Press, Cambridge MA, 1986.
Werbos, P., Beyond regression: New tools for prediction and analysis in the behavioral sciences. Doctoral Dissertation, Appl. Math., Harvard University, November 1974.
Werbos, P., Approximate Dynamic Programming for Real-Time Control and Neural Modeling. Chapter 13 in Handbook of Intelligent Control: Neural, Fuzzy and Adaptive Approaches (White & Sofge, eds.), Van Nostrand Reinhold, New York, NY, 1992.
Werbos, P., New Directions in ACDs: Key to Intelligent Control an Understanding the brain. Proceedings of the International Joint Conference on Neural Networks, Vol. 3, 2000, pp. 61–66.
Salcedo-Sanz, S., Santiago-Mozos, R., Bousoño-Calzón, C., A Hybrid Hopfield Network-Simulated Annealing Approach for Frequency Assignment in Satellite Communications Systems. IEEE Transactions on Systems, Man, and Cybernetics. Part B, Vol. 34, No. 2, April 2004, pp. 1108–1116.
Pu Han, Peng Guo, Dong-Feng Wang. The Research of GPS Based on Hopfield Network and its Application in Unit Load System. Proceedings of the Second Conference on Machine Learning and Cybernetics, Xi'an, 2–5 November 2003, pp. 1226–1230.
Xiaowei Sheng, Minghu Jiang. A Information Retrieval System Based on Automatic Query Expansion and Hopfield Network. Proceedings of the IEEE International Neural Networks and Signal Processing, Nanjing, China, December 14–17, 2003, pp. 1624–1627.
Carpenter, G. A., and Grossberg, S., A massively parallel architecture for a self-organizing neural pattern recognition machine. Computer Vision, Graphics and Image Processing, 37, 1987, pp. 54–115.
Chiung-Yao Fang, Sei-Wang Chen, Chiou-Shann Fuh. Automatic Change Detection of Driving Environments in a Vision-Based Driver Assistance System. IEEE Transactions on Neural Networks, Vol. 14, No. 3, May 2003, pp. 646–657.
Mulder, S. A., Wunsch, D. C., Using Adaptive Resonance Theory and Local Optimization to Divide and Conquer Large Scale Traveling Salesman Problems. Proceedings of the International Joint Conference on Neural Networks, IJCNN?03, 20–24 July 2003, Vol. 2, pp. 1408–1411.
Yu-Hu Cheng, Jian-Qiang Yi, Dong-Bin Zhao. Sensor'Based Navigation System Using a Modified AER-2 Network and Fuzzy Controller for an Indoor Mobile Robot. Proceedings of the Second International Conference on Machine Learning and Cybernetics, Xi'an, 2–5 November 2003, pp. 1374–1379.
Sugisaka, M., Dai, F., The Application of ART Neural Network to Image Processing for Controlling a Mobile Vehicle. Proceedings of the IEEE International Symposium on Industrial Electronics, ISIE 2001, 12–16 June 2001, Vol. 3, pp. 1951–1955.
Siew Chan Woo, Chee Peng Lim, R Osman. Development of a Speaker Recognition System using Wavelets and Artificial Neural Networks. Proceedings of the International Symposium on Intelligent Multimedia, Video and Speech Processing, Hong Kong, 2–4 May 2001, pp. 413–416.
Morra, L., Lamberti, F., Demartini, C., A Neural Network Approach to Unsupervised Segmentation of Single-Channel MR Images. Proceedings of the 1st International IEEE EMBS Conference on Neural Engineering, Capri Island, Italy, March 20–22, 2003, pp. 515–518.
Postolache, O., Girao, P., Pereira, S., Ramos, H., Wireless Water Quality Monitoring System Based on Field Point Technology and Kohonen Maps. Proceedings of IEEE Canadian Conference on Electrical and Computer Engineering, Montreal, May 4–7, Vol. 3, 2003, pp. 1872–1876.
Li Pan, Hong Zheng, Saeid Nahavandi. The Application of Rough Set and Kohonen Network to Feature Selection for Object Extraction. Proceedings of the Second In ternational Conference on Machine Learning and Cybernetics, Xi'an, 2–5 November 2003, pp. 1185–1189.
Sudha, N., An ASIC Implementation of Kohonen's Map based Color Image Compression. Proceedings of the 17th IEEE International Conference on VLSI Design (VLSID'04), 2004, pp. 4.
Hulsen, H., Garnica, St., Fatikow, S., Extended Kohonen Networks for the Pose Control of Microrobots in a Nanohandling Station. Proceedings of the 2003 IEEE International Symposium on Intelligent Control, Houston, Texas, Oct. 5-8, 2003, pp. 116–121.
Hubel, D. H., and Wiesel, T. N., Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. J. Physiology (Lond.), 1962, Vol. 106, No. 1, pp. 106–154.
Hubel, D. H., and Wiesel, T. N., Receptive fields and functional architecture in two nonstriate visual areas (18 and 19) of the cat. J. Neurophysiology, 1965, Vol. 28, No. 2, pp. 229–289.
Watanabe, A., Andoh, M., Chujo, N., Harata, Y., Neocognitron Capableof Position Detection and Vehicle Recognition. Proceedings of the International Joint Conference on Neural Networks, IJCNN?99, 10–16 July 1999, Vol. 5, pp. 3170–3173.
Fukushima, K., Neocognitron of a new version: handwritten digit recognition. Proceedings of the International Joint Conference on Neural Networks, IJCNN?01, 15–19 July 2001, Vol. 2, pp. 1498–1503.
San Kan Lee, Pau-choo Chung, Chein-I Chang, Chien-Shun Lo, Tain Lee, Giu-Cheng Hsu, Chin-Wen Yang. A Shape Cognitron Neural Network for Breast Cancer Detection. Proceedings of the International Joint Conference on Neural Networks, IJCNN?02, 12–17 May 2002, Vol. 1 pp. 822–827.
Fukushima, K., Neural Network Models for Vision. Proceedings of the International Joint Conference on Neural Networks, IJCNN?03, 20–24 July 2003, pp. 2625–2630.
Bintu G. Vasudevan, Bhawani S. Gohil, Vijay K. Agarwal, Backpropagation Neural-Network-Based Retrieval of Atmospheric Water Vapor and Cloud Liquid Water From IRS-P4 MSMR, IEEE Transactions on Geoscience and Remote Sensing, Vol. 42, N 5, May 2004, pp. 985–990.
Hu Y. H. and Hwang J.-N., Eds., Handbook of Neural Network Signal Processing, Boca Raton, FL: CRC, 2002.
Betker, A. L., Moussavi, Z., Szturm, T., On Classification of Simulated Sensory Deficiency, Canadian Conference on Electrical and Computer Engineering, Vol. 2, 2–5 May 2004, pp. 1211–1214.
Helena Grip, Fredrik Ohberg, Urban Wiklund, Ylva Sterner, J. Stefan Karlsson, Bjorn Gerdle, Classification of Neck Movement Patterns Related to Whiplash-Associated Disorders Using Neural Networks, IEEE Transactions on Information Technology in Biomedicine, Vol. 7, No. 4, December 2003, pp. 412–418.
Jiebo Luo, Stephen P. Etz, A Physical Model-Based Approach to Detecting Sky in Photographic Images, IEEE Transactions on Image Processing, Vol. 11, No. 3, March 2002, pp. 201–212.
Dennis S.S. Lee, Brian J. Lightgow, Rob E.Morrison, New Fault Diagnosis of Circuit Breakers, IEEE Transactions on Power Delivery, Vol. 18, No. 2, April 2003, pp. 454–459.
Werbos, P., Neurocontrol and Supervised Learning: An Overview and Evaluation, Ed. by David A. White, Donald A. Sofge, In: Handbook on Intelligent Control: Neural, Fuzzy, and Adaptive Approaches, Part 3, 1992, Van Nostrand Reinhold, pp. 65–89.
Gaurav Saini, S.N. Balakrishnan, Adaptive Critic Based Neurocontroller for Autolanding of Aircrafts, Proceedings of the American Control Conference, Albuquerque, New Mexico, USA, June 1997, pp. 1081–1085.
Dongchen Han, S.N. Balakrishnan, State-Constrained Agile Missile Control with Adaptive-Critic-Based Neural networks, IEEE Transactions on Control Systems Technology, Vol. 10, No. 4, July 2002, pp. 481–489.
Venayagamoorthy, G. K., Harley, R. G., Wunsch, D. C., Implementation of Adaptive Critic-Based Neurocontrollers for Turbogenerators in a Multimachine Power System, IEEE Transactions on Neural Networks, Vol. 14, Issue 5, Sept. 2003, pp. 1047– 1064.
Xiaoqun Liao, Ernst L. Hall, Beyond Adaptive Critic - Creative Learning for Intelligent Autonomous Mobile Robots, Proceedings of ANNIE 2002, St. Louis, Missouri, USA, November 2002, pp. 1–6.
Kussul, E. M., Baidyk, T. N., Lukovitch, V. V., Rachkovskij, D. A., Adaptive high performance classifier based on random threshold neurons. Proc. of Twelfth European Meeting on Cybernetics and Systems Research (EMCSR-94), Austria, Vienna, April 5–8, 1994. In: R. Trappl, ed., Cybernetics and Systems 94, World Scientific Publishing Co. Pte. Ltd, Singapore, pp. 1687–1695.
Kussul, E. M., Baidyk, T. N., Neural Random Threshold Classifier in OCR Application, Proceedings of the Second All-Ukrainian Intern. Conf. UkrOBRAZ'94, Kyjiv, Ukraine, December 20–24, 1994, pp. 154–157.
Baidyk, T., Kussul, E., Makeyev, O., Texture Recognition with Random Subspace Neural Classifier, WSEAS Transactions on Circuits and Sysytems, Issue 4, Vol. 4, April 2005, pp. 319–325.
Baidyk, T., and Kussul, E., Neural Network Based Vision System for Micro Workpieces Manufacturing. WSEAS Transactions on Systems, Issue 2, Volume 3, April 2004, pp. 483–488.
Kussul, E., Baidyk, T., Wunsch, D., Makeyev, O., Martín, A., Permutation coding technique for image recognition systems. IEEE Transactions on Neural Networks, Vol. 17/6, November 2006, pp. 1566–1579.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Kussul, E., Baidyk, T., Wunsch, D.C. (2010). Classical Neural Networks. In: Neural Networks and Micromechanics. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-02535-8_2
Download citation
DOI: https://doi.org/10.1007/978-3-642-02535-8_2
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-02534-1
Online ISBN: 978-3-642-02535-8
eBook Packages: Computer ScienceComputer Science (R0)