## Abstract

This tutorial article deals with the basics of artificial neural networks (ANN) and their applications in pattern recognition. ANN can be viewed as computing models inspired by the structure and function of the biological neural network. These models are expected to deal with problem solving in a manner different from conventional computing. A distinction is made between pattern and data to emphasize the need for developing pattern processing systems to address pattern recognition tasks. After introducing the basic principles of ANN, some fundamental networks are examined in detail for their ability to solve simple pattern recognition tasks. These fundamental networks together with the principles of ANN will lead to the development of architectures for complex pattern recognition tasks. A few popular architectures are described to illustrate the need to develop an architecture specific to a given pattern recognition problem. Finally several issues that still need to be addressed to solve practical problems using ANN approach are discussed.

This is a preview of subscription content, access via your institution.

## References

Abu-Mostafa Y S, St. Jaques J M 1985 Information capacity of the Hopfield model.

*IEEE Trans. Inf. Theor.*31: 461–464Ackley D M, Hinton G E, Sejnowski T J 1985 A learning algorithm for Boltzmann machines.

*Cogn. Sci.*9: 147–169Ahalt S C, Krishnamurthy A K, Chen P, Melton D E 1990 Competitive learning algorithms for vector quantization.

*Neural Networks*3: 277–290Aleksander I, Morton H 1990

*An introduction to neural computing*(London: Chapman and Hall)Bhat N, McAvoy T 1989 Use of neural nets for dynamic modelling and control of chemical process systems.

*Proc. Am. Autom. Contr. Conf.*, Pittsburgh, PA, pp. 1342–1348Bienenstock E, von der Malsburg Ch 1987 A neural network for the retrieval of superimposed connection patterns.

*Euro. Phys. Lett.*3: 1243–1249Bottou L, Soulie F F, Blanchet P, Lienard J S 1990 Speaker independent isolated digit recognition: multilayer perceptrons vs. dynamic time warping.

*Neural Networks*3: 436–465Carpenter G A 1989 Neural network models for pattern recognition and associative memory.

*Neural networks*2: 138–152Carpenter G A, Grossberg S 1987 art2: Self-organization of stable category recognition codes for analog input patterns.

*Appl. Opt.*26: 4919–4930Carpenter G A, Grossberg S 1988 The art of adaptive pattern recognition by a self-organizing neural network.

*IEEE Comput.*21: 77–88Cohen M, Grossberg S 1983 Absolute stability of global pattern formation and parallel storage by competitive neural networks.

*IEEE Trans. Syst., Man Cybern.*SMC-13: 815–825Cole R, Fanty M, Muthuswamy Y, Gopalakrishna M 1992 Speaker-independent recognition of spoken English letters.

*Proc. Int. Joint Conf. Neural Networks*, San Diego, caCollins E, Ghosh S, Scotfield C L 1988 An application of a multiple neural network learning system to emulation of mortgage underwriting judgements.

*IEEE Int. Conf. Neural Networks*(Piscataway, NJ: IEEE Press) 2: 459–466Cybenko G 1989 Continuous value neural networks with two hidden layers are sufficient.

*Math. Control. Signal Syst.*2: 303–314Desai M S 1990

*Noisy pattern retrieval using associative memories*. MSEE thesis, University of Louisville, KentuckyDeuker J, Schwartz D, Wittner B, Solla S, Howard R, Jackel L, Hopfield J 1987 Large automatic learning, rule extraction, and generalization.

*Complex Syst.*1: 877–922Dotsenko V S 1988 Neural networks: translation-, rotation- and scale invariant pattern recognition

*J. Phys.*A21: L783-L787Dutta S, Shekkar S 1988 Bond rating: a non-conservative application of neural networks.

*IEEE Int. Conf. Neural Networks*(Piscataway, NJ: IEEE Press) 2: 443–450Freeman J A, Skupura D M 1991

*Neural network algorithms, applications and programming techniques*(New York: Addison-Wesley)Fukushima K 1975 Cognitron: A self-organizing multilayer neural network.

*Biol. Cybern.*20: 121–136Fukushima K 1988 A neural network for visual pattern recognition.

*IEEE Comput.*21: 65–75Fukushima K 1991 Handwritten alphanumeric character recognition by the neocognitron.

*IEEE Trans. Neural Networks*2: 355–365Fukushima K, Miyake S 1982 Neocognitron: a new algorithm for pattern recognition tolerant of deformations and shifts in position.

*Pattern Recogn.*15: 455–469Geman S, Geman D 1984 Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images.

*IEEE Trans. Pattern Anal. Machine Intell.*PAMI-6: 721–741Gorman R, Sejnowski T 1988 Learned classification of sonar targets using a massively parallel network.

*IEEE Trans. Acoust. Speech Signal Process.*36: 1135–1140Grossberg S 1969 Some networks that can learn, and reproduce any number of complicated space-time patterns.

*Int. J. Math. Mech.*19: 53–91Grossberg S 1980 How does a brain build a cognitive code?

*Psychol. Rev.*87: 1–51Grossberg S 1982

*Studies of mind & brain*(Boston: Reidel)Grossberg S 1988 Nonlinear neural networks: Principles, mechanisms, and architecture.

*Neural networks*1: 17–61Handleman D H, Lane S H, Gelfand J J 1990 Integrating neural networks and knowledge-based systems for intelligent robotic control.

*IEEE Control Syst. Mag.*10(3): 77–87Hassoun M H 1989 Dynamic heteroassociative memories.

*Neural Networks*2: 275–287Hebb D 1949

*Organization of the behaviour*(New York: Wiley)Hecht-Nielson R 1987 Counterpropagation networks.

*Appl. Opt.*26: 4979–4984Hecht-Nielson R 1990

*Neurocomputing*(Reading, MA: Addison-Wesley)Hertz J, Krogh A, Richard G P 1991

*Introduction to the theory of neural computation*(New York: Addison-Wesley)Hinton G E, Sejnowski T J 1986 Learning and relearning in Boltzmann machines. In

*Parallel distributed processing: Explorations in the microstructure of cognition*(eds) D E Rumelhart, J L McClelland (Cambridge, MA: MIT Press) 1: 282–317Hodgkin A L, Huxley A F 1952 A quantitative description of membrane current and its application to conduction and excitation in nerve.

*J. Physiol.*117: 500–544Hopfield J J 1982 Neural networks and physical systems with emergent collective computational capabilities.

*Proc. Natl. Acad. Sci. (USA)*79: 2554–2558Hopfield J J, Tank D W 1985 Neural computation of decisions in optimization problems.

*Biol. Cybern.*52: 141–154Huang Z, Kuh A 1992 A combined self-organizing feature map and multilayer perceptron for isolated word recognition.

*IEEE Trans. Signal Process.*40: 2651–2657Hush D R, Horne B G 1993 Progress in supervised neural networks.

*IEEE Signal Process. Mag.*10: 8–39Kamp Y, Hasler M 1990

*Recursive neural networks for associative memory*(Chichester: John Wiley & Sons)Kennedy M P, Chau L O 1988 Neural networks for nonlinear programming.

*IEEE Trans. Circuits Syst.*CAS-35: 554–562Kirkpatrick S, Gelatt C D Jr, Vecchi M P 1983 Optimization by simulated annealing.

*Science*220: 671–680Kohonen T 1988 An introduction to neural computing.

*Neural Networks*1: 3–16Kohonen T 1989

*Self-organization and associative memory*(3rd edn) (Berlin: Springer-Verlag)Kohonen T 1990 The self-organizing map.

*Proc. IEEE*78: 1464–1480Konishi M, Otsuka Y, Matsuda K, Tamura N, Fuki A, Kadoguchi K 1990 Application of a neural network to operation guidance in a blast furnace.

*3rd European Seminar on Neural Computing: The Marketplace*, LondonKosko B 1988 Bidirectional associative memories.

*IEEE Trans. Syst., Man Cybern.*18: 49–60Kosko B 1990 Unsupervised learning in noise.

*IEEE Trans. Neural Networks*1: 44–57Kosko B 1992

*Neural networks and fuzzy systems*(Englewood Cliffs, NJ: Prentice-Hall)Krzyzak A, Dali W, Yuen C Y 1990 Unconstrained handwritten character classification using modified back propagation model. In

*Frontiers in handwriting recognition*(ed.) C Y Suen (Montreal: CENPARMI)Kung S Y, Hwang J N 1989 Neural network architectures for robotic applications.

*IEEE Trans. Robotics Autom.*5: 641–657Kuperstein M, Wang J 1990 Neural controller for adaptive movements with unforeseen payloads.

*IEEE Trans. Neural Networks*1(1): 137–142Lang K J, Waibel A H, Hinton G E 1990 A time-delay neural network architecture for isolated word recognition.

*Neural Networks*3 (1): 23–44LeCun Y, Boser B, Denker J S, Henderson D, Howard R E, Hubbard W, Jackel L D 1989 Back propagation applied to handwritten zip code recognition.

*Neural Comput.*1:541–551Lippmann R P 1987 An introduction to computing with neural nets.

*IEEE Trans. Acoust. Speech Signal Process. Mag.*(April): 4–22Lippmann R P 1989a Review of neural networks for speech recognition.

*Neural Comput.*1(1): 1–38Lippmann R P 1989b Pattern classification using neural networks.

*IEEE Commun. Mag.*(Nov): 47–64Lisboa P G P 1992

*Neural networks current applications*(London: Chapman & Hall)Maa C Y, Chin C, Shanblatt M A 1990 A constrained optimization neural net techniques for economic power dispatch.

*Proc. 1990*(New York: IEEE Press)Marcus A, van Dam A 1991 User-interface developments for the nineties.

*IEEE Comput.*24: 49–57McCulloch W S, Pitts W 1943 A logical calculus of the ideas immanent in nervous activity.

*Bull. Math. Biophys.*5: 115–133Michel A N, Farrell J A 1990 Associative memories via artificial neural networks.

*IEEE Control Syst. Mag.*(April): 6–17Minsky M, Papert S A 1988

*Perceptron*(Cambridge, MA: MIT Press)Muller B, Reinhardt J 1990

*Neural networks: An introduction*(Berlin: Springer-Verlag)Murakami K, Aibara T 1987 An improvement on the Moore-Penrose generalized inverse associative memory.

*IEEE Trans. Syst. Man. Cybern.*SMC 17: 699–706Naidu S R, Zafiriou E, McAvoy T J 1990 Use of neural networks for sensor failure detection in a control system.

*IEEE Control Syst. Mag.*10 (3): 49–55Nasrabadi N M, King R A 1988 Image coding using vector quantization: A review.

*IEEE Trans. Commun.*36: 957–971Naylor J, Li K P 1988 Analysis of a neural network algorithm for vector quantization of speech parameters.

*Neural Networks*1 (Suppl): 310Pal S K, Mitra S 1992 Multilayer perceptron, fuzzy sets, and classification.

*IEEE Trans. Neural Networks*3: 683–697Raghu P P, Chouhan H M, Yegnanarayana B 1993 Multispectral image classification using neural network.

*Proc. Natl. Conf. on Neural Networks*, Anna University, Madras (NCNN): 1–10Rauch H E, Winarske T 1988 Neural networks for routing communications traffic.

*IEEE Control Syst. Mag.*(April): 26–31Reggia J A, Suttonn G G 1988 III. Self-processing networks and their biomedical implications.

*Proc. IEEE*76: 680–692Rosenblatt F 1958 A probabilistic model for information storage and organization in the brain.

*Psychol. Rev.*65: 386–408Rosenblatt F 1962

*Principles of neurodynamics*(Washington, DC: Spartan)Rumelhart D, McClelland J 1986

*Parallel distributed processing: Explorations in the microstructure of cognition*(Boston: MIT Press) vol. 1Rumelhart D E, Zipser D 1986 Feature discovery by competitive learning.

*Parallel and distributed processing*(eds) J L McClelland, D E Rumelhart 1: 151-193Scalia F, Marconi L, Ridella S, Arrigo P, Mansi C, Mela G S 1988 An example of back propagation: diagnosis of dyspepsia.

*Ist IEE Conf. Neural Networks*(IEE Conf. Publ.) 313: 332–540Schalkoft R 1992

*Pattern recognition — Statistical, structural and neural approaches*(New York: John Wiley & Sons)Seibert M, Waxman A 1989 Spreading activation layers, visual saccades, and invariant representations for neural pattern recognition systems.

*Neural Networks*2: 9–27Sejnowski T, Rosenberg C 1987 Parallel networks that learn to pronounce English text.

*Complex Syst.*1: 145–168Shea P M, Lin V 1989 Detection of explosives in checked airline baggage using an artificial neural system.

*Int. Joint. Conf. on Neural Networks*2: 31–34Simpson K P 1990

*Artificial neural systems*(New York: Pergamon)Simpson K P 1992

*Foundations of neural networks in artificial neural networks*(eds) Edgar Sanchez-Sinencio, Clifford Lau (New York: IEEE Press)Szu H 1986 Fast simulated annealing. In

*Neural networks for computing*(ed.) J S Denker (New York: Snowbird)Tagliarini G A, Page E W 1988 A neural network solution to the concentrator assignment problem.

*Neural information processing systems*(ed.) D Z Anderson (New York: Am. Inst. Phys.)von der Malsburg Ch 1973 Self-organization of orientation sensitive cells in the striate cortex.

*Kybernetik*14: 85–100Waibel A 1989 Modular construction of time-delay neural networks for speech recognition.

*Neural Comput.*1: 39–46Wasserman P D 1988 Combined backpropagation/cauchy machine.

*Neural networks: Abstracts of the first INNS Meeting*, Boston (Elmsford, NY: Pergamon) 1: 556White H 1988 Economic prediction using neural networks: the case of IBM daily stock returns.

*Neural networks: Abstracts of the First INNS Meeting*, Boston (Elmsford, NY: Pergamon) 1: 451–458Widrow B, Hoff M E 1960 Adaptive switching circuits.

*IRE WESCON Convention Record*(4): 96–104Willshaw D J, von der Malsburg Ch 1976 How patterned neural connections can be set up by self-organization.

*Proc. R. Soc. London*B194: 431–445Zurada J M 1992

*Introduction to artificial neural systems*(St. Paul, MN: West)

## Author information

### Affiliations

### Corresponding author

## Additional information

This paper is mostly a consolidation of work reported by several researchers in the literature, some of which is cited in the references. The author has borrowed several ideas and illustrations from the references quoted in this paper.

## Rights and permissions

## About this article

### Cite this article

Yegnanarayana, B. Artificial neural networks for pattern recognition.
*Sadhana* **19, **189–238 (1994). https://doi.org/10.1007/BF02811896

Received:

Revised:

Issue Date:

### Keywords

- Artificial neural network
- pattern recognition
- biological neural network