Journal of Intelligent Manufacturing

, Volume 8, Issue 3, pp 157–165 | Cite as

A practical overview of neural networks



This paper overviews the myths and misconceptions that have surrounded neural networks in recent years. Focusing on backpropagation and the Hopfield network, we discuss the problems that have plagued practical application of these techniques, and review some of the recent progress made. Both real and perceived inadequacies of backpropagation are discussed, as well as the need for an understanding of statistics and of the problem domain in order to apply and assess the neural network properly. We consider alternatives or variants to backpropagation, which overcome some of its real limitations. The Hopfield network's poor performance on the traveling salesman problem in combinatorial optimization has colored its reception by engineers; we describe both new research in this area and promising results in other practical optimization applications. Overall, it is hoped, this paper will aid in a more balanced understanding of neural networks. They seem worthy of consideration in many applications, but they do not deserve the status of a panacea – nor are they as fraught with problems as would now seem to be implied.

Neural networks engineering applications 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Beyer, D. and Ogier, R. (1990) The tabu learning search neural network method applied to the traveling salesman problem, unpublished technical report, SRI International, Stanford, CA.Google Scholar
  2. Blevins, W. and St Clair, D. (1993) Determining the number and placement of functions for radial basis approximation networks, in Intelligent Engineering Systems through Artificial Neural Networks, Vol. 3. Dagli, C., Burke, L., Fernandez, B. and Ghosh, J. (eds), ASME Press, New York, pp. 45–50.Google Scholar
  3. Burke, L.(1993) Comparing neural networks for classification: advantages of RCE-based networks, in Intelligent Systems through Artificial Neural Networks, Vol. 3, ASME Press, New York.Google Scholar
  4. Burke, L. and S. Rangwala (1991) Tool condition monitoring in metal cutting: a neural network approach. Journal of Intelligent Manufacturing Special Issue on Neural Networks, Vol. 2, pp. 269–280.Google Scholar
  5. Fahlman, S. and Lebiere, C. (1990) The cascade-correlation learning architecture, in Advances in Neural Information Processing Systems 2. Touretsky, D. (ed.), Morgan Kaufmann, San Mateo, CA, pp. 524–532.Google Scholar
  6. Fausett, L. (1994) Fundamentals of Neural Networks. Prentice-Hall, Englewood Cliffs, NJ.Google Scholar
  7. Frean, M. (1990) The upstart algorithm: a method for constructing and training feedforward neural networks, Neural Computation, 2, 198–209.Google Scholar
  8. Golden, B., Wasil, E., Coy, S. and Dagli, C. (1997) Neural networks in practice: survey results, in Computer Science and Operations Research: Advances in the Interface, (in press).Google Scholar
  9. Hecht-Nielsen, R. (1990) Neurocomputing, Addison-Wesley, Reading, MA.Google Scholar
  10. Hopfield, J. J. (1982) Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Science, 79, 2554–2558.Google Scholar
  11. Hopfield, J. J. and Tank, D. (1985) Neural computation of decisions in optimization problems. Biological Cybernetics, 5, 141–152.Google Scholar
  12. Joppe, A., Cardon, H. R. A. and Bioch, J. C. (1990) A neural network for solving the traveling salesman problem on the basis of city adjacency in the tour, in Proceedings of the International Neural Network Conference, Paris, IEEE, pp. 254–257.Google Scholar
  13. Lawler, E. L., Lenstra, J. K., Rinnooy Kan, A. H. G. and Shmoys, D. B. (1985) The Traveling Salesman Problem. John Wiley and Sons, Chichester.Google Scholar
  14. Magent, M. (1996) Combining neural networks and tabu search in a fast neural network simulation for combinatorial optimization, PhD Dissertation, Lehigh University.Google Scholar
  15. Mezard, M. and Nadal, J. (1989) Learning feedforward neural networks: the tiling algorithm. Journal of Physics A: Mathematical and General, 22, 2192–2203.Google Scholar
  16. Moody, J. (1992) The effective number of parameters: an analysis of generalization and regularization in nonlinear learning systems, in Advances in Neural Information Processing Systems 4 Moody, J. et al, (eds.), Morgan Kaufman Publishers, San Mateo, CA, pp. 847–854.Google Scholar
  17. Moody, J. and Darken C. (1989) Fast learning in networks of locally tuned processing units. Neural Computation, 1, 281–294.Google Scholar
  18. Reed, R. (1993) Pruning algorithms–a survey. IEEE Transactions on Neural Networks, 4(5), 740–747.Google Scholar
  19. Reilly, D., Cooper, L. and Elbaum, C. (1982) A neural model for category learning. Biological Cybernetics, 45, 35–41.Google Scholar
  20. Rumelhart, D., Hinton, G. and Williams, R. (1986) Learning representations by backpropagating errors. Nature, 323, 533–536.Google Scholar
  21. Smith, K., Palaniswami, M. and Krishnamoorthy, M. (1996) A hybrid neural approach to combinatorial optimization. Computers and Operations Research, 23(6), 597–610.Google Scholar
  22. Specht, D. (1990) Probabilistic neural networks. Neural Networks, 3(1), 109–118.Google Scholar
  23. Weigend, A., Huberman, B. and Rumelhart, D. (1990) Predicting the future: a connectionist approach. International Journal of Neural Systems, 1(3), 193–210.Google Scholar
  24. Wilson, G. V. and Pawley, G. S. (1988) On the stability of the travelling salesman problem of Hopfield and Tank. Biological Cybernetics, 58, 63–70.Google Scholar
  25. Yang, J., Parekh, R. G. and Honavar, V. G. (1996) Mtiling–a constructive neural network learning algorithm for multicategory pattern classification, in Proceedings of World Congress on Neural Networks, Lawrence Erlbaum, Hillsdale, NJ, pp. 182–187.Google Scholar

Copyright information

© Chapman and Hall 1997

Authors and Affiliations

    • 1
    • 2
  1. 1.Department of Industrial and Manufacturing Systems EngineeringLehigh UniversityBethlehemUSA
  2. 2.Department of Systems EngineeringUniversity of VirginiaCharlottesvilleUSA

Personalised recommendations