Abstract
This paper overviews the myths and misconceptions that have surrounded neural networks in recent years. Focusing on backpropagation and the Hopfield network, we discuss the problems that have plagued practical application of these techniques, and review some of the recent progress made. Both real and perceived inadequacies of backpropagation are discussed, as well as the need for an understanding of statistics and of the problem domain in order to apply and assess the neural network properly. We consider alternatives or variants to backpropagation, which overcome some of its real limitations. The Hopfield network's poor performance on the traveling salesman problem in combinatorial optimization has colored its reception by engineers; we describe both new research in this area and promising results in other practical optimization applications. Overall, it is hoped, this paper will aid in a more balanced understanding of neural networks. They seem worthy of consideration in many applications, but they do not deserve the status of a panacea – nor are they as fraught with problems as would now seem to be implied.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Beyer, D. and Ogier, R. (1990) The tabu learning search neural network method applied to the traveling salesman problem, unpublished technical report, SRI International, Stanford, CA.
Blevins, W. and St Clair, D. (1993) Determining the number and placement of functions for radial basis approximation networks, in Intelligent Engineering Systems through Artificial Neural Networks, Vol. 3. Dagli, C., Burke, L., Fernandez, B. and Ghosh, J. (eds), ASME Press, New York, pp. 45–50.
Burke, L.(1993) Comparing neural networks for classification: advantages of RCE-based networks, in Intelligent Systems through Artificial Neural Networks, Vol. 3, ASME Press, New York.
Burke, L. and S. Rangwala (1991) Tool condition monitoring in metal cutting: a neural network approach. Journal of Intelligent Manufacturing Special Issue on Neural Networks, Vol. 2, pp. 269–280.
Fahlman, S. and Lebiere, C. (1990) The cascade-correlation learning architecture, in Advances in Neural Information Processing Systems 2. Touretsky, D. (ed.), Morgan Kaufmann, San Mateo, CA, pp. 524–532.
Fausett, L. (1994) Fundamentals of Neural Networks. Prentice-Hall, Englewood Cliffs, NJ.
Frean, M. (1990) The upstart algorithm: a method for constructing and training feedforward neural networks, Neural Computation, 2, 198–209.
Golden, B., Wasil, E., Coy, S. and Dagli, C. (1997) Neural networks in practice: survey results, in Computer Science and Operations Research: Advances in the Interface, (in press).
Hecht-Nielsen, R. (1990) Neurocomputing, Addison-Wesley, Reading, MA.
Hopfield, J. J. (1982) Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Science, 79, 2554–2558.
Hopfield, J. J. and Tank, D. (1985) Neural computation of decisions in optimization problems. Biological Cybernetics, 5, 141–152.
Joppe, A., Cardon, H. R. A. and Bioch, J. C. (1990) A neural network for solving the traveling salesman problem on the basis of city adjacency in the tour, in Proceedings of the International Neural Network Conference, Paris, IEEE, pp. 254–257.
Lawler, E. L., Lenstra, J. K., Rinnooy Kan, A. H. G. and Shmoys, D. B. (1985) The Traveling Salesman Problem. John Wiley and Sons, Chichester.
Magent, M. (1996) Combining neural networks and tabu search in a fast neural network simulation for combinatorial optimization, PhD Dissertation, Lehigh University.
Mezard, M. and Nadal, J. (1989) Learning feedforward neural networks: the tiling algorithm. Journal of Physics A: Mathematical and General, 22, 2192–2203.
Moody, J. (1992) The effective number of parameters: an analysis of generalization and regularization in nonlinear learning systems, in Advances in Neural Information Processing Systems 4 Moody, J. et al, (eds.), Morgan Kaufman Publishers, San Mateo, CA, pp. 847–854.
Moody, J. and Darken C. (1989) Fast learning in networks of locally tuned processing units. Neural Computation, 1, 281–294.
Reed, R. (1993) Pruning algorithms–a survey. IEEE Transactions on Neural Networks, 4(5), 740–747.
Reilly, D., Cooper, L. and Elbaum, C. (1982) A neural model for category learning. Biological Cybernetics, 45, 35–41.
Rumelhart, D., Hinton, G. and Williams, R. (1986) Learning representations by backpropagating errors. Nature, 323, 533–536.
Smith, K., Palaniswami, M. and Krishnamoorthy, M. (1996) A hybrid neural approach to combinatorial optimization. Computers and Operations Research, 23(6), 597–610.
Specht, D. (1990) Probabilistic neural networks. Neural Networks, 3(1), 109–118.
Weigend, A., Huberman, B. and Rumelhart, D. (1990) Predicting the future: a connectionist approach. International Journal of Neural Systems, 1(3), 193–210.
Wilson, G. V. and Pawley, G. S. (1988) On the stability of the travelling salesman problem of Hopfield and Tank. Biological Cybernetics, 58, 63–70.
Yang, J., Parekh, R. G. and Honavar, V. G. (1996) Mtiling–a constructive neural network learning algorithm for multicategory pattern classification, in Proceedings of World Congress on Neural Networks, Lawrence Erlbaum, Hillsdale, NJ, pp. 182–187.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
BURKE , L., IGNIZIO , J.P. A practical overview of neural networks. Journal of Intelligent Manufacturing 8, 157–165 (1997). https://doi.org/10.1023/A:1018513006083
Issue Date:
DOI: https://doi.org/10.1023/A:1018513006083