Neural Computing and Applications

, Volume 16, Issue 2, pp 109–124 | Cite as

Evaluation of neural network performance and generalisation using thresholding functions

Original Article

Abstract

The application of a simple thresholding technique to help assess the satisfactory performance of classification networks formed from Multi-Layer Perceptron (MLP) artificial neural networks (ANNs) is discussed. Both conventional Maximum Likelihood and Bayesian Evidence based training paradigms were implemented. Firstly a simulated data set drawn from a two-dimensional Gaussian distribution was investigated to illustrate the physical significance of the threshold plots compared to the classifier output probability contours. Secondly a real world application data set comprising of low-frequency vibration measurements on an aircraft wing (a GNAT trainer) is considered. It is demonstrated that simple threshold based plots applied to classifier network outputs may provide a simple yet powerful technique to aid in the rejection of poorly regularised network structures.

Keywords

Neural network training and generalisation Thresholding 

Notes

Acknowledgements

This work was supported by EPSRC grant number GR/R96415/01 in association with DSTL Farnborough who are acknowledged for sample provision and assistance with data collection. The authors gratefully acknowledge the use of the software package NETLAB developed by Ian Nabney of Aston University [http://www.ncrg.aston.ac.uk/netlab/].

References

  1. 1.
    Lippmann RP (1989) Pattern classification using neural networks. IEEE Commun Mag 47(11):47–64CrossRefGoogle Scholar
  2. 2.
    Zhang GP (2000) Neural networks for classification: a survey. IEEE Trans Systems, Man and Cybernetics-Part C: Applications and Reviews 30(4):451–462Google Scholar
  3. 3.
    Lisboa PJG (2002) A review of evidence of health benefit from artificial neural networks in medical intervention. Neural Netw 15:11–39CrossRefPubMedGoogle Scholar
  4. 4.
    Berardi VL, Patuwo BE, Hu MY (2004) A principled approach for building and evaluating neural network classification models. Decis Support Syst 38:233–246CrossRefGoogle Scholar
  5. 5.
    Augusteijn MF, Folkert BA (2002) Neural network classification and novelty detection. Int J Remote Sens 23(14):2891–2902CrossRefGoogle Scholar
  6. 6.
    Masnata A, Sunseri M (1996) Neural network classification of flaws detected by ultrasonic means. NDT&E Int 29(2):97–93Google Scholar
  7. 7.
    Manson G, Worden K, Allman D (2003a) Experimental validation of a structural health monitoring methodology. Part II. Novelty detection on a Gnat aircraft. J Sound Vib 259(2):345–363CrossRefGoogle Scholar
  8. 8.
    Manson G, Worden K, Allman D (2003b) Experimental validation of a structural health monitoring methodology. Part III. Damage location on an aircraft wing. J Sound Vib 259(2):365–385CrossRefGoogle Scholar
  9. 9.
    Dreiseitl S, Ohno-Machado L (2002) Logistic regression and artificial neural network classification models: a methodology review. J Biomed Inform 35:352–359CrossRefPubMedGoogle Scholar
  10. 10.
    Haykin S (1999) Neural networks, a comprehensive foundation, 2nd edn. Prentice Hall, Englewood CliffsMATHGoogle Scholar
  11. 11.
    Schwarzer G, Vach W, Schumacher M (2000) On the misuses of artificial neural networks for prognostic and diagnostic classification in oncology. Stat Med 19:541–561CrossRefPubMedGoogle Scholar
  12. 12.
    Bishop CM (1995) Neural networks for pattern recognition. Oxford University Press, New YorkGoogle Scholar
  13. 13.
    Nabney IT (2002) Netlab-algorithms for pattern recognition. Springer, Berlin Heidelberg New YorkMATHGoogle Scholar
  14. 14.
    MacKay DJC (2003) Information theory, inference, and learning algorithms. Cambridge University Press, CambridgeMATHGoogle Scholar
  15. 15.
    MacKay DJC (1992) A practical Bayesian framework for back-propagation networks. Neural Comput 4:448–472Google Scholar
  16. 16.
    Worden K, Tomlinson GR (2001) Nonlinearity in structural dynamics. Institute of Physics PublishingGoogle Scholar

Copyright information

© Springer-Verlag London Limited 2006

Authors and Affiliations

  1. 1.Dynamics Research Group, Department of Mechanical EngineeringUniversity of SheffieldSheffieldUK

Personalised recommendations