Optimal use of a trained neural network for input selection

  • Mercedes Fernández Redondo
  • Carlos Hernández Espinosa
Engeneering Applications
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1607)


In this paper, we present a review of feature selection methods, based on the analysis of a trained multilayer feedforward network, which have been applied to neural networks. Furthermore, a methodology that allows evaluating and comparing feature selection methods is carefully described. This methodology is applied to the 19 reviewed methods in a total of 15 different real world classification problems. We present an ordination of methods according to its performance and it is clearly concluded which method performs better and should be used. We also discuss the applicability and computational complexity of the methods.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Devena, L.: Automatic selection of the most relevant features to recognize objects. Proc. of the Int. Conf. on Artificial NNs, vol. 2, pp. 1113–1116, 1994.Google Scholar
  2. 2.
    Battiti, R.: Using mutual information for selecting features in supervised neural net learning. IEEE Trans. on Neural Networks, vol. 5, n. 4, pp. 537–550, 1994.CrossRefGoogle Scholar
  3. 3.
    Belue, L.M., Bauer, K.W.: Determining input features for multilayer perceptrons. Neurocomputing, vol. 7, n. 2, pp. 111–121, 1995.CrossRefGoogle Scholar
  4. 4.
    Engelbrecht, AP., Cloete, I.: A sensitivity analysis algorithm for pruning feedforward neural networks. Proc. of the Int. Conf. on Neural Networks, vol. 2, pp. 1274–1277, 1996.Google Scholar
  5. 5.
    Priddy, K.L., Rogers, S.K., Ruck D.W., Tarr G.L., Kabrisky, M.: Bayesian selection of important features for feedforward neural networks. Neurocomputing, vol. 5, n. 2&3, pp. 91–103, 1993.CrossRefGoogle Scholar
  6. 6.
    Lee, H., Mehrotra, K., Mohan, C. Ranka, S.: Selection procedures for redundant inputs in neural networks. Proc. of the World Congress on Neural Networks, vol. 1, pp. 300–303, 1993.Google Scholar
  7. 7.
    Tetko, I.V., Tanchuk, V.Y., Luik, A.I.: Simple heuristic methods for input parameter estimation in neural networks. Proc. of the IEEE Int. Conf. on Neural Networks, vol. 1, pp. 376–380, 1994.Google Scholar
  8. 8.
    Cibas, T., Soulié, F.F., Gallinari, P., Raudys, S.: Variable selection with neural networks. Neurocmputing, vol. 12, pp. 223–248, 1996.MATHCrossRefGoogle Scholar
  9. 9.
    Tetko, I.V., Villa, A.E.P., Livingstone, D.J.: Neural network studies. 2. Variable selection. Journal of Chemical Information and Computer Sciences, vol. 36, n. 4 pp. 794–803, 1996.CrossRefGoogle Scholar
  10. 10.
    El-Deredy, W., Branston, N.M.: Identification of relevant features in HMR tumor spectra using neural networks. Proc. of the 4th Int. Conf. on Artificial Neural Networks, pp. 454–458, 1995.Google Scholar
  11. 11.
    Steppe, J.M., Bauer, K.W.: Improved feature screening in feedforward neural networks. Neurocomputing, vol. 13, pp. 47–58, 1996.CrossRefGoogle Scholar
  12. 12.
    Mao, J., Mohiuddin, K., Jain, A.K.: Parsimonious network design and feature selection through node pruning. Proc. of the 12th IAPR Int. Conf. on Pattern Recognition, vol. 2, pp. 622–624, 1994.Google Scholar
  13. 13.
    Utans, J., Moody, J., Rehfuss, S., Siegelmann, H.: Input variable selection for neural networks: Application to predicting the U.S. business cycle. Proc. of IEEE/IAFE 1995 Comput. Intellig. for Financial Eng., pp. 118–122, 1995.Google Scholar
  14. 14.
    Younes, B., Fabrice, B.: A neural network based variable selector. Proc. of the Artificial Neural Network in Engineering, (ANNIE'95), pp. 425–430, 1995.Google Scholar
  15. 15.
    Bowles, A.: Machine learns which features to select”. Proc. of the 5th Australian Joint Conf. on Artificial Intelligence, pp. 127–132, 1992.Google Scholar
  16. 16.
    Sano, H., Nada, A., Iwahori, Y., Ishii, N.: A method of analyzing information represented in neural networks. Proc. of 1993 Int. Joint Conf. on Neural Networks, pp. 2719–2722, 1993.Google Scholar
  17. 17.
    Bishop, C.: Exact calculation of the hessian matrix for the multilayer perceptron. Neural Computation, vol. 4, pp. 494–501, 1992.Google Scholar
  18. 18.
    Watzel, R., Meyer-Bäse, A., Meyer-Bäse, U., hilberg, H., Scheich, H.: Identification of irrelevant features in phoneme recognition with radial basis classifiers. Proc. of 1994 Int. Symp. on Artificial NNs, pp. 507–512, 1994.Google Scholar
  19. 19.
    Bronshtein, I., Semandiavev, K.: Mathematics Handbook for engineers and students (in Spanish). MIR, Moscow, 1977.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1999

Authors and Affiliations

  • Mercedes Fernández Redondo
    • 1
  • Carlos Hernández Espinosa
    • 1
  1. 1.Departamento de InformáticaUniversidad Jaume-ICastellónSpain

Personalised recommendations