Abstract
This paper focuses on enhancing the effectiveness of filter feature selection models from two aspects. One is to modify feature searching engines based on optimization theory, and the other is to improve the regularization capability using point injection techniques. The second topic is undoubtedly important in the situations where overfitting is likely to be met, for example, the ones with only small sample sets available. Synthetic and real data are used to demonstrate the contribution of our proposed strategies.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Al-Ani, A., Deriche, M.: Optimal feature selection using information maximisation: case of biomedical data. In: Proc. of the 2000 IEEE Signal Processing Society Workshop, vol. 2, pp. 841–850 (2000)
Alon, U., et al.: Broad pattern of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays. Proc. Natl. Acad. Sci. USA 96(12), 6745–6750 (1999)
Battiti, R.: Using mutual information for selecting features in supervised neural net learning. IEEE Trans. Neural Networks 5, 537–550 (1994)
Bishop, C.M.: Neural Networks for Pattern Recognition. Oxford University Press, New York (1995)
Bonnlander, B.: Nonparametric Selection of Input Variables for Connectionist Learning, Ph.D. thesis, CU-CS-812-96, University of Colorado at Boulder (1996)
Chow, T.W.S., Huang, D.: Estimating optimal feature subsets using efficient estimation of high-dimensional mutual information. IEEE Trans. Neural Networks 16(1), 213–224 (2005)
Devijver, P.A., Kittler, J.: Pattern Recognition: a Statistical Approach. Prentice Hall, Englewood Cliffs (1982)
Golub, T.R., et al.: Molecular classification of cancer: Class discovery and class prediction by gene expression monitoring. Science 286, 531–537 (1999)
Glick, N.: Additive estimators for probabilities of correct classification. Pattern recognition 18(2), 151–159 (1985)
Gui, J., Li, H.: Penalized Cox regression analysis in the high-dimensional and low-sample size settings, with application to microarray gene expression data. Bioinformatics 21(13), 3001–3008 (2005)
Guyon, I., Weston, J., Barnhill, S.: Gene selection for cancer classification using support vector machines. Machine Learning 46, 389–422 (2002)
Hall, M.A.: Correlation-based Feature Selection for Machine Learning, Ph.D. thesis, Department of Computer Science, Waikato University, New Zealand (1999)
Han, J.W., Kamber, M.: Data mining: concepts and techniques. Morgan Kaufmann Publishers, San Francisco (2001)
Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning, pp. 308–312. Springer, Heidelberg (2001)
Huang, D., Chow, T.W.S.: Efficiently searching the important input variables using Bayesian discriminant. IEEE Trans. Circuits and Systems 52(4), 785–793 (2005)
Huang, D., Chow, T.W.S., et al.: Efficient selection of salient features from microarray gene expression data for cancer diagnosis. IEEE Trans. Circuits and Systems, part I 52(9), 1909–1918 (2005)
Kim, S., Dougherty, E.R., Barrera, J.Y., et al.: Strong feature sets from small samples. Journal of Computational Biology 9, 127–146 (2002)
Lampariello, F., Sciandrone, M.: Efficient training of RBF neural networks for pattern recognition. IEEE Trans. On Neural Networks 12(5), 1235–1242 (2001)
Liu, H., Motoda, H.: Feature Selection for Knowledge Discovery and Data Mining. Kluwer Academic Publishers, London, GB (1998)
Matsuoka, S.: Noise injection into inputs in back-propagation learning. IEEE Trans. Syst., Man, Cybern. 22, 436–440 (1992)
Molina, L.C., Belanche, L., Nebot, A.: Feature Selection Algorithms: a Survey and Experimental Evaluation, Technical Report (2002), available at: http://www.lsi.upc.es/dept/techreps/html/R02-62.html
Parzen, E.: On the estimation of a probability density function and mode. Ann. Math. Statistics 33, 1064–1076 (1962)
Perkins, S., Lacker, K., Theiler, J.: Grafting: Fast, Incremental feature selection by gradient descent in function space. Journal of machine learning research 3, 1333–1356 (2003)
Pudil, P., Novovicova, J., Kittler, J.: Floating search methods in feature selection. Pattern Recognition Letter 15, 1119–1125 (1994)
Singh, D., et al.: Gene expression correlates of clinical prostate cancer behavior. Cancer Cell 1, 203–209 (2002)
Skurichina, M., Raudys, S., Duin, R.P.: K-nearest neighbours directed noise injection in multilayer perceptron training. IEEE Trans. On Neural Networks 11(2), 504–511 (2000)
Wolf, L., Martin, I.: Regularization through feature knock out, AI memo 2004-2005 (2004), available at http://cbcl.mit.edu/cbcl/publications/ai-publications/2004/
Zagoruiko, N.G., Elkina, V.N., Temirkaev, V.S.: ZET-an algorithm of filling gaps in experimental data tables. Comput. Syst. 67, 3–28 (1976)
Zhou, X., Wang, X., Dougherty, E.: Nonlinear probit gene classification using mutual information and wavelet-based feature selection. Journal of Biological Systems 12(3), 371–386 (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Huang, D., Chow, T.W.S. (2006). An Excellent Feature Selection Model Using Gradient-Based and Point Injection Techniques. In: King, I., Wang, J., Chan, LW., Wang, D. (eds) Neural Information Processing. ICONIP 2006. Lecture Notes in Computer Science, vol 4233. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11893257_76
Download citation
DOI: https://doi.org/10.1007/11893257_76
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-46481-5
Online ISBN: 978-3-540-46482-2
eBook Packages: Computer ScienceComputer Science (R0)