Abstract
The Relief is a popular feature selection algorithm. However, it is ineffective in removing redundant features due to its feature evaluation mechanism that all discriminative features are assigned with high relevance scores, regardless of the correlations in between. In the present study, we develop an orthogonal Relief algorithm (O-Relief) to tackle the redundant feature problem. The basic idea of the O-Relief algorithm is to introduce an orthogonal transform to decompose the correlation between features so that the relevance of a feature could be evaluated individually as it is done in the original Relief algorithm. Experiment results on four world problems show that the orthogonal Relief algorithm provides features leading to better classification results than the original Relief algorithm.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Jain, A.K., Duin, R.P.W., Mao, J.: Statistical Pattern Recognition: A Review. IEEE Transactions on Pattern Analysis and Machine Intelligence 22, 4–37 (2000)
Kohavi, R., John, G.H.: Wrappers for Feature Subset Selection. Artificial Intelligent Journal 1, 273–324 (1997)
Kira, K., Rendell, L.A.: The Feature Selection Problem: Traditional Methods and A New Algorithm. In: Proceedings of AAAI 1992, San Jose, USA, pp. 129–134 (1992)
Bins, J., Draper, B.A.: Feature Selection from Huge Feature Sets. In: Proceedings of ICCV 2001, Vancouver, Canada, pp. 159–165 (2001)
Florez-lopez, R.: Reviewing RELIEF and its Extensions: A New Approach for Estimating Attributes Considering High-Correlated Features. In: Proceedings of IEEE International Conference on Data Mining, Maebashi, Japan, pp. 605–608 (2002)
Kononenko, I.: Estimating Attributes: Analysis and Extensions of RELIEF. In: Proceedings of European Conference on Machine Learning, Catania, Italy, vol. 182, pp. 171–182 (1994)
Chen, S., Billings, S.A., Luo, W.: Orthogonal Least Square Methods and their Applications to Nonlinear System Identification. International Journal of Control 50, 1873–1896 (1989)
Mao, K.Z., Billings, S.A.: Algorithms for Minimal Model Structure Detection in Nonlinear Dynamic System Identification. International Journal of Control 68, 311–330 (1997)
Mao, K.Z., Tan, K.C., Ser, W.: Probabilistic Neural Network Structure Determination for Pattern Classification. IEEE Transactions on Neural Networks 11, 1009–1016 (2000)
Mao, K.Z.: RBF Neural Network Center Selection Based on Fisher Ratio Class Separability Measure. IEEE Transactions on Neural Networks 13, 1211–1217 (2002)
Mao, K.Z.: Orthogonal Forward Selection and Backward Elimination Algorithms for Feature Subset Selection. IEEE Transactions on Systems, Man and Cybernetics, Part B 34, 629–634 (2004)
Blake, C.L., Merz, C.J.: UCI Repository of Machine Learning Databases, Department of Information and Computer Sciences, University of California, Irvene, CA, USA (1998), http://www.ics.uci.edu/~mlearn/MLRepository.html
Golub, T.R., et al.: Molecular Classification of Cancer: Class Discovery and Class Prediction by Gene Expression Monitoring. Science 286, 531–537 (1999)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Yang, J., Li, YP. (2006). Orthogonal Relief Algorithm for Feature Selection. In: Huang, DS., Li, K., Irwin, G.W. (eds) Intelligent Computing. ICIC 2006. Lecture Notes in Computer Science, vol 4113. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11816157_22
Download citation
DOI: https://doi.org/10.1007/11816157_22
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-37271-4
Online ISBN: 978-3-540-37273-8
eBook Packages: Computer ScienceComputer Science (R0)