Orthogonal Relief Algorithm for Feature Selection
The Relief is a popular feature selection algorithm. However, it is ineffective in removing redundant features due to its feature evaluation mechanism that all discriminative features are assigned with high relevance scores, regardless of the correlations in between. In the present study, we develop an orthogonal Relief algorithm (O-Relief) to tackle the redundant feature problem. The basic idea of the O-Relief algorithm is to introduce an orthogonal transform to decompose the correlation between features so that the relevance of a feature could be evaluated individually as it is done in the original Relief algorithm. Experiment results on four world problems show that the orthogonal Relief algorithm provides features leading to better classification results than the original Relief algorithm.
KeywordsFeature Selection Discriminative Feature Redundant Feature Orthogonal Space Wrap Method
Unable to display preview. Download preview PDF.
- 3.Kira, K., Rendell, L.A.: The Feature Selection Problem: Traditional Methods and A New Algorithm. In: Proceedings of AAAI 1992, San Jose, USA, pp. 129–134 (1992)Google Scholar
- 4.Bins, J., Draper, B.A.: Feature Selection from Huge Feature Sets. In: Proceedings of ICCV 2001, Vancouver, Canada, pp. 159–165 (2001)Google Scholar
- 5.Florez-lopez, R.: Reviewing RELIEF and its Extensions: A New Approach for Estimating Attributes Considering High-Correlated Features. In: Proceedings of IEEE International Conference on Data Mining, Maebashi, Japan, pp. 605–608 (2002)Google Scholar
- 6.Kononenko, I.: Estimating Attributes: Analysis and Extensions of RELIEF. In: Proceedings of European Conference on Machine Learning, Catania, Italy, vol. 182, pp. 171–182 (1994)Google Scholar
- 12.Blake, C.L., Merz, C.J.: UCI Repository of Machine Learning Databases, Department of Information and Computer Sciences, University of California, Irvene, CA, USA (1998), http://www.ics.uci.edu/~mlearn/MLRepository.html