A Rough Set Approach to Feature Selection Based on Relative Decision Entropy
Rough set theory has been recognized to be one of the powerful tools for feature selection. The essence of rough set approach to feature selection is to find a minimal subset (or called reduct) of original features, which can predict the decision concepts as well as the original features. Since finding a minimal feature subset is a NP-hard problem, many heuristic algorithms have been proposed, which usually employ the feature significance in rough sets as heuristic information. Shannon’s information theory provides us a feasible way to measure the information of data sets with entropy. Hence, many researchers have used information entropy to measure the feature significance in rough sets, and proposed different information entropy models in rough sets. In this paper, we present a new information entropy model, in which relative decision entropy is defined. Based on the relative decision entropy, we propose a new rough set algorithm to feature selection. To verify the efficiency of our algorithm, experiments are carried out on some UCI data sets. The results demonstrate that our algorithm can provide competitive solutions efficiently.
KeywordsRough sets feature selection information entropy relative decision entropy
Unable to display preview. Download preview PDF.
- 3.Hu, X.H.: Knowledge Discovery in Databases: An Attribute-oriented Rough Set Approach. Ph.D. thesis, Regina University (1995)Google Scholar
- 7.Wang, G.Y.: Rough Set Theory and Knowledge Acquisition. Xian Jiaotong University Press, Xian (2001)Google Scholar
- 8.Liu, S.H., Sheng, Q.J., Wu, B., Shi, Z.Z.: Research on Efficient Algorithms for Rough Set Methods. Chinese Journal of Computers 26(5), 525–529 (2003)Google Scholar
- 9.Miao, D.Q., Hu, G.R.: An Heuristic Algorithm of Knowledge Reduction. Computer Research and Development 36(6), 681–684 (1999)Google Scholar
- 14.Xu, Z.Y., Liu, Z.P., Yang, B.R., Song, W.: A Quick Attribute Reduction Algorithm with Complexity of max(O(|C| |U|),O(|C| 2 |U/C|)). Chinese Journal of Computers 29(3), 391–399 (2006)Google Scholar
- 15.Jensen, R., Shen, Q.: Finding Rough Set Reducts with Ant Colony Optimization. In: Proc. of the 2003 UK Workshop on Computational Intelligence, pp. 15–22 (2003)Google Scholar
- 16.Wroblewski, J.: Finding Minimal Reducts Using Genetic Algorithms. In: Proc. of the Second Annual Join Conf. on Information Sciences, NC, pp. 186–189 (1995)Google Scholar
- 17.Bazan, J.: A Comparison of Dynamic and Non-dynamic Rough Set Methods for Extracting Laws from Decision Table. In: Rough Sets in Knowledge Discovery, pp. 321–365. Physica-Verlag, Heidelberg (1998)Google Scholar
- 19.Nguyen, S.H., Nguyen, H.S.: Some Efficient Algorithms for Rough Set Methods. In: Proceedings of IPMU 1996, Granada, Spain, pp. 1451–1456 (1996)Google Scholar
- 20.Skowron, A., Bazan, J., Son, N.H., Wroblewski, J., et al.: RSES 2.2 User’s Guide. Institute of Mathematics, Warsaw University, Warsaw, Poland (2005)Google Scholar
- 21.Stefanowski, J.: On Rough Set Based Approaches to Induction of Decision Rules. In: Rough Sets in Knowledge Discovery, pp. 500–529. Physica-Verlag, Heidelberg (1998)Google Scholar
- 22.Bay, S.D.: The UCI KDD Repository (1999), http://kdd.ics.uci.edu