Analysis on Classification Performance of Rough Set Based Reducts

  • Qinghua Hu
  • Xiaodong Li
  • Daren Yu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4099)

Abstract

Feature subset selection and data reduction is a fundamental and most explored area in machine learning and data mining. Rough set theory has been witnessed great success in attribute reduction. A series of reduction algorithms were constructed for all kinds of applications based on rough set models. There is usually more than one reduct for some real world data sets. It is not very clear which one or which subset of the reducts should be selected for learning. Neither experimental comparison nor theoretic analysis was reported so far. In this paper, we will review the proposed attribute reduction algorithms and reduction selection strategies. Then a series of numeric experiments are presented. The results show that, statistically speaking, the classification systems trained with the reduct with the least features get the best generalization power in terms of single classifiers. Furthermore, Good performance is observed from combining the classifiers constructed with multiple reducts compared with Bagging and random subspace ensembles.

Keywords

Classification Performance Decision Table Feature Subset Selection Random Subspace Ensemble System 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in to check access.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Kim, D., Bang, S.Y.: A handwritten numeral character classification using tolerant rough set. IEEE transactions on PAMI 22, 923–937 (2000)Google Scholar
  2. 2.
    Kim, D.: Data classification based on tolerant rough set. Pattern recognition 34, 1613–1624 (2001)MATHCrossRefGoogle Scholar
  3. 3.
    Greco, S., Matarazzo, B., Slowinski, R.: Rough sets theory for multicriteria decision analysis. European journal of operational research 129, 1–47 (2001)MATHCrossRefMathSciNetGoogle Scholar
  4. 4.
    Yao, Y.: Relational interpretations of neighborhood operators and rough set approximation operators. Information sciences 111, 239–259 (1998)MATHCrossRefMathSciNetGoogle Scholar
  5. 5.
    Wu, W.Z., Zhang, W.X.: Neighborhood operator systems and approximations. Information sciences 144, 201–217 (2002)MATHCrossRefMathSciNetGoogle Scholar
  6. 6.
    Slowinski, R., Vanderpooten, D.: A generalized definition of rough approximations based on similarity. IEEE transactions on knowledge and data engineering 12, 331–336 (2000)CrossRefGoogle Scholar
  7. 7.
    Dubois, D., Prade, H.: Rough fuzzy sets and fuzzy rough sets. International Journal of general systems 17, 191–209 (1990)MATHCrossRefGoogle Scholar
  8. 8.
    Dubois, D., Prade, H.: Putting fuzzy sets and rough sets together. In: Slowiniski, R. (ed.) Intelligent Decision support, pp. 203–232. Kluwer Academic, Dordrecht (1992)Google Scholar
  9. 9.
    Boixader, D., Jacas, J., Recasens, J.: Upper and lower approximations of fuzzy sets. International journal of general systems 29, 555–568 (2000)MATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    Wu, W., Mi, J., Zhang, W.: Generalized fuzzy rough sets. Information sciences 151, 263–282 (2003)MATHCrossRefMathSciNetGoogle Scholar
  11. 11.
    Yeung, D.S., Chen, D.G., Tsang, E.C.C., et al.: On the generalization of fuzzy rough sets. IEEE transactions on fuzzy systems 13, 343–361 (2005)CrossRefGoogle Scholar
  12. 12.
    Duntsch, I., Gediga, G.: Uncertainty measures of rough set prediction. Artificial intelligence 106, 109–137 (1998)CrossRefMathSciNetGoogle Scholar
  13. 13.
    Pawlak, Z.: Rough sets, decision algorithms and Bayes’ theorem. European Journal of Operational Research 136, 181–189 (2002)MATHCrossRefMathSciNetGoogle Scholar
  14. 14.
    Ziarko, W.: Variable Precision Rough Set Model. J. Computer and System Sciences 46, 39–59 (1993)MATHCrossRefMathSciNetGoogle Scholar
  15. 15.
    Wei, L., Zhang, W.: Probabilistic rough sets characterized by fuzzy sets. International journal of uncertainty, fuzziness and knowledge based systems 12, 47–60 (2004)MATHCrossRefMathSciNetGoogle Scholar
  16. 16.
    Hu, Q., Yu, D., Xie, Z.: Fuzzy probabilistic approximation spaces and their information measures. IEEE transactions on fuzzy systems 14, 191–201 (2006)CrossRefGoogle Scholar
  17. 17.
    Skowron, R., Rauszer, C.: The discernibility matrices and functions in information systems. Intelligent ecision Support: Handbook of Applications and Advances of Rough Set Theory, 331–362 (1991)Google Scholar
  18. 18.
    Hu, X., Cercone, N.: Learning in Relational Databases: A Rough Set Approach. Computational Intelligence 11, 323–338 (1995)CrossRefGoogle Scholar
  19. 19.
    Jelonek, J., Krawiec, K., Slowinski, R.: Rough set reduction of attributes and their domains for neural networks. Computational Intelligence 11, 339–347 (1995)CrossRefGoogle Scholar
  20. 20.
    Wang, J., Miao, D.: Analysis of attribute reduction strategies of rough set. Journal of computer science and technology 13, 189–193 (1998)MATHCrossRefMathSciNetGoogle Scholar
  21. 21.
    Bazan, J.G., Skowron, A., Synak, P.: Dynamic reducts as a tool for extracting laws from decision tables. In: Raś, Z.W., Zemankova, M. (eds.) ISMIS 1994. LNCS, vol. 869, pp. 346–355. Springer, Heidelberg (1994)Google Scholar
  22. 22.
    Wroblewski, J.: Finding minimal reducts using genetic algorithms. In: Proc. Second International Joint Conference on Information Sciences, September 1995, pp. 186–189 (1995)Google Scholar
  23. 23.
    Jensen, R., Shen, Q.: Fuzzy-rough sets for descriptive dimensionality reduction. In: FUZZ-IEEE 2002, vol. 1(12–17), pp. 29–34 (2002)Google Scholar
  24. 24.
    Jensen, R., Shen, Q.: Fuzzy-rough attribute reduction with application to web categorization. Fuzzy sets and systems 141, 469–485 (2004)MATHCrossRefMathSciNetGoogle Scholar
  25. 25.
    Shen, Q., Jensen, R.: Selecting informative features with fuzzy-rough sets and its application for complex systems monitoring. Pattern recognition 37, 1351–1363 (2004)MATHCrossRefGoogle Scholar
  26. 26.
    Bhatt, R.B., Gopal, M.: On fuzzy-rough sets approach to feature selection. Pattern recognition letters 26, 965–975 (2005)CrossRefGoogle Scholar
  27. 27.
    Hu, Q.H., Yu, D.R.: Entropies of fuzzy indiscernibility relation and its operations. International Journal of uncertainty, fuzziness and knowledge-based systems 12, 575–589 (2004)MATHCrossRefMathSciNetGoogle Scholar
  28. 28.
    Hu, Q.H., Yu, D.R., Xie, Z.X.: Information-preserving hybrid data reduction based on fuzzy rough techniques. Pattern recognition letters 27, 414–423 (2006)CrossRefGoogle Scholar
  29. 29.
    Pawlak, Z.: Rough Sets-—Theoretical Aspects of Reasoning about Data. Kluwer Academic, Dordrecht (1991)MATHGoogle Scholar
  30. 30.
    Hu, X.: knowledge discovery in database: an attribute-oriented rough set method. Ph.D. Thesis. University of Regina (1995)Google Scholar
  31. 31.
    Slezak, D.: Approximate decision reducts. Ph.D. Thesis. Warsaw University (2001)Google Scholar
  32. 32.
    Jensen, R., Shen, Q.: Semantics-preserving dimensionality reduction: rough and fuzzy-rough based approaches. IEEE trans. on knowl. and data engin. 16, 1457–1471 (2004)CrossRefGoogle Scholar
  33. 33.
    Hassanien, E.: Rough Set Approach for Attribute Reduction and Rule Generation: A Case of Patients With Suspected Breast Cancer. Journal of the American society for information science and technology 55, 954–962 (2004)CrossRefGoogle Scholar
  34. 34.
    Domingos, P.: The role of Occam’s razor in knowledge discovery. Data mining and knowledge discovery 3, 409–425 (1999)CrossRefGoogle Scholar
  35. 35.
    Skowron, R., Rauszer, C.: The discernibility matrices and functions in information systems. In: Slowinski, R. (ed.) Intelligent decision support—Handbook of applications and advances of the rough sets theory, pp. 331–362 (1991)Google Scholar
  36. 36.
    Hu, X.: Using rough set theory and database operations to construct a good ensemble of classifiers for data mining applications. In: ICDM., pp. 233–240 (2001)Google Scholar
  37. 37.
    Wu, Q., David, B., Martin, M.: Multiknowledge for decision making. Knowledge and information systems 7, 246–266 (2005)CrossRefGoogle Scholar
  38. 38.
    Hu, Q.H., Yu, D.R., Wang, M.Y.: Constructing rough decision forests. In: Ślęzak, D., Yao, J., Peters, J.F., Ziarko, W., Hu, X. (eds.) RSFDGrC 2005. LNCS (LNAI), vol. 3642, pp. 147–156. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  39. 39.
    Ho, T.K.: The Random Subspace Method for Constructing Decision Forests. IEEE Transactions on pattern analysis and machine intelligence 20, 832–844 (1998)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Qinghua Hu
    • 1
  • Xiaodong Li
    • 1
  • Daren Yu
    • 1
  1. 1.Harbin Institute of TechnologyHarbinP.R. China

Personalised recommendations