Feature Selection with Positive Region Constraint for Test-Cost-Sensitive Data

  • Jiabin Liu
  • Fan MinEmail author
  • Hong Zhao
  • William Zhu
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8449)


In many data mining and machine learning applications, data are not free, and there is a test cost for each data item. Due to economic, technological and legal reasons, it is neither possible nor necessary to obtain a classifier with 100 % accuracy. In this paper, we consider such a situation and propose a new constraint satisfaction problem to address it. With this in mind, one has to minimize the test cost to keep the accuracy of the classification under a budget. The constraint is expressed by the positive region, whereas the object is to minimizing the total test cost. The new problem is essentially a dual of the test cost constraint attribute reduction problem, which has been addressed recently. We propose a heuristic algorithm based on the information gain, the test cost, and a user specified parameter \(\lambda \) to deal with the new problem. The algorithm is tested on four University of California - Irvine datasets with various test cost settings. Experimental results indicate that the algorithm finds optimal feature subset in most cases, the rational setting of \(\lambda \) is different among datasets, and the algorithm is especially stable when the test cost is subject to the Pareto distribution.


Feature selection Cost-sensitive learning Positive region Test cost 



This work is partially supported by the Natural Science Foundation of Department of Education of Sichuan Province under Grant No. 13ZA0136, and National Science Foundation of China under Grant Nos. 61379089, 61379049.


  1. 1.
    Chen, X.: An improved branch and bound algorithm for feature selection. Pattern Recogn. Lett. 24(12), 1925–1933 (2003)CrossRefGoogle Scholar
  2. 2.
    Dash, M., Liu, H.: Feature selection for classification. Intell. Data Anal. 1, 131–156 (1997)CrossRefGoogle Scholar
  3. 3.
    Fayyad, U., Piatetsky-Shapiro, G., Smyth, P.: From data mining to knowledge discovery in databases. AI Mag. 17, 37–54 (1996)Google Scholar
  4. 4.
    Greco, S., Matarazzo, B., Slowinski, R., Stefanowski, J.: Variable consistency model of dominance-based rough sets approach. In: Ziarko, W.P., Yao, Y. (eds.) RSCTC 2000. LNCS (LNAI), vol. 2005, pp. 170–181. Springer, Heidelberg (2001)zbMATHCrossRefGoogle Scholar
  5. 5.
    Blake, C.L., Merz, C.J.: UCI repository of machine learning databases (1998).
  6. 6.
    He, H.P., Min, F.: Accumulated cost based test-cost-sensitive attribute reduction. In: Kuznetsov, S.O., Ślȩzak, D., Hepting, D.H., Mirkin, B.G. (eds.) RSFDGrC 2011. LNCS, vol. 6743, pp. 244–247. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  7. 7.
    He, H.P., Min, F., Zhu, W.: Attribute reduction in test-cost-sensitive decision systems with common-test-costs. In: Proceedings of the 3rd International Conference on Machine Learning and Computing, vol. 1, pp. 432–436 (2011)Google Scholar
  8. 8.
    Hu, Q.H., Yu, D.R., Liu, J.F., Wu, C.: Neighborhood rough set based heterogeneous feature subset selection. Inf. Sci. 178(18), 3577–3594 (2008)MathSciNetzbMATHCrossRefGoogle Scholar
  9. 9.
    Hunt, E.B., Marin, J., Stone, P.J. (eds.): Experiments in Induction. Academic Press, New York (1966)Google Scholar
  10. 10.
    Lanzi, P.: Fast feature selection with genetic algorithms: a filter approach. In: IEEE International Conference on Evolutionary Computation 1997. IEEE (1997)Google Scholar
  11. 11.
    Lin, T.Y.: Granular computing on binary relations - analysis of conflict and Chinese wall security policy. In: Alpigini, J.J., Peters, J.F., Skowron, A., Zhong, N. (eds.) RSCTC 2002. LNCS (LNAI), vol. 2475, pp. 296–299. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  12. 12.
    Lin, T.Y.: Granular computing - structures, representations, and applications. In: Wang, G., Liu, Q., Yao, Y., Skowron, A. (eds.) RSFDGrC 2003. LNCS (LNAI), vol. 2639, pp. 16–24. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  13. 13.
    Liu, Q.H., Li, F., Min, F., Ye, M., Yang, W.G.: An efficient reduction algorithm based on new conditional information entropy. Control Decis. (in Chinese) 20(8), 878–882 (2005)zbMATHGoogle Scholar
  14. 14.
    Liu, J.B., Min, F., Liao, S.J., Zhu, W.: A genetic algorithm to attribute reduction with test cost constraint. In: Proceedings of 6th International Conference on Computer Sciences and Convergence Information Technology, pp. 751–754 (2011)Google Scholar
  15. 15.
    Liu, H., Motoda, H.: Feature Selection for Knowledge Discovery and Data Mining. The Springer International Series in Engineering and Computer Science, vol. 454. Kluwer Academic Publishers, Boston (1998)zbMATHCrossRefGoogle Scholar
  16. 16.
    Ma, L.W.: On some types of neighborhood-related covering rough sets. Int. J. Approx. Reason. 53(6), 901–911 (2012)MathSciNetzbMATHCrossRefGoogle Scholar
  17. 17.
    Min, F., He, H.P., Qian, Y.H., Zhu, W.: Test-cost-sensitive attribute reduction. Inf. Sci. 181, 4928–4942 (2011)CrossRefGoogle Scholar
  18. 18.
    Min, F., Hu, Q.H., Zhu, W.: Feature selection with test cost constraint. Int. J. Approximate Reasoning (2013, to appear). doi: 10.1016/j.ijar.2013.04.003 MathSciNetzbMATHCrossRefGoogle Scholar
  19. 19.
    Min, F., Liu, Q.H.: A hierarchical model for test-cost-sensitive decision systems. Inf. Sci. 179, 2442–2452 (2009)MathSciNetzbMATHCrossRefGoogle Scholar
  20. 20.
    Min, F., Zhu, W.: Attribute reduction with test cost constraint. J. Electr. Sci. Technol. China 9(2), 97–102 (2011)Google Scholar
  21. 21.
    Min, F., Zhu, W.: Minimal cost attribute reduction through backtracking. In: Kim, T., et al. (eds.) DTA/BSBT 2011. CCIS, vol. 258, pp. 100–107. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  22. 22.
    Min, F., Zhu, W.: Optimal sub-reducts in the dynamic environment. In: Proceedings of IEEE International Conference on Granular Computing, pp. 457–462 (2011)Google Scholar
  23. 23.
    Min, F., Zhu, W.: Optimal sub-reducts with test cost constraint. In: Yao, J.T., Ramanna, S., Wang, G., Suraj, Z. (eds.) RSKT 2011. LNCS, vol. 6954, pp. 57–62. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  24. 24.
    Min, F., Zhu, W., Zhao, H., Pan, G.Y., Liu, J.B., Xu, Z.L.: Coser: cost-sensitive rough sets (2012).
  25. 25.
    Pan, G.Y., Min, F., Zhu, W.: A genetic algorithm to the minimal test cost reduct problem. In: Proceedings of IEEE International Conference on Granular Computing. pp. 539–544 (2011)Google Scholar
  26. 26.
    Pawlak, Z.: Rough set approach to knowledge-based decision support. Eur. J. Oper. Res. 99, 48–57 (1997)zbMATHCrossRefGoogle Scholar
  27. 27.
    Pawlak, Z.: Rough sets. Int. J. Comput. Inf. Sci. 11, 341–356 (1982)zbMATHCrossRefGoogle Scholar
  28. 28.
    Pawlak, Z.: Rough sets and intelligent data analysis. Inf. Sci. 147(12), 1–12 (2002)MathSciNetzbMATHCrossRefGoogle Scholar
  29. 29.
    Qian, Y.H., Liang, J.Y., Pedrycz, W., Dang, C.Y.: Positive approximation: an accelerator for attribute reduction in rough set theory. Artif. Intell. 174(9–10), 597–618 (2010)MathSciNetzbMATHCrossRefGoogle Scholar
  30. 30.
    Skowron, A., Rauszer, C.: The discernibility matrices and functions in information systems. In: Intelligent Decision Support (1992)CrossRefGoogle Scholar
  31. 31.
    Swiniarski, R.W., Skowron, A.: Rough set methods in feature selection and recognition. Pattern Recogn. Lett. 24(6), 833–849 (2003)zbMATHCrossRefGoogle Scholar
  32. 32.
    Tseng, T.L.B., Huang, C.-C.: Rough set-based approach to feature selection in customer relationship management. Omega 35(4), 365–383 (2007)CrossRefGoogle Scholar
  33. 33.
    Wang, G.Y.: Attribute core of decision table. In: Alpigini, J.J., Peters, J.F., Skowron, A., Zhong, N. (eds.) RSCTC 2002. LNCS (LNAI), vol. 2475, pp. 213–217. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  34. 34.
    Wang, X., Yang, J., Teng, X., Xia, W., Jensen, R.: Feature selection based on rough sets and particle swarm optimization. Pattern Recogn. Lett. 28(4), 459–471 (2007)CrossRefGoogle Scholar
  35. 35.
    Xu, Z.L., Min, F., Liu, J.B., Zhu, W.: Ant colony optimization to minimal test cost reduction. In: Proceedings of the 2011 IEEE International Conference on Granular Computing. pp. 688–693 (2012)Google Scholar
  36. 36.
    Yao, J.T., Zhang, M.: Feature selection with adjustable criteria. In: Ślȩzak, D., Wang, G., Szczuka, M.S., Düntsch, I., Yao, Y. (eds.) RSFDGrC 2005. LNCS (LNAI), vol. 3641, pp. 204–213. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  37. 37.
    Yao, Y.Y., Zhao, Y.: Attribute reduction in decision-theoretic rough set models. Inf. Sci. 178(17), 3356–3373 (2008)MathSciNetzbMATHCrossRefGoogle Scholar
  38. 38.
    Zhang, W.X., Mi, J., Wu, W.: Knowledge reductions in inconsistent information systems. Chin. J. Comput. 26(1), 12–18 (2003)MathSciNetGoogle Scholar
  39. 39.
    Zhao, H., Min, F., Zhu, W.: A backtracking approach to minimal cost feature selection of numerical data. J. Inf. Comput. Sci. 10(13), 4105–4115 (2013)CrossRefGoogle Scholar
  40. 40.
    Zhao, H., Min, F., Zhu, W.: Test-cost-sensitive attribute reduction based on neighborhood rough set. In: Proceedings of the 2011 IEEE International Conference on Granular Computing, pp. 802–806 (2011)Google Scholar
  41. 41.
    Zhao, H., Min, F., Zhu, W.: Test-cost-sensitive attribute reduction of data with normal distribution measurement errors. Math. Prob. Eng. 2013, 12 pp (2013)Google Scholar
  42. 42.
    Zhong, N., Dong, Z.J., Ohsuga, S.: Using rough sets with heuristics to feature selection. J. Intell. Inf. Syst. 16(3), 199–214 (2001)zbMATHCrossRefGoogle Scholar
  43. 43.
    Zhu, W.: Generalized rough sets based on relations. Inf. Sci. 177(22), 4997–5011 (2007)MathSciNetzbMATHCrossRefGoogle Scholar
  44. 44.
    Zhu, W.: Topological approaches to covering rough sets. Inf. Sci. 177(6), 1499–1508 (2007)MathSciNetzbMATHCrossRefGoogle Scholar
  45. 45.
    Zhu, W.: Relationship between generalized rough sets based on binary relation and covering. Inf. Sci. 179(3), 210–225 (2009)MathSciNetzbMATHCrossRefGoogle Scholar
  46. 46.
    Zhu, W., Wang, F.: Reduction and axiomization of covering generalized rough sets. Inf. Sci. 152(1), 217–230 (2003)MathSciNetzbMATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  1. 1.Department of Computer ScienceSichuan University for NationalitiesKangdingChina
  2. 2.Lab of Granular ComputingZhangzhou Normal UniversityZhangzhouChina

Personalised recommendations