Dimensional Reduction Using Conditional Entropy for Incomplete Information Systems

  • Mustafa Mat DerisEmail author
  • Norhalina Senan
  • Zailani Abdullah
  • Rabiei Mamat
  • Bana Handaga
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11657)


Dimension reduction approach is one of the main data reduction approaches in order to reduce the storage and processing time while maintaining the integrity of the original data. A wide range of dimension reduction approaches are based on classical approaches such as PCA and Bayer’s, and machine learning approaches such as clustering, and feature selection techniques. However, many of the approaches do not consider the incomplete information systems where some attribute values are missing or incomplete. Only few studies were proposed for the problem in incomplete information systems due to its complexities, specifically on attribute selection. The most popular approaches is based on probability theory to replace missing values with the most common values, or remove the missing objects from the information systems. However, it needs to know the probability distribution of data in advance. To overcome these issues, we propose a new approach based on conditional entropy to reduce dimensionality. The results show that the proposed approach achieves better data reduction with higher accuracy for objects and dimensionality reduction in incomplete information systems.


Dimension reduction Conditional entropy Incomplete information system 



The research was supported from Ministry of Higher Education through Fundamental Research Grant Scheme (FRGS) vote number 1643.


  1. 1.
    Chandramouli, B., Goldstein, J., Duan, S.: Temporal analytics on Big Data for web advertizing. In: 28th IEEE International Conference on Data Engineering, pp. 90–101 (2012)Google Scholar
  2. 2.
    Pawlak, Z.: Rough sets. Int. J. Comput. Inform. Sci. 11(5), 341–356 (1982)CrossRefGoogle Scholar
  3. 3.
    Lu, Z., Qin, Z.: Rule extraction from incomplete decision system based on novel dominance relation. In: Proceedings of the 4th International Conference on Intelligent Networks and Intelligent Systems, pp. 149–152 (2011)Google Scholar
  4. 4.
    Dai, J., Wang, W., Xu, Q., Tian, H.: Uncertainty measurement for interval-valued decision systems based on extended conditional entropy. Knowl. Based Syst. 27, 443–450 (2012)CrossRefGoogle Scholar
  5. 5.
    Skowron, A., Wasilewski, P.: Toward interactive Rough-Granular Computing. Control Cybern. 40(2), 213–235 (2011)zbMATHGoogle Scholar
  6. 6.
    Skowron, A., Stepaniuk, J., Swiniarski, R.: Approximation spaces in Rough-Granular Computing. Fundamentae Informaticae 100(1–4), 141–157 (2010)MathSciNetzbMATHGoogle Scholar
  7. 7.
    Yanto, I.T.R., Vitasari, H.T., Deris, M.M.: Applying variable precision rough set model for clustering suffering student’s anxiety. Expert Syst. Appl. 39(1), 452–459 (2012)CrossRefGoogle Scholar
  8. 8.
    Herawan, T., Deris, M.M., Abawajy, J.H.: A rough set approach for selecting clustering attributes. Knowl. Based Syst. 23(3), 220–231 (2010)CrossRefGoogle Scholar
  9. 9.
    Parmar, D., Wu, T., Blackhurst, J.: MMR: an algorithm for clustering categorical data using rough set theory. Data Knowl. Eng. 63(3), 879–893 (2007)CrossRefGoogle Scholar
  10. 10.
    Kim, D.: Data classification based on tolerant rough set. Pattern Recogn. 34(8), 1613–1624 (2001)CrossRefGoogle Scholar
  11. 11.
    Trabelsi, S., Elouedi, Z., Lingras, P.: Classification systems based on rough sets under the belief function network. Int. J. Approximate Reasoning 52(9), 1409–1432 (2011)CrossRefGoogle Scholar
  12. 12.
    Kaneiwa, K.: A rough set approach to multiple dataset analysis. J. Appl. Soft Comput. 11(2), 2538–2547 (2011)CrossRefGoogle Scholar
  13. 13.
    Yan, T., Han, C.: A novel approach of rough conditional entropy-based attribute selection for incomplete decision system. Math. Probl. Eng. 2014, 1–15 (2014)Google Scholar
  14. 14.
    Grzymala-Busse, J.W.: Rough set strategies to data with missing attribute values. In: Proceedings of the workshop on Foundation and New Directions in Data Mining, associated with the 3rd IEEE International Conference on Data Mining, pp. 56–63 (2003)Google Scholar
  15. 15.
    Kryszkiewicz, M.: Rough set approach to incomplete information systems. Inf. Sci. 112(1–4), 39–49 (1998)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Kryszkiewicz, M.: Rules in incomplete information systems. Inf. Sci. 113(3–4), 271–292 (1999)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Stefanowski, J., Tsoukiàs, A.: On the extension of rough sets under incomplete information. In: Zhong, N., Skowron, A., Ohsuga, S. (eds.) RSFDGrC 1999. LNCS (LNAI), vol. 1711, pp. 73–81. Springer, Heidelberg (1999). Scholar
  18. 18.
    Stefanowski, J., Tsoukias, A.: Incomplete information table and rough classification. Comput. Intell. 17(3), 545–566 (2001)CrossRefGoogle Scholar
  19. 19.
    Wang, G.Y.: Extension of rough set under incomplete system. In: IEEE International Conference on Fuzzy Systems, pp. 1098–1103 (2002)Google Scholar
  20. 20.
    Yang, X., Song, X., Hu, X.: Generalization of rough set for rule induction in incomplete system. Int. J. Granular Comput. Rough Sets Intell. Syst. 2(1), 37–50 (2011)CrossRefGoogle Scholar
  21. 21.
    Nguyen, D.V., Yamada, K., Unehara, M.: Extended tolerance relation to define a new rough set model in incomplete information systems. Adv. Fuzzy Syst. 2013, 1–11 (2013)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Deris, M.M., Abdullah, Z., Mamat, R., Yuan, Y.: A new limited tolerance relation for attribute selection in incomplete information systems. In: IEEE International Conference on Fuzzy Systems and Knowledge Discovery, pp. 964–969 (2015)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Mustafa Mat Deris
    • 1
    • 2
    • 3
    • 4
    Email author
  • Norhalina Senan
    • 1
  • Zailani Abdullah
    • 2
  • Rabiei Mamat
    • 3
  • Bana Handaga
    • 4
  1. 1.Faculty of Computer Science and Information TechnologyUniversiti Tun Hussein Onn MalaysiaParit RajaMalaysia
  2. 2.Faculty of Entrepreneurship and BusinessUniversiti Malaysia KelantanKota BharuMalaysia
  3. 3.Faculty of Informatics and Applied MathematicsUniversity of Malaysia TerengganuKuala TerengganuMalaysia
  4. 4.Program Studi InformatikaUniversitas Muhammadiah SurakartaSurakartaIndonesia

Personalised recommendations