Skip to main content
SpringerLink
Log in

Complement information entropy for uncertainty measure in fuzzy rough set and its applications

  • Methodologies and Application
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Uncertainty measure is an important tool for analyzing imprecise and ambiguous data. Some information entropy models in rough set theory have been defined for various information systems. However, there are relatively few studies on evaluating uncertainty in fuzzy rough set. In this paper, we propose a new complement information entropy model in fuzzy rough set based on arbitrary fuzzy relation, which takes inner-class and outer-class information into consideration. The corresponding definitions of complement conditional entropy, complement joint entropy, complement mutual information and complement information granularity are also presented. The properties of these definitions are analyzed, which show complement information entropy shares some similar properties with Shannon’s entropy. Moreover, a generalized information entropy model is proposed by introducing probability distribution into fuzzy approximate space. This model can be used to measure uncertainty of data with the different sample distributions. Applications of the proposed entropy measures in feature importance evaluation and feature selection are studied with data set experiments. Experimental results show that the proposed method is effective and adaptable to different classifiers.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  • Beaubouf T, Petry F, Arora G (1998) Information-theoretic measures of uncertainty for rough sets and rough relational databases. J Inf Sci 109:185–195

    Article  Google Scholar 

  • Buckley J, Eslami E (2003) Uncertain probabilities I: the discrete case. Soft Comput 7:500–505

    Article  MATH  Google Scholar 

  • Buckley J, Eslami E (2004) Uncertain probabilities II: the continuous case. Soft Comput 8:193–199

    Article  MATH  Google Scholar 

  • Bertoluzza C, Doldi V, Naval G (2004) Uncertainty measure on fuzzy partitions. Fuzzy Sets Syst 142:105–116

    Article  MATH  MathSciNet  Google Scholar 

  • Cooman G (1997) Possibility theory. Int J Gen Syst 25(4):291–371

    Article  MATH  Google Scholar 

  • Coletti G, Scozzafava R, Vantaggi B (2008) Possibility measures in probabilistic inference. In: Dubois D et al (eds) Soft methods for hand. var. and imprecision, ASC 48, pp 51–58

  • De Luca A, Termini S (1972) A definition of non-probabilistic entropy in the setting of fuzzy theory. Inf Control 20:301–312

    Article  MATH  Google Scholar 

  • Dubois D, Prade H (1990) Rough fuzzy sets and fuzzy rough sets. Int J Gen Syst 17:191–209

    Article  MATH  Google Scholar 

  • Dai J, Wang W, Mi J (2013a) Uncertainty measurement for interval-valued information systems. Inf Sci 251:63–78

    Article  MathSciNet  Google Scholar 

  • Dai J, Wang W, Xu Q (2013b) An uncertainty measure for incomplete decision tables and its applications. IEEE Trans Cybern 43(4):1277–1289

    Article  Google Scholar 

  • Dai J, Tian H (2013c) Entropy measures and granularity measures for set-valued information systems. Inf Sci 240:72–82

    Article  MathSciNet  Google Scholar 

  • Flage R, Aven T, Baraldi P (2012) An imprecision importance measure for uncertainty representations interpreted as lower and upper probabilities, with special emphasis on possibility theory. J Risk Reliab 226(6):656–665

    Google Scholar 

  • Graves K, Nagarajah R (2007) Uncertainty estimation using fuzzy measures for multiclass classification. IEEE Trans Neural Netw 18(1):128–140

    Article  Google Scholar 

  • Halliwell J, Shen Q (2009) Linguistic probabilities: theory and application. Soft Comput 13:169–183

    Article  MATH  Google Scholar 

  • Hu Q, Zhang L, Chen D et al (2010) Gaussian kernel based fuzzy rough sets: model, uncertainty measures and applications. Int J Approx Reason 51:453–471

    Article  MATH  Google Scholar 

  • Jensen R, Shen Q (2007) Fuzzy-rough sets assisted attribute selection. IEEE Trans Fuzzy Syst 15(1):73–89

    Article  Google Scholar 

  • Klir G, Smith R (2001) On measuring uncertainty and uncertainty-based information: recent developments. Ann Math Artif Intell 32:5–33

    Article  MathSciNet  Google Scholar 

  • Klir G (2005) Measuring uncertainty associated with convex sets of probability distributions: a new approach. In: 2005 annual meeting of the North American Fuzzy Information Processing Society, pp 61–64

  • Liang J, Chin K, Dang C et al (2002) A new method for measuring uncertainty and fuzziness in rough set theory. Int J Gen Syst 31(4):331–342

    Article  MATH  MathSciNet  Google Scholar 

  • Liang J, Shi Z, Li D, Wierman M (2006) Information entropy, rough entropy and knowledge granulation in incomplete information systems. Int J Gen Syst 35:641–654

    Article  MATH  MathSciNet  Google Scholar 

  • Liang J, Wang J, Qian Y (2009) A new measure of uncertainty based on knowledge granulation for rough sets. Inf Sci 179:458–470

    Article  MATH  MathSciNet  Google Scholar 

  • Li Y, Chen J, Feng L (2013) Dealing with uncertainty: a survey of theories and practices. IEEE Trans Knowl Data Eng 25(11):2463–2482

    Article  Google Scholar 

  • Liu Y (2013) Uncertain random variables a mixture of uncertainty and randomness. Soft Comput 17:625–634

    Article  MATH  Google Scholar 

  • Mi J, Zhang W (2004) An axiomatic characterization of a fuzzy generalization of rough sets. Inf Sci 160(1–4):235–249

    Article  MATH  MathSciNet  Google Scholar 

  • Mi J, Leung Y, Wu W (2005) An uncertainty measure in partition-based fuzzy rough sets. Int J Gen Syst 34:77–90

    Article  MATH  MathSciNet  Google Scholar 

  • Mi J, Li X, Zhao H, Feng T (2007) Information-theoretic measure of uncertainty in generalized fuzzy rough sets. RSFDGrC 2007. LNAI 4482:63–70

    Google Scholar 

  • Mesiar R (2005) Fuzzy measures and integrals. Fuzzy Sets Syst 156:365–370

    Article  MATH  MathSciNet  Google Scholar 

  • Nguyen H, Bouchon-Meunier B (2003) Random sets and large deviations principle as a foundation for possibility measures. Soft Comput 8:61–70

    Article  MATH  Google Scholar 

  • Pal N, Bezdek J (1994) Measuring fuzzy uncertainty. IEEE Trans Fuzzy Syst 2:107–117

    Article  Google Scholar 

  • Pawlak Z (1982) Rough Sets. Int J Comput Inf Sci 11:341–356

    Article  MATH  MathSciNet  Google Scholar 

  • Quinla J (1993) C4.5: programs for machine learning. The Morgan Kaufmann series in machine learning. Morgan Kaufmann Publishers, San Mateo

    Google Scholar 

  • Qian Y, Liang J, Wierman M (2009) A new method for measuring the uncertainty in incomplete information systems. Int J Uncert, Fuzziness, Knowl-Based Syst 17(6):855–880

    Article  MATH  Google Scholar 

  • Robnik Šikonja M, Kononenko I (2003) Theoretical and empirical analysis of ReliefF and RReliefF. Mach Learn J 53:23–69

  • Wierman M (1999) Measuring uncertainty in rough set theory. Int J Gen Syst 28:283–297

  • Wu W, Leung Y, Mi J (2005) On characterizations of \((\wp ,\Im )\)-fuzzy rough approximation operators. Fuzzy Sets Syst 154:76–102

  • Xu W, Wang Q, Zhang X (2013) Multi-granulation rough sets based on tolerance relations. Soft Comput 17:1241–1252

    Article  Google Scholar 

  • Yu D, Hu Q, Wu C (2007) Uncertainty measures for fuzzy relations and their applications. Appl Soft Comput 7(3):1135–1143

    Article  Google Scholar 

  • Yager R (2002) Uncertainty representation using fuzzy measures. IEEE Trans Syst Man Cybern-Part B: Cybern 1:13–20

    Article  MATH  MathSciNet  Google Scholar 

  • Yager R (2011) A measure based approach to the fusion of possibilistic and probabilistic uncertainty. Fuzzy Optim Decis Making 10:91–113

  • Yager R, Alajlan N (2012a) Measure based representation of uncertain information. Fuzzy Optim Decis Mak 11:363–385

  • Yager R (2012b) Entailment principle for measure-based uncertainty. IEEE Trans Fuzzy Syst 20(3):526–535

    Article  Google Scholar 

  • Zadeh L (1968) Probability measures of fuzzy events. Int J Math Anal Appl 23:421–427

    Article  MATH  MathSciNet  Google Scholar 

  • Zadeh L (1978) Fuzzy sets as a basis for a theory of possibility. Fuzzy Sets Syst 1:3–28

    Article  MATH  MathSciNet  Google Scholar 

  • Zhao J, Zhang Z (2007) Fuzzy-rough Data Reduction based on information entropy. In: Proceedings of the 6th international conference on machine learning and cybernetics. Hong Kong, pp 3708–3712

  • Zhai D, Mendel J (2011) Uncertainty measures for general type-2 fuzzy sets. Inf Sci 181:503–518

    Article  MATH  MathSciNet  Google Scholar 

Download references

Acknowledgments

This work is supported by the China Postdoctoral Science Foundation (2013M532048), Foundation for Innovative Research Groups of the National Natural Science Foundation of China (60921003), the National Natural Science Foundation of China (61074176, 41174162).

Author information

Authors and Affiliations

  1. Ministry of Education Key Lab For Intelligent Networks and Network Security (MOE KLINNS Lab), Institute of Integrated Automation, School of Electronic and Information Engineering, Xi’an Jiaotong University, Xi’an , 710049, China

    Junyang Zhao & Chongzhao Han

  2. Xi’an Research Inst. of Hi-tech Hongqing Town, Xi’an , 710025, China

    Junyang Zhao, Zhili Zhang & Zhaofa Zhou

Authors
  1. Junyang Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhili Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chongzhao Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhaofa Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Junyang Zhao.

Additional information

Communicated by V. Loia.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, J., Zhang, Z., Han, C. et al. Complement information entropy for uncertainty measure in fuzzy rough set and its applications. Soft Comput 19, 1997–2010 (2015). https://doi.org/10.1007/s00500-014-1387-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-014-1387-5

Keywords

Search

Navigation