# Fuzzy cross-entropy

- 1.7k Downloads
- 11 Citations

## Abstract

This paper deals with the divergence of fuzzy variables from *a priori* one. Within the framework of credibility theory, a fuzzy cross-entropy is defined to measure the divergence, and some mathematical properties are investigated. Furthermore, a minimum cross-entropy principle is proposed, which tells us that out of all membership functions satisfying given moment constraints, we should choose the one that is closest to the given *a priori* membership function.

## Keywords

Fuzzy variable Cross-entropy Entropy Credibility theory## Introduction

Fuzzy entropy provides a quantitative measure of the uncertainty associated with each fuzzy variable. Since Zadeh [1] introduced the fuzzy entropy as a weighted shannon entropy, researchers gave several definitions from different angles, such as De Luca and Termini [2], Yager [3], Kaufmann [4], Kosko [5], Pal and Pal [6]. The above definitions characterize the uncertainty resulting primarily from the linguistic vagueness rather than resulting from information deficiency and vanish when the fuzzy variable is an equipossible one. However, Liu [7] suggested that a fuzzy entropy should meet at least the following three basic requirements: the entropy of a crisp number is zero; the entropy of an equipossible fuzzy variable is maximum; and the entropy is applicable not only to finite and infinite cases but also to discrete and continuous cases. In order to meet these requirements, within the framework of credibility theory, Li and Liu [8] provided a new definition of fuzzy entropy to characterize the uncertainty resulting from information deficiency which is caused by the impossibility to predict the specified value that a fuzzy variable takes. Based on this definition, Li and Liu [9] proposed the fuzzy maximum entropy principle and proved some maximum entropy theorems.

This paper is devoted to formulate a fuzzy cross-entropy characterized by credibility measure. For this purpose, we organize this paper as follows. The ‘Preliminaries’ section recalls some useful definitions and properties about credibility theory. The ‘Fuzzy cross-entropy’ section defines the fuzzy cross-entropy and studies some useful properties. In the ‘Minimum cross-entropy principle’ section, the minimum cross-entropy principle is proposed. At the end of this paper, a brief summary is given.

## Preliminaries

Credibility theory [10] is a branch of mathematics for studying the behavior of fuzzy phenomena. Let *Θ* be a nonempty set, and let Open image in new window be the power set of *Θ*. Each element *A* of Open image in new window is called an event. In 2002, Liu and Liu [11] presented a credibility measure Cr{*A*} to express the chance that event *A* occurs. Furthermore, Li and Liu [12] proved that a set function is a credibility measure if and only if it satisfies the following axioms:

Axiom 1. (Normality) Cr{*Θ*}=1;

Axiom 2. (Monotonicity) Cr{*A*}≤Cr{*B*} whenever *A*⊂*B*;

Axiom 3. (Self-duality) Cr is self-dual, i.e., Cr{*A*}+Cr{*A* ^{ c }}=1 for any event *A*;

Axiom 4. (Countable subadditivity) Cr{∪_{ i } *A* _{ i }}= sup*i*Cr{*A* _{ i }} for any events {*A* _{ i }} with sup*i*Cr{*A* _{ i }}<0.5.

*Θ*,,Cr) is called a credibility space. A fuzzy variable is defined as a function from a credibility space (

*Θ*,,Cr) to the set of real numbers. Let

*ξ*be a fuzzy variable. Then, its membership function is derived from the credibility measure by:

*ξ*is a fuzzy variable with membership function

*μ*, then, for any set

*B*⊆ℜ, we have:

This formula is also called the credibility inversion theorem.

### **Definition** **2.1.**

*ξ*be a fuzzy variable taking values in {

*x*

_{1},

*x*

_{2},⋯,

*x*

_{ n }} (Li and Liu [8]). Then, its fuzzy entropy is defined as:

*S*(

*t*)=−

*t*ln

*t*−(1−

*t*) ln(1−

*t*).

Fuzzy entropy is used to quantify the uncertainty associated to fuzzy variables.

### **Theorem** **2.1.**

*ξ*be a fuzzy variable taking values in {

*x*

_{1},

*x*

_{2},⋯,

*x*

_{ n }} (Li and Liu [8]). Then, we have:

Especially, *H*[ *ξ*] attains its minimum value 0 if and only if *ξ* is a crisp number, and *H*[ *ξ*] attains its maximum value *n* ln2 if and only if *ξ* is an equipossible fuzzy variable.

### **Definition** **2.2.**

*ξ*be a continuous fuzzy variable (Li and Liu [8]). Then, its fuzzy entropy is defined as:

### **Theorem** **2.2**.

*ξ*be a continuous fuzzy variable taking values in [

*a,b*] (Li and Liu [8]). Then, we have:

Especially, *H*[ *ξ*] attains its minimum value if and only if *ξ* is a crisp number, and *H*[ *ξ*] attains its maximum value if and only if *ξ* is an equipossible fuzzy variable.

In 2007, Li and Liu [9] proposed a fuzzy maximum entropy principle, which tells us that out of all the membership functions satisfying the given constraints, we should select the one that maximizes the entropy.

## Fuzzy cross-entropy

In this section, we define a fuzzy cross-entropy for quantifying the divergence of fuzzy variables from an *a priori* one. The relation between fuzzy entropy and fuzzy cross-entropy is also discussed.

### **Definition** **3.1.**

*ξ*and

*η*be two discrete fuzzy variables taking values in {

*x*

_{1},

*x*

_{2},⋯,

*x*

_{ n }}. Then, the fuzzy cross-entropy of

*ξ*from

*η*is defined as:

*T*(

*s*,

*t*)=

*s*ln(

*s*/

*t*)+(1−

*s*) ln((1−

*s*)/(1−

*t*)).

It is easy to prove that *D*[ *ξ*;*η*] is permutationally symmetric, i.e., the value does not change if the outcomes are labeled differently.

### **Definition** **3.2.**

*ξ*and

*η*be two continuous fuzzy variables taking values in [

*a,b*]. Then, the cross-entropy of

*ξ*from

*η*is defined as:

*μ*and

*ν*be the membership functions of continuous fuzzy variables

*ξ*and

*η*, respectively. Since Cr{

*ξ*=

*x*}=

*μ*(

*x*)/2 and Cr{

*η*=

*x*}=

*ν*(

*x*)/2, the cross-entropy of

*ξ*from

*η*can be rewritten as:

### **Remark** **3.1.**

*=(*

**ξ***ξ*

_{1},

*ξ*

_{2},⋯,

*ξ*

_{ m }) and

*=(*

**η***η*

_{1},

*η*

_{2},⋯,

*η*

_{ m }) are discrete, we have:

*and*

**ξ***are continuous variables, we have:*

**η**### **Remark** **3.2.**

*T*(

*s,t*) is a function from [ 0,1]×[ 0,1] to [ 0,+

*∞*). Please also mention that:

Then, the following properties about *T*(*s,t*) can be easily proved: (a) *T*(*s,t*) is strictly convex with respect to (*s,t*) and attains its minimum value zero on the line *s*=*t*; and (b) for any 0≤*s*≤1 and 0≤*t*≤1, we have *T*(*s*,*t*)=*T*(1−*s*,1−*t*).

### **Theorem** **3.1.**

For any fuzzy variables *ξ* and *η*, we have *D*[ *ξ*;*η*]≥0, and the equality holds if and only if *ξ* and *η* have the same membership function.

### *Proof*.

*μ*and

*ν*be the membership functions of discrete fuzzy variables

*ξ*and

*η*, respectively. Since

*T*(

*s*,

*t*) is strictly convex about (

*s*,

*t*) and attains its minimum value zero on the line

*s*=

*t*, we have

*T*(Cr{

*ξ*=

*x*

_{ i }},Cr{

*η*=

*x*

_{ i }})≥0 for all

*i*, which implies that:

*s*

^{∗}≤1, the unique minimum point of

*T*(

*s*

^{∗},

*t*) is

*t*=

*s*

^{∗}. Thus, we have

*D*[

*ξ*;

*η*]=0 if and only if

*T*(Cr{

*ξ*=

*x*

_{ i }},Cr{

*η*=

*x*

_{ i }})=0, that is:

*i*=1,2,⋯,

*n*. If

*ξ*and

*η*are continuous fuzzy variables, the theorem can be proved in a similar way. The proof is complete. □

### **Theorem** **3.2.**

*τ*be the equipossible fuzzy variable with membership function

*ν*(

*x*

_{ i })=1 for all

*i*=1,2,⋯,

*n*. Then, for any discrete fuzzy variable

*ξ*taking values in {

*x*

_{1},

*x*

_{2},⋯,

*x*

_{ n }}, we have:

### *Proof*.

*τ*=

*x*

_{ i }}=0.5 for all

*i*=1,2,⋯,

*n*. It follows from the definition of cross-entropy that

*D*[

*ξ*,

*τ*] is:

The proof is complete. □

### **Theorem** **3.3.**

*τ*be the equipossible fuzzy variable with membership function

*ν*(

*x*)=1 for all

*x*∈[

*a*,

*b*]. Then, for any continuous fuzzy variable

*ξ*taking values in [

*a*,

*b*], we have:

### *Proof*.

*D*[

*ξ*,

*τ*] is:

The proof is complete. □

## Minimum cross-entropy principle

In many real problems, the membership function of a fuzzy variable is unavailable except some partial information, for example, moment constraints, which may be based on observations. In this case, the maximum entropy principle (Li and Liu [9]) tells us that out of all the membership functions satisfying given constraints, choose the one that has maximum entropy. However, there may be another type of information, for example, *a priori* membership function, which may be based on intuition or experience with the problem. If both the *a priori* membership function and the moment constraints are given, which membership function should we choose? The following minimum cross-entropy principle tells us that out of all membership functions satisfying given moment constraints, choose the one that is closest to the given *a priori* membership function.

*a priori*experience or intuition to guide us, we choose the membership function that is nearest to the equipossible one. In this sense, if the

*a priori*membership function is not prescribed and the fuzzy variable is simple (bounded for continuous case), the maximum entropy principle and minimum cross-entropy principle are consistent because:

*υ*is the equipossible fuzzy variable.

## Conclusion

Based on credibility measure, a definition of cross-entropy was proposed in this paper to measure the divergence of fuzzy variables from *a priori* one, and some properties were investigated. Furthermore, a minimum cross-entropy principle was proposed as an important entropy optimization principle.

## Notes

### Acknowledgements

This work was supported by the National Natural Science Foundation of China (No. 71371027) and Program for New Century Excellent Talents in University under Grant No. NCET-13-0649.

## References

- 1.Zadeh, LA: Probability measures of fuzzy events. J. Math. Anal. Appl. 23, 421–427 (1968).CrossRefzbMATHMathSciNetGoogle Scholar
- 2.De Luca, A, Termini, S: A definition of nonprobabilistic entropy in the setting of fuzzy sets theory. Inf. Control. 20, 301–312 (1972).CrossRefzbMATHMathSciNetGoogle Scholar
- 3.Yager, RR: On measures of fuzziness and negation, part I: membership in the unit interval. Int. J. General Syst. 5, 221–229 (1979).CrossRefzbMATHMathSciNetGoogle Scholar
- 4.Kaufmann, A: Introduction to the Theory of Fuzzy Subsets. Academic Press, New York (1975).zbMATHGoogle Scholar
- 5.Kosko, B: Fuzzy entropy and conditioning. Inf. Sci. 40, 165–174 (1986).CrossRefzbMATHMathSciNetGoogle Scholar
- 6.Pal, NR, Pal, SK: Higher order fuzzy entropy and hybrid entropy of a set. Inf. Sci. 61, 211–231 (1992).CrossRefzbMATHGoogle Scholar
- 7.Liu, B: A survey of entropy of fuzzy variables. J. Uncertain Syst. 1(1), 4–13 (2007).Google Scholar
- 8.Li, P, Liu, B: Entropy of credibility distributions for fuzzy variables. IEEE Trans. Fuzzy Syst. 16(1), 123–129 (2008).CrossRefGoogle Scholar
- 9.Li, X, Liu, B: Maximum entropy principle for fuzzy variables. Int. J. Uncertainty Fuzziness Knowledge-Based Syst. 15(Supp 2), 40–48 (2007).Google Scholar
- 10.Liu, B: Uncertainty Theory. Springer-Verlag, Berlin (2004).CrossRefzbMATHGoogle Scholar
- 11.Liu, B, Liu, YK: Expected value of fuzzy variable and fuzzy expected value models. IEEE Trans. Fuzzy Syst. 10(4), 445–450 (2002).CrossRefGoogle Scholar
- 12.Li, X, Liu, B: A sufficient and necessary condition for credibility measures. Int. J. Uncertainty Fuzziness Knowledge-Based Syst. 14(5), 527–535 (2006).CrossRefzbMATHGoogle Scholar

## Copyright information

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited.