R-norm entropy and R-norm divergence in product MV-algebras

The aim of the paper is to extend the results concerning the Shannon entropy and Kullback–Leibler divergence in product MV-algebras to the case of R-norm entropy and R-norm divergence. We define the R-norm entropy of finite partitions in product MV-algebras and its conditional version and derive the basic properties of these entropy measures. In addition, we introduce the concept of R-norm divergence in product MV-algebras and we prove basic properties of this quantity. In particular, it is proven that the Kullback–Leibler divergence and Shannon’s entropy of partitions in a given product MV-algebra can be obtained as the limits of their R-norm divergence and R-norm entropy, respectively.


Introduction
MV-algebras were introduced by Chang (1958) as the semantical counterpart of the Łukasiewicz many-valued propositional logic. This algebraic structure is currently being studied by many researchers, and it is natural that there are many results regarding entropy in this structure; for instance, we refer to Di Nola et al. (2005), Riečan (2005), cf. also Markechová et al. (2018a). An important case of MV-algebras is the so-called product MV-algebra introduced independently by Riečan (1999) and Montagna (2000), see also Di Nola and Dvurečenskij (2001) and Jakubík (2002). This notion generalizes some classes of fuzzy sets (Zadeh 1965); an example of product MV-algebra is a full tribe of fuzzy sets (see, e.g., Riečan and Neubrunn 2002).
In this paper, we continue to study entropy in product MV-algebras, which began in Petrovičová (2000), see also Petrovičová (2001), by defining and studying the R-norm entropy and R-norm divergence in this structure. We recall that the R-norm entropy (cf. Arimoto 1971; Boekke and Van Der Lubbe 1980) of a probability distribution P ¼ p 1 ; p 2 ; . . .; p n f gis defined, for a positive real number R not equal to 1, by the formula: Some results regarding the R-norm entropy measure and its generalizations can be found in Hooda and Ram (2002), Hooda and Sharma (2008), Hooda and Bajaj (2008), Kumar and Choudhary (2012), Kumar et al. (2014), Markechová et al. (2018b). We note that in the recently published paper Markechová and Riečan (2017), the results regarding the Shannon entropy of partitions in product MV-algebras were exploited to define the notions of Kullback-Leibler divergence and mutual information of partitions in this structure. The Kullback-Leibler divergence (K-L divergence, for short) was proposed in Kullback and Leibler (1951) as the distance measure between two probability distributions, and it is currently one of the most basic quantities in information theory (Gray 2009). We remark that the concepts of the R-norm entropy and Rnorm divergence are extensions of the notions of Shannon entropy (Shannon 1948) and K-L divergence, respectively.
The aim of the present article is to study the R-norm entropy and R-norm divergence in product MV-algebras. The rest of the article is organized as follows. Sect 2 Communicated by A. Di Nola. contains basic definitions, notations and some known facts that will be used in the succeeding sections. Our results are presented in Sects. 3 and 4. In Sect. 3, we define and study the R-norm entropy and conditional R-norm entropy of finite partitions in product MV-algebras and examine their properties. In Sect. 4, the notion of the R-norm divergence in product MV-algebras is introduced and the properties of this distance measure are studied. It is proved that the Shannon entropy and the K-L divergence in product MValgebras can be derived from their R-norm entropy and Rnorm divergence, respectively, as the limiting cases for R ! 1: We illustrate results with numerical examples. Finally, the last section provides brief closing remarks.

Basic definitions and related works
Let us begin by recalling the definitions of the basic terms and some known results that will be used in the following parts. We mention in this section also some works connected with the issue of this article, of course, with no claim for completeness.
For defining the notion of MV-algebra, several different (but equivalent) axiom systems have been used (cf., e.g., Cattaneo and Lombardo 1998;Gluschankof 1993;Riečan 1999). In this paper, we apply the definition of MV-algebra in accordance with the definition given by Riečan (2012), which is based on Mundici's representation theorem (Mundici 1986; see also Mundici 2011). According to the Mundici theorem, MV-algebras can be viewed as intervals of a commutative lattice ordered groups (shortly l-group). We recall that by an l-group (Anderson and Feil 1988) we mean a triplet ðG; þ; Þ; where ðG; þÞ is a commutative group, ðG; Þ is a partially ordered set being a lattice and x y ) x þ z y þ z: Definition 1 (Riečan 2012) An MV-algebra is an algebraic system ðA; È; Ã; 0; uÞ satisfying the following conditions: (i) there exists an l-group ðG; þ; Þ such that A ¼ ½0; u ¼ fx 2 G; 0 x ug; where 0 is the neutral element of ðG; þÞ and u is a strong unit of G (i.e., u 2 G such that u [ 0 and to each x 2 G there exists a positive integer n with x nu); (ii) È; Ã are binary operation on A satisfying the following identities: x È y ¼ ðx þ yÞ^u; x Ã y ¼ðx þ y À uÞ _ 0: Definition 2 (Riečan and Mundici 2002) A state on an MV-algebra ðA; È; Ã; 0; uÞ is a map s : A ! ½0; 1 with the properties: (i) sðuÞ ¼ 1; (ii) if x; y 2 A such that x þ y u; then sðx þ yÞ ¼ sðxÞ þ sðyÞ: Definition 3 (Riečan 2012) A product MV-algebra is an algebraic structure ðA; È; Ã; Á ; 0; uÞ; where ðA; È; Ã; 0; uÞ is an MV-algebra and Á is a commutative and associative binary operation on A with the following properties: For the sake of brevity, we write in the following ðA; Á Þ instead of ðA; È; Ã; Á ; 0; uÞ: A relevant probability theory for the product MV-algebras was developed in Riečan (2000), see also Kroupa (2005) and Vrábelová (2000). A suitable entropy theory of Shannon type for the product MV-algebras has been provided in Petrovičová (2000Petrovičová ( , 2001, Riečan (2005). The main idea and some results of this theory follow.
Following Petrovičová (2000), by a partition in a product MV-algebra ðA; Á Þ; we will mean any n-tuple a ¼ ðx 1 ; x 2 ; . . .; x n Þ of (not necessarily different) elements of A with the property x 1 þ x 2 þ Á Á Á þ x n ¼ u: In the system of all partitions in a given product MV-algebra ðA; Á Þ; we define the refinement partial order 1 in a standard way (cf. Markechová et al. 2018a). If a ¼ ðx 1 ; x 2 ; . . .; x n Þ; and b ¼ ðy 1 ; y 2 ; . . .; y m Þ are two partitions in ðA; Á Þ; then we write b 1 a (and we say that b is a refinement of a Þ; if there exists a partition Ið1Þ; Ið2Þ; . . .; IðnÞ f g of the set 1; 2; . . .; m f gsuch that x i ¼ P j2I i ð Þ y j ; for i ¼ 1; 2; . . .; n: Further, we define a _ b as a k-tuple (where k ¼ n Á mÞ consisting of the elements x ij ¼ x i Á y j ; i ¼ 1; 2; . . .; n; j ¼ 1; 2; . . .; m: is a partition in ðA; Á Þ; it represents an experiment consisting of a realization of a and b: Proposition 1 Let a ¼ ðx 1 ; x 2 ; . . .; x n Þ be a partition in a product MV-algebra ðA; Á Þ and s be a state on ðA; Á Þ: Then, for any element y 2 A; it holds sðyÞ ¼ P n i¼1 sðx i Á yÞ: Proof The proof can be found in Markechová et al. (2018a). Proposition 2 If a; b are partitions in a product MValgebra ðA; Á Þ such that b 1 a; then for every partition c in ðA; Á Þ; it holds b _ c 1 a _ c: Proof The proof can be found in Markechová et al. (2018a).
Definition 4 Let s be a state on a product MV-algebra ðA; Á Þ: We say that partitions a; b in ðA; Á Þ are statistically independent with respect to s, if sðx Á yÞ ¼ sðx Þ Á sðyÞ; for every x 2 a; and y 2 b: The following definition of entropy of Shannon type has been introduced in Petrovičová (2000).
Definition 5 Let a ¼ ðx 1 ; x 2 ; . . .; x n Þ be a partition in a product MV-algebra ðA; Á Þ and s be a state on ðA; Á Þ: Then the entropy of a with respect to s is defined by Shannon's formula: where . . .; x n Þ; and b ¼ ðy 1 ; y 2 ; . . .; y m Þ are two partitions in ðA; Á Þ; then the conditional entropy of a given b is defined by: The base b of the logarithm can be any positive real number; depending on the selected base b of the logarithm, information is measured in bits (b = 2), nats (b = e), or dits (b = 10). Note that we use the convention (based on continuity arguments) that 0 log The entropy and the conditional entropy of partitions in a product MV-algebra satisfy all properties corresponding to properties of Shannon's entropy of measurable partitions in the classical case; for more details, see Petrovičová (2000). The notion of K-L divergence in product MValgebras was defined in Markechová and Riečan (2017) as follows.
Definition 6 Let s; t be states defined on a given product MV-algebra ðA; Á Þ; and a ¼ ðx 1 ; x 2 ; . . .; x n Þ be a partition in ðA; Á Þ: Then, we define the K-L divergence D a ðs k tÞ by the formula: The logarithm in this formula is taken to the base b ¼ 2 if information is measured in units of bits, to the base b ¼ 10 if information is measured in dits, or to the base b ¼ e if information is measured in nats. We use the convention that 3 The R-norm entropy of partitions in product MV-algebras In this section, we shall introduce the concept of R-norm entropy in product MV-algebras and prove basic properties of this measure of information. It is shown that it has properties that correspond to properties of Shannon's entropy of measurable partitions, with the exception of additivity. In particular, we prove that the R-norm entropy H s R ðaÞ is a concave function on the family of all states defined on a given product MV-algebra ðA; Á Þ: Definition 7 Let a ¼ ðx 1 ; x 2 ; . . .; x n Þ be a partition in a given product MV-algebra ðA; Á Þ: The R-norm entropy of a with respect to a state s defined on ðA; Á Þ is defined for R 2 ð0; 1Þ [ ð1; 1Þ as the number: Remark 1 For the sake of brevity, we write Remark 2 It is easy to verify that the R-norm entropy H s R ðaÞ is always nonnegative. Namely, for 0\R\1; it . . .; x n Þ; and b ¼ ðy 1 ; y 2 ; . . .; y m Þ be two partitions in ðA; Á Þ and s be a state defined on ðA; Á Þ: The conditional R-norm entropy of a given b with respect to s is defined for R 2 ð0; 1Þ [ ð1; 1Þ by the formula: Remark 3 Consider any product MV-algebra ðA; Á Þ; and a state s : A ! ½0; 1: It is easy to see that the set e ¼ u f g is a partition in ðA; ÁÞ with the property a 1 e; for any partition a in ðA; Á Þ; and with the R-norm entropy H s R ðeÞ ¼ The following theorem shows that as the limiting case of the conditional R-norm entropy H s R ða=bÞ for R ! 1; we get the conditional Shannon entropy H s b a=b ð Þ expressed in nats.
Theorem 1 Let a ¼ ðx 1 ; x 2 ; . . .; x n Þ; and b ¼ ðy 1 ; y 2 ; . . .; y m Þ be two partitions in a product MV-algebra ðA; Á Þ, and s be a state defined on ðA; Á Þ: Then: and gðRÞ ¼ 1 À 1 R ; for every R 2 ð0; 1Þ: Then the functions f ; g are differentiable, and for every R 2 ð0; 1Þ [ ð1; 1Þ; we can write: Using L'Hôpital's rule, this implies that under the assumption that the right-hand side exists. Let us calculate the derivative of the function f ðRÞ: Since lim The following theorem states that the R-norm entropy H s R ðaÞ converges for R ! 1 to the Shannon entropy H s b ðaÞ expressed in nats.
Theorem 2 Let a ¼ ðx 1 ; x 2 ; . . .; x n Þ be a partition in a product MV-algebra ðA; Á Þ; and s be a state defined on ðA; Á Þ: Then: Proof The claim follows immediately from Theorem 1 by substituting u f g for b: h In the following part, basic properties of the R-norm entropy H s R ðaÞ are derived.
Theorem 4 Let a 1 ; a 2 ; . . .; a n and c be partitions in a product MV-algebra ðA; Á Þ; and s be a state defined on ðA; Á Þ: Then, for n ¼ 2; 3; . . .; the following equality holds: Remark 4 Let a 1 ; a 2 ; . . .; a n be partitions in a product MValgebra ðA; Á Þ: If we put c ¼ u f g in Eq. (7), we get the following equality: Putting n ¼ 2 in (8), we obtain the following property of the R-norm entropy H s R ðaÞ: Theorem 5 Let a and b be two partitions in a product MValgebra ðA; Á Þ and s be a state defined on ðA; Á Þ: Then: To illustrate the result of the previous theorem, we provide the following example.
Example 1 Consider the measurable space ð½0; 1; B Þ; where B is the r-algebra of all Borel subsets of the unit interval ½0; 1: Let A be the family of all Borel measurable functions f : ½0; 1 ! ½0; 1; so-called full tribe of fuzzy sets (Riečan and Neubrunn 2002). The family A is closed also under the natural product of fuzzy sets and represents a special case of product MV-algebras. We define a state s : A ! ½0; 1 by the equality sðf Þ ¼ R 1 0 f ðxÞdx; for any element f of A: Evidently, the pairs a ¼ ðf 1 ; f 2 Þ and b ¼ ðg 1 ; g 2 Þ; where f 1 ðxÞ ¼ x; f 2 ðxÞ ¼ 1 À x; g 1 ðxÞ ¼ x 2 ; g 2 ðxÞ ¼ 1 À x 2 ; x 2 ½0; 1; are two partitions in ðA; ÁÞ with the s-state values 1 2 ; 1 2 and 1 3 ; 2 3 of the corresponding elements, respectively. By simple calculations, we get that H s 1=2 The join of partitions a and b is the quadruple a sðy j Þ R ; for i ¼ 1; 2; . . .; n; and consequently: R-norm entropy and R-norm divergence in product MV-algebras 6089 Since R RÀ1 [ 0 for R [ 1; we conclude that: The case of 0\R\1 can be obtained using similar techniques.
(ii) Let a; b and c be partitions in a product MV-algebra ðA; Á Þ such that b 1 a: Then, according to Proposition 2, we have b _ c 1 a _ c: Therefore, by Theorem 5 and the property (i), we get: Proof Let a ¼ ðx 1 ; x 2 ; . . .; x n Þ; b ¼ ðy 1 ; y 2 ; . . .; y m Þ: By the assumption, sðx i Á y j Þ ¼ sðx i Þ Á sðy j Þ; for i ¼ 1; 2; . . .; n; j ¼ 1; 2; . . .; m: Therefore, we can write: h One of the most important properties of Shannon entropy is its additivity: the entropy of combined experiment consisting of the realization of two independent experiments is equal to the sum of the entropies of these experiments. In the case of the R-norm entropy H s R ðaÞ; the following property (so-called pseudo additivity) applies.
Theorem 8 Let s be a state defined on a product MValgebra ðA; Á Þ; and a; b be statistically independent partitions in ðA; Á Þ with respect to s. Then: Proof The claim follows by combining Theorem 5 with Theorem 7. h Let us denote by the symbol S ðAÞ the class of all states defined on a given product MV-algebra ðA; Á Þ: It is very easy to verify that if s; t 2 S ðAÞ; then, for every real number k 2 ½0; 1; ks þ ð1 À kÞt 2 S ðAÞ: In the following, we prove that the R-norm entropy H s R ðaÞ is a concave function on the family S ðAÞ: In the proof, we will use the known Minkowski inequality which states that for nonnegative real numbers a 1 ; a 2 ; . . .; a n ; b 1 ; b 2 ; . . .; b n ; we have: ; for 0\R\1: Theorem 9 Let a be a partition in a product MV-algebra ðA; Á Þ: Then, for every s; t 2 S ðAÞ; and for every real number k 2 ½0; 1; the following inequality holds: Proof Assume that a ¼ ðx 1 ; x 2 ; . . .; x n Þ; and k 2 ½0; 1: Putting a i ¼ ksðx i Þ; and b i ¼ ð1 À kÞtðx i Þ; i ¼ 1; 2; . . .; n; in the Minkowski inequality, we get for R [ 1 : and for 0\R\1 : This means that the function s7 ! P n i¼1 sðx i Þ R In this section, we define the concept of the R-norm divergence of states defined on a given product MV-algebra ðA; Á Þ: We will prove basic properties of this quantity. In order to avoid expressions like 0 0 ; we will use in this section the following simplification: for any partition a ¼ ðx 1 ; x 2 ; . . .; x n Þ in ðA; Á Þ; we assume that sðx i Þ [ 0, for i ¼ 1; 2; . . .; n: Definition 9 Let s; t be two states defined on a given product MV-algebra ðA; Á Þ; and a ¼ ðx 1 ; x 2 ; . . .; x n Þ be a partition in ðA; Á Þ: The R-norm divergence of states s;t with respect to a is defined for R 2 ð0; 1Þ [ ð1; 1Þ as the number: Remark 5 As can be easily seen, for any partition a in a product MV-algebra ðA; Á Þ; the R-norm divergence d a R ðs k sÞ is zero.
The following theorem shows that the K-L divergence D a ðs k tÞ measured in nats can be obtained as the limiting case of R-norm divergence d a R ðs k tÞ for R going to 1. Theorem 10 Let s; t be two states defined on a given product MV-algebra ðA; Á Þ; and a ¼ ðx 1 ; x 2 ; . . .; x n Þ be a partition in ðA; Á Þ: Then lim Proof For every R 2 ð0; 1Þ [ ð1; 1Þ; we can write: where f ; g are continuous functions defined for R 2 ð0; 1Þ by the formulas: By continuity of the functions f ; g; we get that lim Using L'Hôpital's rule, it follows that: under the assumption that the right-hand side exists. Let us calculate the derivative of the function f ðRÞ: Since lim R!1 g 0 ðRÞ ¼ lim h Let a ¼ ðx 1 ; x 2 ; . . .; x n Þ be a partition in ðA; Á Þ: In Markechová and Riečan (2017), it has been shown that for the K-L divergence D a ðs k tÞ it holds the inequality D a ðs k tÞ ! 0: The equality holds if and only if sðx i Þ ¼ tðx i Þ; for i ¼ 1; 2; . . .; n: We remark that the previous inequality is known in information theory as the Gibbs inequality. An analogous result also applies to the case of R-norm entropy and R-norm divergence in product MV-algebras 6091 the R-norm divergence d a R ðs k tÞ; as shown in the following theorem.
Theorem 11 Let s; t be two states defined on a given product MV-algebra ðA; Á Þ; and a ¼ ðx 1 ; x 2 ; . . .; x n Þ be a partition in ðA; Á Þ: Then d a R ðs k tÞ ! 0 with the equality if and only if sðx i Þ ¼ tðx i Þ; for i ¼ 1; 2; . . .; n: Proof In the proof, we use the Jensen inequality for the function w defined by wðxÞ ¼ x 1ÀR ; for every x 2 ½0; 1Þ: We shall consider two cases: the case of R [ 1; and the case of 0\R\1: Consider the case of R [ 1: The assumption that R [ 1 implies 1 À R\0; hence the function w is convex. Therefore, applying the Jensen inequality, we obtain: and consequently: For 0\R\1; the function w is concave. Hence, using the Jensen inequality, we obtain: Since R RÀ1 \0 for 0\R\1; this yields that: The equality in (12) holds if and only if tðx i Þ sðx i Þ is constant, for i ¼ 1; 2; . . .; n; i.e., if and only if tðx i Þ ¼ csðx i Þ; for i ¼ 1; 2; . . .; n: Taking the sum over all i ¼ 1; 2; . . .; n; we get the equality P n i¼1 tðx i Þ ¼c P n i¼1 sðx i Þ; which implies that c ¼ 1: Therefore tðx i Þ ¼ sðx i Þ; for i ¼ 1; 2; . . .; n: This means that d a R ðs k tÞ ¼ 0 if and only if sðx i Þ ¼ tðx i Þ; for i ¼ 1; 2; . . .; n: h In the following example, it is shown that the triangle inequality for the R-norm divergence d a R ðs k tÞ does not hold, in general, which means that the R-norm divergence d a R ðs k tÞ is not a metric. Example 2 Consider the product MV-algebra ðA; Á Þ from Example 1 and the real functions F 1 ; F 2 ; F 3 defined by F 1 ðxÞ ¼ x; F 2 ðxÞ ¼ x 2 ; F 3 ðxÞ ¼ x 3 ; for every x 2 ½0; 1: On the product MV-algebra ðA; Á Þ; we define the states s 1 ; s 2 ; s 3 by the formulas s i ðf Þ ¼ R 1 0 f ðxÞdF i ðxÞ; i ¼ 1; 2; 3; for any element f of A: In addition, we will consider the partition a in ðA; Á Þ from Example 1. It can be easily calculated that it has the s 1 -state values 1 2 ; 1 2 ; the s 2 -state values 2 3 ; 1 3 ; and the s 3 -state values 3 4 ; 1 4 of the corresponding elements. We get: Theorem 12 Let s; t be two states defined on a given product MV-algebra ðA; Á Þ; and a ¼ ðx 1 ; x 2 ; . . .; x n Þ be a partition in ðA; Á Þ: In addition, let t be uniform over a; i.e., tðx i Þ ¼ 1 n ; for i ¼ 1; 2; . . .; n: Then, it holds: Proof Let us calculate: From this, it follows Eq. (13). h By combining the results of Theorems 11 and 12, we obtain the following property of R-norm entropy.
Corollary 1 Let s be a state defined on a product MValgebra ðA; Á Þ: Then for any partition a ¼ ðx 1 ; x 2 ; . . .; x n Þ in ðA; Á Þ; it holds: with the equality if and only if the state s is uniform over a.
Theorem 13 Let s be a state defined on a product MValgebra ðA; Á Þ: Then, for every partition a ¼ ðx 1 ; x 2 ; . . .; x n Þ in ðA; Á Þ; it holds: (i) 0\R\1 implies d a R ðs k tÞ D a ðs k tÞ; (ii) R [ 1 implies d a R ðs k tÞ ! D a ðs k tÞ; where D a ðs k tÞ ¼ X n i¼1 sðx i Þ ln sðx i Þ tðx i Þ : Proof By using the inequality ln x x À 1; that applies for all real numbers x [ 0; we get: Suppose that 0\R\1: Then R RÀ1 \0: Therefore, using the inequality (14) and the Jensen inequality for the concave function w defined by wðxÞ ¼ ln x; x 2 ð0; 1Þ; we can write: The case of R [ 1 can be obtained using similar techniques. h To illustrate the result of previous theorem, let us consider the following example which is a continuation to Examples 1 and 2.
Example 3 Consider the product MV-algebra ðA; Á Þ from Example 1 and the real functions F 1 ; F 2 defined by F 1 ðxÞ ¼ x; F 2 ðxÞ ¼ x 2 ; for every x 2 ½0; 1: On the product MV-algebra ðA; Á Þ; we define two states s 1 ; s 2 by the formulas s i ðf Þ ¼ R 1 0 f ðxÞdF i ðxÞ; i ¼ 1; 2; for any element f of A: In addition, we consider the partition a ¼ we have d a R ðs 1 k s 2 Þ \D a ðs 1 k s 2 Þ; d a R ðs 2 k s 1 Þ \D a ðs 2 k s 1 Þ; and for R ¼ 2 we have d a R ðs 1 k s 2 Þ [ D a ðs 1 k s 2 Þ; d a R ðs 2 k s 1 Þ [ D a ðs 2 k s 1 Þ; which is consistent with the claim of Theorem 13. Based on the previous results, we see that the K-L divergence D a ðs k tÞ and the R-norm divergence d a R ðs k tÞ are not symmetrical.

Conclusions
In the paper, we have extended the results concerning the Shannon entropy and K-L divergence in product MV-algebras to the case of R-norm entropy and R-norm divergence. We introduced the notion of R-norm entropy of finite partitions in product MV-algebras and derived its basic properties. In addition, we introduced the notion of Rnorm divergence in product MV-algebras and we proved basic properties of this quantity. In particular, it was shown that the K-L divergence and Shannon's entropy of partitions in a given product MV-algebra can be obtained as the limits of their R-norm divergence and R-norm entropy, respectively. We have provided some numerical examples to illustrate the results as well. As mentioned above, the full tribe of fuzzy sets represents a special case of product MV-algebras; so the obtained results can therefore be immediately applied to this important case of fuzzy sets.
link to the Creative Commons license, and indicate if changes were made.