1. Introduction and Lemma

Let be a tree which is infinite, connected and contains no circuits. Given any two vertices , there exists a unique path from to with distinct. The distance between and is defined to , the number of edges in the path connecting and . To index the vertices on , we first assign a vertex as the "root" and label it as . A vertex is said to be on the th level if the path linking it to the root has edges. The root is also said to be on the 0th level.

Definition 1.1.

Let be a tree with root , and let be a sequence of positive integers. is said to be a generalized Bethe tree or a generalized Cayley tree if each vertex on the th level has branches to the th level. For example, when and (), is rooted Bethe tree on which each vertex has neighboring vertices (see Figure 1, ), and when (), is rooted Cayley tree on which each vertex has branches to the next level.

In the following, we always assume that is a generalized Bethe tree and denote by the subgraph of containing the vertices from level 0 (the root) to level . We use () to denote the th vertex at the th level and denote by the number of vertices in the subgraph . It is easy to see that, for ,

(1.1)

Let , , , where is a function defined on and taking values in , and let be the smallest Borel field containing all cylinder sets in . Let be the coordinate stochastic process defined on the measurable space ; that is, for any , define

(1.2)
(1.3)

Now we give a definition of Markov chain fields on the tree by using the cylinder distribution directly, which is a natural extension of the classical definition of Markov chains (see [1]).

Figure 1
figure 1

Bethe tree .

Full size image

Definition 1.2.

Let . One has a strictly positive stochastic matrix on ,   a strictly positive distribution on , and a measure on . If

(1.4)

Then will be called a Markov chain field on the tree determined by the stochastic matrix and the distribution .

Let be an arbitrary probability measure defined as (1.3), denote

(1.5)

is called the entropy density on subgraph with respect to . If , then by (1.4), (1.5) we have

(1.6)

The convergence of in a sense ( convergence, convergence in probability, or almost sure convergence) is called the Shannon-McMillan theorem or the entropy theorem or the asymptotic equipartition property (AEP) in information theory. The Shannon-McMillan theorem on the Markov chain has been studied extensively (see [2, 3]). In the recent years, with the development of the information theory scholars get to study the Shannon-McMillan theorems for the random field on the tree graph (see [4]). The tree models have recently drawn increasing interest from specialists in physics, probability and information theory. Berger and Ye (see [5]) have studied the existence of entropy rate for G-invariant random fields. Recently, Ye and Berger (see [6]) have also studied the ergodic property and Shannon-McMillan theorem for PPG-invariant random fields on trees. But their results only relate to the convergence in probability. Yang et al. [7–9] have recently studied a.s. convergence of Shannon-McMillan theorems, the limit properties and the asymptotic equipartition property for Markov chains indexed by a homogeneous tree and the Cayley tree, respectively. Shi and Yang (see [10]) have investigated some limit properties of random transition probability for second-order Markov chains indexed by a tree.

In this paper, we study a class of Shannon-McMillan random approximation theorems for arbitrary random fields on the generalized Bethe tree by comparison between the arbitrary measure and Markov measure on the generalized Bethe tree. As corollaries, a class of Shannon-McMillan theorems for arbitrary random fields and the Markov chains field on the generalized Bethe tree are obtained. Finally, some limit properties for the expectation of the random conditional entropy are discussed.

Lemma 1.3.

Let and be two probability measures on , , and let be a positive-valued stochastic sequence such that

(1.7)

then

(1.8)

In particular, let , then

(1.9)

Proof (see [11]).

Let

(1.10)

is called the sample relative entropy rate of relative to . is also called the asymptotic logarithmic likelihood ratio. By (1.9)

(1.11)

Hence can be look on as a type of measures of the deviation between the arbitrary random fields and the Markov chain fields on the generalized Bethe tree.

2. Main Results

Theorem 2.1.

Let be an arbitrary random field on the generalized Bethe tree. and are, respectively, defined as (1.5) and (1.10). Denote , the random conditional entropy of relative to on the measure , that is,

(2.1)

Let

(2.2)
(2.3)

when ,

(2.4)
(2.5)

In particular,

(2.6)

where is the natural logarithmic, is expectation with respect to the measure .

Proof.

Let be the probability space we consider, an arbitrary constant. Define

(2.7)

denote

(2.8)

We can obtain by (2.7), (2.8) that in the case ,

(2.9)
(2.10)

Therefore, , are a class of consistent distributions on . Let

(2.11)

then is a nonnegative supermartingale which converges almost surely (see [12]). By Doob's martingale convergence theorem we have

(2.12)

Hence by (1.3), (1.9), (2.9), and (2.11) we get

(2.13)

By (1.4), (2.8), and (2.11), we have

(2.14)

By (1.10), (2.2), (2.13), and (2.14) we have

(2.15)

By (2.15) we have

(2.16)

By the inequality

(2.17)

()  and (2.16), (2.17), (2.3), we have in the case of ,

(2.18)

When , we get by (2.18)

(2.19)

Let , in the case , then it is obvious attains, at , its smallest value on the interval . We have

(2.20)

When , we select such that (). Hence for all , it follows from (2.19) that

(2.21)

It is easy to see that (2.20) also holds if from (2.21).

Analogously, when , it follows from (2.18) if ,

(2.22)

Setting in (2.14), by (2.14) we have

(2.23)

Noticing

(2.24)

By (1.4), (1.5), (2.20), and (2.23), we obtain

(2.25)

Hence (2.4) follows from (2.25). By (1.4), (1.5), (1.10), (2.2), and (2.22), we have

(2.26)

Therefore (2.5) follows from (2.26). Set in (2.4) and (2.5), (2.6) holds naturally.

Corollary 2.2.

Let be the Markov chains field determined by the measure on the generalized Bethe tree , are, respectively, defined as (1.6) and (2.3), and is defined by (2.1). Then

(2.27)

Proof.

We take , then . It implies that (2.2) always holds when . Therefore holds. Equation (2.27) follows from (2.3) and (2.6).

3. Some Shannon-McMillan Approximation Theorems on the Finite State Space

Corollary 3.1.

Let be an arbitrary random field which takes values in the alphabet on the generalized Bethe tree. , and are defined as (1.5), (1.10), and (2.2). Denote , . is defined as above. Then

(3.1)
(3.2)

Proof.

Set we consider the function

(3.3)

Then

(3.4)

Let thus . Accordingly it can be obtained that

(3.5)

By (2.3) and (3.5) we have

(3.6)

Therefore, (2.3) holds naturally. By (2.18) and (3.6) we have

(3.7)

In the case of , by (3.7) we have

(3.8)

Let , in the case , then it is obvious attains, at , its smallest value on the interval . That is

(3.9)

By the similar means of reasoning (2.21), it can be concluded that (3.9) also holds when . According to the methods of proving (2.4), (3.1) follows from (1.5), (2.23), and (3.9). Similarly, when , , by (3.7) we have

(3.10)

Imitating the proof of (2.5), (3.2) follows from (1.5), (1.10), (2.2), and (3.10).

Corollary 3.2 (see [9]).

Let be the Markov chains field determined by the measure on the generalized Bethe tree is defined as (1.6), and is defined as (2.1). Then

(3.11)

Proof.

By (3.1) and (3.2) in Corollary 3.1, we obtain that when ,

(3.12)

Set , then . It implies (2.2) always holds when . Therefore holds. Equation(3.11) follows from (3.12).

Corollary 3.3.

Under the assumption of Corollary 3.1, if , then

(3.13)

Proof.

It can be obtained that . holds if (see Gray 1990 [13]), therefore . Equation (3.13) follows from (3.12).

Let be a Markov chains field on the generalized Bethe tree with the initial distribution and the joint distribution with respect to the measure as follows:

(3.14)
(3.15)

where is a strictly positive stochastic matrix on , is a strictly positive distribution. Therefore, the entropy density of with respect to the measure is

(3.16)

Let the initial distribution and joint distribution of with respect to the measure be defined as (1.4) and (1.5), respectively.

We have the following conclusion.

Corollary 3.4.

Let be a Markov chains field on the generalized Bethe tree whose initial distribution and joint distribution with respect to the measure and are defined by (3.14), (3.15) and (1.4), (1.5), respectively. is defined as (3.16). If

(3.17)

then

(3.18)
(3.19)

Proof.

Let in Corollary 3.1, and by (1.5), (3.15) we get (3.16). By the inequalities , , (3.17), and (1.10), we obtain

(3.20)

By (3.17) and (3.20) we have

(3.21)

It follows from (2.2) and (3.21) that ; therefore (3.18), (3.19) follow from (3.1), (3.2).

4. Some Limit Properties for Expectation of Random Conditional Entropy on the Finite State Space

Lemma 4.1 (see [8]).

Let be a Markov chains field defined on a Bethe tree , be the number of in the set of random variables . then for all ,

(4.1)

where is the stationary distribution determined by .

Theorem 4.2.

Let be a Markov chains field defined on a Bethe tree , and let be defined as above. Then

(4.2)

Proof.

Noticing now , for all , , that therefore we have

(4.3)

Noticing that , by (4.3) we have

(4.4)

Equation(4.2) follows from (4.4).

Theorem 4.3.

Let be a Markov chains field defined on a Bethe tree , defined as above. Then

(4.5)

Proof.

By the definition of and properties of conditional expectation, we have

(4.6)

Accordingly we have by (4.6)

(4.7)

Therefore (4.5) also holds.