Skip to main content
Log in

Localizing the latent structure canonical uncertainty: entropy profiles for hidden Markov models

  • Published:
Statistics and Computing Aims and scope Submit manuscript

An Erratum to this article was published on 26 June 2015

Abstract

This paper addresses state inference for hidden Markov models. These models rely on unobserved states, which often have a meaningful interpretation. This makes it necessary to develop diagnostic tools for quantification of state uncertainty. The entropy of the state sequence that explains an observed sequence for a given hidden Markov chain model can be considered as the canonical measure of state sequence uncertainty. This canonical measure of state sequence uncertainty is not reflected by the classic multidimensional posterior state (or smoothed) probability profiles because of the marginalization that is intrinsic in the computation of these posterior probabilities. Here, we introduce a new type of profiles that have the following properties: (i) these profiles of conditional entropies are a decomposition of the canonical measure of state sequence uncertainty along the sequence and makes it possible to localise this uncertainty, (ii) these profiles are unidimensional and thus remain easily interpretable on tree structures. We show how to extend the smoothing algorithms for hidden Markov chain and tree models to compute these entropy profiles efficiently. The use of entropy profiles is illustrated by sequence and tree data examples.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  • Brushe, G., Mahony, R., Moore, J.: A soft output hybrid algorithm for ML/MAP sequence estimation. IEEE Trans. Inf. Theory 44(7), 3129–3134 (1998)

    Article  MATH  MathSciNet  Google Scholar 

  • Cappé, O., Moulines, E., Rydén, T.: Inference in Hidden Markov Models. Springer Series in Statistics. Springer, New York (2005)

    Google Scholar 

  • Celeux, G., Soromenho, G.: An entropy criterion for assessing the number of clusters in a mixture model. J. Classif. 13(2), 195–212 (1996)

    Article  MATH  MathSciNet  Google Scholar 

  • Cover, T., Thomas, J.: Elements of Information Theory, 2nd edn. Wiley, Hoboken (2006)

    MATH  Google Scholar 

  • Crouse, M., Nowak, R., Baraniuk, R.: Wavelet-based statistical signal processing using hidden Markov models. IEEE Trans. Signal Process. 46(4), 886–902 (1998)

    Article  MathSciNet  Google Scholar 

  • Devijver, P.A.: Baum’s forward-backward algorithm revisited. Pattern Recognit. Lett. 3, 369–373 (1985)

    Article  MATH  Google Scholar 

  • Durand, J.-B., Girard, S., Ciriza, V., Donini, L.: Optimization of power consumption and device availability based on point process modelling of the request sequence. Appl. Stat. 62(2), 151–162 (2013)

    MathSciNet  Google Scholar 

  • Durand, J.-B., Gonçalvès, P., Guédon, Y.: Computational methods for hidden Markov tree models: an application to wavelet trees. IEEE Trans. Signal Process. 52(9), 2551–2560 (2004)

    Article  MathSciNet  Google Scholar 

  • Durand, J.-B., Guédon, Y., Caraglio, Y., Costes, E.: Analysis of the plant architecture via tree-structured statistical models: the hidden Markov tree models. New Phytol. 166(3), 813–825 (2005)

    Article  Google Scholar 

  • Durand, J.-B., Guédon, Y.: Localizing the latent structure canonical uncertainty: entropy profiles for hidden Markov odels. Available: hal.inria.fr/hal-00675223/en, Inria technical report (2012)

  • Ephraim, Y., Merhav, N.: Hidden Markov processes. IEEE Trans. Inf. Theory 48, 1518–1569 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  • Guédon, Y., Caraglio, Y., Heuret, P., Lebarbier, E., Meredieu, C.: Analyzing growth components in trees. J. Theor. Biol. 248(3), 418–447 (2007a)

    Article  Google Scholar 

  • Guédon, Y.: Exploring the state sequence space for hidden Markov and semi-Markov chains. Comput. Stat. Data Anal. 51(5), 2379–2409 (2007b)

  • Guédon, Y.: Segmentation uncertainty in multiple change-point models. Stat. Comput. (2013, in press)

  • Hernando, D., Crespi, V., Cybenko, G.: Efficient computation of the hidden Markov model entropy for a given observation sequence. IEEE Trans. Inf. Theory 51(7), 2681–2685 (2005)

  • Lauritzen, S.: Graphical Models. Clarendon Press, Oxford (1996)

    Google Scholar 

  • Le Cadre, J.-P., Tremois, O.: Bearing-only tracking for maneuvering sources. IEEE Trans. Aerosp. Electron. Syst. 34(1), 179–193 (1998)

  • McLachlan, G., Peel, D.: Finite Mixture Models. Wiley Series in Probability and Statistics. Wiley, New York (2000)

    Book  Google Scholar 

  • Zucchini, W., MacDonald, I.: Hidden Markov Models for Time Series: An Introduction Using R. Chapman & Hall/CRC, Boca Raton (2009)

    Book  Google Scholar 

Download references

Acknowledgments

The authors are indebted to Yves Caraglio for useful comments on modelling the Aleppo pines dataset and for providing this dataset.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jean-Baptiste Durand.

Appendices

Appendix 1: Computation of the global entropy of state subtrees in hidden Markov tree models

Proposition 2

Let \({\mathcal {V}}\) be a subtree of \({\mathcal {T}}\) with root vertex \(r\). Then for any possible value \({\bar{\varvec{s}}}_{\mathcal {V}}\) of \(\bar{\varvec{S}}_{\mathcal {V}}\) and for any \({\varvec{x}}\),

$$\begin{aligned}&P\left( \bar{\varvec{S}}_{\mathcal {V}}= {\bar{\varvec{s}}}_{\mathcal {V}}| {\varvec{X}}={\varvec{x}}\right) \\&\quad = P(S_r=s_r| {\varvec{X}}={\varvec{x}})\\&\quad {\mathop {\mathop {\prod }\limits _{u \in {\mathcal {V}}}}\limits _{u \ne r}} P\big (S_u=s_u|S_{\rho (u)}=s_{\rho (u)}, {\varvec{X}}={\varvec{x}}\big ) \end{aligned}$$

and

$$\begin{aligned} H(\bar{\varvec{S}}_{\mathcal {V}}| {\varvec{X}}={\varvec{x}}) = H(S_r | {\varvec{X}}={\varvec{x}}) + \mathop {\mathop {\sum }\limits _{u \in {\mathcal {V}}}}\limits _{u \ne r} H(S_u|S_{\rho (u)},{\varvec{X}}={\varvec{x}}). \end{aligned}$$

Proof

This is proved by induction on the vertices in \({\mathcal {V}}\). The induction step is as follows: let \(\ell \) be a leaf vertex of \({\mathcal {V}}\). Then for any possible value \({\bar{\varvec{s}}}_{\mathcal {V}}\) of \(\bar{\varvec{S}}_{\mathcal {V}}\),

$$\begin{aligned}&P(\bar{\varvec{S}}_{\mathcal {V}}= {\bar{\varvec{s}}}_{\mathcal {V}}| {\varvec{X}}={\varvec{x}}) \\&\qquad \quad = P(S_{\ell }=s_{\ell }|\bar{\varvec{S}}_{{\mathcal {V}}\backslash \{\ell \}} = {\bar{\varvec{s}}}_{{\mathcal {V}}\backslash \{\ell \}}, {\varvec{X}}={\varvec{x}})\\&\qquad \qquad \quad P(\bar{\varvec{S}}_{{\mathcal {V}}\backslash \{\ell \}} = {\bar{\varvec{s}}}_{{\mathcal {V}}\backslash \{\ell \}} | {\varvec{X}}={\varvec{x}}) \\&\qquad \quad = P(S_{\ell }=s_{\ell }|S_{\rho (\ell )} = s_{\rho (\ell )}, {\varvec{X}}={\varvec{x}}) \\&\qquad \qquad \quad P(\bar{\varvec{S}}_{{\mathcal {V}}\backslash \{\ell \}} = {\bar{\varvec{s}}}_{{\mathcal {V}}\backslash \{\ell \}} | {\varvec{X}}={\varvec{x}}) \end{aligned}$$

since \(S_{\ell }\) is conditionally independent from the other vertices in \({\mathcal {V}}\) given \(S_{\rho (\ell )}\) and \({\varvec{X}}\).

The induction step is completed by observing that \({\mathcal {V}}\backslash \{\ell \}\) is a subtree of \({\mathcal {T}}\).

The decomposition of the entropy of \(\bar{\varvec{S}}_{\mathcal {V}}\) yielded by the chain rule

$$\begin{aligned} H(\bar{\varvec{S}}_{\mathcal {V}}| {\varvec{X}}={\varvec{x}}) = H(S_r | {\varvec{X}}={\varvec{x}}) + {\mathop {\mathop {\sum }\limits _{u \in {\mathcal {V}}}}\limits _{u \ne r}} H(S_u|S_{\rho (u)}, {\varvec{X}}={\varvec{x}}) \end{aligned}$$

is proved similarly as Corollary 1. \(\square \)

Appendix 2: Direct computation of global state tree entropy in hidden Markov tree models

Direct computation of global state tree entropy is based on recursive computation of the entropies of children state subtrees given each state. This recursion relies on conditional independence properties between hidden and observed variables in HMT models, and particularly the following relations: for any internal, non-root vertex \(u\) and fo \(j = 1,\ldots ,J\),

$$\begin{aligned} P(\bar{\varvec{S}}_{c(u)}&= {\bar{\varvec{s}}}_{c(u)} | S_u=j, \bar{\varvec{S}}_{0 \backslash u} = {\bar{\varvec{s}}}_{0 \backslash u}, {\varvec{X}}={\varvec{x}}) \nonumber \\&= P(\bar{\varvec{S}}_{c(u)} = {\bar{\varvec{s}}}_{c(u)} | S_u=j, S_{\rho (u)} = s_{\rho (u)} ,{\varvec{X}}={\varvec{x}}) \nonumber \\&= P(\bar{\varvec{S}}_{c(u)} = {\bar{\varvec{s}}}_{c(u)} | S_u=j, {\varvec{X}}={\varvec{x}}) \nonumber \\&= \prod \limits _{v \in c(u)} P(\bar{\varvec{S}}_v = {\bar{\varvec{s}}}_v | S_u=j, {\varvec{X}}={\varvec{x}}) \nonumber \\&= \prod \limits _{v \in c(u)} P(\bar{\varvec{S}}_v = {\bar{\varvec{s}}}_v | S_u=j, \bar{\varvec{X}}_v=\bar{\varvec{x}}_v) \end{aligned}$$
(19)
$$\begin{aligned}&= P(\bar{\varvec{S}}_{c(u)} = {\bar{\varvec{s}}}_{c(u)} | S_u=j, \bar{\varvec{X}}_u=\bar{\varvec{x}}_u), \end{aligned}$$
(20)
$$\begin{aligned} P(\bar{\varvec{S}}_u&= {\bar{\varvec{s}}}_u | \bar{\varvec{S}}_{0 \backslash u} = {\bar{\varvec{s}}}_{0 \backslash u}, {\varvec{X}}={\varvec{x}}) \\&= P(\bar{\varvec{S}}_u = {\bar{\varvec{s}}}_u | S_{\rho (u)}= s_{\rho (u)} , {\varvec{X}}={\varvec{x}})\\&= P(\bar{\varvec{S}}_u = {\bar{\varvec{s}}}_u | S_{\rho (u)} = s_{\rho (u)}, \bar{\varvec{X}}_u=\bar{\varvec{x}}_u). \end{aligned}$$

Entropies \(H(\bar{\varvec{S}}_{c(u)} | S_{u}=j, \bar{\varvec{X}}_u=\bar{\varvec{x}}_u)\) can be computed for any \(u \in {\mathcal {U}}, u \ne 0\) and for \(j=0,\ldots ,J-1\), by an upward algorithm initialised at the leaf vertices \(u\) by

$$\begin{aligned} H(\bar{\varvec{S}}_{c(u)}|S_{u}=j,\bar{\varvec{X}}_u=\bar{\varvec{x}}_u) = 0. \end{aligned}$$

As a consequence from (20), we have for any state \(j\), \(H(\bar{\varvec{S}}_{c(u)}|S_{u}=j,\bar{\varvec{X}}_u=\bar{\varvec{x}}_u) = H(\bar{\varvec{S}}_{c(u)}|S_{u}=j,{\varvec{X}}={\varvec{x}})\). Thus, it is deduced from (19) that

$$\begin{aligned} H(\bar{\varvec{S}}_{c(u)}|S_{u}&=j,\bar{\varvec{X}}_u=\bar{\varvec{x}}_u)\nonumber \\&= H(\bar{\varvec{S}}_{c(u)}|S_{u}=j,\bar{\varvec{X}}_{c(u)}=\bar{\varvec{x}}_{c(u)}) \nonumber \\&= \sum _{v \in c(u)}H(\bar{\varvec{S}}_{v}|S_{u}=j,\bar{\varvec{X}}_v=\bar{\varvec{x}}_v). \end{aligned}$$
(21)

Moreover, for any \(v \in c(u)\) with \(c(v) \ne \emptyset \) and for \(j=0,\ldots ,J-1\),

$$\begin{aligned}&H(\bar{\varvec{S}}_{v}|S_{u}=j,\bar{\varvec{X}}_u=\bar{\varvec{x}}_u) \nonumber \\&\quad = -\sum _{{\bar{\varvec{s}}}_{c(v)}, s_v} P(\bar{\varvec{S}}_{c(v)}={\varvec{s}}_{c(v)}, S_v=s_v | S_u=j,\bar{\varvec{X}}_u=\bar{\varvec{x}}_u) \nonumber \\&\quad \quad \times \log P(\bar{\varvec{S}}_{c(v)}={\varvec{s}}_{c(v)}, S_v=s_v | S_u=j,\bar{\varvec{X}}_u=\bar{\varvec{x}}_u) \nonumber \\&\quad = -\sum _{{\varvec{s}}_{c(v)}} \sum \limits _{k=0}^{J-1} P\left( \bar{\varvec{S}}_{c(v)}={\varvec{s}}_{c(v)} | S_v = k, S_u=j, \bar{\varvec{X}}_u=\bar{\varvec{x}}_u \right) \nonumber \\&\ \quad \qquad P(S_v = k | S_u=j, \bar{\varvec{X}}_u=\bar{\varvec{x}}_u) \nonumber \\&\qquad \quad \times \{ \log P(\bar{\varvec{S}}_{c(v)}={\varvec{s}}_{c(v)} | S_v = k, S_u=j, \bar{\varvec{X}}_u=\bar{\varvec{x}}_u)\nonumber \\&\qquad \qquad \, \qquad + \log P(S_v = k | S_u=j, \bar{\varvec{X}}_u=\bar{\varvec{x}}_u) \} \nonumber \\&\qquad = -\sum \limits _{k=0}^{J-1} P(S_v = k | S_u=j, \bar{\varvec{X}}_v=\bar{\varvec{x}}_v)\nonumber \\&\qquad \quad \times \left\{ \sum _{{\varvec{s}}_{c(v)}} P(\bar{\varvec{S}}_{c(v)}={\varvec{s}}_{c(v)} | S_v = k, \bar{\varvec{X}}_v=\bar{\varvec{x}}_v) \right. \nonumber \\&\quad \qquad \times \log P(\bar{\varvec{S}}_{c(v)}={\varvec{s}}_{c(v)} | S_v = k, \bar{\varvec{X}}_v=\bar{\varvec{x}}_v)\nonumber \\&\qquad \quad \left. + \log P(S_v = k | S_u=j, \bar{\varvec{X}}_v=\bar{\varvec{x}}_v) \right\} \nonumber \\&\quad = \sum \limits _{k=0}^{J-1} P(S_v = k | S_u=j, \bar{\varvec{X}}_v=\bar{\varvec{x}}_v) \nonumber \\&\qquad \qquad \qquad \big \{ H(\bar{\varvec{S}}_{c(v)} | S_v= k, \bar{\varvec{X}}_v=\bar{\varvec{x}}_v)\nonumber \\&\qquad \qquad \qquad - \log P(S_v = k | S_u=j, \bar{\varvec{X}}_v=\bar{\varvec{x}}_v) \big \}. \end{aligned}$$
(22)

Thus, the recursion of the upward algorithm is given by

$$\begin{aligned}&H(\bar{\varvec{S}}_{c(u)}|S_{u}=j,\bar{\varvec{X}}_u=\bar{\varvec{x}}_u)\nonumber \\&\quad = \sum _{v \in c(u)} \left\{ \sum _{s_v} P(S_v=s_v | S_u=j, \bar{\varvec{X}}_v=\bar{\varvec{x}}_v)\right. \nonumber \\&\qquad \qquad \qquad \qquad \times \left. \left[ H(\bar{\varvec{S}}_{c(v)}|S_v=s_v, \bar{\varvec{X}}_v=\bar{\varvec{x}}_v) \right. \right. \nonumber \\&\qquad \qquad \qquad \qquad \left. \left. - \log P(S_v=s_v | S_u=j, \bar{\varvec{X}}_v=\bar{\varvec{x}}_v) \right] \right\} , \end{aligned}$$
(23)

where \(P(S_v=k | S_u=j, \bar{\varvec{X}}_v=\bar{\varvec{x}}_v) = P(S_v=k | S_u=j, {\varvec{X}}={\varvec{x}})\) is given by equation (17).

The termination step is obtained by similar arguments as equation (21):

$$\begin{aligned} H({\varvec{S}}| {\varvec{X}}= {\varvec{x}})&= H(\bar{\varvec{S}}_{c(0)} | S_0, {\varvec{X}}= {\varvec{x}}) + H(S_0 | {\varvec{X}}= {\varvec{x}}) \\&= \sum \limits _{j=0}^{J-1} \beta _{0}\left( j\right) \left\{ H\left( \bar{\varvec{S}}_{c\left( 0\right) }|S_{0}=j,{\varvec{X}}= {\varvec{x}}\right) \right. \\&\qquad \qquad \qquad \quad \,\,\, \left. -\log \beta _{0}\left( j\right) \right\} . \end{aligned}$$

Using similar arguments as in (22), the partial state tree entropy \(H(\bar{\varvec{S}}_u | {\varvec{X}}= {\varvec{x}})\) can be deduced from the conditional entropies \(H(\bar{\varvec{S}}_{c(u)}|S_{u}=j,\bar{\varvec{X}}_u=\bar{\varvec{x}}_u)\) (with \(j=0,\ldots ,J-1\)) as follows:

$$\begin{aligned} H(\bar{\varvec{S}}_u | {\varvec{X}}= {\varvec{x}})&= H(\bar{\varvec{S}}_{c(u)} | S_u, {\varvec{X}}= {\varvec{x}}) + H(S_u | {\varvec{X}}= {\varvec{x}}) \nonumber \\&= \sum _j L_{u}(j) \left\{ H\left( \bar{\varvec{S}}_{c\left( u\right) }|S_{u}=j,{\varvec{X}}= {\varvec{x}}\right) \right. \nonumber \\&\quad \qquad \qquad \qquad \left. -\log L_{u}\left( j\right) \right\} \nonumber \\&= \sum _j L_{u}(j) \left\{ H\left( \bar{\varvec{S}}_{c\left( u\right) }|S_{u}=j,\bar{\varvec{X}}_u= \bar{\varvec{x}}_u \right) \right. \nonumber \\&\quad \qquad \qquad \qquad \left. -\log L_{u}\left( j\right) \right\} , \end{aligned}$$
(24)

where the \(\left\{ L_u(j)\right\} _{j=0,\ldots ,J-1}\) are directly extracted from the downward recursion (14).

The profile of conditional entropies \(H(S_{u} | S_{\rho (u)}, {\varvec{X}}= {\varvec{x}})\) is deduced from

$$\begin{aligned}&H\left( \bar{\varvec{S}}_{\rho (u)} | {\varvec{X}}= {\varvec{x}}\right) = H\left( S_{\rho (u)}, \bar{\varvec{S}}_{b(u)}, \bar{\varvec{S}}_u | {\varvec{X}}= {\varvec{x}}\right) \\&\quad \quad \quad \qquad \qquad \qquad = H(S_{\rho (u)} | {\varvec{X}}= {\varvec{x}}) + H\left( \bar{\varvec{S}}_{b(u)} | S_{\rho (u)}, {\varvec{X}}= {\varvec{x}})\right. \\&\qquad \qquad \qquad \qquad \qquad \left. + H(\bar{\varvec{S}}_{u} |S_{\rho (u)}, {\varvec{X}}= {\varvec{x}}\right) , \end{aligned}$$

where

$$\begin{aligned} H(\bar{\varvec{S}}_{b(u)} | S_{\rho (u)}, {\varvec{X}}= {\varvec{x}})&= \sum _{v \in b(u)} H(\bar{\varvec{S}}_v | S_{\rho (v)}, {\varvec{X}}= {\varvec{x}}) \\&= \sum _{v \in b(u)} \!\left\{ \!\sum _j L_{\rho (v)}(j) H(\bar{\varvec{S}}_v | S_{\rho (v)}=j, {\varvec{X}}= {\varvec{x}})\! \right\} \!, \end{aligned}$$

and where for any brother vertex \(v\) of \(u\), \(H(\bar{\varvec{S}}_v | S_{\rho (v)}=j, {\varvec{X}}= {\varvec{x}})\) is given by (22). Since

$$\begin{aligned}&H(\bar{\varvec{S}}_{v} | S_{\rho (v)}, {\varvec{X}}= {\varvec{x}})\\&\quad = \sum _j H(\bar{\varvec{S}}_{v} | S_{\rho (v)}=j, {\varvec{X}}= {\varvec{x}}) L_{\rho (v)}(j) \end{aligned}$$

and since

$$\begin{aligned}&H\left( \bar{\varvec{S}}_u | S_{\rho (u)}, {\varvec{X}}= {\varvec{x}}\right) \nonumber \\&\qquad \qquad \qquad \qquad \!=\! H\left( S_u | S_{\rho (u)}, {\varvec{X}}\!=\! {\varvec{x}}\right) \!+\! H\left( \bar{\varvec{S}}_c(u) | S_u, {\varvec{X}}\!=\! {\varvec{x}}\right) \nonumber \\&\qquad \qquad \qquad \qquad = H\left( S_u | S_{\rho (u)}, {\varvec{X}}= {\varvec{x}}\right) \nonumber \\&\qquad \qquad \qquad \qquad + \sum _j L_u(j) H\left( \bar{\varvec{S}}_{c(u)} | S_u = j, {\varvec{X}}= {\varvec{x}}\right) , \end{aligned}$$
(25)

\(H(S_{u} | S_{\rho (u)}, {\varvec{X}}= {\varvec{x}})\) is directly extracted from the partial state entropies \(H(\bar{\varvec{S}}_{c(u)} | S_u=j, {\varvec{X}}\!=\! {\varvec{x}})\) and \(H(\bar{\varvec{S}}_{\rho (u)} | {\varvec{X}}\!=\! {\varvec{x}})\) and from the marginal entropy \(H(S_{\rho (u)} | {\varvec{X}}= {\varvec{x}})\).

In summary, the partial subtrees entropies \(\big \{H(\bar{\varvec{S}}_{c(u)}|S_{u}= j,\bar{\varvec{X}}_u=\bar{\varvec{x}}_u)\big \}_{u \in {\mathcal {U}};}\) \({}_{j=0,\ldots ,J-1}\) are firstly computed using (23). The partial state tree entropies \(\left\{ H(\bar{\varvec{S}}_u | {\varvec{X}}= \right. \left. {\varvec{x}})\right\} _{u \in {\mathcal {U}}}\) and then the profile of conditional entropies \(\left\{ H(S_{u} | S_{\rho (u)}, {\varvec{X}}= {\varvec{x}})\right\} _{u \in {\mathcal {U}}}\) are deduced from these entropies and the posterior state probabilities, using (24) and (25). The time complexity of the algorithm is in \({\mathcal {O}}(J^2n)\).

Appendix 3: Application of HMT model to Aleppo pines: path containing female shoots

A path containing a female shoot is considered. This path corresponds to the main axis of the third individual (for which \(H({\varvec{S}}|{\varvec{X}}={\varvec{x}}) = 29.6\)). The path contains 6 vertices, referred to as \(\{0, \ldots , 5\}\). The female shoot is at vertex 2, and vertex 3 is a bicyclic shoot. Shoots 4 and 5 are unbranched, monocyclic, sterile shoots.

The contribution of the vertices of the considered path \({\mathcal {P}}\) to the global state tree is equal to 0.48 (that is, 0.08 per vertex on average). The global state tree entropy for this individual is 0.21 per vertex, against 0.20 per vertex in the whole dataset. The mean marginal state entropy for this individual is 0.37 per vertex, which strongly overestimates the mean state tree entropy.

Since a female shoot necessarily is in state 0, \(H(S_2 | {\varvec{X}}= {\varvec{x}})=0\) (no uncertainty). The states of shoots 0 and 1 can be deduced from \(S_2\) using the transition matrix \({\hat{P}}\), thus their marginal entropy is null. Since shoot 3 is bicyclic, it is in state 0 with a very high probability \((H(S_3 | {\varvec{X}}= {\varvec{x}}) \approx 0)\). Uncertainty remains concerning the states of shoots 4 and 5, which thus have high marginal entropies. However, \(S_5\) can be deduced from \(S_4\) using \({\hat{P}}\) and inversely, which results into high mutual information between \(S_4\) and \(S_5\) given \({\varvec{X}}= {\varvec{x}}\). This is illustrated by conditional and marginal entropy profiles in Fig. 9.

Fig. 9
figure 9

Path containing a female shoot: profiles of conditional and of marginal entropies

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Durand, JB., Guédon, Y. Localizing the latent structure canonical uncertainty: entropy profiles for hidden Markov models. Stat Comput 26, 549–567 (2016). https://doi.org/10.1007/s11222-014-9494-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11222-014-9494-9

Keywords

Navigation