Statistics and Computing

, Volume 26, Issue 1–2, pp 549–567 | Cite as

Localizing the latent structure canonical uncertainty: entropy profiles for hidden Markov models

  • Jean-Baptiste Durand
  • Yann Guédon


This paper addresses state inference for hidden Markov models. These models rely on unobserved states, which often have a meaningful interpretation. This makes it necessary to develop diagnostic tools for quantification of state uncertainty. The entropy of the state sequence that explains an observed sequence for a given hidden Markov chain model can be considered as the canonical measure of state sequence uncertainty. This canonical measure of state sequence uncertainty is not reflected by the classic multidimensional posterior state (or smoothed) probability profiles because of the marginalization that is intrinsic in the computation of these posterior probabilities. Here, we introduce a new type of profiles that have the following properties: (i) these profiles of conditional entropies are a decomposition of the canonical measure of state sequence uncertainty along the sequence and makes it possible to localise this uncertainty, (ii) these profiles are unidimensional and thus remain easily interpretable on tree structures. We show how to extend the smoothing algorithms for hidden Markov chain and tree models to compute these entropy profiles efficiently. The use of entropy profiles is illustrated by sequence and tree data examples.


Conditional entropy Hidden Markov chain model Hidden Markov tree model Plant structure analysis 



The authors are indebted to Yves Caraglio for useful comments on modelling the Aleppo pines dataset and for providing this dataset.


  1. Brushe, G., Mahony, R., Moore, J.: A soft output hybrid algorithm for ML/MAP sequence estimation. IEEE Trans. Inf. Theory 44(7), 3129–3134 (1998)zbMATHMathSciNetCrossRefGoogle Scholar
  2. Cappé, O., Moulines, E., Rydén, T.: Inference in Hidden Markov Models. Springer Series in Statistics. Springer, New York (2005)Google Scholar
  3. Celeux, G., Soromenho, G.: An entropy criterion for assessing the number of clusters in a mixture model. J. Classif. 13(2), 195–212 (1996)zbMATHMathSciNetCrossRefGoogle Scholar
  4. Cover, T., Thomas, J.: Elements of Information Theory, 2nd edn. Wiley, Hoboken (2006)zbMATHGoogle Scholar
  5. Crouse, M., Nowak, R., Baraniuk, R.: Wavelet-based statistical signal processing using hidden Markov models. IEEE Trans. Signal Process. 46(4), 886–902 (1998)MathSciNetCrossRefGoogle Scholar
  6. Devijver, P.A.: Baum’s forward-backward algorithm revisited. Pattern Recognit. Lett. 3, 369–373 (1985)zbMATHCrossRefGoogle Scholar
  7. Durand, J.-B., Girard, S., Ciriza, V., Donini, L.: Optimization of power consumption and device availability based on point process modelling of the request sequence. Appl. Stat. 62(2), 151–162 (2013)MathSciNetGoogle Scholar
  8. Durand, J.-B., Gonçalvès, P., Guédon, Y.: Computational methods for hidden Markov tree models: an application to wavelet trees. IEEE Trans. Signal Process. 52(9), 2551–2560 (2004)MathSciNetCrossRefGoogle Scholar
  9. Durand, J.-B., Guédon, Y., Caraglio, Y., Costes, E.: Analysis of the plant architecture via tree-structured statistical models: the hidden Markov tree models. New Phytol. 166(3), 813–825 (2005)CrossRefGoogle Scholar
  10. Durand, J.-B., Guédon, Y.: Localizing the latent structure canonical uncertainty: entropy profiles for hidden Markov odels. Available:, Inria technical report (2012)Google Scholar
  11. Ephraim, Y., Merhav, N.: Hidden Markov processes. IEEE Trans. Inf. Theory 48, 1518–1569 (2002)zbMATHMathSciNetCrossRefGoogle Scholar
  12. Guédon, Y., Caraglio, Y., Heuret, P., Lebarbier, E., Meredieu, C.: Analyzing growth components in trees. J. Theor. Biol. 248(3), 418–447 (2007a)CrossRefGoogle Scholar
  13. Guédon, Y.: Exploring the state sequence space for hidden Markov and semi-Markov chains. Comput. Stat. Data Anal. 51(5), 2379–2409 (2007b)Google Scholar
  14. Guédon, Y.: Segmentation uncertainty in multiple change-point models. Stat. Comput. (2013, in press)Google Scholar
  15. Hernando, D., Crespi, V., Cybenko, G.: Efficient computation of the hidden Markov model entropy for a given observation sequence. IEEE Trans. Inf. Theory 51(7), 2681–2685 (2005) Google Scholar
  16. Lauritzen, S.: Graphical Models. Clarendon Press, Oxford (1996)Google Scholar
  17. Le Cadre, J.-P., Tremois, O.: Bearing-only tracking for maneuvering sources. IEEE Trans. Aerosp. Electron. Syst. 34(1), 179–193 (1998)Google Scholar
  18. McLachlan, G., Peel, D.: Finite Mixture Models. Wiley Series in Probability and Statistics. Wiley, New York (2000)CrossRefGoogle Scholar
  19. Zucchini, W., MacDonald, I.: Hidden Markov Models for Time Series: An Introduction Using R. Chapman & Hall/CRC, Boca Raton (2009)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  1. 1.Laboratoire Jean Kuntzmann and Inria, MistisUniv. Grenoble AlpesGrenoble Cedex 9France
  2. 2.CIRAD, UMR AGAP and Inria, Virtual PlantsMontpellierFrance

Personalised recommendations