Context Trees, Variable Length Markov Chains and Dynamical Sources

  • Peggy CénacEmail author
  • Brigitte Chauvin
  • Frédéric Paccaut
  • Nicolas Pouyanne
Part of the Lecture Notes in Mathematics book series (LNM, volume 2046)


Infinite random sequences of letters can be viewed as stochastic chains or as strings produced by a source, in the sense of information theory. The relationship between Variable Length Markov Chains (VLMC) and probabilistic dynamical sources is studied. We establish a probabilistic frame for context trees and VLMC and we prove that any VLMC is a dynamical source for which we explicitly build the mapping. On two examples, the “comb” and the “bamboo blossom”, we find a necessary and sufficient condition for the existence and the uniqueness of a stationary probability measure for the VLMC. These two examples are detailed in order to provide the associated Dirichlet series as well as the generating functions of word occurrences.


Variable length Markov chains Dynamical systems of the interval Dirichlet series Occurrences of words Probabilistic dynamical sources 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.



We are very grateful to Antonio Galves, who introduced us to the challenging VLMC topics. We warmly thank Brigitte Vallée for valuable and stormy discussions.


  1. 1.
    M. Abadi, A. Galves, Inequalities for the occurrence times of rare events in mixing processes. The state of the art. Markov Proc. Relat. Field. 7(1), 97–112 (2001)Google Scholar
  2. 2.
    M. Abadi, B. Saussol, Stochastic processes and their applications. 121(2), 314–323Google Scholar
  3. 3.
    P. Billingsley, Probability and Measure, 3rd edn. Wiley Series in Probability and Mathematical Statistics (Wiley, New York, 1995)Google Scholar
  4. 4.
    G. Blom, D. Thorburn, How many random digits are required until given sequences are obtained? J. Appl. Probab. 19, 518–531 (1982)Google Scholar
  5. 5.
    P. Bühlmann, A.J. Wyner, Variable length markov chains. Ann. Statist. 27(2), 480–513 (1999)Google Scholar
  6. 6.
    J. Clément, P. Flajolet, B. Vallee, Dynamical sources in information theory: Analysis of general tries. Algorithmica 29, 307–369 (2001)Google Scholar
  7. 7.
    F. Comets, R. Fernandez, P. Ferrari, Processes with long memory: Regenerative construction and perfect simulation. Ann. Appl. Prob. 12(3), 921–943 (2002)Google Scholar
  8. 8.
    P. Flajolet, M. Roux, B. Vallée, Digital trees and memoryless sources: From arithmetics to analysis. DMTCS Proc. AM, 233–260 (2010)Google Scholar
  9. 9.
    J.C. Fu, Bounds for reliability of large consecutive-k-out-of-n : f system. IEEE Trans. Reliab. 35, 316–319 (1986)Google Scholar
  10. 10.
    J.C. Fu, M.V. Koutras, Distribution theory of runs: A markov chain approach. J. Amer. Statist. Soc. 89, 1050–1058 (1994)Google Scholar
  11. 11.
    S. Gallo, N. Garcia, Perfect simulation for stochastic chains of infinite memory: Relaxing the continuity assumptions. pp. 1–20 (2010) [arXiv:1105.5459v1]Google Scholar
  12. 12.
    A. Galves, E. Löcherbach, Stochastic chains with memory of variable length. TICSP Series 38, 117–133 (2008)Google Scholar
  13. 13.
    H. Gerber, S. Li, The occurrence of sequence patterns in repeated experiments and hitting times in a markov chain. Stoch. Process. Their Appl. 11, 101–108 (1981)Google Scholar
  14. 14.
    T.E. Harris, On chains of infinite order. Pac. J. Math. 5, 707–724 (1955)Google Scholar
  15. 15.
    P. Jacquet, W. Szpankowski, Autocorrelation on words and its applications. Analysis of suffix trees by string-ruler approach. J. Combin. Theor. A.66, 237–269 (1994)Google Scholar
  16. 16.
    M.V. Koutras, Waiting Times and Number of Appearances of Events in a Sequence of Discrete Random Variables. Advances in Combinatorial Methods and Applications to Probability and Statistics, Stat. Ind. Technol., (Birkhäuser Boston, Boston, 1997), pp. 363–384Google Scholar
  17. 17.
    A. Lambert, S. Siboni, S. Vaienti, Statistical properties of a nonuniformly hyperbolic map of the interval. J. Stat. Phys. 72(5/6), 1305–1330 (1993)Google Scholar
  18. 18.
    S.-Y.R. Li, A martingale approach to the study of occurrence of sequence patterns in repeated experiments. Ann. Probab. 8(6): 1171–1176 (1980)Google Scholar
  19. 19.
    V. Pozdnyakov, J. Glaz, M. Kulldorff, J.M. Steele, A martingale approach to scan statistics. Ann. Inst. Statist. Math. 57(1), 21–37 (2005)Google Scholar
  20. 20.
    M. Régnier, A unified approach to word occurrence probabilities. Discrete Appl. Math. 104, 259–280 (2000)Google Scholar
  21. 21.
    G. Reinert, S. Schbath, M.S. Waterman, Probabilistic and statistical properties of words: An overview. J. Comput. Biol. 7(1/2), 1–46 (2000)Google Scholar
  22. 22.
    D. Revuz, Markov Chains. (North-Holland Mathematical Library, Amsterdam, 1984)Google Scholar
  23. 23.
    J. Rissanen, A universal data compression system. IEEE Trans. Inform. Theor. 29(5), 656–664 (1983)Google Scholar
  24. 24.
    S. Robin, J.J. Daudin, Exact distribution of word occurrences in a random sequence of letters. J. Appl. Prob. 36, 179–193 (1999)Google Scholar
  25. 25.
    V. Stefanov, A.G. Pakes, Explicit distributional results in pattern formation. Ann. Appl. Probab. 7, 666–678 (1997)Google Scholar
  26. 26.
    W. Szpankowski, A generalized suffix tree and its (un)expected asymptotic behaviors. SIAM J. Comput. 22(6), 1176–1198 (1993)Google Scholar
  27. 27.
    X-J. Wang, Statistical physics of temporal intermittency. Phys. Rev. A 40(11), 6647–6661 (1989)Google Scholar
  28. 28.
    D. Williams, Probability with Martingales. Cambridge Mathematical Textbooks (Cambridge University Press, Cambridge, 1991)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Peggy Cénac
    • 1
    Email author
  • Brigitte Chauvin
    • 2
    • 3
  • Frédéric Paccaut
    • 4
  • Nicolas Pouyanne
    • 3
  1. 1.Institut de Mathématiques de Bourgogne IMB UMR 5584 CNRSUniversité de BourgogneDIJON CEDEX, BourgogneFrance
  2. 2.INRIA Rocquencourtproject AlgorithmsLe Chesnay CEDEXFrance
  3. 3.Laboratoire de Mathématiques de Versailles, CNRS, UMR 8100Université de Versailles - St-QuentinVersailles CEDEXFrance
  4. 4.LAMFA, CNRS, UMR 6140Université de Picardie Jules VerneAmiensFrance

Personalised recommendations