, Volume 16, Issue 2, pp 148–172 | Cite as

Markov constraints: steerable generation of Markov sequences

  • François PachetEmail author
  • Pierre Roy
Open Access


Markov chains are a well known tool to model temporal properties of many phenomena, from text structure to fluctuations in economics. Because they are easy to generate, Markovian sequences, i.e. temporal sequences having the Markov property, are also used for content generation applications such as text or music generation that imitate a given style. However, Markov sequences are traditionally generated using greedy, left-to-right algorithms. While this approach is computationally cheap, it is fundamentally unsuited for interactive control. This paper addresses the issue of generating steerable Markovian sequences. We target interactive applications such as games, in which users want to control, through simple input devices, the way the system generates a Markovian sequence, such as a text, a musical sequence or a drawing. To this aim, we propose to revisit Markov sequence generation as a branch and bound constraint satisfaction problem (CSP). We propose a CSP formulation of the basic Markovian hypothesis as elementary Markov Constraints (EMC). We propose algorithms that achieve domain-consistency for the propagators of EMCs, in an event-based implementation of CSP. We show how EMCs can be combined to estimate the global Markovian probability of a whole sequence, and accommodate for different species of Markov generation such as fixed order, variable-order, or smoothing. Such a formulation, although more costly than traditional greedy generation algorithms, yields the immense advantage of being naturally steerable, since control specifications can be represented by arbitrary additional constraints, without any modification of the generation algorithm. We illustrate our approach on simple yet combinatorial chord sequence and melody generation problems and give some performance results.


CSP Markov chains Sequence constraints Global constraints Interactive applications 


  1. 1.
    Addessi, A.-R., & Pachet, F. (2005). Experiments with a musical machine: Musical style replication in 3/5 year old children. British Journal of Music Education, 22(1), 21–46.CrossRefGoogle Scholar
  2. 2.
    Allauzen, C., Crochemore, M., & Raffinot, M. (1999). Factor oracle: A new structure for pattern matching. In J. Pavelka, G. Tel, & M. Bartosek (Eds.), Proceedings of the 26th conference on current trends in theory and practice of informatics (November 27–Dec. 4), LNCS (Vol. 1725, pp. 295–310). London: Springer.Google Scholar
  3. 3.
    Anders, T., & Miranda, E. R. (2008). Constraint-based composition in real time. In Proc. of int. computer music conference, Belfast, UK.Google Scholar
  4. 4.
    Anders, T., & Miranda, E. R. (2010). Constraint programming systems for modeling music theories and composition. ACM Computing Surveys (in press).Google Scholar
  5. 5.
    Assayag, G., Dubnov, S., & Delerue, O. (1999). Guessing the composer’s mind: Applying universal prediction to musical style. In Proc. of int. computer music conference, Bejing, China, ICMA.Google Scholar
  6. 6.
    Bejerano, G. (2004). Algorithms for variable length Markov chain modeling. Bioinformatics, 20(5), 788–789.CrossRefGoogle Scholar
  7. 7.
    Beldiceanu, N., Carlsson, M., & Demassey, S. (2007). Global constraint catalogue: Past, present and future. Constraints, 12(1), 32–62MathSciNetGoogle Scholar
  8. 8.
    Brooks, F. P., Hopkins, A. L., Neumann, P. G., & Wright, W. V. (1993). An experiment in musical composition. In S. M. Schwanauer & D. A. Levitt (Eds.), Machine models of music (pp. 23–40). Cambridge: MIT.Google Scholar
  9. 9.
    Conklin, D. (2003). Music generation from statistical models. In Proc. of AISB 2003 symposium on artificial intelligence and creativity in the arts and sciences (pp. 30–35).Google Scholar
  10. 10.
    Conklin, D., & Witten, I. H. (1995). Multiple viewpoint systems for music prediction. Journal of New Music Research, 24(1), 51–73.CrossRefGoogle Scholar
  11. 11.
    Cont, A., Dubnov, S., & Assayag, G. (2007). Anticipatory model of musical style imitation using collaborative and competitive reinforcement learning. In Anticipatory behavior in adaptive learning systems. LNCS (Vol. 4520/2007, pp. 285–306). Berlin: Springer.CrossRefGoogle Scholar
  12. 12.
    Demassey, S., Pesant, G., & Rousseau, L.-M. (2006). A cost-regular based hybrid column generation approach. Constraints, 11(4), 315–333.CrossRefzbMATHMathSciNetGoogle Scholar
  13. 13.
    Di Meola, A., McLaughlin, J., & De Lucia, P. (1992). Friday night in San Francisco. Milwaukee: Hal Leonard Corporation.Google Scholar
  14. 14.
    Eigenfield, A., & Pasquier, P. (2010). Realtime generation of harmonic progressions using controlled markov selection. In Proc. of the 1st international conference on computational creativity (ICCCX) (pp. 16–25). Lisbon: ACM.Google Scholar
  15. 15.
    Farbood, M., & Schoner, B. (2001). Analysis and synthesis of palestrina-style counterpoint using Markov chains. In Proc. of international computer music conference, Cuba (pp. 471–474).Google Scholar
  16. 16.
  17. 17.
    Johnson, D. S. (1974). Approximation algorithms for combinatorial problems. Journal of Computer and System Sciences, 9(3), 256–278.CrossRefzbMATHMathSciNetGoogle Scholar
  18. 18.
    Marinescu, R., & Dechter, R. (2005). Advances in AND/OR branch-and-bound search for constraint optimization. In The 7th international workshop on preferences and soft constraints of the eleventh international conference on principles and practice of constraint programming, CP’2005.Google Scholar
  19. 19.
    Markov Constraints (2010). Available online at Accessed September 2010.
  20. 20.
    Menana, J., & Demassey, S. (2009). Sequencing and counting with the multicost-regular constraint. In Lecture notes in computer science, 6th international conference on integration of AI and OR techniques in constraint programming for combinatorial optimization problems (CPAIOR’09), Pittsburgh, USA.Google Scholar
  21. 21.
    Nierhaus, G. (2009). Algorithmic composition—paradigms of automated music generation. Berlin: Springer.zbMATHGoogle Scholar
  22. 22.
    Pachet, F. (2002). The continuator: Musical interaction with style. In Proc. of international computer music conference (pp. 211–218). Goteborg, Sweden. September, Best Paper Award.Google Scholar
  23. 23.
    Pachet, F., Roy, P., & Cazaly, D. (2000). A combinatorial approach to content-based music selection. IEEE Multimedia, 7(1), 44–51.CrossRefGoogle Scholar
  24. 24.
    Parker, C. (2001). Charlie Parker omnibook: For C instruments. Van Nuys: Alfred Publishing.Google Scholar
  25. 25.
    Pearce, M., & Wiggins, G. (2004). Improved methods for statistical modeling of monophonic music. Journal of New Music Research, 33(4), 367–385.CrossRefGoogle Scholar
  26. 26.
    Perrot, J.-F. (1977). Informatique et Algèbre, La théorie des codes à longueur variable. In Theoretical computer science, G. I. conference, lecture notes in computer science (3rd Ed., no. 48, pp. 27–44).Google Scholar
  27. 27.
    Pesant, G. (2004). A regular language membership constraint for finite sequences of variables. Lecture Notes in Computer Science, 32(58), 482–495.CrossRefGoogle Scholar
  28. 28.
    Rabiner, L. R. (1989). A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2), 257–286.CrossRefGoogle Scholar
  29. 29.
    Radicioni, D. P., & Lombardo, V. (2007). A constraint-based approach for annotating music scores with gestural information. Constraints, 12(4), 405–428.CrossRefzbMATHMathSciNetGoogle Scholar
  30. 30.
    Régin, J.-C. (1994). A filtering algorithm for constraints of difference in CSPs. In Proc. of the 12th national conference on artificial intelligence. AAAI 1994 (pp. 362–367).Google Scholar
  31. 31.
    Ron, D., Singer, Y., & Tishby, N. (1996). The power of amnesia: Learning probabilistic automata with variable memory length. Machine Learning, 25(2–3), 117–149.CrossRefzbMATHGoogle Scholar
  32. 32.
    Rossi, F., van Beek, P., & Walsh, T. (Eds.) (2006). Handbook of constraint programming. New York: Elsevier.zbMATHGoogle Scholar
  33. 33.
    Simhon, S., & Dudek, G. (2004). Sketch interpretation and refinement using statistical models. In H. W. Jensen, & A. Keller (Eds.), Eurographics symposium on rendering.Google Scholar
  34. 34.
    Steedman, M. J. (1984). A generative grammar for jazz chord sequences. Music Perception, 2(1), 52–77.Google Scholar
  35. 35.
    van Hoeve, W. J., & Katriel, I. (2006). Global constraints. Handbook of constraint programming (chap. 6). New York: Elsevier Science Inc.Google Scholar
  36. 36.
    van Hoeve, W.-J., Pesant, G., Rousseau, L.-M., & Sabharwal, A. (2006). Revisiting the sequence constraint. In Proc. of the 12th international conference on principles and practice of constraint programming (CP 2006). LNCS 4204 (pp. 620–634).Google Scholar
  37. 37.
    Witten, I. H., & Bell, T. C. (1991). The zero-frequency problem: Estimating the probabilities of novel events in adaptive text compression. IEEE Transaction on Information Theory, 37(4), 1085–1094.CrossRefGoogle Scholar

Copyright information

© The Author(s) 2010

Authors and Affiliations

  1. 1.Sony CSLParisFrance

Personalised recommendations