Markov chains are a well known tool to model temporal properties of many phenomena, from text structure to fluctuations in economics. Because they are easy to generate, Markovian sequences, i.e. temporal sequences having the Markov property, are also used for content generation applications such as text or music generation that imitate a given style. However, Markov sequences are traditionally generated using greedy, left-to-right algorithms. While this approach is computationally cheap, it is fundamentally unsuited for interactive control. This paper addresses the issue of generating steerable Markovian sequences. We target interactive applications such as games, in which users want to control, through simple input devices, the way the system generates a Markovian sequence, such as a text, a musical sequence or a drawing. To this aim, we propose to revisit Markov sequence generation as a branch and bound constraint satisfaction problem (CSP). We propose a CSP formulation of the basic Markovian hypothesis as elementary Markov Constraints (EMC). We propose algorithms that achieve domain-consistency for the propagators of EMCs, in an event-based implementation of CSP. We show how EMCs can be combined to estimate the global Markovian probability of a whole sequence, and accommodate for different species of Markov generation such as fixed order, variable-order, or smoothing. Such a formulation, although more costly than traditional greedy generation algorithms, yields the immense advantage of being naturally steerable, since control specifications can be represented by arbitrary additional constraints, without any modification of the generation algorithm. We illustrate our approach on simple yet combinatorial chord sequence and melody generation problems and give some performance results.
Addessi, A.-R., & Pachet, F. (2005). Experiments with a musical machine: Musical style replication in 3/5 year old children. British Journal of Music Education, 22(1), 21–46.
Allauzen, C., Crochemore, M., & Raffinot, M. (1999). Factor oracle: A new structure for pattern matching. In J. Pavelka, G. Tel, & M. Bartosek (Eds.), Proceedings of the 26th conference on current trends in theory and practice of informatics (November 27–Dec. 4), LNCS (Vol. 1725, pp. 295–310). London: Springer.
Anders, T., & Miranda, E. R. (2008). Constraint-based composition in real time. In Proc. of int. computer music conference, Belfast, UK.
Anders, T., & Miranda, E. R. (2010). Constraint programming systems for modeling music theories and composition. ACM Computing Surveys (in press).
Assayag, G., Dubnov, S., & Delerue, O. (1999). Guessing the composer’s mind: Applying universal prediction to musical style. In Proc. of int. computer music conference, Bejing, China, ICMA.
Bejerano, G. (2004). Algorithms for variable length Markov chain modeling. Bioinformatics, 20(5), 788–789.
Beldiceanu, N., Carlsson, M., & Demassey, S. (2007). Global constraint catalogue: Past, present and future. Constraints, 12(1), 32–62
Brooks, F. P., Hopkins, A. L., Neumann, P. G., & Wright, W. V. (1993). An experiment in musical composition. In S. M. Schwanauer & D. A. Levitt (Eds.), Machine models of music (pp. 23–40). Cambridge: MIT.
Conklin, D. (2003). Music generation from statistical models. In Proc. of AISB 2003 symposium on artificial intelligence and creativity in the arts and sciences (pp. 30–35).
Conklin, D., & Witten, I. H. (1995). Multiple viewpoint systems for music prediction. Journal of New Music Research, 24(1), 51–73.
Cont, A., Dubnov, S., & Assayag, G. (2007). Anticipatory model of musical style imitation using collaborative and competitive reinforcement learning. In Anticipatory behavior in adaptive learning systems. LNCS (Vol. 4520/2007, pp. 285–306). Berlin: Springer.
Demassey, S., Pesant, G., & Rousseau, L.-M. (2006). A cost-regular based hybrid column generation approach. Constraints, 11(4), 315–333.
Di Meola, A., McLaughlin, J., & De Lucia, P. (1992). Friday night in San Francisco. Milwaukee: Hal Leonard Corporation.
Eigenfield, A., & Pasquier, P. (2010). Realtime generation of harmonic progressions using controlled markov selection. In Proc. of the 1st international conference on computational creativity (ICCCX) (pp. 16–25). Lisbon: ACM.
Farbood, M., & Schoner, B. (2001). Analysis and synthesis of palestrina-style counterpoint using Markov chains. In Proc. of international computer music conference, Cuba (pp. 471–474).
Ilog (2001) Ilog solver 5.2 reference manual. Available online at https://www.enseignement.polytechnique.fr/local/documentation/ilog/solver52/doc/refman/html/IlcTableConstraint.html.
Johnson, D. S. (1974). Approximation algorithms for combinatorial problems. Journal of Computer and System Sciences, 9(3), 256–278.
Marinescu, R., & Dechter, R. (2005). Advances in AND/OR branch-and-bound search for constraint optimization. In The 7th international workshop on preferences and soft constraints of the eleventh international conference on principles and practice of constraint programming, CP’2005.
Markov Constraints (2010). Available online at http://www.csl.sony.fr/MarkovCt. Accessed September 2010.
Menana, J., & Demassey, S. (2009). Sequencing and counting with the multicost-regular constraint. In Lecture notes in computer science, 6th international conference on integration of AI and OR techniques in constraint programming for combinatorial optimization problems (CPAIOR’09), Pittsburgh, USA.
Nierhaus, G. (2009). Algorithmic composition—paradigms of automated music generation. Berlin: Springer.
Pachet, F. (2002). The continuator: Musical interaction with style. In Proc. of international computer music conference (pp. 211–218). Goteborg, Sweden. September, Best Paper Award.
Pachet, F., Roy, P., & Cazaly, D. (2000). A combinatorial approach to content-based music selection. IEEE Multimedia, 7(1), 44–51.
Parker, C. (2001). Charlie Parker omnibook: For C instruments. Van Nuys: Alfred Publishing.
Pearce, M., & Wiggins, G. (2004). Improved methods for statistical modeling of monophonic music. Journal of New Music Research, 33(4), 367–385.
Perrot, J.-F. (1977). Informatique et Algèbre, La théorie des codes à longueur variable. In Theoretical computer science, G. I. conference, lecture notes in computer science (3rd Ed., no. 48, pp. 27–44).
Pesant, G. (2004). A regular language membership constraint for finite sequences of variables. Lecture Notes in Computer Science, 32(58), 482–495.
Rabiner, L. R. (1989). A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2), 257–286.
Radicioni, D. P., & Lombardo, V. (2007). A constraint-based approach for annotating music scores with gestural information. Constraints, 12(4), 405–428.
Régin, J.-C. (1994). A filtering algorithm for constraints of difference in CSPs. In Proc. of the 12th national conference on artificial intelligence. AAAI 1994 (pp. 362–367).
Ron, D., Singer, Y., & Tishby, N. (1996). The power of amnesia: Learning probabilistic automata with variable memory length. Machine Learning, 25(2–3), 117–149.
Rossi, F., van Beek, P., & Walsh, T. (Eds.) (2006). Handbook of constraint programming. New York: Elsevier.
Simhon, S., & Dudek, G. (2004). Sketch interpretation and refinement using statistical models. In H. W. Jensen, & A. Keller (Eds.), Eurographics symposium on rendering.
Steedman, M. J. (1984). A generative grammar for jazz chord sequences. Music Perception, 2(1), 52–77.
van Hoeve, W. J., & Katriel, I. (2006). Global constraints. Handbook of constraint programming (chap. 6). New York: Elsevier Science Inc.
van Hoeve, W.-J., Pesant, G., Rousseau, L.-M., & Sabharwal, A. (2006). Revisiting the sequence constraint. In Proc. of the 12th international conference on principles and practice of constraint programming (CP 2006). LNCS 4204 (pp. 620–634).
Witten, I. H., & Bell, T. C. (1991). The zero-frequency problem: Estimating the probabilities of novel events in adaptive text compression. IEEE Transaction on Information Theory, 37(4), 1085–1094.
About this article
Cite this article
Pachet, F., Roy, P. Markov constraints: steerable generation of Markov sequences. Constraints 16, 148–172 (2011). https://doi.org/10.1007/s10601-010-9101-4
- Markov chains
- Sequence constraints
- Global constraints
- Interactive applications