Enforcing Structure on Temporal Sequences: The Allen Constraint
Recent applications of constraint programming to entertainment, e.g., music or video, call for global constraints describing the structure of temporal sequences. A typical constraint approach is to model each temporal event in the sequence with one variable, and to state constraints on these indexed variables. However, this approach hampers the statement of constraints involving events based on temporal position, since the position depends on preceding events rather than on the index. We introduce Allen, a global constraint relating event indexes with temporal positions. Allen maintains two set-variables: the set of events occurring at a position defined by an Allen relation, and the set of their indexes. These variables enable defining structural and temporal synchronization properties that cannot be stated on indexed variables. We show that a model based on a local scheduling approach does not solve the problem, even for very small instances, highlighting the need for complex filtering. We present a model that uses Multi-valued Decision Diagrams (MDDs) to compile the Allen constraint. We show that this model can be used to state and solve two complex musical tasks: audio track synchronization and musical score generation.
KeywordsGlobal constraints Temporal sequences Music MDD
This research is conducted within the Flow Machines project which received funding from the European Research Council under the European Unions Seventh Framework Programme (FP/2007-2013)/ERC Grant Agreement n. 291156.
- 1.Derrien, A., Fages, J.-G., Petit, T., Prud’homme, C.: A global constraint for a tractable class of temporal optimization problems. In: Pesant, G. (ed.) CP 2015. LNCS, vol. 9255, pp. 105–120. Springer, Heidelberg (2015)Google Scholar
- 2.Galvane, Q., Christie, M., Lino, C., Ronfard, R.: Camera-on-rails: automated computation of constrained camera paths. In: ACM SIGGRAPH Conference on Motion in Games, Paris, France, November 2015Google Scholar
- 3.Galvane, Q., Ronfard, R., Lino, C., Christie, M.: Continuity editing for 3D animation. In: AAAI Conference on Artificial Intelligence. AAAI Press, Austin, January 2015Google Scholar
- 4.Berrani, S.A., Boukadida, M.H., Gros, P.: Constraint satisfaction programming for video summarization. In: IEEE International Symposium on Multimedia, Anaheim, California, United States. IEEE, December 2013Google Scholar
- 5.Dixon, S.: Onset detection revisited. In: Proceedings of the 9th International Conference on Digital Audio Effects, Citeseer, vol. 120, pp. 133–137 (2006)Google Scholar
- 10.Roy, P., Pachet, F.: Enforcing meter in finite-length markov sequences. In: des Jardins, M., Littman, M.L. (eds.) AAAI. AAAI Press (2013)Google Scholar
- 11.Papadopoulos, A., Pachet, F., Roy, P., Sakellariou, J.: Exact sampling for regular and markov constraints with belief propagation. In: Pesant, G. (ed.) CP 2015. LNCS, vol. 9255, pp. 341–350. Springer, Heidelberg (2015)Google Scholar
- 12.Puget, J.F.: PECOS: a high level constraint programming language. In: Proceedings of Singapore International Conference on Intelligent Systems, SPICIS 1992, pp. 137–142 (1992)Google Scholar
- 14.Perez, G., Régin, J.C., Antipolis, U.N.S., Umr, I.S.: Efficient operations on MDDs for building constraint programming models. In: IJCAI International Joint Conference on Artificial Intelligence, Buenos Aires, Argentina, pp. 374–380 (2015)Google Scholar
- 15.Perez, G., Régin, J.-C.: Improving GAC-4 for table and MDD constraints. In: O’Sullivan, B. (ed.) CP 2014. LNCS, vol. 8656, pp. 606–621. Springer, Heidelberg (2014)Google Scholar
- 16.Perez, G., Régin, J.C.: Relations between MDDs and Tuples and Dynamic Modifications of MDDs based constraints. arXiv preprint (2015). arXiv:1505.02552
- 17.Gómez, E.: Tonal Description of Music Audio Signals. Ph.D. thesis, Universitat Pompeu Fabra (2006)Google Scholar