Foundations for Reliable and Flexible Interactive Multimedia Scores
Interactive Scores (IS) is a formalism for composing and performing interactive multimedia scores with several applications in video games, live performance installations, and virtual museums. The composer defines the temporal organization of the score by asserting temporal relations (TRs) between temporal objects (TOs). At execution time, the performer may modify the start/stop times of the TOs by triggering interaction points and the system guarantees that all the TRs are satisfied. Implementations of IS and formal models of their behavior have already been proposed, but these do not provide usable means to reason about their properties. In this paper we introduce ReactiveIS, a programming language that fully captures the temporal structure of IS during both composition and execution. For that, we propose a semantics based on tree-like structures representing the execution state of the score at each point in time. The semantics captures the hierarchical aspects of IS and provides an intuitive representation of their execution. We also endow ReactiveIS with a logical semantics based on linear logic, thus widening the reasoning techniques available for IS. We show that ReactiveIS is general enough to capture the full behavior of IS and it also provides declarative ways to increase the expressivity of IS with, for instance, conditional statements and loops.
KeywordsOperational Semantic Execution Model Temporal Organization Label Tree Interactive Score
- 1.Allombert, A., Marczak, R., Desainte-Catherine, M., Baltazar, P.: GarnierLaurent: virage: designing an interactive intermedia sequencer from users requirements and theoretical background. In: International Computer Music Conference (2010)Google Scholar
- 3.Arias, J., Desainte-Catherine, M., Olarte, C., Rueda, C.: Foundations for reliable and flexible interactive multimedia scores. Technical Report, LaBRI, University of Bordeaux, March 2015Google Scholar
- 5.De la Hogue, T., Baltazar, P., Desainte-Catherine, M., Chao, J., Bossut, C.: OSSIA: open scenario system for interactive applications. In: Journées d’Informatique Musicale, pp. 78–84. Bourges (2014)Google Scholar
- 8.Hoare, T., Menzel, G., Misra, J.: A tree semantics of an orchestration language. In: Broy, M., Gruenbauer, J., Harel, D., Hoare, T. (eds.) Engineering Theories of Software Intensive Systems. NATO Science Series II: Mathematics, Physics and Chemistry, vol. 195, pp. 331–350. Springer, Netherlands (2005) CrossRefGoogle Scholar
- 9.Marczak, R., Desainte-Catherine, M., Allombert, A.: Real-time temporal control of musical processes. In: Proceedings of the Third International Conferences on Advances in Multimedia, MMEDIA 2011, pp. 12–17 (2011)Google Scholar