Foundations for Reliable and Flexible Interactive Multimedia Scores

  • Jaime Arias
  • Myriam Desainte-Catherine
  • Carlos Olarte
  • Camilo Rueda
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9110)


Interactive Scores (IS) is a formalism for composing and performing interactive multimedia scores with several applications in video games, live performance installations, and virtual museums. The composer defines the temporal organization of the score by asserting temporal relations (TRs) between temporal objects (TOs). At execution time, the performer may modify the start/stop times of the TOs by triggering interaction points and the system guarantees that all the TRs are satisfied. Implementations of IS and formal models of their behavior have already been proposed, but these do not provide usable means to reason about their properties. In this paper we introduce ReactiveIS, a programming language that fully captures the temporal structure of IS during both composition and execution. For that, we propose a semantics based on tree-like structures representing the execution state of the score at each point in time. The semantics captures the hierarchical aspects of IS and provides an intuitive representation of their execution. We also endow ReactiveIS with a logical semantics based on linear logic, thus widening the reasoning techniques available for IS. We show that ReactiveIS is general enough to capture the full behavior of IS and it also provides declarative ways to increase the expressivity of IS with, for instance, conditional statements and loops.


Operational Semantic Execution Model Temporal Organization Label Tree Interactive Score 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Allombert, A., Marczak, R., Desainte-Catherine, M., Baltazar, P.: GarnierLaurent: virage: designing an interactive intermedia sequencer from users requirements and theoretical background. In: International Computer Music Conference (2010)Google Scholar
  2. 2.
    Andreoli, J.M.: Logic programming with focusing proofs in linear logic. J. Log. Comput. 2(3), 297–347 (1992)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Arias, J., Desainte-Catherine, M., Olarte, C., Rueda, C.: Foundations for reliable and flexible interactive multimedia scores. Technical Report, LaBRI, University of Bordeaux, March 2015Google Scholar
  4. 4.
    Danos, V., Joinet, J., Schellinx, H.: The structure of exponentials: uncovering the dynamics of linear logic proofs. In: Mundici, D., Gottlob, G., Leitsch, A. (eds.) KGC 1993. LNCS, vol. 713, pp. 159–171. Springer, Heidelberg (1993) CrossRefGoogle Scholar
  5. 5.
    De la Hogue, T., Baltazar, P., Desainte-Catherine, M., Chao, J., Bossut, C.: OSSIA: open scenario system for interactive applications. In: Journées d’Informatique Musicale, pp. 78–84. Bourges (2014)Google Scholar
  6. 6.
    Desainte-Catherine, M., Allombert, A., Assayag, G.: Towards a hybrid temporal paradigm for musical composition and performance: the case of musical interpretation. Comput. Music J. 37(2), 61–72 (2013)CrossRefGoogle Scholar
  7. 7.
    Girard, J.: Linear logic. Theor. Comput. Sci. 50, 1–102 (1987)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Hoare, T., Menzel, G., Misra, J.: A tree semantics of an orchestration language. In: Broy, M., Gruenbauer, J., Harel, D., Hoare, T. (eds.) Engineering Theories of Software Intensive Systems. NATO Science Series II: Mathematics, Physics and Chemistry, vol. 195, pp. 331–350. Springer, Netherlands (2005) CrossRefGoogle Scholar
  9. 9.
    Marczak, R., Desainte-Catherine, M., Allombert, A.: Real-time temporal control of musical processes. In: Proceedings of the Third International Conferences on Advances in Multimedia, MMEDIA 2011, pp. 12–17 (2011)Google Scholar
  10. 10.
    Nigam, V., Olarte, C., Pimentel, E.: A general proof system for modalities in concurrent constraint programming. In: D’Argenio, P.R., Melgratti, H. (eds.) CONCUR 2013 – Concurrency Theory. LNCS, vol. 8052, pp. 410–424. Springer, Heidelberg (2013) CrossRefGoogle Scholar
  11. 11.
    Olarte, C., Rueda, C.: A declarative language for dynamic multimedia interaction systems. In: Chew, E., Childs, A., Chuan, C.-H. (eds.) MCM 2009. CCIS, vol. 38, pp. 218–227. Springer, Heidelberg (2009) CrossRefGoogle Scholar
  12. 12.
    Sénac, P., de Saqui-Sannes, P., Willrich, R.: Hierarchical time stream petri net: a model for hypermedia systems. In: DeMichelis, G., Díaz, M. (eds.) ICATPN 1995. LNCS, vol. 935, pp. 451–470. Springer, Heidelberg (1995) CrossRefGoogle Scholar
  13. 13.
    Toro, M., Desainte-Catherine, M., Rueda, C.: Formal semantics for interactive music scores: a framework to design, specify properties and execute interactive scenarios. J. Math. Music 8(1), 93–112 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Tran, D.A., Hua, K.A., Vu, K.: VideoGraph: a graphical object-based model for representing and querying video data. In: Laender, A.H.F., Liddle, S.W., Storey, V.C. (eds.) ER 2000. LNCS, vol. 1920, pp. 383–396. Springer, Heidelberg (2000) CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Jaime Arias
    • 1
  • Myriam Desainte-Catherine
    • 1
  • Carlos Olarte
    • 2
  • Camilo Rueda
    • 3
  1. 1.Université de Bordeaux, LaBRI, UMR 5800TalenceFrance
  2. 2.ECTUniversidade Federal do Rio Grande do NorteNatalBrazil
  3. 3.DECCPontificia Universidad Javeriana CaliCaliColombia

Personalised recommendations