ASE — Audio and Synchronization Extension of Compound Documents

Abstract

This paper describes extensions of CDA/DDIF by audio and synchronization information. Audio information can be split into analog audio primitives (like LPs or tapes), digital audio primitives (sampled sound, like that on CD or DAT), and music primitives. Music data are described by two different content types: the logical content and the layout (visual and gestural) content. The logical content describes audio/music events by means of basic events like notes, rests, chords, or user-defined events, which allows for describing not only western tonal music but also a wide variety of musical forms like Hindu or African music. To this logical content the layout (score and/or the dynamic interpretation) can be related. For the layout, more hardware-oriented encodings like MIDI (Musical Instrument Digital Interface) and more revisable symbolic notations are provided. The synchronization aspects, imported with the audio extensions into the compound document format, have been generalized to be applicable to the other document domains. Three different forms can be used: trigger-point, continuous, and related synchronization. Trigger-point synchronization aligns events (like the start of a music output or the display of a picture) at certain points in time. Continuous or relative synchronization allows duration times to be adjusted (e.g., display some image as long as some sound is heard), while related synchronization couples the start times of specific events with termination times of other events.

This project has been funded by Digital’s European External Research Project 377/AU-016.