A Framework for Managing Multimodal Digitized Music Collections

  • Frank Kurth
  • David Damm
  • Christian Fremerey
  • Meinard Müller
  • Michael Clausen
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5173)


In this paper, we present a framework for managing heterogeneous, multimodal digitized music collections containing visual music representations (scanned sheet music) as well as acoustic music material (audio recordings). As a first contribution, we propose a preprocessing workflow comprising feature extraction, audio indexing, and music synchronization (linking the visual with the acoustic data). Then, as a second contribution, we introduce novel user interfaces for multimodal music presentation, navigation, and content-based retrieval. In particular, our system offers high quality audio playback with time-synchronous display of the digitized sheet music. Furthermore, our system allows a user to select regions within the scanned pages of a musical score in order to search for musically similar sections within the audio documents. Our novel user interfaces and search functionalities will be integrated into the library service system of the Bavarian State Library as part of the Probado project.


Audio Recording Musical Work Chroma Feature Sheet Music Music Retrieval 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Wang, P., Bunke, H.: Handbook on Optical Character Recognition and Document Image Analysis. World Scientific, Singapore (1997)Google Scholar
  2. 2.
    Witten, I.H., Moffat, A., Bell, T.C.: Managing Gigabytes, 2nd edn. Van Nostrand Reinhold (1999)Google Scholar
  3. 3.
    Baeza-Yates, R.A., Ribeiro-Neto, B.A.: Modern Information Retrieval. ACM Press, Addison-Wesley (1999)Google Scholar
  4. 4.
    Ohta, M., Takasu, A., Adachi, J.: Retrieval methods for English-text with missrecognized OCR characters. In: ICDAR 1997: Proceedings of the 4th International Conference on Document Analysis and Recognition, Washington, DC, USA, pp. 950–956. IEEE Computer Society, Los Alamitos (1997)CrossRefGoogle Scholar
  5. 5.
    Harding, S.M., Croft, W.B., Weir, C.: Probabilistic Retrieval of OCR Degraded Text Using N-Grams. In: Peters, C., Thanos, C. (eds.) ECDL 1997. LNCS, vol. 1324, pp. 345–359. Springer, Heidelberg (1997)CrossRefGoogle Scholar
  6. 6.
    Google Inc.: Google Book Search (2007),
  7. 7.
    Kurth, F., Müller, M., Fremerey, C., Chang, Y., Clausen, M.: Automated Synchronization of Scanned Sheet Music with Audio Recordings. In: Proc. ISMIR, Vienna, Austria, pp. 261–266 (September 2007)Google Scholar
  8. 8.
    Krottmaier, H., Kurth, F., Steenweg, T., Appelrath, H.J., Fellner, D.: PROBADO - A Generic Repository Integration Framework. In: Proceedings of the 11th European Conference on Digital Libraries (September 2007)Google Scholar
  9. 9.
    Bartsch, M.A., Wakefield, G.H.: Audio thumbnailing of popular music using chroma-based representations. IEEE Trans. on Multimedia 7(1), 96–104 (2005)CrossRefGoogle Scholar
  10. 10.
    Hu, N., Dannenberg, R., Tzanetakis, G.: Polyphonic audio matching and alignment for music retrieval. In: Proc. IEEE WASPAA, New Paltz, NY (October 2003)Google Scholar
  11. 11.
    Müller, M.: Information Retrieval for Music and Motion. Springer, Heidelberg (2007)Google Scholar
  12. 12.
    Choudhury, G., DiLauro, T., Droettboom, M., Fujinaga, I., Harrington, B., MacMillan, K.: Optical music recognition system within a large-scale digitization project. In: Proc. ISMIR, Plymouth, MA, USA (2000)Google Scholar
  13. 13.
    Byrd, D., Schindele, M.: Prospects for improving OMR with multiple recognizers. In: Proc. ISMIR, Victoria, Canada, pp. 41–46 (2006)Google Scholar
  14. 14.
    Jones, G.: SharpEye Music Reader (2008),
  15. 15.
    Kurth, F., Müller, M.: Efficient Index-based Audio Matching. IEEE Transactions on Audio, Speech, and Language Processing 16(2), 382–395 (2008)CrossRefGoogle Scholar
  16. 16.
    Gracenote: WWW (2008),
  17. 17.
    Krajewski, E.: DE-PARCON Softwaretechnologie (2008),
  18. 18.
    Arifi, V., Clausen, M., Kurth, F., Müller, M.: Synchronization of music data in score-, MIDI- and PCM-format. Computing in Musicology 13 (2004)Google Scholar
  19. 19.
    Dunn, J.W., Byrd, D., Notess, M., Riley, J., Scherle, R.: Variations2: Retrieving and using music in an academic setting. Special Issue, Commun. ACM 49(8), 53–58 (2006)Google Scholar
  20. 20.
    Diet, J., Kurth, F.: The Probado Music Repository at the Bavarian State Library. In: Proc. ISMIR, Vienna, Austria, pp. 501– 504 (September 2007)Google Scholar
  21. 21.
    IFLA Study Group on the Functional Requirements of Bibliographic Records: Functional Requirements for Bibliographic Records; Final Report. Saur, Munich (1998),

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Frank Kurth
    • 1
  • David Damm
    • 2
  • Christian Fremerey
    • 2
  • Meinard Müller
    • 3
  • Michael Clausen
    • 2
  1. 1.Research Establishment for Applied Science (FGAN), FKIE-KOMWachtbergGermany
  2. 2.Department of Computer Science IIIUniversity of BonnBonnGermany
  3. 3.Max-Planck-Institut für Informatik,Department D4 - Computer GraphicsSaarbrückenGermany

Personalised recommendations