Score-PCM Music Synchronization Based on Extracted Score Parameters

  • Vlora Arifi
  • Michael Clausen
  • Frank Kurth
  • Meinard Müller
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3310)

Abstract

In this paper we present algorithms for the automatic time-synchronization of score-, MIDI- or PCM-data streams which represent the same polyphonic piano piece. In contrast to related approaches, we compute the actual alignment by using note parameters such as onset times and pitches. Working in a score-like domain has advantages in view of the efficiency and accuracy: due to the expressiveness of score-like parameters only a small number of such features is sufficient to solve the synchronization task. To obtain a score-like representation from the waveform-based PCM-data streams we use a preprocessing step to extract note parameters. In this we use the concept of novelty curves for onset detection and multirate filter banks in combination with note templates for pitch extraction. Also the data streams in MIDI- and score-format have to be suitably preprocessed. In particular, we suggest a data format which handles possible ambiguities such as trills or arpeggios by introducing the concept of fuzzy-notes. Further decisive ingredients of our approach are carefully designed cost functions in combination with an appropriate notion of alignment which is more flexible than the classical DTW concept. Our synchronization algorithms have been tested on a variety of classical polyphonic piano pieces recorded on MIDI- and standard acoustic pianos or taken from CD-recordings.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Arifi, V.: Algorithmen zur Synchronisation von Musikdaten im Paritur-, MIDI- und PCM-Format. PhD thesis, Universität Bonn, Institut für Informatik (2002), http://hss.ulb.uni-bonn.de:90/ulb_bonn/dis_sonline/math_nat_fak/2002/arifivlora
  2. 2.
    Bobrek, M., Koch, D.: Music Signal Segmentation Using Tree-Structured Filter Banks. Journal of Audio Engineering Society 46(5) (May 1998)Google Scholar
  3. 3.
    Dannenberg, R.B.: An On-Line Algorithm for Real-Time Accompaniment. In: Proceedings of ICMC (1984)Google Scholar
  4. 4.
    Dannenberg, R.B., Mukaino, H.: New Techniques for Enhanced Quality of Computer Accompaniment. In: Proceedings of ICMC (1988)Google Scholar
  5. 5.
    Desain, P., Honing, H., Heijink, H.: Robust Score-Performance Matching: Taking Advantage of Structural Information. In: ICMC Proceedings 1997, pp. 377–340 (1997)Google Scholar
  6. 6.
    Foote, J.: Automatic Audio Segmentation Using a Measure of Audio Novelity. In: Proceedings of IEEE International Conference on Multimedia and Expo., vol. I, pp. 452–455 (2000)Google Scholar
  7. 7.
    Foster, S., Schloss, W.A., Rockmore, A.J.: Towards an Intelligent Editor of Digital Audio: Signal Processing Methods Computer Music Journal 6(1) (spring 1982)Google Scholar
  8. 8.
    Kurth, F.: A Matlab Toolbox for Handling Standard MIDI files, Department of Computer Science III, Bonn University, Germany (2003), http://www-mmdb.iai.uni-bonn.de/download/miditools/
  9. 9.
    Large, E.W.: Dynamic programming for the analysis of serial behaviours. Behaviour Research Methods, Instruments, & Computers (1993)Google Scholar
  10. 10.
    Klapuri, A., Virtanen, T., Holm, J.: Robust Multipitch Estimation for the Analysis and Manipulation of Polyphonic Musical Signals. In: Proc. COST-G6 Conference of Digital Audio Effects, DAFx 2000, Verona, Italy (2000)Google Scholar
  11. 11.
    Mazzola, G.: The Topos of Music. Birkhäuser verlag, Basel (2002)MATHGoogle Scholar
  12. 12.
    Orio, N., Lemouton, S., Schwarz, D.: Score Following: State of the Art and New Developments. In: Proc. of the Conference of New Interfaces for Musical Expression NIME, Montreal (2003)Google Scholar
  13. 13.
    Rabiner, L.R., Juang, B.-H.: Fundamentals of Speech Recognition. Prentice Hall, Englewood Cliffs (1993)Google Scholar
  14. 14.
    Raphael, C.: Automatic Segmentation of Acoustic Musical Signals Using Hidden Markov Models. IEEE Transactions on Pattern Analysis and Machine Intelligence 21(4) (April 1999)Google Scholar
  15. 15.
    Raphael, C.: Automatic Transcription of Piano Music. In: ISMIR 2002, IRCAM, Conference Proceedings, Paris (2002)Google Scholar
  16. 16.
    Scheirer, E.D.: Extracting Expressive Performance Information from Recorded Music. Unpublished M. S. thesis, MIT Media Laboratory (1995), http://web.media.mit.edu/eds/thesis.pdf
  17. 17.
    Soulez, F., Rodet, X., Schwarz, D.: Improving polyphonic and poly-instrumental music to score alignment. In: International Conference on Music Information Retrieval, Baltimore (2003)Google Scholar
  18. 18.
    Turetsky, R.J., Ellis, D.P.: Force-Aligning MIDI Syntheses for Polyphonic Music Transcription Generation. In: International Conference on Music Information Retrieval, Baltimore, USA (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Vlora Arifi
    • 1
  • Michael Clausen
    • 1
  • Frank Kurth
    • 1
  • Meinard Müller
    • 1
  1. 1.Institut für Informatik IIIUniversität BonnBonnGermany

Personalised recommendations