Real Time Tracking of Musical Performances

  • Antonio Camarena-Ibarrola
  • Edgar Chávez
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6438)


Real time tracking of musical performances allows for implementation of virtual teachers of musical instruments, automatic accompanying of musicians or singers, and automatic adding of special effects in live presentations.

State of the art approaches make a local alignment of the score (the target audio) and a musical performance, such procedure induce cumulative error since it assumes the rendition to be well tracked up to the current time. We propose searching for the k-nearest neighbors of the current audio segment among all audio segments of the score then use some heuristics to decide the current tracked position of the performance inside the score.

We tested the method with 62 songs, some pop music but mostly classical. For each song we have two performances, we use one of them as the score and the other one as the music to be tracked with excellent results.


entropy index proximity 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Sethares, W.A., Morris, R.D., Sethares, J.C.: Beat tracking of musical performances using low level audio features. IEEE Transactions on Speech and Audio Processing (2) (March 2005)Google Scholar
  2. 2.
    Cano, P., Loscos, A., Bonada, J.: Score-performance matching using hmms. In: ICMC 1999, Audiovisual Institute, Pompeu Fabra University, Spain (1999)Google Scholar
  3. 3.
    Orio, N., Déchelle, F.: Score following using spectral analysis and hidden Markov models. In: Proceedings of the ICMC, pp. 151–154 (2001)Google Scholar
  4. 4.
    Orio, N., Lemouton, S., Schwarz, D.: Score following: state of the art and new developments. In: Proceedings of the, conference on New Interfaces for Musical Expression, National University of Singapore, p. 41 (2003)Google Scholar
  5. 5.
    Sakoe, H., Chiba, S.: Dynamic programming algortihm optimization for spoken word recognition. IEEE Transactions on Acoustics and Speech Signal Processing (ASSP), 43–49 (1978)Google Scholar
  6. 6.
    Dixon, S.: Live tracking of musical performances using on-line time warping. In: 8th International Conference on Digital Audio Effects (DAFx 2005), Austrian Research Institute for Artificial Intelligence, Vienna (September 2005)Google Scholar
  7. 7.
    Dixon, S., Widmer, G.: Match: A music alignment tool chest. In: 6th International Conference on Music Information Retrieval (ISMIR), Austrian Research Institute for Artificial Intelligence, Vienna (2005)Google Scholar
  8. 8.
    Rabiner, L., Juang, B.: An introduction to hidden markov models. IEEE ASSP Magazine 3(1), 4–16 (2003)CrossRefGoogle Scholar
  9. 9.
    Bilmes, J.A.: A gentle tutorial of the EM algorithm and its application to parameter estimation for Gaussian mixture and hidden Markov models. Technical Report TR-97-021, Department of Electrical Engineering and Computer Science U.C. Berkeley (April 1998)Google Scholar
  10. 10.
    Rabiner, R.L.: A tutorial on hidden markov models and selected aplications in speech recognition. Proceedings of the IEEE 77(2), 257–286 (1989)CrossRefGoogle Scholar
  11. 11.
    Camarena-Ibarrola, A., Chavez, E.: On musical performances identification, entropy and string matching. In: Gelbukh, A., Reyes-Garcia, C.A. (eds.) MICAI 2006. LNCS (LNAI), vol. MICAI, pp. 952–962. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  12. 12.
    Camarena-Ibarrola, A., Chavez, E., Tellez, E.S.: Robust radio broadcast monitoring using a multi-band spectral entropy signature. In: 14th Iberoamerican Congress on Pattern Recognition, pp. 587–594. Springer, Heidelberg (2009)Google Scholar
  13. 13.
    Figueroa, K., Chávez, E., Navarro, G.: The sisap metric indexing library,
  14. 14.
    Burkhard, W.A., Keller, R.M.: Some approaches to best-match file searching. ACM Commun. 16(4), 230–236 (1973)CrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Antonio Camarena-Ibarrola
    • 1
  • Edgar Chávez
    • 1
  1. 1.Universidad Michoacana de San Nicolás de HidalgoMoreliaMéxico

Personalised recommendations